Very detailed and thoughtful discussion of the issue in regards to how challenging it is to prove something in the social sciences including the challenges of isolating variables, proving causality, etc.
The spanking literature, however, has addressed itself to this problem in several ways. First, in the absence of true experimentation, an argument for causality can still be supported indirectly if three conditions are met: first, there’s a link between behavior A and outcome B. Second, behavior A appears before outcome B in the timeline (which can be documented using longitudinal studies following the same kids over time). Third, other explanations for the A-B link are ruled out (for example stress, which may cause parents to spank and children to deteriorate).
Spanking research has by now produced robust evidence for all three propositions. Spanking is correlated strongly and quite exclusively with multiple negative outcomes for children. The negative outcomes often appear only after the spanking has begun, and the effects of spanking remain significant and sizable even after controlling for the influence of other variables such as parental age, child age, sex, race, family structure, poverty, emotional support, cognitive stimulation, etc.