Abstract
Adversarial attacks have emerged in the field of visual object tracking to mislead the tracker and result in its failure. Black-box attacks in particular have attracted increasing attention for their affinity with real-world applications. In the paradigm of decision-based black-box attacks, the magnitude of perturbation is gradually amplified, while the optimisation direction is predefined by an initial adversarial sample. Considering the pivotal role played by the initial adversarial sample in determining the success of an attack, we utilise the noise generated from the reverse process of a diffusion model as a better attacking direction. On the one hand, the diffusion model generates Gaussian noise, which formulate global information interaction, with a comprehensive impact on Transformer-based trackers. On the other hand, the diffusion model pays more attention to the target region during the inverse process, resulting in a more powerful perturbation of the target object. Our method, which is widely applicable, has been validated on a range of trackers using several benchmarking datasets. It is shown to deliver more extensive tracking performance degradation, compared to other state-of-the-art methods. We also investigate different approaches to the problem of generating the initial adversarial sample, confirming the effectiveness and rationality of our proposed diffusion initialisation method.