Abstract
Deep image-based modeling received lots of attention in recent years, yet the
parallel problem of sketch-based modeling has only been briefly studied, often
as a potential application. In this work, for the first time, we identify the
main differences between sketch and image inputs: (i) style variance, (ii)
imprecise perspective, and (iii) sparsity. We discuss why each of these
differences can pose a challenge, and even make a certain class of image-based
methods inapplicable. We study alternative solutions to address each of the
difference. By doing so, we drive out a few important insights: (i) sparsity
commonly results in an incorrect prediction of foreground versus background,
(ii) diversity of human styles, if not taken into account, can lead to very
poor generalization properties, and finally (iii) unless a dedicated sketching
interface is used, one can not expect sketches to match a perspective of a
fixed viewpoint. Finally, we compare a set of representative deep single-image
modeling solutions and show how their performance can be improved to tackle
sketch input by taking into consideration the identified critical differences.