Abstract
We present the first competitive drawing agent Pixelor that exhibits human-level
performance at a Pictionary-like sketching game, where the participant whose sketch is recognized first is a winner. Our AI agent can autonomously
sketch a given visual concept, and achieve a recognizable rendition as quickly
or faster than a human competitor. The key to victory for the agent’s goal
is to learn the optimal stroke sequencing strategies that generate the most
recognizable and distinguishable strokes first. Training Pixelor is done in two
steps. First, we infer the stroke order that maximizes early recognizability of
human training sketches. Second, this order is used to supervise the training
of a sequence-to-sequence stroke generator. Our key technical contributions
are a tractable search of the exponential space of orderings using neural
sorting; and an improved Seq2Seq Wasserstein (S2S-WAE) generator that
uses an optimal-transport loss to accommodate the multi-modal nature of the
optimal stroke distribution. Our analysis shows that Pixelor is better than the
human players of the Quick, Draw! game, under both AI and human judging
of early recognition. To analyze the impact of human competitors’ strategies,
we conducted a further human study with participants being given unlimited
thinking time and training in early recognizability by feedback from an AI
judge. The study shows that humans do gradually improve their strategies
with training, but overall Pixelor still matches human performance. The code
and the dataset are available at http://sketchx.ai/pixelor.