Abstract
Sketch is a powerful communication tool naturally embedded in humans, allowing the expression of complex ideas through relatively simple forms. Unlike raster images, which are dense grid-based representations captured by cameras or sensors, sketches should be treated as inherently different and abstract data structures. In this work, we explore and develop sketch-based generation frameworks in vector drawing formats that better emulate the human creative process.
The motivation for this research is twofold: (i) to emphasise the intrinsic creativity of human sketches and the importance of strokes in creative vector drawing generation tasks, and (ii) to develop vector-based sketch generation models that enhance 2D and 3D digital workflows for both artists and non-experts. Our research consists of three main works: (i) SketchXAI tries to explore creativity by reorganising the strokes of sketches with our proposed model, which introduces a novel framework where classification models are adapted to generate vector drawings, emphasising the importance of strokes in creative sketch generation tasks. (ii) SketchDreamer extends this approach by introducing a text-driven framework for vector sketch generation. By integrating vision and language models into our framework, SketchDreamerallows users to describe their creative ideation through text prompts and transforms the simple drawings into detailed vector drawings step by step. (iii) DreamWire, which represents a significant leap into the 3D space, enabling the generation of multi-view wire art -- an intricate form of sculpture that reveals different interpretations when viewed from various angles.
We look forward to our research culminate in a vision of future where AI plays a central role in expanding the possibilities of artistic expression, making sketch -- a complex and abstract form of art accessible to all.