We propose GroundUp, the first sketch-based ideation tool for 3D city "massing" of urban areas. The focus here is on early-stage urban design, where sketching is a common tool and the design starts from balancing planned building volumes (masses) and open spaces. With Human-Centered AI in mind, we aim to help architects quickly revise their ideas by easily switching between 2D sketches and 3D models, allowing for smoother iteration and sharing of ideas. Inspired by feedback from architects and existing workflows, our system takes as a first input a user sketch of multiple buildings in a top-down view. The user then draws a perspective sketch of the envisioned site. Our method is designed to exploit the complementarity of information in the two sketches and allows users to quickly preview and adjust the inferred 3D shapes. Our model, driving the proposed urban massing system, has two main components. First, we propose a novel sketch-to-depth prediction network for perspective sketches that exploits top-down view cues. Second, we amalgamate the complementary but sparse 2D signals to condition a customarily trained latent diffusion model. The diffusion model works in the domain of a heightfield, allowing users to construct the city "from the ground up".