Semantic image editing for reshaping architecture of power: Lesson learned from selected cases

Objectives We explore an evaluation scheme for assessment of generative computer vision models in architecture-related tasks with a focus on text-conditioned image editing for use cases relating to architecture of power. It is an umbrella term for building ranging from Socialist Realism to Post-War Modernism. While some of them can be considered landmarks on former Eastern Bloc countries, they often lack modern features, such as accessibility. With a recent progress in generative vision, the diffusion pipelines can be used to reimagine such buildings with pictures, which may later provide a blueprint for transforming such sites. Prior work While an intense effort can be observed in image generation models (including semantic image editing) and their applications (such as architecture), evaluating domain-specific benchmarks is still cumbersome. The case of architecture of power carries unique challenges, as it is a domain rather underrepresented in the publicly available datasets on which many models are pretrained. Results We present selected results of our evaluation schema for assessing generative vision models for various tasks related to improving mid-20th century architecture, which consist of taxonomy of tasks. We also demonstrate the proposed approach on a several state-of-the-art text- and image-conditioned diffusion models and pipelines (such as DiffEdit, Kandinsky, or ControlNet) for selected buildings in Warsaw, Cracow, Riga, and Bucharest. Implications While the presented evaluation scheme is rather intended to be used by researchers, the results of such an assessment can be used to select models most suitable for the architecture and urban planning communities. Since we focus on text-conditioned models, they can be used by general audience to help reimagining the buildings according to their need.