FaceLift: a transparent deep learning framework to beautify urban scenes


In the area of computer vision, deep learning techniques have recently been used to predict whether urban scenes are likely to be considered beautiful: it turns out that these techniques are able to make accurate predictions. Yet they fall short when it comes to generating actionable insights for urban design. To support urban interventions, one needs to go beyond predicting beauty, and tackle the challenge of recreating beauty. Unfortunately, deep learning techniques have not been designed with that challenge in mind. Given their ‘black-box nature’, these models cannot be directly used to explain why a particular urban scene is deemed to be beautiful. To partly fix that, we propose a deep learning framework (which we name FaceLift1) that is able to both beautify existing urban scenes (Google Street Views) and explain which urban elements make those transformed scenes beautiful. To quantitatively evaluate our framework, we cannot resort to any existing metric (as the research problem at hand has never been tackled before) and need to formulate new ones. These new metrics should ideally capture the presence (or absence) of elements that make urban spaces great. Upon a review of the urban planning literature, we identify five main metrics: walkability, green spaces, openness, landmarks and visual complexity. We find that, across all the five metrics, the beautified scenes meet the expectations set by the literature on what great spaces tend to be made of. This result is further confirmed by a 20-participant expert survey in which FaceLift has been found to be effective in promoting citizen participation. All this suggests that, in the future, as our framework’s components are further researched and become better and more sophisticated, it is not hard to imagine technologies that will be able to accurately and efficiently support architects and planners in the design of the spaces we intuitively love.

Royal Society Open Science
Sagar Joglekar
Sagar Joglekar
Former PhD student (now Research Scientist at Bell Labs Cambridge)

I am a Research Scientist at Nokia Bell labs, Cambridge UK, working with the social dynamics team. I am mainly interested in projects that deal with quantification of human processes from web scale data using methods from complex networks, machine learning and computer vision. I was a King’s India scholar at King’s College London, where I worked on my Ph.D. in computer science at the Department of Informatics, under the guidance of Dr. Nishanth Sastry. I have graduated with a Masters of Science (M.S.) degree from University of California at Santa Barbara - USA, majoring in signals processing and networks, and a Bachelors of Engineering (B.Eng) from University of Pune - India, majoring in Electronics engineering .

Miriam Redi
Miriam Redi
Visiting Researcher, Wikimedia Research