Researchers have used Google Street View and AI to study how buildings, green spaces, pavements and roads in US cities affect heart disease in people living in urban neighbourhoods. The study, including major US cities in the country’s Midwest, South and Western regions, found that urban environmental factors can predict 63% of variation in heart risk between areas.

Machine vision and convolutional neural network-based AI were used to analyse the Google Street View images and align hundreds of separate urban and built environmental factors with disease incidence.

The study was conducted by a team from Case Western Reserve University, Ohio and University Hospitals Harrington Heart and Vascular Institute. It has been published in the European Heart Journal.

According to the researchers, using Google Street View reveals the heart disease risk factors in the urban environment and could help adapt towns and cities to reduce the risk of heart disease.

The study was led by Prof Sadeer Al-Kindi and Prof Sanjay Rajagopalan from University Hospitals Harrington Heart & Vascular Institute and Case Western Reserve University, Ohio, USA, and Dr Zhuo Chen, a post-doctoral fellow in Prof Rajagopalan’s laboratory.

The study included more than half a million Google Street View images of Detroit, Michigan; Kansas City, Missouri; Cleveland, Ohio; Brownsville, Texas; Fremont, California; Bellevue, Washington State; and Denver, Colorado.

Researchers also collected data on rates of coronary heart disease according to ‘census tracts’. These are areas smaller than a US zip code that are home to an average of 4,000 people. Using convolutional neural networks provided recognition and analysis of predictive patterns in the Street View images.

Prof Rajagopalan said: “We have always been interested in how the environment, both the built and natural environment, influences cardiovascular disease. Here in America, they say that the zip code is a better predictor of heart disease than even personal measures of health. However, to investigate how the environment influences large populations in multiple cities is no mean task. Hence, we used machine vision-based approaches to assess the links between the built environment and coronary heart disease prevalence in US cities.”

Prof Al-Kindi said: “We also used an approach called attention mapping, which highlights some of the important regions in the image, to provide a semi-qualitative interpretation of some of the thousands of features that may be important in coronary heart disease. For instance, features like green space and walkable roads were associated with lower risk, while other features, such as poorly paved roads, were associated with higher risk. However, these findings need further investigation.

“We’ve shown that we can use computer vision approaches to help identify environmental factors influencing cardiovascular risk and this could play a role in guiding heart-healthy urban planning. The fact that we can do this at scale is something that is absolutely unique and important for urban planning.”

“With upcoming challenges including climate change and a shifting demographic, where close to 70% of the world’s population will live in urban environments, there is a compelling need to understand urban environments at scale, using computer vision approaches that can provide exquisite detail at an unparalleled level,” said Prof Rajagopalan.

Dr Rohan Khera from Yale University School of Medicine, USA said: “The association of residential location with outcomes often supersedes that of known biological risk factors. This is often summarised with the expression that a person’s postal code is a bigger determinant of their health than their genetic code. However, our ability to appropriately classify environmental risk factors has relied on population surveys that track wealth, pollution, and community resources.

“The study by Chen and colleagues presents a novel and more comprehensive evaluation of the built environment. This work creatively leverages Google’s panoramic street-view imagery that supplements its widely used map application.

“An AI-enhanced approach to studying the physical environment and its association with cardiovascular health highlights that across our communities, measures of cardiovascular health are strongly encoded in merely the visual appearance of our neighbourhoods. It is critical to use this information wisely, both in defining strategic priorities for identifying vulnerable communities and in redoubling efforts to improve cardiovascular health in communities that need it most.”