Road inventory data is collected manually to identify the road environment near crash locations. In this study, we automatically detect median on roads and presence of intersections from aerial images. Data is manually as well as automatically collected from two different cities (Chennai and Trichy) in the state of Tamil Nadu in India using Google Application Programming Interface (API). An image recognition model is built using Convolutional Neural Networks (CNN). Multiple models were built using ResNet architecture comprising training dataset from same city as of test set and mixed dataset of both the cities to test the model’s generalizability. F1 scores are used to rate the model performance. The results reveal that the model’s F1 scores increase when training data comprises images from both cities. This work makes two contributions. Firstly, it describes how CNN can be utilized for road safety research and secondly, the proposed dataset can be used in future to build the model for other cities so that manual data collection can be minimized. Lastly, recommendations are made for more such work in future.