Crawler for Nodes in the Internet of Things

[Abstract] Determining the application and version of nodes in the Internet of Things (IoT) is very important for warning about and managing vulnerabilities in the IoT. This article defines the attributes for determining the application and version of nodes in the IoT. By improving the structure of the Internet web crawler, which obtains raw data from nodes, we can obtain data from nodes in the IoT. We improve on the existing strategy, in which only determinations are stored, by also storing downloaded raw data locally in MongoDB. This stored raw data can be conveniently used to determine application type and node version when a new determination method emerges or when there is a new application type or node version. In such instances, the crawler does not have to scan the Internet again. We show through experimentation that our crawler can crawl the IoT and obtain data necessary for determining the application type and node version.

[Keywords] crawler; local storage; nodes; Internet of Things