What are Web Reconnaissance Tools?
There are several tools out there that continuously scan the internet and make their findings available through a search engine. They can be described in different ways, such as IoT search engine, Search engine for service banners, Certificate search engine, Internet scanner, etc. We will refer to them here generically as Web Reconnaissance Platforms.
The most popular tools of this type include Shodan, ZoomEye, Censys, and Greynoise. The feature set and capabilities of each can vary but generally include a search functionality. Some platforms also include monitoring or alerting features.
What Are The Risks Using Web Reconnaissance Tools?
We described in a previous article how these platforms are at the intersection of port scanning and OSINT, and why organizations should be monitoring them. At a high level, the risk of an exposed service being exploited increases significantly when it is publicly available and indexed by these search engines since malicious actors use these platforms to build lists of targets. An actor looking to exploit a vulnerability on RDP services might, for example, query Shodan for all available 3389 ports, the default RDP port, which returns at the time of writing over 4 million results. An actor may also search for a specific vulnerability, identified by Shodan by looking at the service banner versions, and use this to start a campaign.
How to Monitor Web Reconnaissance Tools?
1. Monitor All Platforms
There are a number of platforms that perform similar tasks and provide this intelligence to malicious actors. A new web reconnaissance tool appears every year or so and has varying coverage, features, and scanning frequency, making monitoring a single one only partially useful. Some platforms require subscriptions, while others only provide a Russian or Chinese interface. Finally, not all platforms provide an alerting or monitoring feature, so recurring manual searches or custom integration with their APIs is required.
Covering all platforms is the first step to achieving a comprehensive view of the potential threats.
2. Monitor All Infrastructure, Domains, and Hosts
The tools usually provide a simple search feature that enables searching by domain or IP address. Unfortunately, organizations tend to have hundreds or thousands of potential hosts to monitor through the same amount of domains, subdomains, and addresses. Organizations often do not even have a comprehensive list of what should be monitored.
To achieve complete coverage, the ideal approach is to combine the scanning with a solution that automatically detects and discovers all relevant URLs and FQDNs, and uses this list as a starting point to monitor the scanning tools. Running this continuous discovery process will also allow coverage for changing DNS A records, and keep the list of monitored IP addresses in sync with the actual footprint of the organization.
3. Prioritize Results
The larger the infrastructure, the higher the number of results from the reconnaissance platforms. Many, if not the majority, of results, will represent expected open ports such as the main corporate website of the organization, or an often-used SFTP server. The challenge, therefore, lies in finding unexpected network exposures, such as a publicly accessible database or a mistakenly opened RDP port. A contextual prioritization layer, similar to what can be found in a Vulnerability Management solution, will therefore be very useful to identify which open ports pose an actual risk and eliminate the noise caused by normal internet activity.
4. Streamline Threat Mitigation
There are a finite number of threats that can be found by web reconnaissance tools. Integrating the scanning results in a SOAR, SIEM, TIP or XDR will enable the organization to attach playbooks and automation to the different use cases and streamline, or even automate, the response. Common use cases that should be considered include ports for protocols like RDP, SSH, FTP, and ports for open databases like MySQL, PostgreSQL, and ElasticSearch. This last database, hosted by default on 9200, has been repeatedly responsible for large data breaches in recent months, including Cognyte and The Telegraph.
5. Store Activity for Incident Response and Investigations
Finally, web reconnaissance tool data should be stored and saved to help when responding to incidents. The search feature of the tool typically only provides the most recent scanning data. In cases where older data is available (the case for Shodan and Censys), it tends to be difficult to browse or require an active subscription. Having historical data in a clear, prioritized, and digestible format can provide significant insights into past network activity and help understand what was visible to malicious actors at a defined point in time.
The Flare Platform
The Flare platform searches and monitors web reconnaissance platforms based on IP addresses, domains, and auto-discovered subdomains. It provides wide coverage for your organization and can help streamline response to the threat of web reconnaissance platforms. The alerting system will notify operations teams as soon as potential high-risk events occur and help them remediate quickly before malicious actors can take advantage of them.
