03-How to quickly build information collection of network assets

Article Directory

I have sorted out the knowledge points related to "information collection" before, and I personally feel that it is not particularly perfect. Compiling the knowledge points related to "information collection" again is also a supplement to the previous one, and it is also a summary of common methods for quick asset collection in actual combat.

https://blog.csdn.net/weixin_42250835/article/details/111411855 First knowledge of information collection
https://blog.csdn.net/weixin_42250835/article/details/111460459 How to bypass CDN to query the real IP of the website
https:// blog.csdn.net/weixin_42250835/article/details/111464945 Information collection of subdomains
https://blog.csdn.net/weixin_42250835/article/details/111474285 Git information leakage vulnerability and GitHack usage
https: //blog.csdn .net/weixin_42250835/article/details/111488925 Information disclosure vulnerability
https://blog.csdn.net/weixin_42250835/article/details/111566350 Summary of information collection articles

Common basic information collection

Operating system recognition: upper and lower case, TTL value, combination correspondence, etc.

1、Linux对大小写敏感,Windows对大小写不敏感
  实例:通过URL改变大小写查看页面响应是否正常
  TTL值:通过ping命令得到对方服务的TTL回显值(不是很标准)

  组合对应:根据Linux、Windows服务器搭建特性或常见的搭建组合去猜解
  aspx mssql windows iis
  php mysql windows/linux apache
  jsp mssql/oracle windows/linux tomcat
  javaee mysql/oracle/ windows/linux weblogic/jboos/tomcat

2.中间件平台识别:返回数据包,端口探针,组合对应等
3.数据库类型识别:端口探针,组合对应,应用规模猜测等
4.脚本开发语言识别:URL获取,搜索引擎探针,组合对应等
5.WEB类接口信息识别:扫描探针,特定访问,端口扫描等

Be sure to understand the common building asset identification technologies, common building portfolio classification, and some information collection technologies

Architectural asset information collection

1.IP类站点获取:直接利用返回IP进行测试访问
	举例:比如一个IP站点不同的端口开放的是不同的站点,这样的情况下就是一个IP对应多个可渗透站点,可以扩大攻击面。
2.目录类站点获取:利用目录爬行或目录扫描等进行获取
	举例:有的时候,不同的目录也是不同的站点,同样也可以利用来进行攻击面的扩大。比如一个域名下面存在xxx.com/bbs/系统

    目录爬行:通过爬虫爬取网站地图获取目录信息,代表工具AWVS简称“WVS”
    目录爆破:利用工具通过字典爆破,根据返回的响应状态获取有效的目录信息,代表  工具7kbscan

3.端口类站点获取:利用端口扫描进行服务探针进行获取
    通过端口扫描发现开放的对应端口,发现对应的服务站点,可通过端口进行访问
4.子域名站点获取:利用子域名查询平台或工具进行获取
    通过子域名资产工具针对子域名资产进行收集。

Understand the differences in site architecture and how to obtain such information sources, some information collection techniques

Source code asset information collection

1.如何获取源码类信息:CMS名称,特定文件,版本信息等
	当知道一个网站使用的CMS(内容管理系统)的时候,就可以根据当前网站所使用的CMS去搜集对应的版本信息、源码以及公开的可利用的漏洞,直接一步到位。
	
	CMS检测平台(网上有很多可自行百度)
	国外:https://whatcms.org/ -> 
	国内:http://whatweb.bugscaner.com/ -> 
	工具:https://github.com/Tuhinshubhra/CMSeeK ->

	CMS检测小技巧:当网站没有明确的标识使用的CMS内容管理系统,可通过访问网站的请求包里的特定的路径通过搜索引擎找到该网站使用CMS。
	
2.如何下载到源码程序:源码站(黑白),备份,泄漏,监控,手工分析搜索等
	详见下文“源码安全”		

Tool platform recognition cooperates with manual specific search and other operations, and method selection according to different WEB application types

Domain name asset information collection

Domain name information: whois, record, subdomain, similar domain names, etc.

Understand the ideas and expansion of asset information collection on domain names, involving manual operations such as tools and platforms

第三方平台使用,工具或脚本使用,类似域名获取测试
https://github.com/shmilylty/OneForAll
	python oneforall.py --target=xiaodi8.com run

In addition to OneForAll, this subdomain mining artifact, there are also Recon-ng, Layer excavator, DNSenum, Yujian, etc., which are all standing information collection tools. I won't describe too much about the usage here. There are many on the Internet. For example , there are many detailed introductions in Xie Gongzi's blog column .
The registration interface cooperates with search engine keywords and other technologies to obtain other similar domain names.

Source code security information collection

Reference: https://www.secpulse.com/archives/124398.html

Digression:

信息泄露,是指网站在无意间的情况下向用户泄露的敏感信息。结合上下流量包信息,网站可能会将各种各样的信息泄漏给潜在的攻击者。

包括但不限于:

    有关其他用户的数据,例如用户名或财务信息
    敏感的商业信息或商业数据
    有关网站服务器及其基础架构的技术细节

泄露敏感信息或业务数据给用户,本身就是相当危险的行为,同样的泄露相关技术信息有时也会造成很严重的危险后果。
可能某些信息用途有限,但它可能是暴露其他攻击面的起点。当攻击者试图发起一次高强度攻击没有收获时,往往收集的这些基础信息就可以解决这个问题。

有时,敏感信息可能会不慎泄露给仅以正常方式浏览网站的用户。但是更常见的是,攻击者需要通过以意外或恶意的方式与网站进行交互来触发信息泄露。然后,攻击者会仔细研究网站的响应,以尝试找出可以利用的信息来作为攻击的支撑。
  • git source code leaked
  • svn source code leaked
  • hg source code leak
  • Website backup compressed files
  • WEB-INF/web.xml leaked
  • DS_Store file leak
  • SWP file leaked
  • CVS leak
  • Bzr leaked
  • GitHub source code leak

Among them, hg, SWP, and CVS are relatively old version control systems compared to others, and few people use them now. I won't do too much introduction here, and interested students can Baidu by themselves.

Website backup compressed files

In the process of using the website, it is often necessary to modify and upgrade the files in the website. At this time, you need to back up the entire site or one of its pages. When the backup file or the cache file in the modification process is left in the web directory of the website for various reasons, and the directory is not set to access permissions, it may cause the backup file or the cache file of the editor to be downloaded, resulting in sensitive Information leakage lays hidden dangers to the security of the server.
Causes and hazards of
loopholes : There are two main causes for the loopholes:

The server administrator mistakenly placed the backup file of the website or web page in the server web directory.
The backup files or temporary files automatically saved by the editor during use are not deleted for various reasons and are saved in the web directory.

漏洞检测:
该漏洞往往会导致服务器整站源代码或者部分页面的源代码被下载,利用。源代码中所包含的各类敏感信息。

如服务器数据库连接信息,服务器配置信息等会因此而泄露,造成巨大的损失。被泄露的源代码还可能会被用于代码审计。

进一步利用而对整个系统的安全埋下隐患。

.rar
.zip
.7z
.tar.gz
.bak
.swp
.txt
.html

git source code leaked

Git is an open source distributed version control system. When the git initinitialization directory is executed , a directory is automatically created under the current directory .gitto record code changes. When publishing the code, if .gitthis directory is not deleted, it is directly published to the server, and the attacker can use it to restore the source code.

漏洞利用工具:GitHack

github项目地址:https://github.com/lijiejie/GitHack

用法示例:

GitHack.py http://www.openssl.org/.git/

Repair suggestion: delete the .git directory or modify the middleware configuration to access the hidden folder of .git.

SVN source code leaked

SVN is an open source version control system. In the process of using SVN to manage local code, a hidden folder named .svn will be automatically generated, which contains important source code information. When the webmaster released the code, he did not use the “export” function, but directly copied the code folder to the WEB server, which made the .svn hidden folder exposed to the external network environment and could use the .svn/entries file, Obtain the server source code.

When deploying code on the server. If the svn checkout function is used to update the code, and the directory access permissions are not configured, this vulnerability exists. Hackers can use this vulnerability to download the source code of the entire website.

After using svn checkout, a hidden .svn folder will be generated under the project directory (you can't see it with the ls command on Linux, you need to use the ls -al command).
svn1.6 and previous versions will generate a .svn folder in each folder of the project, which contains the backup of all files, the file name is .svn/text-base/file name.svn-base

漏洞利用工具:Seay SVN漏洞利用工具
漏洞利用工具:svnExploit漏洞利用工具

Repair suggestion: Delete all hidden folders of .svn in the web directory. Developers should strictly use the export function when using SVN, and it is forbidden to copy code directly.

Reference: https://www.cnblogs.com/hilfloser/p/10517856.html SVN information disclosure vulnerability analysis

Bazaar/bzr leak

Bzr is also a version control tool. Although it is not very popular, it also supports multiple platforms and has a good graphical interface.

Run example:

rip-bzr.pl -v -u http://www.example.com/.bzr/

WEB-INF/web.xml leaked

WEB-INF is the secure directory of Java's WEB application. If you want to directly access the files in the page, you must map the files you want to access through the web.xml file to access it.

WEB-INF 主要包含一下文件或目录:

WEB-INF/web.xml : Web应用程序配置文件, 描述了servlet和其他的应用组件配置及命名规则.
WEB-INF/database.properties : 数据库配置文件
WEB-INF/classes/ : 一般用来存放Java类文件(.class)
WEB-INF/lib/ : 用来存放打包好的库(.jar)
WEB-INF/src/ : 用来放源代码(.asp和.php等)
通过找到 web.xml 文件,推断 class 文件的路径,最后直接 class 文件,再通过反编译 class 文件,得到网站源码。

Causes of vulnerability:

Usually some web applications we will use multiple web servers together to solve the performance defects of one of the web servers and the advantages of balancing load and complete some hierarchical security policies. When using this architecture, due to improper configuration of the mapping of static resource directories or files, some security issues may arise, resulting in files such as web.xml being able to be read.
Vulnerability detection and utilization method:
by finding the web.xml file, inferring the path of the class file, and finally directing the class file, and decompiling the class file to obtain the website source code.

In general, the JSP engine is forbidden to access the WEB-INF directory by default. When Nginx cooperates with Tomcat for load balancing or clustering, the cause of the problem is actually very simple. Nginx will not consider configuring other types of engines (Nginx is not a jsp engine). The security issue of the web site is introduced into its own security specification (so the coupling is too high), just modify the Nginx configuration file to prohibit access to the WEB-INF directory: location ~ ^/WEB-INF/* {deny all;} or return 404; or other!

DS_Store file leak

".DS_Store" is a data file used by Finder under Mac to save how to display files/folders, one for each folder. If the .DS_Store is uploaded and deployed to the server, the file directory structure may be leaked, especially the backup files and source code files.

漏洞利用工具:

github项目地址:https://github.com/lijiejie/ds_store_exp

用法示例:

ds_store_exp.py http://hd.zj.qq.com/themes/galaxyw/.DS_Store

GitHub source code leak

GitHub is a hosting platform for open source and private software projects. Many people like to upload their code to the platform for hosting. Attackers can search through keywords to find sensitive information about the target site and even download the source code of the website.

There are many similar code hosting platforms, and talent is the biggest loophole.

https://github.com/[email protected]&type=code

There is also a GitHub monitoring
reference: http://sc.ftqq.com/3.version Server sauce application-configuration-write-test

WAF recognition-post-test bypass point

How to judge whether the website has WAF and identify -wafw00f or look at the picture to identify

wafw00f

wafw00f-GIT address https://github.com/EnableSecurity/wafw00f
https://blog.csdn.net/weixin_42250835/article/details/115313988 wafw00f introduction and firewall detection

Look at the picture to recognize WAF

1.D shield

D shield

2. Cloud lock

Cloud lock

3.UPUPW security protection

UPUPW security protection

4. Pagoda website firewall

Pagoda website firewall

5. Net defense G01

Net Defense G01

6. Guardian

Guardian

7. Website security dog

Website security dog

8. Smart firewall

8. Smart firewall

9.360 host guard or 360webscan

9.360 host guard or 360webscan

10. Western Digital WTS-WAF

10. Western Digital WTS-WAF

11.Naxsi WAF

11.Naxsi WAF

12.Tencent Cloud

12.Tencent Cloud

13. Tencent Aegis

13. Tencent Aegis

14.Baidu Cloud

14.Baidu Cloud

15.Huawei Cloud

15.Huawei Cloud

16. Wangsu Cloud

16. Wangsu Cloud

17. Chuangyudun

17. Chuangyudun

18. Basalt shield

18. Basalt shield

19. Alibaba Cloud Shield

19. Alibaba Cloud Shield

20.360 website guard

20.360 website guard

21. Qi'anxin Website Guardian

21. Qi'anxin Website Guardian

22. Anyu Cloud WAF

22. Anyu Cloud WAF

23. Iridium WAF

23. Iridium WAF

24. Long Pavilion SafeLine

24. Long Pavilion SafeLine

25. Anheng Mingyu WAF

25. Anheng Mingyu WAF

26.F5 BIG-IP

27.Mod_Security

27.Mod_Security

28.OpenRASP

28.OpenRASP

29.dotDefender

29.dotDefender

30. Unknown Cloud WAF

30. Unknown Cloud WAF

Side note, section C, IP reverse check-post-test ideas

It is found that several related articles collected before have been saved. The articles are relatively similar, with the same methods, so we can summarize and add some self-accumulation.

旁注:同服务器下不同站点
旁注:服务器上存在A,B两个站点,A是测试目标,没有发现漏洞无法测试,这个时候可以借助旁注查询获取B站点,通过B站点的测试进一步获取A目标的权限。

IP反查:利用IP获取服务器解析域名 a 

C段:如果服务器是独立站点或其他站点也无漏洞利用时,可以获取同网段服务器下的站点信息,进行安全测试,获取到同网段下的某台服务器权限,进行内网安全测试获取指定服务器权限。
C段:同网段不同服务器下不同站点

CDN bypass-post-test bypass point

1.传统访问:用户访问域名–>解析服务器IP–>访问目标主机
2.普通CDN:用户访问域名–>CDN节点–>真实服务器IP–>访问目标主机
3.带WAF的CDN:用户访问域名–>CDN节点(WAF)–>真实服务器IP–>访问目标主机
如何判断网站是否存在CDN
nslookup,各地ping
如何获取网站真实IP地址

子域名,去掉www,邮件服务器,国外访问,证书查询,APP抓包

Reference: https://zhuanlan.zhihu.com/p/33440472 Summary of methods to find the real IP of a website by bypassing CDN

Reference: https://blog.csdn.net/weixin_42250835/article/details/111460459 How to bypass CDN to query the real IP of a website

Dark space engine, obtained through loopholes or leaks, sweeping the entire network, looking at it, third-party interface query, etc.

https://www.17ce.com/
https://www.wepcc.com/
https://get-site-ip.com/
https://x.threatbook.cn/
https://tools.ipip.net/cdn.php
https://github.com/Tai7sy/fuckcdn
https://github.com/boy-hack/w8fuckcdn

Port scanning, service detection

Scanning artifact, eye of the gods-no explanation

http://www.nmap.com.cn/doc/manual.shtm Nmap Chinese Manual

https://blog.csdn.net/weixin_42250835/article/details/115339302 (Eye of the Gods) super detailed port scanning

https://blog.csdn.net/weixin_42250835/article/details/115364161 nmap (eyes of the gods) advanced application-IDS and firewall evasion and exploit script

端口扫描-系统版本服务探针
端口扫描-自带防火墙服务探针
端口扫描-网段系统端口信息探针

nmap -sV www.xxx.com
nmap -O -A www.xxx.com
nmap -Pn -O -A www.xxx.com

Dark Engine-Fofa, shadan, zoomeye, 360

语法,玩法,功能等-关联关系,自动数据,白引擎外等
https://fofa.so/
https://www.shodan.io/
https://www.zoomeye.org/
https://quake.360.cn/quake/#/index

ARL lighthouse-use of automated asset reconnaissance lighthouse system

安装,使用,功能,补充等
安装docker docker-compose
修改项目的版本 3修改成2
修改docker源 保证速度问题
注意:服务器记得安全组配置放行对应端口
https://github.com/TophantTechnology/ARL
https://www.cnblogs.com/zhengjim/p/13678257.html

APP, small program capture, search information extraction

Tools:
Charles [Real-time capture of operands to report]
Packet Capture Wizard [Convenient, install on a mobile phone or emulator, ready to use]
Burpsuite [Easy to link with other security tools later]
Wireshark [Capture all network card traffic]

Centralized information collection: information collection-APP/small program: external display and internal source code analysis.
Basic application version function reinforcement, involving reference protocols (WEB, IP, interface, etc.), code logic security issues, involving resource files, critical keys , Download source, application to obtain operation permissions and other information.

Here is a brief introduction, APP and applet penetration discrimination is not that difficult. After all, WEB protocol is also used for data interaction. After obtaining the HTTP proxy-based URL and request response, it can be used as a WEB end to infiltrate.

The following are common ideas for server-side vulnerability detection

Based on the HTTP proxy idea:

水平权限风险检测
垂直权限风险检测
SQL注入漏洞检测
XSS漏洞检测
敏感信息检测(硬编码、用户密码、银行卡、身份证等敏感信息明文传输)

Proxy thinking based on TCP or UDP (socket):

敏感信息检测(硬编码、用户密码、银行卡、身份证等敏感信息明文传输)

html5 vulnerability detection

APK static debugging security detection-platform

https://www.cnblogs.com/xiaozi/p/12749801.html Free and easy-to-use APP security online detection platform

https://console.cloud.tencent.com/ms/scan Tencent Cloud Mobile Application Security Evaluation

External display of packet capture history-two tools

https://github.com/huolizhuminh/NetWorkPacketCapture/releases
Packet capture wizard, Burpsuite configuration capture test
Example "Fofa-"index/login/login/token"-121.127.227.20-MetaTrade 5" to
capture historical data packets and compare The packet capture tool captures the results and analyzes the follow-up ideas

Internal key search features-automatic extraction

https://github.com/TheKingOfDuck/ApkAnalyser
uses the apkAnalyser automated script to extract the corresponding other results (it needs to be combined with packet capture for analysis, and the combination of the two can ensure more complete information collection. After all, tools are tools, and there are certain errors in the results of the analysis. Newspaper.)

Search fuliqh, MetaTrade (this is the APK package for testing) after two decompilation, IDEA searches for specific keywords globally (using one-click decompilation tool)

6/9/2021 10:47:26 PM

! ! ! —Work more, just watching and listening is not enough —! ! !
Three days of fishing, without in-depth understanding of the principle, only using tools can only become a script kid