- Location
- scripts/news_crawler.js:124
- Finding
- Unvalidated article URLs permit server-side request forgery<![CDATA[
## Vulnerability Details
**File Location**: `scripts/news_crawler.js`, lines 9-29, 66-87, and 124-143
**Vulnerability Type**: Server-Side Request Forgery (SSRF) through unvalidated remote links
**Risk Level**: Medium
### Vulnerable Code
```javascript
const pageUrls = soup.querySelectorAll('li a').slice(1).map(a => a.getAttribute('href') || '');
const headers = {
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
'Accept-Encoding': 'gzip, deflate',
'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8',
'Cache-Control': 'no-cache',
'Cookie': 'cna=DLYSGBDthG4CAbRVCNxSxGT6',
'Host': 'tv.cctv.com',
'Pragma': 'no-cache',
'Proxy-Connection': 'keep-alive',
'Upgrade-Insecure-Requests': '1',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36'
};
const data = await Promise.all(pageUrls.map(async pageUrl => {
try {
const pageResponse = await fetch(pageUrl, { headers });
```
The same vulnerable pattern also occurs in the older and mid-period crawler paths:
```javascript
const pageUrls = rawList.slice(1).map(item => item.match(/(http.*)/)?.[0].split('\'')[0] || '');
```
```javascript
const pageUrls = soup.querySelectorAll('#contentELMT1368521805488378 li a')
.slice(1)
.map(a => a.getAttribute('href') || '');
```
Each resulting value is subsequently passed to `fetch(pageUrl, { headers })`.
### Technical Analysis
The crawler retrieves an index page from a fixed CCTV URL but then trusts article links extracted from that remotely supplied HTML. Before issuing secondary requests, it does not validate:
- The URL scheme.
- The destination hostname.
- The destination port.
- Whether the URL contains embedded credentials.
- Whether DNS resolution points to loopback, private, link-local, or other restricted addresses.
- Whether redirects remain on a
...[truncated 2211 chars]
- Remediation
- <![CDATA[
## Remediation Suggestions
1. Parse every extracted link with `new URL()` and reject malformed URLs.
2. Permit only the `https:` scheme.
3. Apply an explicit hostname allowlist, limited to required CCTV domains such as `tv.cctv.com` and `cctv.cntv.cn`.
4. Reject URLs containing usernames or passwords and reject unexpected ports.
5. Resolve destination hostnames and reject loopback, private, link-local, multicast, reserved, and cloud-metadata address ranges for both IPv4 and IPv6.
6. Disable automatic redirects or manually process them, applying the same validation to every redirect target.
7. Reject empty links and cap the number of links fetched from each index page.
8. Add request timeouts, response-size limits, status-code checks, and concurrency limits.
9. Enforce equivalent destination restrictions at the runtime or network-sandbox layer so the process cannot access internal networks even if application validation is bypassed.
10. Remove the manually assigned `Host` header and let the HTTP implementation derive it from the validated URL.
]]>