- Location
- scripts/rag-service.mjs:45
- Finding
- Unbounded Request Bodies and Ingestion Work Permit Denial of Service<![CDATA[
## Vulnerability Details
**File Location**: `scripts/rag-service.mjs:45-49`, `scripts/rag-service.mjs:64-76`, and `scripts/rag-service.mjs:122-151`
**Vulnerability Type**: Uncontrolled resource consumption
**Risk Level**: Medium
### Vulnerable Code
```js
async function readJsonBody(req) {
const chunks = [];
for await (const c of req) chunks.push(c);
const raw = Buffer.concat(chunks).toString('utf8') || '{}';
return JSON.parse(raw);
}
```
```js
function listTextFiles(dir) {
if (!fs.existsSync(dir)) return [];
const out = [];
const stack = [path.resolve(dir)];
while (stack.length) {
const cur = stack.pop();
for (const ent of fs.readdirSync(cur, { withFileTypes: true })) {
const p = path.join(cur, ent.name);
if (ent.isDirectory()) stack.push(p);
else if (/\.(txt|md)$/i.test(ent.name)) out.push(p);
}
}
return out;
}
```
```js
async function ingestFromDir(dir, reset = true) {
const files = listTextFiles(dir);
if (!files.length) return { files: 0, chunks: 0 };
const rows = [];
for (const f of files) {
const text = fs.readFileSync(f, 'utf8');
const chunks = chunkText(text);
chunks.forEach((c, idx) => rows.push({
id: `${path.basename(f)}#${idx + 1}`,
source: path.resolve(f),
chunkIndex: idx + 1,
text: c,
}));
}
if (!rows.length) return { files: files.length, chunks: 0 };
const firstVec = await embed(rows[0].text);
ensureCollection(firstVec.length, reset);
const docs = [{
id: rows[0].id,
vectors: { embedding: firstVec },
fields: { source: rows[0].source, chunkIndex: rows[0].chunkIndex, text: rows[0].text },
}];
for (let i = 1; i < rows.length; i++) {
const vec = await embed(rows[i].text);
docs.push({
id: rows[i].id,
vectors: { embedding: vec },
fields: { source: rows[i].source, chunkIndex: rows[i].chunkIndex, text: rows[i].text },
});
}
```
### Technical Analysis
`readJsonBody()` buffers the compl
...[truncated 2256 chars]
- Remediation
- <![CDATA[
## Remediation Suggestions
1. Enforce a small maximum HTTP body size while streaming the request. Abort and return HTTP `413 Payload Too Large` as soon as the limit is exceeded.
2. Configure header, request, idle, and overall operation timeouts.
3. Limit concurrent ingestion jobs and embedding requests. Consider allowing only one authenticated administrative ingestion job at a time.
4. Establish server-side limits for directory depth, file count, individual file size, total bytes, generated chunks, and total embeddings per request.
5. Reject ingestion before processing when preflight enumeration exceeds configured limits.
6. Stream or incrementally process files rather than retaining every file, chunk, vector, and document in memory simultaneously.
7. Replace synchronous filesystem and database operations on the request path where supported, or move ingestion to a bounded worker queue.
8. Add cancellation support so disconnected clients and expired jobs stop consuming resources.
9. Apply per-client rate limits and operating-system resource constraints.
10. Add tests covering oversized bodies, slow clients, very large files, deeply nested directories, and excessive file counts.
]]>