Security audit
Security checks for vulnerabilities and agentic risk
The skill is a coherent NL-to-SQL toolkit, but it handles database queries, audit logs, and clarification sessions with weak controls that should be reviewed before installation.
Review this skill carefully before installing in any environment with real business data. Use only with sandbox or read-only databases unless you add authorization, SQL safety checks, log redaction, retention limits, safe export paths, and non-debug authenticated clarification routes. Note that audit_logger.py also has a syntax error and will not load unchanged.
schema_context_builder.py:96Database connection information is persisted without redaction
schema_context_builder.py:45Database identifiers are interpolated into SQL without safe quoting
disambiguation_ui.py:151Flask application starts with the development debugger enabled
disambiguation_ui.py:119Clarification sessions are accessible without authentication or ownership checks
audit_logger.py:39Audit records retain complete sensitive query data in plaintext
fallback_handler.py:164Negative review identifiers bypass review bounds validation
Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.
### Features
- **Multi-domain support**: Tested across 10+ industries
- **Confidence thresholds**: 85% auto-execute, 60% clarify, below 60% human review
- **Disambiguation flow**: Interactive UI for clarifying ambiguous queries
- **Error recovery**: Automatic retry with modified prompts
- **Full audit trail**: Compliance-ready logging
The export function writes the full audit trail, including user queries, generated SQL, user IDs, session IDs, and errors, to any caller-supplied file path with no validation, access control, or safety checks. In an NL-to-SQL context, these records can contain sensitive business data, credentials embedded in queries, or behavioral history, so unrestricted export increases the risk of data exfiltration or accidental disclosure.
Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.
score: float # 0-100
factors: List[str]
warnings: List[str]
recommended_action: str # 'auto_execute', 'clarify', 'human_review'
class ConfidenceScorer:
Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.
score: float # 0-100
factors: List[str]
warnings: List[str]
recommended_action: str # 'auto_execute', 'clarify', 'human_review'
class ConfidenceScorer:
Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.
score: float # 0-100
factors: List[str]
warnings: List[str]
recommended_action: str # 'auto_execute', 'clarify', 'human_review'
class ConfidenceScorer:
Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.
score: float # 0-100
factors: List[str]
warnings: List[str]
recommended_action: str # 'auto_execute', 'clarify', 'human_review'
class ConfidenceScorer:
Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.
score: float # 0-100
factors: List[str]
warnings: List[str]
recommended_action: str # 'auto_execute', 'clarify', 'human_review'
class ConfidenceScorer:
Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.
"""Calculate confidence scores for NL-to-SQL queries."""
# Thresholds from production system
AUTO_EXECUTE_THRESHOLD = 85
CLARIFY_THRESHOLD = 60
def __init__(self, schema: Dict, threshold_override: Optional[Dict] = None):
Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.
"""Calculate confidence scores for NL-to-SQL queries."""
# Thresholds from production system
AUTO_EXECUTE_THRESHOLD = 85
CLARIFY_THRESHOLD = 60
def __init__(self, schema: Dict, threshold_override: Optional[Dict] = None):
Allowing caller-supplied threshold_override to lower the auto-execute threshold can weaken safety policy and increase the chance that ambiguous NL-to-SQL requests are automatically executed. In a system where this scorer gates database actions, an attacker or misconfigured component could set permissive thresholds and bypass intended human review.
self.relationships = schema.get('relationships', [])
if threshold_override:
self.AUTO_EXECUTE_THRESHOLD = threshold_override.get('auto', 85)
self.CLARIFY_THRESHOLD = threshold_override.get('clarify', 60)
def score(self, query: str, intent: Dict, constraints: Dict) -> ConfidenceResult:
This branch explicitly authorizes an 'auto_execute' recommendation based on a heuristic score derived from simple string matching and lightweight penalties. In NL-to-SQL contexts, heuristic confidence scoring can misclassify ambiguous or adversarial prompts, leading to unsafe autonomous execution of database queries without adequate review.
total_score = max(0, min(100, total_score))
# Determine recommended action
if total_score >= self.AUTO_EXECUTE_THRESHOLD:
action = 'auto_execute'
elif total_score >= self.CLARIFY_THRESHOLD:
action = 'clarify'
The top-level documentation frames the file as UI components for a larger NL-to-SQL skill, implying an embedded helper role. In practice, the code defines a global Flask app, registers a public route, and runs a debug web server when executed directly, which is a materially broader behavior than 'components' alone.
This code path allows automatic approval when confidence exceeds a threshold and ambiguity is not detected, relying only on a boolean safety_check_passed supplied by the caller. In an NL-to-SQL skill, confidence is not a security boundary; misclassification or weak upstream checks could allow unintended or over-privileged SQL to be auto-executed without human review.
fallback_queue=None
)
# Auto-execute if high confidence
if confidence >= self.auto_execute_threshold and not ambiguity_detected:
return FallbackDecision(
should_fallback=False,
The conditional grants an execution path based on confidence and ambiguity rather than explicit query risk or authorization. In this skill context, that makes the component more dangerous because NL-to-SQL systems can generate harmful or privacy-impacting queries even when they appear semantically confident.
)
# Auto-execute if high confidence
if confidence >= self.auto_execute_threshold and not ambiguity_detected:
return FallbackDecision(
should_fallback=False,
reason=None,
Returning recommended_action='auto_execute' operationalizes autonomous execution and may be consumed by downstream components as approval to run SQL without additional safeguards. In an NL-to-SQL environment, this can lead to unauthorized data access or unintended query execution if downstream systems trust this recommendation too broadly.
should_fallback=False,
reason=None,
confidence=confidence,
recommended_action='auto_execute',
fallback_queue=None
)
The export_log method writes raw user queries, user IDs, timestamps, and arbitrary context to any supplied filepath without validation, access control, redaction, or safety prompts. In an NL-to-SQL workflow, this data can contain sensitive business queries, schema clues, or personal data, so unrestricted export increases the risk of data leakage or unsafe file writes by callers.
The docstring advertises broad compliance tracking for all queries and interpretations, which implies comprehensive capture of NL-to-SQL reasoning artifacts. In practice, the logger only stores a fixed set of fields passed into log_query and has no logic to independently capture full interpretations, reasoning traces, or guarantee that all queries are recorded.
This code displays the user's original query in the web UI, which may contain sensitive business or personal data, but there is no accompanying notice, comment, or visible disclosure about that exposure. For a code file, user-data display/handling that could affect privacy should have at least some warning or explanation when not obviously disclosed elsewhere in the file.
The docstring at L127 says the method 'Route query to human review queue', which implies routing behavior, but the implementation also stores the full review item in an internal log (L138-L139) and a separate method can export that accumulated log to a JSON file (L187-L191). This is not merely an implementation detail of queueing because it changes data handling behavior by retaining and later writing user queries, IDs, and context.