eml to sqlite indexer

Indexes EML emails into an SQLite database, providing a web interface for searching, management, Excel export, and file deletion, with IP access control and...

Security Scan

The code, runtime instructions, and declared requirements are coherent with an EML→SQLite indexer web app; no evidence of hidden exfiltration or unrelated credential requests, but there are several operational security issues (weak default admin password, file-deletion logic, backup behavior) you should address before running in production.

Purpose & Capability

The name/description (index EML into SQLite, provide web search, export, deletion, backups) match the provided files (indexer.py for indexing; app.py for Flask UI, search, export, deletion, scheduled backup). No unrelated environment variables, binaries, or external services are requested.

Instruction Scope

SKILL.md and the code instruct the agent/user to index local EML files, run a Flask web app, create/serve backups, and allow admin-triggered physical file deletion. These behaviors are within the stated purpose, but file-deletion and import/restore functionality have security/operational impact (see details below). The app does not attempt outbound network exfiltration or read unrelated system credentials.

Install Mechanism

No install spec; the project is instruction-and-code only and uses standard Python libraries listed in requirements.txt (Flask, tqdm). This is proportionate for a Python Flask application and minimizes install-time risk.

Credentials

No environment variables or external service credentials are requested; configuration is handled via local config.json. The requested resources are proportional to the stated purpose.

Persistence & Privilege

The app launches a background thread that performs scheduled backups while the server runs; this is consistent with the described scheduled-backup feature. The skill does not request forced-permanent inclusion (always:true) or modify other skills. Note that running the app grants it persistent local disk write access for backups and the database while it runs.

Assessment

This package appears to do what it claims, but treat it as high-impact tooling and do not run it on untrusted hosts or exposed servers without changes. Before installing/running: 1) Change the default admin password in config.json (default is 'change_me_now'). 2) Restrict allowed_ips to only the hosts that must access the UI (do not include broad network ranges). 3) Run the web app only on a trusted host (or inside an isolated container/VM) because it reads and can delete local files and writes backups to disk. 4) Review and test the delete/restore behavior: the code deletes the DB record before attempting file removal and uses simple checks ('..' and .eml suffix) which may be insufficient for all path-traversal or symlink scenarios—consider hardening path validation and restricting deletion to known mail directories. 5) Protect backups (encrypt or move to secure storage) if they contain sensitive content. 6) If you must expose the UI remotely, add transport security (TLS), strong authentication, and audit logging. If you want, I can list exact code lines to change to harden deletion, authentication, and backup handling.

EML to SQLite Indexer Skill (V7.0.0 - Management & Export Edition)

This skill indexes EML email files from a specified directory into an SQLite database and provides a feature-rich web interface for searching and management. It includes automatic deduplication, IP access control, Excel export, and a JSON-formatted scheduled backup and restore system configurable via the web interface.

Features

• Efficient Indexing: Uses MD5 fingerprinting for automatic deduplication, ensuring no duplicate emails are imported. Supports processing millions of email records.
• Key Information Extraction: Automatically parses and stores email sender, recipient, subject, body content, and sent time.
• Web Query Interface: Provides a Flask-based web interface with:
  • Advanced Search: Keywords (subject/body, case-insensitive), sender (fuzzy matching), recipient (fuzzy matching), and date range filtering.
  • Excel Export: Export search results to an Excel-compatible CSV file, including the original file path.
  • File Deletion: Delete specific emails from both the database and the physical disk (Admin only).
  • Pagination: Optimized for large datasets to prevent browser slowdowns.
• IP Access Control: Configurable whitelist of allowed IP addresses for web access, enhancing security. By default, only localhost and 127.0.0.1 are allowed.
• Web-Configurable Scheduled Backup:
  • Scheduled Mode: Configure via the web interface to set the backup frequency (e.g., every X days) and specific hour (0-23) for execution.
  • JSON Format: Backups export email data as structured JSON and compress it into a ZIP file, named eml-indexer_YYYYMMDD.zip, offering excellent cross-platform compatibility.
  • Automatic Circular Overwrite: The system automatically retains a configured number of backups, deleting the oldest one when the limit is exceeded.
  • Manual Management: One-click download of JSON backup ZIPs and upload of ZIPs for database restoration via the web interface.
• Admin-Exclusive Interface: When accessed from localhost or 127.0.0.1, a "⚙️ System Settings" tab is displayed, providing IP management, backup configuration, and deletion functionalities.

Installation & Deployment

1. Environment Requirements

• Python 3.8+ (latest stable version recommended)
• Recommended OS: Windows, Linux

2. Dependency Installation

Ensure your Python environment has the following packages installed:

pip install -r requirements.txt
# Or install manually:
pip install Flask tqdm

3. File Structure

eml_indexer/
├── app.py              # Web application (Flask) - includes scheduled backup thread
├── indexer.py          # Core EML indexing script
├── requirements.txt    # Python dependencies list
├── SKILL.md            # Skill documentation (English)
├── config.json         # Runtime configuration (allowed IPs, backup frequency, retention)
├── emails.db           # SQLite database file (generated after running indexer.py)
├── backups/            # Directory for JSON backups (automatically created)
└── templates/
    ├── detail.html     # Email detail page template
    └── index.html      # Email search and management main page template
└── references/
    └── SKILL-TW.md     # Traditional Chinese version of the skill documentation

Usage

1. Index EML Emails

Run indexer.py to import EML files from a specified directory into the database. It automatically skips already indexed emails on subsequent runs.

python indexer.py <EML_DIRECTORY_PATH> <DATABASE_PATH (default: emails.db)>

2. Start Web Query Interface (with Scheduled Backup)

Execute app.py to start the Flask web server. The background scheduled backup thread will also start automatically.

python app.py

After starting, visit http://localhost:5000 in your browser.

3. Manage System Settings

When accessing the web interface from localhost or 127.0.0.1, click the "⚙️ System Settings" tab. Here you can:

• IP Management: Add or remove allowed IP addresses.
• Backup Settings: Configure "Backup Interval (Days)", "Backup Time (Hour)", and "Number of Backups to Retain".
• Manual Backup: Click "Create and Download Backup (ZIP)" to generate an immediate JSON backup.
• Manual Restore: Upload a JSON backup ZIP file to restore the database.

Version History

• V7.0.1 (2026-03-30):
  • Added Excel Export functionality (includes file paths).
  • Added Physical File Deletion functionality (Admin only).
  • Updated documentation to include Traditional Chinese version in references/SKILL-TW.md.
• V6.0.0 (2026-03-29):
  • Web-configurable scheduled backups (days interval and specific hour).
  • Backup filenames formatted as eml-indexer_YYYYMMDD.zip.
  • Integrated scheduled check logic into the background thread.
  • SKILL.md updated to English, with author changed to "威廉陳". Original Traditional Chinese SKILL.md moved to references/SKILL-TW.md.

License

MIT License