Module file_index

Module file_index 

Source
Expand description

File index for efficient log navigation (Option 3 Implementation)

This module provides a lightweight index structure that scans the file once to identify log boundaries and metadata without full parsing.

§How it works:

  1. Fast Initial Scan: On file open, scan all lines to find log boundaries
  2. Extract Metadata: For each log, extract timestamp and layer (cheap operation)
  3. Build Index: Store line ranges and metadata for each log
  4. Lazy Parsing: Parse full log content only when needed (on navigation)
  5. Smart Caching: Cache parsed logs to avoid re-parsing

§Benefits:

  • Fast file loading (~0.5s for 10k logs)
  • Know total log count immediately
  • Efficient layer filtering (no need to parse disabled layers)
  • Instant navigation (index knows where each log is)

Structs§

FileIndex
Index structure for efficient file navigation
LogMetadata
Metadata for a single log entry