Log rotation and generations
When a log file is rotated on a remote host - renamed, truncated, or replaced by a fresh file at the same path - Log Lens must not confuse the new file with the old one. If it appended the new bytes onto the old mirror, offsets would drift and events would be corrupted. Generations solve this: each distinct incarnation of a file is tracked and indexed separately, and older generations are kept intact.
How rotation is detected
During each sync, the connector discovers every remote file and reports two things about it: its path and its identity (a stable fingerprint such as the underlying inode). Log Lens compares these against the latest stream it already tracks for that path. A new generation is started when either condition holds:
| Signal | Meaning |
|---|---|
remote_identity changed | A different file now occupies the same path (rotated/replaced) |
remote size < fetched offset | The file shrank - it was truncated in place |
When neither is true and the file has simply grown, Log Lens does an ordinary append, fetching only the new tail.
Generations
Every stream carries a generation number. The first time a path is seen it is generation 1; each detected rotation increments it. Each generation gets its own isolated mirror directory, so previously indexed bytes and their events are preserved:
{sources}/{connectorId}/{sha256(path)[:16]}-g{generation}/{basename}
The new generation starts from offset 0 and re-fetches the current file in full, while the prior generation's mirror and indexed events remain untouched and searchable.
Renames
If a file keeps its identity but appears under a new path - the classic app.log to app.log.1 rotation - Log Lens recognizes the same underlying file and follows it. The existing stream is repointed to the new path and its generation is set to the next value for that path, rather than being re-fetched as brand new.
What you see in preview
The sync preview labels each file with the action it will take, including rotation, rename, append, unchanged, and index_local. Reviewing this before syncing lets you confirm a rotation was detected as expected.