Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
bug fix
1,修复配置修改或重启后sincedb中记录的last_check_at被重置为当前时间的问题。
2,修复tail模式下配置修改或重启后导致当前采集的日志在rotate后重复采集的问题
  • Loading branch information
cliffordsun committed May 10, 2019
commit 1f82b449508d156c1841ae8fe78bb17c69c4803a
9 changes: 8 additions & 1 deletion lib/filewatch/sincedb_collection.rb
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,14 @@ def flush_at_interval

def handle_association(sincedb_value, watched_file)
watched_file.update_bytes_read(sincedb_value.position)
sincedb_value.set_watched_file(watched_file)
if watched_file.all_read?
# avoid the last_changed_at of sincedb_value which record the watched file has bean read over being changed when reload sincedb
sincedb_value.set_watched_file_without_touch(watched_file)
logger.trace("handle_association call set_watched_file_without_touch")
else
sincedb_value.set_watched_file(watched_file)
logger.trace("handle_association call set_watched_file")
end
watched_file.initial_completed
if watched_file.all_read?
watched_file.ignore
Expand Down
4 changes: 4 additions & 0 deletions lib/filewatch/sincedb_value.rb
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,10 @@ def set_watched_file(watched_file)
@watched_file = watched_file
end

def set_watched_file_without_touch(watched_file)
@watched_file = watched_file
end

def touch
@last_changed_at = Time.now.to_f
end
Expand Down
3 changes: 3 additions & 0 deletions lib/filewatch/tail_mode/processor.rb
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,9 @@ def process_rotation_in_progress(watched_files)
potential_sdb_value.set_watched_file(watched_file)
end
end
# Clean the path_in_sincedb because a file with the same name appears, this file must have been renamed
# Fix the duplicating collection after reloading sincedb(logstash restart or change logstash.conf)
sdb_value.add_path_in_sincedb(nil) unless sdb_value.nil?
logger.trace("---------- >>>> Rotation In Progress: after handling rotation", "this watched_file details" => watched_file.details, "sincedb_value" => (potential_sdb_value || sdb_value))
end
end
Expand Down