Skip to content

Improve logging #4921

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

matekungl-byborg
Copy link
Contributor

Fix for "Too Many Open File Descriptors" Issue

Changes

  • Refactored request logging implementation: Replaced global requestLogger with per-request logger instances
    • Removed global requestLogger variable that was previously initialized once
    • Now creating a new logger instance for each request in expressRequestLogger function
    • This prevents file descriptor accumulation over time

Technical Details

  • The previous implementation used a single shared logger instance for all requests
  • The new implementation creates a fresh logger for each request, ensuring proper resource cleanup
  • Request metadata is now directly included in the logger's defaultMeta rather than passed as a parameter
  • All other logging functionality (S3, GCS, local storage) remains unchanged

Benefits

  • Fixes the "Too many open file descriptors" error in high-traffic environments
  • Maintains all existing logging capabilities while improving resource management
  • No configuration changes required - works with all existing storage backends

Left is the original version, right is the fixed (simple lsof -w | wc -l)

image

Copy link
Contributor

@HenryHengZJ HenryHengZJ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thnak you!

@HenryHengZJ HenryHengZJ merged commit 8846fd1 into FlowiseAI:main Jul 25, 2025
2 checks passed
@matekungl-byborg matekungl-byborg deleted the bugfix/too-many-open-file-descriptor branch July 27, 2025 09:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants