The lab-shipper log collector forwards the logs of a hundred shop services from /var/log/shop to a central store. Your colleagues only see the logs of some of the services. Look at what the shipper itself writes, at your own open-file limit and at how many files the shipper has open.
Look at the shipper
The lab-shipper collector forwards the logs from /var/log/shop. Look at the last lines of its log, at your own open-file limit and at how many files the shipper process has open.
First, predict
The shipper cannot open the next log: Too many open files. How many files does the shipper process already have open?
Now start the exercise, run the command and compare your prediction with the output.
The shipper process has 64 files open and cannot open the next one. Your ulimit -n is 1024. What follows from this?
-
Linux does not forbid opening a file that another process has open: a service writes a log while the shipper reads it. A refused access would look like Permission denied.
-
The logs are already on the disk; the shipper only reads them.
df -hshows a lack of space, and the error would read No space left on device. -
Then other programs could not open files either, yet you open them freely. The shipper stops at exactly 64 — that is its own ceiling.
The shipper process has its own ceiling — 64 open files, which is lower than yours.