I had downloaded a open source tool pylint. When I got it to run, I was always given this stack trace “Too many open files”. After searching for a clue on the web, I was pretty positive too many file descriptors created by the tool since I saw a similar problem. Wanting to be sure I did:
$ watch lsof +D .
while running the tool. It looked right the output filled up the screen.
With the help of my coworkers, I discovered that there is a setting to increase the max opened files allowed bye a process. The command is
It’s a bash command to limit system-wide resources. Another thing that is new to me is that there is a concept of soft limit and hard limit. Not sure why there are such things, I seem to notice that the soft limit is what preventing pylint to work on huge number of files. It seems to me that the soft limit is the real max value that a user can set.
Below are ways to find out the lower and upper extremes.
$ ulimit -a -H
$ ulimit -a -S
Let say the soft limit for open files is 1024, you cannot do
$ ulimit -n 1025 because that is greater than the soft limit. That was a problem for me since I needed more so what I need to do first is to increase the soft limit and then use ulimit to increase it for my session.
After more research, I found a solution.
Basically, since I’m on Mac Leopard I need to do:
$ launchctl limit maxfiles 10240 unlimited
or set it directly in:
$ ulimit -n 10000
That solves my problem. Aside from that I probably need to ask pylint developers why it needs to open so many files because it could be I am using it wrong but at least I discovered something new.