As of 10.10, the maximum allowed number of simultaneous open
files, or maximum number of file descriptors, has been raised from
2048 to 60000 per process. This value is specified by the constant
MAXFUPLIM (and
the equivalents _MAXFUPLIM
and FD_SETSIZE).
This feature is optional because it will not be often used
and to minimize the impact on existing code. An application requiring
more than 2048 file descriptors must be recompiled with the new
symbol _USE_BIG_FDS
defined. To do this, add the flag D_USE_BIG_FDS
to the compile command in the application's makefile.
Or, the symbol can be defined at the beginning of every application
source file (via #define _USE_BIG_FDS).
As in previous releases, there are also per-application resource
limits, independent of the MAXFUPLIM
limit. To use a large amount of file descriptors, your application
might need to modify its RLIMIT_NOFILE
resource limits with getrlimit(2)
and setrlimit(2).
See the manpages for these system calls for more information. Alternatively,
the system-wide RLIMIT_NOFILE
defaults can be changed (as in previous releases) by modifying the
kernel tunables maxfiles
and maxfiles_lim;
see the SAM on-line kernel configuration help for more information.
Any machine running an application that uses a large amount
of file descriptors might need to be reconfigured with a larger
value for the kernel tunable nfile.
This tunable specifies the per-machine (as opposed to per-process)
maximum number of simultaneous open files and is by default much
less than 60000. See the SAM online kernel configuration help for
more information.
Use of this new feature is not compatible with all libraries.
Users need to review the libraries used by their application to
determine if they can use this feature. See the section "Impacts"
below for more information.
You can use SAM to implement this feature. The kernel tunables
table reflect new maximum values for the tunables maxfiles
and maxfiles_lim,
from 2048 to 60000.
Impacts |
 |
This new functionality might have three possible affects:
For all user-space code (commands, libraries, applications, and user code):
User-space code that raises its RLIMIT_NOFILE
resource limit with setrlimit
(see above) beyond MAXFUPLIM
might fail.
The RLIMIT_NOFILE
resource limit cannot exceed MAXFUPLIM.
In previous releases, the kernel and user space MAXFUPLIM
values were the same. To support 60K file descriptors, the kernel
MAXFUPLIM is
now 60000. However, to maintain compatibility with previous releases,
user applications by default will still use a MAXFUPLIM
of 2048. If an application with a small MAXFUPLIM
"succeeds" in raising its resource limit above 2048, subsequent
failures might occur. For example, you can now open more than 2048
files (assuming that the kernel tunable nfile
is sufficiently large), though select(2)
data structures are sized for only 2048 file descriptors. A select
call might hang or might cause a segmentation violation depending
on the arguments passed to it. See the select
manpage and the <sys/types.h>
header file for more information.
There are no workarounds for this impact; impacted code must
be modified.
For code opting for 60K file descriptor functionality:
The use of 60K file descriptors in conjunction with some libraries
is not supported. A library compiled with the old MAXFUPLIM
value might not work with applications using the new MAXFUPLIM
value. Any calls to select(2)
made by a library function on behalf of the application would fail
if you had more than 2048 open files, as in the preceding item above.
The use of 60K file descriptors in conjunction with the following
libraries is not supported:
X Window System libraries
Pascal and Fortran libraries
3D graphics libraries (Starbase, HP PEX, and HP-PHIGS)
For code opting for 60K file descriptor functionality:
Programs that opt for 60K file descriptors might experience
excessive memory usage or performance problems.
The fd_set
data structure used in select(2)
is sized with the FD_SETSIZE
constant (equivalent to MAXFUPLIM).
Increasing this constant from 2048 to 60000 increases the amount
of memory used by an fd_set
from 256 bytes to about 7500 bytes. Most applications that use fd_sets
use only a few. However, there might be applications that use many.
Recompiling those applications to use 60K file descriptors will
cause a large increase in memory usage, which may be problematic.
The same problem exists for applications that use MAXFUPLIM
or FD_SETSIZE
to size other data structures.
For similar reasons, applications opting for 60K file descriptors
might experience performance degradation if the application uses
MAXFUPLIM or
FD_SETSIZE as
a loop bound or as a select nfds
argument (see the select
manpage for details).
Alternatives |
|
To minimize the impact on existing code, the 60K file descriptor
feature is optional. Existing code, even as it is recompiled, continues
to use the old MAXFUPLIM
value of 2048 unless a different value is selected by defining _USE_BIG_FDS.
Whether code is recompiled with or without _USE_BIG_FDS
or even if the code is not recompiled (such as existing binaries),
it may be vulnerable to the first impact described above. There
is no workaround for this impact.
Size Requirement |
|
Approximately 1 KB of memory is used by the kernel to store
32 file descriptors, both in the per-process file descriptor table
and the system-wide file table (sized by nfile,
above). 60K file descriptors can use as much as 2 MB. Thus, excessive
use of this feature may not be possible on small-memory machines.
Printable version
