As of 10.10, the maximum allowed number of simultaneous open files, or maximum number of file descriptors,
has been raised from 2048 to 60000 per process.
This value is specified by the constant MAXFUPLIM (and
the equivalents _MAXFUPLIM and FD_SETSIZE).
This feature is optional because
it will not be often used and to minimize
the impact on existing code. An application requiring more than 2048 file
descriptors must be recompiled with the new symbol _USE_BIG_FDS
defined. To do this, add the
flag D_USE_BIG_FDS to the compile command in the application's
makefile. Or, the symbol can be defined at the beginning of every
application source file (via #define _USE_BIG_FDS).
As in previous releases, there are also per-application resource
limits, independent of the MAXFUPLIM limit. To use a large
amount of file descriptors, your application might need to
modify its RLIMIT_NOFILE resource limits with
getrlimit(2) and setrlimit(2). See
the manpages for these system calls for more information.
Alternatively, the system-wide RLIMIT_NOFILE defaults can be changed
(as in previous releases) by modifying the kernel tunables maxfiles
and maxfiles_lim; see the SAM on-line kernel configuration help for
more information.
Any machine running an application that uses a large amount of file
descriptors might need to be reconfigured with a larger value for the
kernel tunable nfile. This tunable specifies the per-machine (as
opposed to per-process) maximum number of simultaneous open files
and is by default much less than 60000. See the SAM online kernel
configuration help for more information.
Use of this new feature is not compatible with all libraries. Users
need to review the libraries used by their application to
determine if they can use this feature. See the section "Impacts"
below for more information.
You can use SAM to implement this feature. The kernel tunables
table reflect new maximum values for the tunables maxfiles and
maxfiles_lim, from 2048 to 60000.
Impacts |
 |
This new functionality might have three possible affects:
For all user-space code (commands, libraries,
applications, and user code):
User-space code that raises its RLIMIT_NOFILE resource limit with
setrlimit (see above) beyond MAXFUPLIM might fail.
The RLIMIT_NOFILE resource limit cannot exceed MAXFUPLIM.
In previous releases, the kernel and user space MAXFUPLIM values
were the same. To support 60K file descriptors, the kernel
MAXFUPLIM is now
60000. However, to maintain compatibility with previous releases,
user applications by default will still use a MAXFUPLIM of 2048.
If an application with a small MAXFUPLIM "succeeds" in raising
its resource limit above 2048, subsequent failures might occur. For
example, you can now open more than 2048 files (assuming that
the kernel tunable nfile is sufficiently large),
though select(2) data structures are sized for only 2048 file
descriptors. A select call might hang or might cause a
segmentation violation depending on the arguments passed to it. See the select
manpage and the <sys/types.h> header file for more
information.
There are no workarounds for this impact; impacted code
must be modified.
For code opting for 60K file descriptor functionality:
The use of 60K file descriptors in conjunction with some libraries
is not supported. A library compiled with the old MAXFUPLIM
value might not work with applications using the new MAXFUPLIM value. Any calls to
select(2) made by a library function on behalf of the application would fail
if you had more than 2048 open files, as in the preceding
item above.
The use of 60K file descriptors in conjunction with the following
libraries is not supported:
X Window System libraries
Pascal and Fortran libraries
3D graphics libraries (Starbase, HP PEX, and HP-PHIGS)
For code opting for 60K file descriptor functionality:
Programs that opt for 60K file descriptors might experience
excessive memory usage or performance problems.
The fd_set data structure used in select(2) is sized with the
FD_SETSIZE constant (equivalent to MAXFUPLIM). Increasing this
constant from 2048 to 60000 increases the amount of memory used by
an fd_set from 256 bytes to about 7500 bytes.
Most applications that use fd_sets use only a few. However,
there might be applications that use many. Recompiling those
applications to use 60K file descriptors will cause a large increase
in memory usage, which may be problematic.
The same problem exists for applications that use MAXFUPLIM or
FD_SETSIZE to size other data structures.
For similar reasons, applications opting for 60K file descriptors might
experience performance degradation if the application uses
MAXFUPLIM or FD_SETSIZE
as a loop bound or as a select nfds argument
(see the select manpage for details).
Alternatives |
|
To minimize the impact on existing code,
the 60K file descriptor feature is optional.
Existing code, even as it is
recompiled, continues to use the old MAXFUPLIM value of 2048
unless a different value is selected by defining _USE_BIG_FDS.
Whether code is recompiled with or without _USE_BIG_FDS or even if
the code is not recompiled (such as existing binaries), it may be
vulnerable to the first impact described above. There is no workaround for this impact.
Size Requirement |
|
Approximately 1 KB of memory is used by the kernel to store 32 file
descriptors, both in the per-process file descriptor table and the
system-wide file table (sized by nfile, above). 60K file
descriptors can use as much as 2 MB. Thus, excessive use of this
feature may not be possible on small-memory machines.
Printable version
