select() max number of file descriptors

[Charles Wilkins]

It seems that my system's select call has a limitation of 1024 open
file descriptors.

I need to learn about this restriction as I am programming a
multiplexed I/O server.

Is the 1024 open file descriptor limitation a per process limitation
or a system wide limitation?

How can I find out how many open files my system supports?

Where do I look to find out if my C / C++ implementation has similar
such file limits?

I have done some googling and have a general idea of the implications,
but I need details for my project.

What are some solutions for handling a large number ( 5000+ ) of
simultaneously open sockets?

Thank you for reading,
Charles Wilkins

[DINH Viêt Hoà]

Charles Wilkins wrote :

> Is the 1024 open file descriptor limitation a per process limitation
> or a system wide limitation?
>
> How can I find out how many open files my system supports?

sysconf(_SC_OPEN_MAX)

> What are some solutions for handling a large number ( 5000+ ) of
> simultaneously open sockets?

could you use poll() ?
But I am wondering if there is no such limit with poll()

--
DINH V. Hoa,

"s/^\(\(\\.\|[^\[]\|\[\(\^.\|[^^]\)[^]]*\]\)*\)\(\\[^\*[]\)/\1\\\4/"
-- Stéphane CHAZELAS

[Casper H.S. Dik]

DINH =?iso-8859-1?Q?Vi=EAt_Ho=E0?= <dinh.viet.hoa@free.fr> writes:

>Charles Wilkins wrote :

>> Is the 1024 open file descriptor limitation a per process limitation
>> or a system wide limitation?
>>
>> How can I find out how many open files my system supports?

>sysconf(_SC_OPEN_MAX)

Might just return your current soft limit and ntot the real system limit.

Generally, there are three limits:

- max numbers of fds in a single process (could be as low as 1024)
- process hard limit (equal or less than the first)
- process soft limit.

>> What are some solutions for handling a large number ( 5000+ ) of
>> simultaneously open sockets?

>could you use poll() ?
>But I am wondering if there is no such limit with poll()

poll has no restriction on the size of the pollset but your process
may stil be limited.

Solaris does not have a kernel enforced hard limit but a per-proces
hard limit of 1024 which can be raised or lowered.


Using larger fd sets generally requires a recompile with a certain
flags (typeically -DFD_SETSIZE=value)

Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

[Brian C]

:::Snip:::

> What are some solutions for handling a large number ( 5000+ ) of
> simultaneously open sockets?

It depends on what type of processing will be needed to be done on the
messages received by the clients. For instance, I've got servers at work
that handle very little load, and then some that need to process 400-1000+
blocked messages a second, and do alot with each message in the block.
If you want to have 5000+ connections, even if it's the simplest, I
would probably suggest multithreading the application with pthreads. You
could for instance have a thread accepting new connections and then either
pass the connection off to an existing thread (i.e. have 20 connections
per), or create a new thread if that limit has been reached. I've done this
and it allows me to change the connections per thread, and maximum threads
via a config file.