[afnog] Understanding the "Stack Size" ulimit Parameter wrt Multithreading

[Gerald Begumisa]

Hi,

I ran into an issue while stress testing a multithreaded application
and I was hoping I could find a few answers here.

I found the "stack size" ulimit parameter abit non-intuitive.  I found
that with the default stack size setting, the application could only
create about 300 threads after which the error "Cannot allocate
memory" is encountered.  Initially I thought that by increasing the
stack size setting, I would be able to create more threads, but this
was not the case, instead I could create fewer threads.  See demo
below (uses attached test.c program):

------->8----------
[root at switch34-billing tmp]# gcc -o tes test.c -lpthread
[root at switch34-billing tmp]# ulimit -s
10240
[root at switch34-billing tmp]# ./tes
Created thread 0
Created thread 1
...
Created thread 303
Created thread 304
Error creating thread 305: Cannot allocate memory
[root at switch34-billing tmp]# ulimit -s 4096
[root at switch34-billing tmp]# ./tes
Created thread 0
Created thread 1
...
Created thread 760
Created thread 761
Error creating thread 762: Cannot allocate memory
--------8<------

Therefore I figured that the stack size parameter should probably be
interpreted on a per-thread or per-process basis.  This meaning that a
bigger stack size meant fewer threads could be created (perhaps
because there is some kind of per-process memory limit). And likewise
a smaller stack size meaning more threads created.

My question is: which ulimit setting governs the per-process memory
limit? Any clues welcome - I may be interpreting this wrongly.  Below
is the output of "ulimit -a" for the same server:

------
[root at switch34-billing tmp]# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
max nice                        (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 8192
max locked memory       (kbytes, -l) 32
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
max rt priority                 (-r) 0
stack size              (kbytes, -s) 4096
cpu time               (seconds, -t) unlimited
max user processes              (-u) 8192
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
file locks                      (-x) unlimited
----


Regards,
Gerald.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.c
Type: application/octet-stream
Size: 481 bytes
Desc: not available
URL: <http://afnog.org/pipermail/afnog/attachments/20080903/2792bc8c/attachment-0002.obj>