home | list info | list archive | date index | thread index

Interesting little feature (bug?) in GNU "make"

Hi,

Yeah, yeah, I'm bored.

GNU Make has a "-jn" option that lets you run up to "n" parallel make
processes in parallel.  I use it a lot because my workstation is stupidly
powerful (32 cores / 64 threads).

Then I started trying higher and higher values of "n".  All went well
up to about 65535.  But higher numbers hung the process.

OK, pause reading and see if you can come up with a theory as to why this
magic number would cause "make" to lock up.

.

.

.

.

.

Turns out, make keeps track of how many processes are running using a pipe.
At the start, it fills the pipe with n copies of the "+" character.
When it wants to start a new process, it reads one character from the
pipe.  When that process exits, it writes another "+" to the pipe.  That
way, it never runs more than "n" processes at a time, because if it
tried to, the "n+1th" process would block until some other process
exited and wrote a "+" to the pipe.  Clever!

But on Linux, by default, a pipe's capacity is around 64K.  If you set n
too high, then the initial fill of the pipe blocks and make hangs.

Hope some of you found that interesting.  So the caveat is: Don't buy
a computer with more than 64K CPU threads.  "make" won't be able to take
full advantage of it. :)

Regards,

Dianne.

To unsubscribe send a blank message to linux+unsubscribe [ at ] linux-ottawa [ dot ] org
To get help send a blank message to linux+help [ at ] linux-ottawa [ dot ] org
To visit the archives: https://lists.linux-ottawa.org