On Tue, Mar 28, 2006 at 01:44:38PM -0500, Peter Sjoberg wrote: > On Tue, 2006-03-28 at 12:21 -0500, Martin Hicks wrote: > > On Tue, Mar 28, 2006 at 11:27:08AM -0500, Peter Sjoberg wrote: > > > I just wonder if anyone knows what state the linux NUMA implementation > > > is at. I looked on some linux numa sites but it seems very dated (many > > > 2001 and 2004 the latest). > > > I had a discussion with a friend about how good numa is on linux and his > > > opinion is that we should run x86 Solaris on all opterons or replace the > > > 4way with UP system since numa is so immature under linux. > > > > NUMA works quite well under Linux. There is room for improvement, > > perhaps, but people like HP, AMD and SGI all have NUMA machines, and > > SGI's are even Linux *only* (and they're actually the biggest computers > > too, with up to 1024CPUs in a single machine) > I'm interested in how far it goes. As I understand it you have some > stickiness when it comes to process cpu but don't know how strong it is. Its pretty strong. With cpusets you can limit it to a single CPU. > If a process starts to access a file, are the cache buffers allocated > randomly or going to the same cpu(socket). Is memory migration planned? I think buffer cache is allocated round-robin, although you might be able to get it to allocate only in a specific set of nodes (cpuset) Some parts of migration work already. I'm not exactly sure where things are right now. Ray Bryant was working on that for SGI and AMD. > I seen some talk about making it more aware about near -far-farther > memory, and make it work better with multi-core and multi threading. The locality of memory is part of ACPI. Linux reads this information from ACPi tables and uses it to build zonelists appropriately. > Solaris has implemented at least some of it in the latest version (to > work better with there 8 core 32 thread cpu). > Where does Linux kernel stand on this? Linux worked fine on 512p machines a two years ago. > > > > There is libNUMA to do manual intervention on how you want memory > > policies to be inforced (local first, round robin, local only) as well > > as assigning jobs to run on certain CPUs. > Guess that works for program written for it but what about normal > cases. numactl mh -- Martin Hicks || mort [ at ] bork [ dot ] org || PGP/GnuPG: 0x4C7F2BEE
Attachment:
signature.asc
Description: Digital signature