1

I know that there is a hardware capacity of 2^48 bits, and I know that there isn't a lot of native limiting of app memory constraints in general by the kernel, but is there an upper bound of the memory that an app can consume in general besides 2^48 because that's the number of channels available on the DIMM sockets?

For reference, Redmond limits their products to ~2TB on most server products.

6
  • 3
    On what OS and what architecture? I'd expect a theoretical limit at 8EB (2^63) on most OSes, but this is entirely dependent on the unix variant, and possibly on versions or kernel compilation options. The architecture is going to be a limiting factor in practice (you've got to fit the MMU table), unless you use huge pages (which should let you reach the theoretical maximum on some OS/architecture combinations). Of course, the real limiting factor will probably be your budget. Commented Sep 27, 2011 at 0:42
  • The only architecture I can think of is either x86_64 or AMD64. I know that Microsoft has a specific max size on their processes, and I thought the 32 bit Unix and Linux variants had a max process size below 2^32 (especially since PAE allows for more than 2^32 addressing) Commented Sep 27, 2011 at 14:39
  • Linux does allow a 4G/4G split, which gives the process and kernel space each almost a full 4GB, with a bit of the space reserved to help switch between the two. I'm not aware that any Unices allow this. Commented Sep 28, 2011 at 1:35
  • FWIW there was a question on Database Administrators that asked about running a 500GB in memory database, so that was the impetus to this Q (cc @Gilles @Ignacio) Commented Sep 28, 2011 at 1:39
  • @jcolebrand Link? This is well above my pay grade, but my first reflex would be to arrange to split the database over about 100 consumer-grade PCs (the Google way). If you want 500GB in the same machine (about 10 times what you can get on current PC-style servers), the machine will come with a working OS anyway, probably Linux or perhaps AIX (or Solaris?) (also: you have a dedicated pro working for you). Commented Sep 28, 2011 at 7:16

2 Answers 2

5

By definition (at least, by a common definition), a 64-bit process can only see 264 bytes of memory. In practice, most operating systems limit each process to a little less than the available address space, so that the kernel can manipulate 64-value that can be pointers to kernel memory, process memory or device memory. A 4EB or 8EB limit is common.

Well before that, you'll probably hit an architecture-dependent limit. In particular, a process's memory map has to fit in the machine's MMU, and hardware tends to have only as many bits as necessary. MMUs come with up to four levels of page tables, and Linux (amongst others) can support all four: a PGD (page global directory) listing PUDs (page upper directories) listing PMDs (page mid-level directories) listing PTEs (page table entries). Since each level of indirection is one more step to take on most pointer dereferences, systems . It is common for the size of a page to be 4kB (21242 addressable bytes, however most 64-bit architectures can be configured for larger page tables, so it is hard to give any kind of general answer to the question of how high you can go.

Note that the per-process address limit is independent of how much physical memory there can be in the machine. For example, the number of channels available on the DIMM sockets in a PC are a theoretical limit to how much RAM you can have in that PC (although the width of the CPU memory bus is usually the limiting factor). It has no bearing on how much memory or how much address space a process can consume (the memory could be swapped out, or shared between several nodes of a NUMA multiprocessor machine).

2

The theoretical maximum is limited by the memory segmentation used on the CPU, e.g. X86-64 uses an absolute mechanism (the segments start at 0 and have a capacity of 264), but the OS itself can enforce artificial limits on the process size.

2
  • define "OS" and how that's different from the kernel. Do any of the current kernels limit this behavior on 64 bit systems? Commented Sep 27, 2011 at 0:41
  • The "OS" is the kernel plus any modules and libraries required to support operation of the machine. I am not aware of any *nices that enforce a process limit, but I do know that some enforce a physical limit. Commented Sep 27, 2011 at 0:56

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.