numactl sources are on github:
https://github.com/numactl/numactl
I looked how "-m" option is implemented and got things wrong. Then I saw numademo.c allocating memory on specified numa node with "numa_alloc_onnode()". Looking into implementation of that
https://github.com/numactl/numactl/blob ... 1150-L1168
likely the missing memory mapping done there was what made my code not work. No need to go into the detail, just use that function:I created new node_mem.c demo:
https://github.com/Hermann-SW/RR/blob/m ... node_mem.c
Arguments are size to be allocated on each node, and number of seconds to sleep at the end.
This was initial free memory on 4 numa nodes of 4-socket system:Then I did run "./node_mem 10000000000 10", and after some time did run above command again:The 10GiB are allocated on node 0 and 1 already, partially on node 2 and not in node 3.
After the number of nodes got output the 10 second sleep started:Free memory then was as it should:Finally, after program completed, automatically all memory allocated on numa nodes got freed:
As stated before I want to run code on systems I thought having no numa nodes (Pi5 and AMD 7950X CPU PC). But "numa_available()" returns 0 on those systems as it does on 2-socket and 4-socket systen. So I installed numactl on 7950X PCand saw a single numa node:
But to my big surprise the Pi5 reports 8 numa nodes?!?!?
https://github.com/numactl/numactl
I looked how "-m" option is implemented and got things wrong. Then I saw numademo.c allocating memory on specified numa node with "numa_alloc_onnode()". Looking into implementation of that
https://github.com/numactl/numactl/blob ... 1150-L1168
likely the missing memory mapping done there was what made my code not work. No need to go into the detail, just use that function:
Code:
... void *numa_alloc_onnode(size_t size, int node);... numa_alloc_onnode() allocates memory on a specific node. The size ar‐ gument will be rounded up to a multiple of the system page size. if the specified node is externally denied to this process, this call will fail. This function is relatively slow compared to the malloc(3) fam‐ ily of functions. The memory must be freed with numa_free(). On er‐ rors NULL is returned....https://github.com/Hermann-SW/RR/blob/m ... node_mem.c
Arguments are size to be allocated on each node, and number of seconds to sleep at the end.
This was initial free memory on 4 numa nodes of 4-socket system:
Code:
hermann@x3950-X6:~$ numactl -H | grep freenode 0 free: 62967 MBnode 1 free: 63769 MBnode 2 free: 63153 MBnode 3 free: 63117 MBhermann@x3950-X6:~$Code:
hermann@x3950-X6:~$ numactl -H | grep freenode 0 free: 53426 MBnode 1 free: 54218 MBnode 2 free: 60952 MBnode 3 free: 63117 MBhermann@x3950-X6:~$After the number of nodes got output the 10 second sleep started:
Code:
hermann@x3950-X6:~/RR/tsp/pthread$ ./node_mem 10000000000 10numnodes=4Code:
hermann@x3950-X6:~$ numactl -H | grep freenode 0 free: 53426 MBnode 1 free: 54218 MBnode 2 free: 53519 MBnode 3 free: 53573 MBhermann@x3950-X6:~$Code:
hermann@x3950-X6:~$ numactl -H | grep freenode 0 free: 62957 MBnode 1 free: 63750 MBnode 2 free: 63082 MBnode 3 free: 62960 MBhermann@x3950-X6:~$ As stated before I want to run code on systems I thought having no numa nodes (Pi5 and AMD 7950X CPU PC). But "numa_available()" returns 0 on those systems as it does on 2-socket and 4-socket systen. So I installed numactl on 7950X PCand saw a single numa node:
Code:
hermann@7950x:~$ numactl -Havailable: 1 nodes (0)node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31node 0 size: 31197 MBnode 0 free: 29341 MBnode distances:node 0 0: 10 hermann@7950x:~$But to my big surprise the Pi5 reports 8 numa nodes?!?!?
8 numa nodes with 512MB for my 4GB Pi5 ...pi@raspberrypi5:~ $ numactl -H
available: 8 nodes (0-7)
node 0 cpus: 0 1 2 3
node 0 size: 487 MB
node 0 free: 100 MB
node 1 cpus: 0 1 2 3
node 1 size: 501 MB
node 1 free: 107 MB
node 2 cpus: 0 1 2 3
node 2 size: 509 MB
node 2 free: 132 MB
node 3 cpus: 0 1 2 3
node 3 size: 509 MB
node 3 free: 120 MB
node 4 cpus: 0 1 2 3
node 4 size: 509 MB
node 4 free: 131 MB
node 5 cpus: 0 1 2 3
node 5 size: 509 MB
node 5 free: 83 MB
node 6 cpus: 0 1 2 3
node 6 size: 509 MB
node 6 free: 115 MB
node 7 cpus: 0 1 2 3
node 7 size: 507 MB
node 7 free: 172 MB
node distances:
node 0 1 2 3 4 5 6 7
0: 10 10 10 10 10 10 10 10
1: 10 10 10 10 10 10 10 10
2: 10 10 10 10 10 10 10 10
3: 10 10 10 10 10 10 10 10
4: 10 10 10 10 10 10 10 10
5: 10 10 10 10 10 10 10 10
6: 10 10 10 10 10 10 10 10
7: 10 10 10 10 10 10 10 10
pi@raspberrypi5:~ $
Statistics: Posted by HermannSW — Tue Aug 26, 2025 4:15 pm