[20191202]关于hugepages相关问题.txt
--//上午在测试hugepages相关问题时遇到一个问题,自己以前疏忽了,就是在use_large_pages=true的情况下.
--//如果在/etc/sysctl.conf参数文件中vm.nr_hugepages定义不够大,即使定义vm.nr_overcommit_hugepages足够.
--//也不会向vm.nr_overcommit_hugepages借用内存,导致进入混合模式.
--//通过例子说明:
1.环境:
SCOTT@book> @ ver1
PORT_STRING VERSION BANNER
------------------------------ -------------- --------------------------------------------------------------------------------
x86_64/Linux 2.4.xx 11.2.0.4.0 Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
SCOTT@book> alter system set use_large_pages=true scope=spfile;
System altered.
# grep "^vm.nr" /etc/sysctl.conf
vm.nr_hugepages = 104
vm.nr_overcommit_hugepages = 1000
# sysctl -p
2.测试:
--//重启数据库看看,检查alert文件发现:
Adjusting the default value of parameter parallel_max_servers
from 480 to 170 due to the value of parameter processes (200)
Starting ORACLE instance (normal)
************************ Large Pages Information *******************
Per process system memlock (soft) limit = 51 GB
Total Shared Global Region in Large Pages = 208 MB (33%)
Large Pages used by this instance: 104 (208 MB)
Large Pages unused system wide = 0 (0 KB)
Large Pages configured system wide = 104 (208 MB)
Large Page size = 2048 KB
RECOMMENDATION:
Total System Global Area size is 618 MB. For optimal performance,
prior to the next instance restart:
1. Increase the number of unused large pages by
at least 205 (page size 2048 KB, total size 410 MB) system wide to
get 100% of the System Global Area allocated with large pages
********************************************************************
--//可以发现仅仅使用定义vm.nr_hugepages = 104.
$ grep -i page /proc/meminfo
AnonPages: 165700 kB
PageTables: 18904 kB
AnonHugePages: 0 kB
HugePages_Total: 104
HugePages_Free: 49
HugePages_Rsvd: 49
HugePages_Surp: 0
Hugepagesize: 2048 kB
$ ps -ef | grep pmo[n] | awk '{print $2}' | xargs -IQ grep SYSV /proc/Q/numa_maps
60000000 default file=/SYSV00000000\040(deleted) huge dirty=1 mapmax=25 N0=1
60c00000 default file=/SYSV00000000\040(deleted) huge
6a400000 default file=/SYSV00000000\040(deleted) huge
6c400000 default file=/SYSV00000000\040(deleted) huge
6cc00000 default file=/SYSV00000000\040(deleted) huge
--//仅仅5个共享内存段使用hugepages.
6d000000 interleave:0-1 file=/SYSV00000000\040(deleted) dirty=785 mapmax=25 active=245 N0=400 N1=385
86800000 interleave:0-1 file=/SYSVe8a8ec10\040(deleted) dirty=1 mapmax=25 N1=1
$ sysresv
IPC Resources for ORACLE_SID "book" :
Shared Memory:
ID KEY
413040641 0x00000000
413073410 0x00000000
413106179 0x00000000
413138949 0x00000000
413171718 0x00000000
413204487 0x00000000
413237256 0xe8a8ec10
Semaphores:
ID KEY
335151104 0x6aa88594
Oracle Instance alive for sid "book"
--//也就是在use_large_pages=true的情况下不会向vm.nr_overcommit_hugepages参数借用内存.
3.测试:
SYS@book> alter system set use_large_pages=only scope=spfile;
System altered.
# grep "^vm.nr" /etc/sysctl.conf
vm.nr_hugepages = 104
vm.nr_overcommit_hugepages = 204
# sysctl -p
..
SYS@book> startup
ORA-27137: unable to allocate large pages to create a shared memory segment
Linux-x86_64 Error: 12: Cannot allocate memory
Additional information: 2097152
--//104+204 = 308,还不够大.
# grep "^vm.nr" /etc/sysctl.conf
vm.nr_hugepages = 104
vm.nr_overcommit_hugepages = 1000
# sysctl -p
..
SYS@book> startup
ORACLE instance started.
Total System Global Area 643084288 bytes
Fixed Size 2255872 bytes
Variable Size 205521920 bytes
Database Buffers 427819008 bytes
Redo Buffers 7487488 bytes
Database mounted.
Database opened.
--//alert输出如下:
Adjusting the default value of parameter parallel_max_servers
from 480 to 170 due to the value of parameter processes (200)
Starting ORACLE instance (normal)
************************ Large Pages Information *******************
Parameter use_large_pages = ONLY
Per process system memlock (soft) limit = 51 GB
Large Pages unused system wide = 308 (616 MB)
Large Pages configured system wide = 308 (616 MB)
Large Page size = 2048 KB
ERROR:
Failed to allocate shared global region with large pages, unix errno = 12.
Aborting Instance startup.
ORA-27137: unable to allocate Large Pages to create a shared memory segment
ACTION:
Total System Global Area size is 624 MB. Prior to next instance restart:
1. Increase the number of unused large pages (page size 2048 KB)
to at least 312 (624 MB) to allocate 100% System Global Area
with large pages.
********************************************************************
--//以上是报ORA-27137的情况.
Mon Dec 02 15:59:56 2019
Adjusting the default value of parameter parallel_max_servers
from 480 to 170 due to the value of parameter processes (200)
Starting ORACLE instance (normal)
************************ Large Pages Information *******************
Parameter use_large_pages = ONLY
Per process system memlock (soft) limit = 51 GB
Total Shared Global Region in Large Pages = 618 MB (100%)
Large Pages used by this instance: 309 (618 MB)
Large Pages unused system wide = 0 (0 KB)
Large Pages configured system wide = 309 (618 MB)
Large Page size = 2048 KB
********************************************************************
$ ps -ef | grep pmo[n] | awk '{print $2}' | xargs -IQ grep SYSV /proc/Q/numa_maps
60000000 default file=/SYSV00000000\040(deleted) huge dirty=1 mapmax=25 N0=1
60c00000 default file=/SYSV00000000\040(deleted) huge dirty=37 mapmax=25 N0=30 N1=7
86800000 interleave:0-1 file=/SYSVe8a8ec10\040(deleted) huge dirty=1 mapmax=25 N0=1
$ sysresv
IPC Resources for ORACLE_SID "book" :
Shared Memory:
ID KEY
413401089 0x00000000
413433858 0x00000000
413466627 0xe8a8ec10
Semaphores:
ID KEY
335314944 0x6aa88594
Oracle Instance alive for sid "book"
3.总结:
--//设置内核参数vm.nr_overcommit_hugepages ,仅仅在use_large_pages=only的情况下才会有效.
来自 “ ITPUB博客 ” ,链接:/267265/viewspace-2666518/,如需转载,请注明出处,否则将追究法律责任。