Not Another Post Configuring HugePages for Oracle on Linux (x86-64)

USE_LARGE_PAGES (TRUE/FALSE/ONLY)

Test case is only for one database instance on server.

DB is using memory_target.

SQL> show parameter use_large_pages
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
use_large_pages string TRUE
SQL> show parameter sga
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
lock_sga boolean FALSE
pre_page_sga boolean FALSE
sga_max_size big integer 768M
sga_target big integer 0
SQL> show parameter pga
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
pga_aggregate_target big integer 0
SQL> show parameter target
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
archive_lag_target integer 900
db_flashback_retention_target integer 1440
fast_start_io_target integer 0
fast_start_mttr_target integer 0
memory_max_target big integer 1G
memory_target big integer 1G
parallel_servers_target integer 16
pga_aggregate_target big integer 0
sga_target big integer 0
SQL> show sga
Total System Global Area 801701888 bytes
Fixed Size 2257520 bytes
Variable Size 276827536 bytes
Database Buffers 520093696 bytes
Redo Buffers 2523136 bytes
SQL> exit

Gather memory configuration.

$ grep Huge /proc/meminfo
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
++++++++++
$ grep PageTables /proc/meminfo
PageTables: 24428 kB
++++++++++
$ free
total used free shared buffers cached
Mem: 1534604 1484476 50128 1512 20352 335916
-/+ buffers/cache: 1128208 406396
Swap: 4194300 0 4194300

Calculate memlock.

SQL> select round(1534604*.875) from dual;
ROUND(1534604*.875)
-------------------
1342779

Edit /etc/security/limits.conf to add memlock as shown below.

Logout, login, check ulimit -l

$ grep memlock /etc/security/limits.conf
# - memlock - max locked-in-memory address space (KB)
oracle soft memlock 1342779
oracle hard memlock 1342779
++++++++++
$ ulimit -l
134217728

Run hugepages_settings.sh ERROR due to memory target being used.

$ ./hugepages_settings.sh
This script is provided by Doc ID 401749.1 from My Oracle Support
(http://support.oracle.com) where it is intended to compute values for
the recommended HugePages/HugeTLB configuration for the current shared
memory segments on Oracle linux. Before proceeding with the execution please note following:
* For ASM instance, it needs to configure ASMM instead of AMM.
* The 'pga_aggregate_target' is outside the SGA and
you should accommodate this while calculating SGA size.
* In case you changes the DB SGA size,
as the new SGA will not fit in the previous HugePages configuration,
it had better disable the whole HugePages,
start the DB with new SGA size and run the script again.
And make sure that:
* Oracle Database instance(s) are up and running
* Oracle Database 11g Automatic Memory Management (AMM) is not setup
(See Doc ID 749851.1)
* The shared memory segments can be listed by command:
# ipcs -m
Press Enter to proceed...
***********
** ERROR **
***********
Sorry! There are not enough total of shared memory segments allocated for
HugePages configuration. HugePages can only be used for shared memory segments
that you can list by command:
# ipcs -m
of a size that can match an Oracle Database SGA. Please make sure that:
* Oracle Database instance is up and running
* Oracle Database 11g Automatic Memory Management (AMM) is not configured

Remove memory target configuration from DB.

Note reset was used as shown:

alter system reset memory_target scope=spfile sid=’*’;

alter system reset memory_max_target scope=spfile sid=’*’;

oracle@arrow1:HAWKA:/home/oracle
$ sysdba
SQL*Plus: Release 11.2.0.4.0 Production on Fri Jan 6 21:33:31 2017
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning and Real Application Testing options
SQL> show parameter spfile
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
spfile string /u01/app/oracle/product/11.2.0
.4/db_1/dbs/spfileHAWKA.ora
SQL> alter system reset memory_target scope=spfile sid='*';
System altered.
SQL> alter system reset memory_max_target scope=spfile sid='*';
System altered.
SQL> alter system set sga_max_size=768M scope=spfile sid='*';
System altered.
SQL> alter system set sga_target=768M scope=spfile sid='*';
System altered.
SQL> alter system set pga_aggregate_target=256M scope=spfile sid='*';
System altered.
SQL> shu immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup;
ORACLE instance started.
Total System Global Area 801701888 bytes
Fixed Size 2257520 bytes
Variable Size 276827536 bytes
Database Buffers 520093696 bytes
Redo Buffers 2523136 bytes
Database mounted.
Database opened.
SQL> @show.sql
SQL> show parameter use_large_pages
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
use_large_pages string TRUE
SQL> show parameter sga
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
lock_sga boolean FALSE
pre_page_sga boolean FALSE
sga_max_size big integer 768M
sga_target big integer 768M
SQL> show parameter pga
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
pga_aggregate_target big integer 256M
SQL> show parameter target
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
archive_lag_target integer 900
db_flashback_retention_target integer 1440
fast_start_io_target integer 0
fast_start_mttr_target integer 0
memory_max_target big integer 0
memory_target big integer 0
parallel_servers_target integer 16
pga_aggregate_target big integer 256M
sga_target big integer 768M
SQL> show sga
Total System Global Area 801701888 bytes
Fixed Size 2257520 bytes
Variable Size 276827536 bytes
Database Buffers 520093696 bytes
Redo Buffers 2523136 bytes
SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning and Real Application Testing options
oracle@arrow1:HAWKA:/home/oracle
$

Run hugepages_settings.sh Recommended setting: vm.nr_hugepages = 388

$ ./hugepages_settings.sh
This script is provided by Doc ID 401749.1 from My Oracle Support
(http://support.oracle.com) where it is intended to compute values for
the recommended HugePages/HugeTLB configuration for the current shared
memory segments on Oracle Linux. Before proceeding with the execution please note following:
* For ASM instance, it needs to configure ASMM instead of AMM.
* The 'pga_aggregate_target' is outside the SGA and
you should accommodate this while calculating SGA size.
* In case you changes the DB SGA size,
as the new SGA will not fit in the previous HugePages configuration,
it had better disable the whole HugePages,
start the DB with new SGA size and run the script again.
And make sure that:
* Oracle Database instance(s) are up and running
* Oracle Database 11g Automatic Memory Management (AMM) is not setup
(See Doc ID 749851.1)
* The shared memory segments can be listed by command:
# ipcs -m
Press Enter to proceed...
Recommended setting: vm.nr_hugepages = 388
Manually calculate vm.nr_hugepages in KB using [sga_max_size(768M) * 1024 * Hugepagesize(2048 kB)] SQL> select round(768*1024/2048)+1 from dual;
ROUND(768*1024/2048)+1
----------------------
385
SQL>

From alert log vm.nr_hugepages=385

Fri Jan 06 21:34:33 2017
Starting ORACLE instance (normal)
************************ Large Pages Information *******************
Per process system memlock (soft) limit = 128 GB
Total Shared Global Region in Large Pages = 0 KB (0%)
Large Pages used by this instance: 0 (0 KB)
Large Pages unused system wide = 0 (0 KB)
Large Pages configured system wide = 0 (0 KB)
Large Page size = 2048 KB
RECOMMENDATION:
Total System Global Area size is 770 MB. For optimal performance,
prior to the next instance restart:
1. Increase the number of unused large pages by at least 385
(page size 2048 KB, total size 770 MB) system wide to get
100% of the System Global Area allocated with large pages
********************************************************************

Configure Dynamic vm.nr_hugepages=385

[root@arrow1 ~]# sysctl -w vm.nr_hugepages=385
vm.nr_hugepages = 385
[root@arrow1 ~]# grep Huge /proc/meminfo
HugePages_Total: 353
HugePages_Free: 353
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB

Configure Static vm.nr_hugepages=385 and restart server Oracle recommended.

[root@arrow1 ~]# grep vm.nr_hugepages /etc/sysctl.conf
[root@arrow1 ~]# vi /etc/sysctl.conf
[root@arrow1 ~]# grep vm.nr_hugepages /etc/sysctl.conf
vm.nr_hugepages=385
[root@arrow1 ~]# cat /etc/system-release
Oracle Linux Server release 6.6
[root@arrow1 ~]#