The VMS SharkOpenVMS Notes: System Tuning

  1. The information and software presented on this web site are intended for educational use only by OpenVMS application developers and OpenVMS system attendants.
  2. The information and software presented on this web site are provided free of charge.
  3. The information and software presented on this web site are presented to you as-is. I will not be held responsible in any way if the information or software presented on this web site damages your computer system, business or organization (sounds like the legal warning from a Microsoft shrink-wrap seal, eh?)
  4. The information presented on this page can be really dangerous if you don't know what you are doing.
  5. The information on this page will never be more than a simple overview. Much of this info came from a third party (Raxco) tuning course back in 1987 but is still valid. If I've made any mistakes here then it was because I was drinking a Heineken while typing this.
*** This is a work in progress ***

A Very Simple Overview of OpenVMS Memory

  1. VMS and OpenVMS are multi-user "virtual memory" operating systems.

  2. VAX pages are always 512 bytes in size. Why? This was the industry standard size of a disk sector in the mid 1970s. Thirty five years later the disk drive business has proposed increasing the value from 512 to 4096 for disks manufactured in 2011.
     
  3. Alpha and Itanium make a distinction between pages and pagelets. Pagelets (smaller than a page) are always 512 bytes but "pages" are larger and vary between machines of the same type. Here are three ways to determine page size:

    1. $ mcr sysgen
      SYSGEN>  SHOW WSMAX
      Parameter Name            Current    Default     Min.       Max.   Unit  Dynamic
      --------------            -------    -------   -------    -------  ----  -------
      WSMAX                      100000       8192      1024  134217728 Pagelets   
       internal value              6250        512        64    8388608 Pages      
      SYSGEN>  EXIT
      $ write sys$output 100000/6250*512
      8192
      $
    2. $ show memory/phy
      System Memory Resources on 20-JUN-2010 06:22:01.58
      
      Physical Memory Usage (pages): Total Free In Use Modified
      Main Memory (128.00MB) 16384 3977 11327 1080
      $ write sys$output 128 * 1024 * 1024 / 16384
      8192
      $  
    3. $ write sys$output f$getsysi("PAGE_SIZE")
      8192
      $
    Speculation: "I think" DEC did it this way so that existing SYSUAF account settings on VAX would automatically receive 16 times more memory on Alpha. But this would only work if pagelets were larger than pages (see Caveat in the next line item)

  4. An excerpt from OpenVMS 8.3 System Manager's Manual, Volume 2 about pages and pagelets

    Pages and Pagelets

    On VAX systems, the operating system allocates and deallocates memory for processes in units called pages. A page on a VAX system is 512 bytes. Some system parameter values are allocated in units of pages.

    On Alpha and I64 systems, some system parameter values are allocated in units of pages, while others are allocated in units of pagelets. Both Alpha and I64 support a variety of page sizes. The OpenVMS operating system currently uses 8KB (8192 bytes) pages on Alpha and I64 systems. A pagelet is a 512-byte unit of memory. One Alpha or I64 pagelet is the same size as one VAX page. On an Alpha or I64 system with a page size of 8KB, 16 pagelets equal one page. When reviewing parameter values, especially those parameters related to memory management, be sure to note the units required for each parameter. Showing Parameter Values with SYSMAN and Showing Parameter Values with SYSGEN explain how to show parameter values and their units of allocation.

  5. It doesn't matter if you start a process using SYSUAF as a template or run SYS$SYSTEM:LOGINOUT with your own parameters, the sysgen parameter WSMAX will place an upper limit on the maximum amount of RAM (physical memory) every process in the system.

  6. If you add more physical memory to an OpenVMS system, your system will not see/use any of it until you first invoke the tuning script:

            @sys$update:autogen { more parameters expected on this line }
     
    Caveat:
    do not execute this script unless you know what you are doing -OR- can experiment with a non-production platform. You might think you can run this program without hurting your system but doing so might prevent your system from rebooting at some future time. More information about autogen can be found below.

Process Creation

VMS processes are created in two primary ways.
  1. Logging in with a username + password. A program called sys$system:loginout.exe will look up your authorized parameters in SYSUAF (system User Authorization File) then use them to create your process.

  2. Detached processes can be created without referencing the UAF and might be started like this real-world example (parameters referenced here were defined elsewhere in the script)
    $ RUN SYS$SYSTEM:LOGINOUT -
    /PROCESS_NAME=FTP_LISTENER -
    /NOACCOUNTING -
    /NOAUTHORIZE -
    /DETACHED -
    /INPUT=TCPWARE:FTP_LISTENER.COM -
    /OUTPUT='LOGFILE' -
    /ERROR=NLA0: -
    /PRIORITY=5 -
    /PRIVILEGES=(NOSAME,ACNT,ALTPRI,BYPASS,DETACH,GROUP,LOG_IO,NETMBX,-
    SYSPRV,SHARE,TMPMBX,WORLD,EXQUOTA,PRMMBX,OPER,CMKRNL,-
    CMEXEC,SYSNAM,OPER,PHY_IO,SETPRV,PSWAPM) -
    /UIC=[1,3] -
    /AST_LIMIT='ASTLM' -
    /BUFFER_LIMIT='BYTLM' -
    /ENQUEUE_LIMIT='ENQLM' -
    /EXTENT='WSEXTENT' -
    /FILE_LIMIT='FILLM' -
    /IO_BUFFERED='BIOLM' -
    /IO_DIRECT='DIOLM' -
    /MAXIMUM_WORKING_SET='WSQUOTA' -
    /PAGE_FILE='PAGEFILE' -
    /QUEUE_LIMIT='TQELM' -
    /RESOURCE_WAIT -
    /SUBPROCESS_LIMIT='PRCLM' -
    /WORKING_SET='WSLIMIT'
    $!

System User Authorization File (SYSUAF)

Here is a sample UAF entry for my non-priv account:
$ set def sys$system
$ run authorize
UAF>show ns_rieck

Username: NS_RIECK                         Owner:  NSR_N123119_TEK
Account:  ADMCSM                           UIC:    [346,305] ([NS_RIECK])
CLI:      DCL                              Tables: 
Default:  CSMIS$USER3:[ADMCSM.NS_RIECK]
LGICMD:   CSMIS$COM:LOGIN.COM
Flags:    DisPwdDic DisPwdHis
Primary days:   Mon Tue Wed Thu Fri        
Secondary days:                     Sat Sun
Primary   000000000011111111112222  Secondary 000000000011111111112222
Day Hours 012345678901234567890123  Day Hours 012345678901234567890123
Network:  ##### Full access ######            ##### Full access ######
Batch:    ##### Full access ######            ##### Full access ######
Local:    -----  No access  ------            -----  No access  ------
Dialup:   -----  No access  ------            -----  No access  ------
Remote:   -----  No access  ------            -----  No access  ------
Expiration:            (none)    Pwdminimum:  8   Login Fails:     0
Pwdlifetime:           (none)    Pwdchange:   3-OCT-2003 16:03 
Last Login:  9-DEC-2004 17:45 (interactive), 30-SEP-2002 13:55 (non-interactive)
Maxjobs:         0  Fillm:       300  Bytlm:        99000
Maxacctjobs:     0  Shrfillm:      0  Pbytlm:           0
Maxdetach:       0  BIOlm:       100  JTquota:       8192
Prclm:          10  DIOlm:       100  WSdef:         2000
Prio:            4  ASTlm:       100  WSquo:         4000
Queprio:         0  TQElm:        10  WSextent:     16000
CPU:        (none)  Enqlm:       400  Pgflquo:      99000
Authorized Privileges: 
  GROUP        NETMBX       TMPMBX       WORLD
Default Privileges: 
  GROUP        NETMBX       TMPMBX       WORLD
UAF> 

Notes for some UAF Parameters:

A few of our Alpha BASIC programs consist of more than 60k lines so the memory allotment for developers must be higher on our development system like so:
$ set def sys$system
$ run authorize
UAF>show neil

Username: NEIL                             Owner:  NSR_N123119_ADM
Account:  ADMCSM                           UIC:    [346,1] ([NEIL])
CLI:      DCL                              Tables: 
Default:  CSMIS$USER3:[ADMCSM.NEIL]
LGICMD:   CSMIS$ADM:LOGIN.COM
Flags:  DisPwdDic DisPwdHis
Primary days:   Mon Tue Wed Thu Fri        
Secondary days:                     Sat Sun
No access restrictions
Expiration:            (none)    Pwdminimum:  8   Login Fails:     0
Pwdlifetime:           (none)    Pwdchange:   4-DEC-2006 15:02 
Last Login: 19-JUN-2010 19:03 (interactive), 18-JUN-2010 23:05 (non-interactive)
Maxjobs:         0  Fillm:       300  Bytlm:      2000000
Maxacctjobs:     0  Shrfillm:      0  Pbytlm:           0
Maxdetach:       0  BIOlm:      1024  JTquota:       8192
Prclm:          20  DIOlm:      1024  WSdef:         2000
Prio:            4  ASTlm:       100  WSquo:        20000
Queprio:         0  TQElm:        10  WSextent:    200000
CPU:        (none)  Enqlm:      2000  Pgflquo:    2000000 <-- yikes
Authorized Privileges: 
  ACNT         ALLSPOOL     ALTPRI       AUDIT        BUGCHK       BYPASS
  CMEXEC       CMKRNL       DIAGNOSE     DOWNGRADE    EXQUOTA      GROUP
  GRPNAM       GRPPRV       IMPERSONATE  IMPORT       LOG_IO       MOUNT
  NETMBX       OPER         PFNMAP       PHY_IO       PRMCEB       PRMGBL
  PRMMBX       PSWAPM       READALL      SECURITY     SETPRV       SHARE
  SHMEM        SYSGBL       SYSLCK       SYSNAM       SYSPRV       TMPMBX
  UPGRADE      VOLPRO       WORLD
Default Privileges: 
  ACNT         ALLSPOOL     ALTPRI       AUDIT        BUGCHK       BYPASS
  CMEXEC       CMKRNL       DIAGNOSE     DOWNGRADE    EXQUOTA      GROUP
  GRPNAM       GRPPRV       IMPERSONATE  IMPORT       LOG_IO       MOUNT
  NETMBX       OPER         PFNMAP       PHY_IO       PRMCEB       PRMGBL
  PRMMBX       PSWAPM       READALL      SECURITY     SETPRV       SHARE
  SHMEM        SYSGBL       SYSLCK       SYSNAM       SYSPRV       TMPMBX
  UPGRADE      VOLPRO       WORLD
Identifier                         Value           Attributes
  DFU_ALLPRIV                      %X8001001D      
UAF>

SYSGEN (System Generation)

$ def/user sys$output neil.txt ---+-- necessary to see parameters that will scroll off the page
$ mcr sysgen                      |
SYSGEN>  SHO /MAJOR               |
SYSGEN>  SHO AW                   |
SYSGEN>  EXIT                     |
$ type neil.txt ------------------+
Parameters in use: Active
Parameter Name            Current    Default    Min.         Max. Unit       Dynamic
--------------            -------    -------   -------    ------- ----       -------
GBLSECTIONS                   850        250        80      65535 Sections   
MAXPROCESSCNT                 346         32        12      16384 Processes  
MULTITHREAD                     2          1         0        256 KThreads   
SMP_CPUS                       -1         -1         0         -1 CPU bitmas 
SYSMWCNT                     4003       2048       512    1048576 Pagelets   
 internal value               251        128        32      65536 Pages      
BALSETCNT                     344         30         8      16384 Slots      
WSMAX                      100000       4096      1024    8388608 Pagelets   
 internal value              6250        256        64     524288 Pages      
NPAGEDYN                 13524992    1048576    163840         -1 Bytes      
PAGEDYN                   3416064     524288     65536         -1 Bytes      
FILE_CACHE                      0          0         0        100 Percent    
S2_SIZE                         0          0         0         -1 MBytes     
FREELIM                       364         32        16         -1 Pages      
LOCKIDTBL                    4455       3840      1792   16776959 Entries    
RESHASHTBL                   4096         64         1   16777216 Entries    
PFCDEFAULT                     64         64         0       2032 Pagelets   D
 internal value                 4          4         0        127 Pages      D
GBLPAGES                  1787424      65536     10240         -1 Pagelets   D
 internal value            111714       4096       640         -1 Pages      D
QUANTUM                        20         20         2      32767 10Ms       D
PFRATL                          0          0         0         -1 Flts/10Sec D
PFRATH                          8          8         0         -1 Flts/10Sec D
WSINC                        2400       2400         0         -1 Pagelets   D
 internal value               150        150         0         -1 Pages      D
WSDEC                        4000       4000         0         -1 Pagelets   D
 internal value               250        250         0         -1 Pages      D
FREEGOAL                     1376        200        16         -1 Pages      D
GROWLIM                       364         63         0         -1 Pages      D
BORROWLIM                     364        300         0         -1 Pages      D
IO_PREFER_CPUS                 -1         -1         0         -1 CPU bitmas D
LCKMGR_MODE                     0          0         0        255 CPU Count  D
LCKMGR_CPUID                    0          0         0         -1 CPU Id     D
AWSMIN                        512        512         0         -1 Pagelets   D
 internal value                32         32         0         -1 Pages      D
AWSTIME                        20         20         1         -1 10Ms       D
$ 

A very simple overview:

  1. When a process starts, it is initially allowed WSDEF pages of physical memory (RAM) and can demand more by faulting up to WSQUO (this amount is guaranteed if AUTOGEN was done properly "and" no one was making manual changes to the SYSGEN parameters). If the system is lightly loaded a heavily faulting process may be allowed to borrow more pages up to WSEXTENT.
     
    1. SYSUAF values are only read when a process starts. If you change them the user will need to log off then back in for them to take effect
       
    2. WSMAX places an upper limit on process growth. Setting WSDEF or WSEXT larger than WSMAX will have no effect.
       
  2. How does OpenVMS know when to allow borrowing? Free pages in OpenVMS are maintained in a structure known as the page cache or FREE LIST. When the system has more than GROWLIM pages, actively "faulting" processes (at a rate higher than PFRATH) can acquire more pages up to a maximum of WSEXTENT. On the flip side, when the system gets busy and the FREE LIST shrinks lower than FREELIM, the SWAPPER will begin to trim back all processes (extended processes first, then low faulting processes (lower than PFRATL) next). This can continue and may even involve trimming dormant (idle for longer than DORMANTWAIT) processes back to their working-set-defaults (WSDEF). After this the system will resort to swapping out the process entirely. The system will continue this until the FREE LIST size gets to FREEGOAL.

              $mon sys (to watch the FREE LIST grow or shrink)

    Note: GROWLIM and BORROWLIM work together to enable or disable borrowing. The borrow feature is first enabled when the system has more than BORROWLIM pages and then disabled when the system has fewer than GROWLIM pages. Together they provide a form of hysteresis to prevent the system from acting erratically. (e.g. quickly switching from borrowing to trimming). Also note that FREELIM = GROWLIM = BORROWLIM which is a common AUTOGEN setting for systems with lots of memory (e.g. 512 MB or greater)
     
  3. Program code and read-only data (constants) reside on the FREE LIST while program variable data reside on a smaller structure known as the MODIFIED LIST. This so-called dirty page cache will need to be written to the PAGEFILE before these pages are given to another process.
     
  4. When a processes needs more pages but is denied, it can get more by giving up some existing pages back to the FREE LIST. If it needs the page later and the desired page is on the FREE LIST, it can get it back by a process of "soft faulting". However, if the page has been given to someone else, the process will need to "hard fault" the data from disk.
     
  5. Busy systems may want to enable Automatic Working Set Adjustment (AWSA) in both directions (expansion and contraction). The value of PFRATL has been defaulted to zero for many years now but can be enabled to ensure that more free memory is usually available (at the expense of more faulting).
     
  6. SYSMWCNT sets the work set size of the system. If this value is too large, VMS will use too much physical memory; if this value is too small, user processes will be given too much memory which means the OS will become somewhat crippled.
     
  7. You should really read a tuning-guide before you mess with these parameters. That said, if you must hack then check out the following online help:
$mcr sysgen
SYSGEN>help sys freelim
            sys freegoal
            sys growlim
            sys borrowlim
            sys freelim
            sys awstim
            sys awsmin
            sys wsinc
            sys wsdec
            sys pfratl
            sys pfrath

Automatic Working Set Adjustment (AWSA)

In the parameter list above, you can see...

OpenVMS Processes

OpenVMS Swapper

Pool

Retuning The System (via AUTOGEN)

Introduction

When To Retune How To Retune (overview) How To Retune (details)

Manually Retuning The System (via SYSGEN)

Overview and Warnings
  1. Warning: manual tuning without AUTOGEN is for experts only. I have witnessed two occasions where someone (not me) made system parameter changes to OpenVMS which resulted in the system being unbootable. The only way to fix this is to do a "conversational boot" of VMS which drops the console into SYSGEN on the way up. At this point the operator needs to make manual SYSGEN changes to make things right before continuing with the boot process which happens as soon as you exit SYSGEN.
     
  2. Warning: the safest tuning method is to place your SYSGEN commands in file "sys$system:modparams.dat" then invoke the AUTOGEN script. This will ensure that dangerous settings and/or parameter conflicts are written to the AUTOGEN report file titled "sys$system:agen$params.report"
     
  3. That said, there are some situations where it will not be convenient of possible to reboot the system. After all, this isn't a PC operating system.
Manual Tuning
$ mcr sysgen
SYSGEN>  SH PFRATL
Parameter Name           Current Default    Min.    Max. Unit       Dynamic
--------------           ------- ------- ------- ------- ----       -------
PFRATL                         0       0       0      -1 Flts/10Sec D
SYSGEN>  SET PFRATL   2
SYSGEN>  SH PFRATL     
Parameter Name           Current Default    Min.    Max. Unit       Dynamic
--------------           ------- ------- ------- ------- ----       -------
PFRATL                         2       0       0      -1 Flts/10Sec D
SYSGEN>  WRITE ACTIVE    (writes dynamic parameters to the active system in memory)
SYSGEN>  WRITE CURRENT   (writes all parameters to the current system parameter file on disk)
SYSGEN>  EXIT
$

An example MODPARAMS.DAT from one small system with almost no users 

!
! SYS$SYSDEVICE:[SYS0.SYSEXE]MODPARAMS.DAT
! Created during installation of OpenVMS AXP V7.2-1  8-JUL-1999 12:38:47.03
!
VAXCLUSTER=0             !
SCSNODE="KAWC99"         ! our webserver machine
SCSSYSTEMID=15335        ! 14.999 (14 x 1024 +  999 = 15335)
AUTO_DLIGHT_SAV = 1      !
!
maxprocesscnt   = 64     !
balsetcnt       = 62     ! all our systems MUST HAVE room for at least 40 processes
!
wsmax           = 200000 !
tty_buf         = 132    !
TTY_TYPAHDSZ    = 4096   ! was 78
TTY_ALTYPAHD    = 4096   ! was 200
TTY_ALTALARM    = 2000   ! was 64
! TTY_DEFCHAR2  = 528386 x enable system password on terminals
TTY_DEFCHAR2    = 4098   !
Notes:
  1. MAXPROCESSCNT must be at least 2 units higher than BALSETCNT
  2. BALSETCNT determines how many processes will be resident in RAM at one time. If MAXPROCESSCNT was 99 and BALSETCNT was 33, then as many as 99-33-2=64 processes "could" be swapped out (waiting for a turn from the processor). You can reduce one form of swapping by making sure that MAXPROCESSCNT is 2 above BALSETCNT
  3. Autogen will tune this system to ensure that 64 processes have 200,000 virtual memory pagelets. It will do this by making sure the PAGEFILE is large enough. Remember that individual account settings in UAF may be set up to use much less. This way, some larger processes will now be able to borrow past their WSQUOTA (provided WSEXTENT and Pgflquo are set high enough)

Links:


Back to OpenVMS
Back to Home
Neil Rieck
Kitchener - Waterloo - Cambridge, Ontario, Canada.