OpenVMS Demos: Programmer's Corner
This entire sub-domain is a private effort of free information.
There are no cookies, no advertisements, and nothing is for sale.
non-code OpenVMS resources can be found here
- The information presented here is intended for educational use by qualified OpenVMS programmers.
- The information presented here is provided free of charge, as-is, with no warranty of any kind.
- To the best of my knowledge, the programs presented on this web site compile, link and run properly. Please send me an email if they do not (I only support this stuff in my spare time).
- Is this text too small? You have two options:
- hold down the CTRL key while rolling the mouse wheel (zoom-in, zoom-out)
- use your keyboard like so:
- hit: CTRL with "-" key to zoom smaller
- hit: CTRL with "+" key to zoom larger
- hit: CTRL with zero key to reset zoom
- SPECIAL ALERT: this area went through a major reorganization on 2015-11-07 with many of the 200+ sample programs being more-sensibly renamed
- If something has gone missing then send me an email
- Meanwhile, click here for a directory listing
OpenVMS Source Code (presented in HTML format)Presented in HTML format so:
- Google can index them
- you can search them with "Google"
- you can search then with "Google site-search" like so: site:www3.sympatico.ca/n.rieck/ lib$spawn
- you can display them with a browser (tab characters render properly with Firefox, Chrome, and newer versions of IE)
- you can
drag a cursor over source text then copy-paste into an EDT editor (provided you are using a good terminal emulator)
How my source code pages were webified
My demo programs were converted to HTML pages by running my source code through this VMS-BASIC-based utility:
What does it do? For the most part, it converts certain HTML-sensitive characters into HTML entities (meta data) like so:
|Original Character||HTML Entities|
The utility also inserts <pre> before my first line and appends </pre> after my last line in order to preserve spaces and tabs. Optionally, this utility can build a complete web page which saves me lots of time.
General Info Link: OpenVMS Notes: VMS-BASIC (coding techniques)
File I/O Demos
RMS :: Record Management Services for OpenVMS (ISAM)
- shows how to manipulate sequential RMS files (plain green screen)
- shows how to manipulate relative RMS files (plain green screen)
- shows how to manipulate indexed RMS files (plain green screen)
- shows how to manipulate indexed RMS files (FMS green screen)
- you will need this FMS form: rms_indexed_demo_fms.zip
- you will need this FMS include: fms_fdvdef.inc
- you will need this other include: device_controls.inc
- FMS (Forms Management System) Notes:
- during the VAX/VMS era, developers were encouraged to insert their various FMS forms (name1.frm) into a developer-created "forms library" (name.flb). Just after image activation, your program would call FDV$LOPEN to open the library, then FDV$LOAD to fetch the forms into various workspace buffers, then FDV$LCLOS to close the library channel. In those days I went to great lengths to reduce the cost of image activations on our underpowered while overworked VAXs. One solution was to bypass the form library by creating a binary version of each form then linking them to the executable. This demo shows you how to do this.
- There are ~ 66 FMS functions and you can read more about them by downloading manuals from here:
- Many FMS-based solutions were migrated to a newer API called DECForms
- a little hacking with the FSP$ function (originally provided to do trial opens on BASIC-PLUS-2 programs)
- do a trial file-open using a USEROPEN function
- an intermediate version includes a hack to allow inclusion of XABSUM and XABKEY (this last one is for inspecting keys a.k.a. indexes)
- recently added code to include XAMPRO (to inspect protection bits)
- this program works in two modes:
- simply tack my pointer onto the existing FAB:XAB pointer (overwriting what was their previously)
- do my thing
- restore original pointer before exiting useropen
- start at FAB:XAB then scan the linked list until you hit a zero. Then use a home-grown POKE routine to tack my list onto the end of the existing list
- do my thing
- restore original pointer before exiting useropen
- doing file i/o in BASIC without using BASIC's file i/o routines
- directly reading SYSUAF records for audit purposes only (you need lots of privs just to open SYSUAF.DAT)
RDB :: Oracle-Rdb (Relational Data Base)
MariaDB :: An alternate fork of MySQL
- Using BASIC to call a MySQL or MariaDB client software written in C
- VMS-BASIC has built-in commands (eg. OPEN, READ, WRITE, UPDATE, CLOSE) to manipulate RMS files; this demo will show how to do the same thing with MariaDB (or MySQL)
Tools: RMS Export - MySQL Import
- our current system consists of 70+ BASIC programs reading/writing 150+ RMS indexed files. In order for us to do a parallel (RMS=production, MySQL=daily snapshot) operation over the next 12 months, we needed a semi-autonomous method of tool generation. These four BASIC programs do the following:
- generate 99% complete BASIC programs to extract data from RMS files then write it to CSV files
- you need to edit one line before compiling (the input file name)
- generic open/read of your RMS files (bypasses indexes/keys)
- this tool is preferable to opening up your source code to insert customized export routines
- once built, there is no need to do this again unless the RMS file format changes; which means these can be called from batch
- generate 99% complete SQL scripts to create the tables then import the data from CSV files
- since my tool does not (yet) analyze the OPEN statements, you will need to insert SQL statements to create constraints, indexes and keys, etc.
- CAVEAT: you could modify the table creation statements before the import but this would cause the import to be hundreds of times slower (lots of recursion while it maintains each index AND you could still end up with a lop sided index if the inserted data was already ordered). All large imports should always be indexless and constraint-less; you tack them on as a finishing step.
- I placed all my post-import SQL statements in a second script which won't be overwritten by the tool
I/P Stack Demos (this code is written in BASIC; check out my C/C++ demos farther down this page)
TCPIP Services for OpenVMS
- a third-party stack from Process Software (supports IPv4 and IPv6)
- more feature-rich than TCPIP Services for OpenVMS from HP
- No specific demos yet since my "TCPIP Services" demos already work with this stack
- a third-party stack from Process Software (only supports IPv4)
- this was the only IP Stack for VMS in the 1980s and first half of the 1990s
- the Telnet API and FTP API provide a solution midway between Sockets and System Services
- Employs the TELNET API to demo CHARGEN, ECHO, HTTP, and TELNET. Also contains a minimal NVT handshake necessary for TELNETing to UNIX systems (Solaris will not let you connect unless you can provide a TERM TYPE via IAC WILL-WONT-DO-DONT; makes me wonder if this isn't some sort of simplistic anti-hacking effort)
- Employs the TELNET API to demo connecting to a POP3 server to retrieve email
- Employs the TELNET API to demo connecting to an SMTP server to send email
- Employs the FTP API to demo doing FTP from within a BASIC executable
Apache Glue Example
- drop this DCL script into any one of your apache user-accessible scripts directories (I use "[.scripts]")
- rename the file to match the URL you wish to use. For example, if the link is "/demo123" then rename the file to "demo123.com"
- modify the script to run this next program called "read_html_apache.exe". It will read/decode the received HTML page then create DCL symbols representing the contents of HTML objects on the web page.
- if the program ran properly, now run your desired program which will read the DCL symbols then do whatever you want
- use a browser to test your application like so:
- Have the script above run this program which will:
- read data sent by the browser to the web server
- decode the HTML page
- create DCL symbols representing HTML data contents of HTML objects on the web page
Apache No-Glue Example
- This stand-alone application does not require any glue (DCL script). You could do one of the following:
- create a one-line DCL script which does nothing other than run your executable program
- drop the executable program directly into the Apache scripts directory
Password Change Demo
- CGI Glue
- can be used standalone (does not use CGI glue)
- can be used not standalone (see Apache Glued Example above)
- can hook into an external database (RMS in my case) to validate users
- be sure to see my SYSUAF hack on this same page
- coming soon
- Got this working in production but it needs to be cleaned up before publishing
- update: it is so ugly that you might be better off using this C function which can be called from BASIC
- coming soon
- Since all my SOAP services are written in C (via gSOAP) or Java (via AXIS or AXIS2 running on Tomcat), I may publish a little BASIC program used to send SOAP test cases (messages) to a published SOAP service
- coming soon
- AJAX can send back ANY text document. Plain, XML, HTML, whatever.
- Back in the day, I wrote many VMS BASIC applications which employed FMS (form management system) forms on character cell terminals like so:
|Desired Action||BASIC Code|
|display a form||call fdv$disp( fname$ )|
|populate a form (field-by-field)||call fdv$put( last_name$, "f$last_name" )|
|populate a form (all at once)||call fdv$putal( whole_form$ )|
|read a form (field-by-field)||call fdv$get( last_name$, terminator%, "f$last_name" )|
- I recently moved a BASIC application from FMS on VT320 to HTML/JAXA on browser by first creating a static web page which contains an HTML version of my FMS form (the FMS field name became the "id" of the HTML input object). My app first loads the static web page then issues (based upon user input) AJAX transactions requesting tagged data. Some of my scripts process the tagged data then:
populate INPUT objects via:
document.getElementById(fn).value = fd
or populate DIV objects via:
document.getElementById(fn).innerHTML = fd
fn = field name
fd = field data
Data conversion / manipulation
- programmers in 2016 are still interchanging UTF-8 with Unicode and this is causing huge problems
- UTF-8 is a Unicode encoding (so talking about 2-byte (or 16-bit) Unicode has nothing to do with talking about 2-octet UTF-8)
- Microsoft states that all written languages can be represented by 16-bit-Unicode but this will require 3-octet UTF-8
- Both MySQL and MariaDB will set aside 24-bits if you declare the data to be UTF8 (but this will change if you store binary)
- Both MySQL and MariaDB will set aside 32-bits if you declare the data to be UTF8mb4 (but this will change if you store binary)
- programmers in 2016 are still interchanging iso-8859-1 with cp1252 (a.k.a. Windows-1252)
- cp1252 is a 32 byte superset of iso-8859-1 (but browsers treat everything as cp1252)
- Unicode and UTF-8
- this external BASIC function assumes the input data contains a mixture of UTF-8 and CP1252 (which is a superset of ISO-8859-1)
- comment: this should not happen but I have seen UTF-8 mixed with cp-1252 coming from Microsoft database engines like SQL Server
- anything between x00 and x7F is converted to 32-bit Unicode as-is (obviously)
- anything between x80 and xFF is assumed to be a mixture of UTF-8 and CP1252
- everything which appears to be a valid UTF-8 sequence is converted to 32-bit Unicode
- any remaining stuff between x80 and x9F is assumed to be CP1252 which is mapped back to equivalent 32-bit Unicode code points
- any remaining stuff is assumed to be ISO-8859-1 and so is converted to 32-bit Unicode
- this external BASIC function converts from 32-bit Unicode to UTF-8
- this demo program shows how to use the functions just described
- this external function is misnamed. It really should have been called utf8_to_cp1252.fun
- converts 2-octect UTF-8 to a single 8-bit character then (optionally) maps the result to 7-bit characters
- this older program incorrectly uses the phrases
- ISO-8859-1 instead of CP-1252 (a.k.a. Windows-1252)
- Unicode instead of UTF-8
- this older program was developed in the middle of a Microsoft data ecosystem and incorrectly assumes we will only see 2-byte UTF-8. If you want to represent all European characters (both East and West) then you will need to support 3-byte UTF-8. It you have any contact with Asian supply chains (think Alibaba) then you will need to support 4-byte UTF-8
- so in brief: DO NOT USE THIS MISNAMED PROGRAM
- calling VMS sort routines (hey, sometimes it is best to leave it to the experts)
- allows a BASIC program to read DCL command-line parameters like "C" on UNIX
Diffie-Hellman Key Exchange Demo
- Diffie-Hellman Key Exchange Demo (limited to 64-bit math)
- note: this happens at the start of every encrypted data transmission on the net
Displaying System errors
- can translate any system error number into the associated text message. This program is no longer required since modern OpenVMS versions can do this for you from the DCL prompt.
Lock Demos (CEF :: Common Event Flag)
Lock Demos (DLM :: Distributed Lock Manager)
- creates a null lock then synchronously converts it to more restrictive forms
- creates a null lock then synchronously converts it to more restrictive forms
- asynchronously requests an exclusive lock then polls DLM to see if it was granted
- asynchronously requests an exclusive lock, then arms a timer, then waits for one of two event flags
Linked Lists and Memory Queues
- demos: lib$insqhi, lib$insqti, lib$remqhi, lib$remqti but these is better done in C/C++
Mailbox Demo (a.k.a. pipe)
- client program to send DCL requests to a server program via a VMS mailbox; note: a VMS mailbox is similar to, but not identical to, a UNIX PIPE
- server program to receive DCL requests from a client program (via a VMS mailbox)
- "support" scripts to start the programs above
When you need larger DCL symbols
- Some apps like Apache save session data (like HTTP_COOKIE) as a DCL symbol. When the data is greater than 930 bytes, Apache will save the data as multi-line logical names. This program demos how you would do something similar from BASIC
- all operating systems support shell variables known as symbols
- VMS (and OpenVMS) also support a different data-passing method known as logical names
- programs wishing to create or modify logical names will need to call lib$set_logical or sys$crelnm
Caveat: lib$spawn will allow you to make changes on copied data which is discarded upon exit of that spawned task
- While everyone knows how to create single line logical names from BASIC, multi-line logical names are a little more difficult
- Unlike most other BASIC implementations, VMS BASIC doesn't have a PEEK function but sometimes you need it
- see how variables are stored in an OpenVMS system
- see how the Alpha compiler is able to achieve such high levels of performance relative to VAX
- see how fixed-point decimals are stored in OpenVMS and RMS (no difference by the way)
system calls you can't live without
- demos how to properly call sys$getjpi (get job process information)
- demos how a program can do the equivalent of the DCL command "$show default"
- comment: anyone coming from RSX-11 or CP/M will remember that using SET without any parameters is the same as SHOW
- calling sys$qio from BASIC allows more control than PRINT and INPUT
Source Code Tools
- generate "code change" reports to justify your existence
- This horribly written quick-hack has proved invaluable as a tool for locating coding problems introduced by other authors. Anyone who has inherited source code from another organization knows what I am typing about.
- Caveat: I've been meaning to rewrite this tool for a decade now but can never find the time. But this tool can format itself so maybe it's good enough for occasional use
- When run from a DCL script, this program goes great lengths to ensure that your process, and only your process, gets the next number in a numerical sequence. I use this program so that an intercepted email account can temporarily access one of twenty subdirectories before I call MUNPACK to extract MIME attachments (if any) from cell phone messages. Of course, I need to run a different program to read the plain-text email if nothing was extracted
- my own watchdog process (demos lots of systems calls)
OpenVMS BASIC standalone functions (file extension ".fun")
Some BASIC demos may require these standalone (most BASIC) functions:
- compute Easter or Good Friday for any date after 1582. Necessary for writing an annual holiday generator.
- returns the current date-time in ccyymmddhhmmss format
- mime time function for smtp use
- returns the value of a desired DCL symbol
- returns the 64-bit data work found at the specified 32-bit address
- this function will attempt to save your data as a DCL symbol. If it can't (usually because the data is too large for your machine) then it will shift modes and save your data as multi-line logical names (emulates functionality found in the CGI interface of Apache for OpenVMS)
- submitting to batch from within BASIC
- translates a logical name
caveat: we used this function in the VAX days because it was much more efficient than calling lib$get_logical and easier to set up than sys$trnlnm. But there seems to be no performance hit when using lib$get_logical on Alpha so this function is only included here incase I've included a few demos still calling it.
- undo so-called percent encoding (as well as some other stuff)
Question: So just what does the prefix WCSM stand for?
Answer: Waterloo Computer System Maintenance. That department morphed into something greater but the function prefix remains
OpenVMS BASIC includes (file extension ".inc")
Some BASIC demos may require these standalone includes:
- data structures required for item lists in system calls, etc.
- system declarations required to do system calls from BASIC (should not be used for new code development)
- escape sequences required to embed VT terminal and LA printer inside your programs. (written for use on small VAX systems where it was not practical to control devices by system calls to the terminal driver)
- FMS (forms management system) definitions: functions and constants
- TCPware control block definitions
My BASIC Development Environment
Here is a snapshot of my development environment. It will help you understand my includes
|basic source code|
DCL scripts for building some applications
|FMS forms (created by dcl command: $fms yada.frm)|
FMS form libraries
|mapped string declarations (records) for relative and indexed files|
open statements for relative and indexed files
file description language files for tuning rms files
BASIC Compiler bugs
- I just (2012-08-01) discovered a bug in HP Alpha BASIC-1.7-000 (friends tell me it also exists on Itanium)
- Although you can divide large QUADWORD integers, the modulus function fails
- Here is a "C" version which works properly: basic_quad_mod_bug.c
- I just (2012-09-14) tested an experimental version of HP Alpha BASIC 1.7-001 which fixes this problem.
|1.7-000||test fails at 2^32||bug|
|1.7-001||test works up to 2^62||fixed|
- CAVEAT: HP told me they will not be releasing this fix to the world so "if you have a support contract and want it fixed then you must input a support request"
- I just (2012-09-01) discovered a bug in HP Alpha BASIC-1.7-000 (friends tell me it also exists on Itanium)
- Although you can divide large DECIMAL integers, the modulus function fails
- I just (2012-09-14) tested an experimental version of HP Alpha BASIC 1.7-001 which fixes this problem.
|1.7-000||decimal(16,0)||test fails at 2^32||bug|
|1.7-001||decimal(16,0)||test works up to 2^53||fixed|
| ||decimal(17,0)||test works up to 2^56 with compile warnings||safer to use quad|
| ||decimal(18,0)||test works up to 2^59 with compile warnings||safer to use quad|
| ||decimal(19,0)||test works up to 2^62 with compile warnings||safer to use quad|
- CAVEAT: HP told me they will not be releasing this fix to the world so "if you have a support contract and want it fixed then you must input a support request"
BASIC RTL (run-time library) bug
- This affects OpenVMS-8.4 ECO 900 (a.k.a. consolidated patch kit 900 and earlier)
- In BASIC, you can use the open statement to create an indexed file with one or more keys.
- If any one of the keys is a QUAD (signed 64-bit integer) then any attempt to reopen the file will fail
- I placed a support request with HP (OpenVMS Engineering) who provided me with an experimental BASRTL 10-days later.
- Not sure if this will be released to the public so if you need it, PLACE A SUPPORT REQUEST with HP.
p.s. I was attempting to write code which would directly open SYSUAF which contains a key (er, index) based upon an UNSIGNED QUAD. Since VMS-BASIC has no unsigned data types an updated BASRTL would be of no use. Even still, I am surprised to learn that this bug was hanging around since 2005 when HP added the QUAD data type to the language (up until then you needed to use BASIC$QUADWORD defined in the STARLET library)
Hybrid Programming (BASIC calling C)
Some BASIC solutions are found anywhere between impractical to impossible (LDAP springs to mind). In many cases you could "rewrite C programs into BASIC" or "could create a fancy BASIC-to-C wrapper" or "you could just call the C program directly". To do this you need to know how to pass variables back and forth between the two without resorting to "shell variables" or "logical names".
- Hybrid Demo 1
- this BASIC program basic_calling_c_demo1_part1.bas calls C functions defined here: basic_calling_c_demo1_part2.c
- demonstrates passing variables (long, word, byte, and VMS string) to C; one function returns a result while one does not
- also demonstrates how C can directly access global variables declared in BASIC (ugh!)
- caveat: the string can only be accessed as read only; see demo-4 below to learn how to properly modify the BASIC string from C
- Hybrid Demo 2
- Hybrid Demo 3
- Hybrid Demo 4
- this BASIC program basic_calling_c_demo4_part1.bas calls a C function defined here: basic_calling_c_demo4_part2.c
- a variable length VMS string is passed from BASIC to C where the string is extended then sent back
- teaser: new string storage space must be allocated via str$get1_dx rather than malloc
- p.s. write a small BASIC program which only manipulates a few strings then compile it with the "/list/machine" command-line parameters. For the most part the BASIC program will be making calls to the STR$ run-time library
- Hybrid Demo 5
VAX-C / DEC-C / VMS-C
DEC-C++ / VMS-C++
GeneralCaveats for developers:
- both C and C++ languages "appear" to be more complicated on OpenVMS than what you will see on UNIX, Linux or Windows so be sure to read OpenVMS specific documentation.
- C on OpenVMS ( http://h41379.www4.hpe.com/commercial/c/c_index.html )
- C++ on OpenVMS ( http://h41379.www4.hpe.com/commercial/cplus/cplus_index.html )
- HP-C Run-Time-Library Reference Manual For OpenVMS which is used by both languages ( http://h41379.www4.hpe.com/doc/732final/5763/5763pro_001.html )
- tells you where your reference libraries are located
- compares UNIX I/O to RMS I/O
- introduces optional library directives which will allow UNIX-dependent c programs to run on OpenVMS
eg. automatically translate a full filename specification from "//one/two/three/four.txt" to "[one.two.three]four.txt"
- One definition of Sensei (a Black Belt in Karate) is "one who has gone before". Many times you will attempt to solve a problem which has already been solved by someone else long ago but you didn't know where to look. Some of those solutions can be seen here:
- www.decuslib.com/freeware/freewarev40/srh_examples/ which includes lots of C sources like:
- www.decuslib.com/freeware/freewarev50/srh_examples/ which includes lots of C sources like:
- When calling OpenVMS routines defined in the STARLET library, do yourself a favor and enable strict type-checking. This is done by inserting ...
#define __NEW_STARLET 1
... before the library references you wish to be strict (for new programs it should be before all libraries). This way, your code will be in better shape to be ported to another OpenVMS hardware platform. In one of the programs below (sys_create_uid_101.c) I was forced to add a few more lines (resorted to a union) to satisfy the compiler but the optimizer was able to generate a program which was byte-for-byte the same size as an earlier relaxed version.
- Some large projects (especially code originally written for UNIX/Linux) may require you to build your program with a so-called "make utility". MMK is the name of the OpenVMS utility used by many developers and can be downloaded here:
p.s. many old-school OpenVMS programmers prefer to invoke the complier and linker from a DCL prompt. When their link commands become too long they will often move their link commands into a DCL build script. While there is nothing wrong with this, older applications like "$LINK" do not support DCL symbol substitution after the "sys$input/opt" directive which then requires the developer to resort to temporary "logical names" or generating-then-executing temporary scripts. Even then you will have some difficulty squelching a noisy link (lots of warnings but still a successful build). All these problems can be handled by moving to a make-tool like MMK
- GNV (Gnu Not VMS) hosts a unix-like environment on OpenVMS: gnv.sourceforge.net
- Lots of open-source software has already been ported to OpenVMS: vms-ports.sourceforge.net
C Demo Programs
- changing VMS file protection bits
- reads command line arguments (and named logicals and/or DCL symbols)
- the $DESCRIPTOR MACRO is dumber than most people realize so use it with caution or NOT AT ALL
- Diffie-Hellman Key Exchange Demo (limited to 64-bit integer math)
- A very-cool LINUX socket programming to do Name Server lookups. It was not written by me but it works as-is on OpenVMS-8.4 Alpha with "TCPIP Services for OpenVMS v 5.7". It won't work from behind a firewall (without a few mods) but you'll get the basic idea.
Caveat: be sure to see RES_SEARCH_DEMO.c below which may be a bit more useful. Why? Library functions res_search() and res_query() work properly with IPv4 -and- I assume they also work properly with IPv6
- convert between EBCDIC and ASCII character sets
- demos use of getuai (get user authorization information)
- GUID - UID
- Hello World
- HTTP Related
- This little test file came from OpenVMS Engineering (I kept it in homage to them) when we reported a 100k problem with CSWS 2.0 (Apache for OpenVMS). Drop this file into the scripts directory of your Apache server then trigger it from a browser like so:
It will help you test your Apache server as well as assist in confidence tests with your network admin people.
- HTTP-based CGI File Upload
- this is the webpage used to start the file-picker
- http_file_upload_31.c (broken when uploading certain binary files; see caveat below)
- this is a c version of my BASIC code (it can be built as either "a standalone executable program" or "a function which can be called from any so-called DEC language including BASIC)
- caveat: on 2016-03-xx I realized this program contains a small bug when saving uploaded binary files to OpenVMS. With regard to RMS files of the "512-byte fixed" variety, this broken program appends nulls after the last data byte in the last block then places an EOF at the end of the block. Most programmers already know that many files (with extensions like ".exe" and ".xslx" etc.) expect to see an EOF after the last data byte in the last block. I fear the only way around this is to switch from "standard-i/o functions like fprintf" to "OpenVMS functions like sys$put and sys$write". If anyone is aware of a simpler way only using c then please send me an email. Anyway, a solution is just around the corner.
- solution: I now have a working version (his is just a spare time project) but I have not gotten around to publishing it
- MariaDB - An alternate fork of MySQL (installation tips and security quirk)
- client demos
|lib_mysqludf_build_vms.com||DCL script to build lib_mysqludf_sys on OpenVMS (user defined function to do OpenVMS system calls from within Stored Procedures) on MySQL and MariaDB|
note: this script shows how to force the OpenVMS linker to produce both uppercase and lowercase symbols
| || |
- PIPE and VFORK
- playing with quads (long long) in "C"
- this program was used to verify some problems with HP Alpha BASIC 1.7-000
- As of October-2014 I think this is the only reasonably complete OpenVMS demo on the internet showing how to use res_search() and res_query() then extracting A, MX and NS records. If anyone knows a better/easier way to to extract this information on OpenVMS (perhaps using library functions I don't know about) then please send me the details.
- Many library functions are available for Linux (and I was tempted for a short while to begin porting a half dozen or so but why reinvent the wheel if others have gone before me? I certainly couldn't be the first OpenVMS application programmer standing before this chasm, right? Also, the responsibility of keeping the C/C++ libraries up-to-date squarely lays with HP
- Anyone who has viewed the official ISC source code for nslookup.c or dig.c might be wondering WTF? I'm certain they are kept remark-poor to thwart modifications by non-specialists. Meanwhile, I am certain that professional spammers already know how to extract MX records.
- This program is an annotated version of RMSEXP.C found in chapter 2 of the HP C User's Guide. Any newbie wishing to use RMS from C or C++ should first look at this program before you write a single line of code
- sndjbc - submitting a job to a batch queue (this will enable you to submit a job on behalf of another user for the express purpose of having that user send a file to another system via SFTP or SCP)
- demos function tcpip$send_from_file() which is a nifty way to send SMTP messages to your own stack without opening a connection to port #25
- sudoku1.c and sudoku2.c
- to benchmark hardware/software performance
- this little gem performs 20k iterations on a Sudoku puzzle algorithm
- my presentation of a client demo program found in vms folder sys$examples (copyright by HP)
- this program uses sys$qiow rather than Berkeley sockets and is synchronous
- a synchronous version (based on sys$qio) will be published soon
- display the current time translated to any other time zone
- does the "C" example in the UNIX Hater's Handbook actually compile? (yes)
- Old School VT Programming
- display colored text on real VT terminals (or any good emulator) without SMG$
- this file contains escape sequences used to control popular DEC peripheral devices
- note: SMG$ is one of five OpenVMS RTLs. This limited demo employs SMG$ routines to display colored text on a virtual terminal
- Accessing the OpenVMS DLM (Distributed Lock Manager)
- UTF-8 Encode + Decode
- no matter how you configure them, most popular browsers will push Unicode via UTF-8 back to your programs. This will cause a problem if your databases are configured to store single characters so you will want to convert from UTF-8 to ANSI/Windows-1252 (which is a superset of iso-8859-1). But what happens when you are unable to convert everything? Do you throw away anything over 255 or do you map to something else?
- this program can convert from ANSI/Windows-1252 back to UTF-8
- Internationalization Demos for DECC/HP-C/HP-CXX
- calls: setlocale(), iconv_open(), icon_close(), and iconv()
- internationalization_demo_100.c - demos converting from "iso-8859-1" to "ucs-4"
- internationalization_demo_101.c - demos converting from "iso-8859-1-euro" to "utf-8"
- okay for ISO-8859-1 to UTF-8; not okay for Windows-1252 (also called ANSI)
- for this reason alone, I will keep these Internationalization demos close to my UTF-8 routines
C++ Demo Programs
- This string demo is very interesting. Why?
- at first glance these C++ strings appear much more BASIC-like than those found in traditional C.
- Many traditional third generation languages, like HP-BASIC for OpenVMS, limit string size to 32,767 bytes. I have tested this C++ demo up to a string size of 1,999,000 bytes (I could have gone higher but became bored) and found that it worked equally well with:
- "HP C++ V7.3-009 for OpenVMS Alpha V8.4" (complied and linked in 32-bit mode)
- "CodeBlocks 12.11 on Windows 7"
Apache Specific (c)
Apache Module Authentication
- In 2014 we transitioned 100 users from green-screen to the web. Our web applications do their own authentication by hooking into SYSUAF then setting an encrypted session cookie to be used for the next hour:
- we record every transaction and can "log out" inactive web users by deleting their specific cookie from our "current sessions" database
- this database is also used to implement an administrative dashboard
- The negative here is that old-school access to documents stored in protected directories still require a second authentication challenge from mod_auth_openvms which cannot be checked for inactivity -AND- doesn't set a cookie. Our clients are becoming really annoyed at this.
- The success of our transition from green-screen to the web has resulted in the creation of new business opportunities resulting in ~1300 new users. However, the usual account churn has become somewhat of a burden so we would like to transition to self-serve account maintenance (think Amazon or eBay). This could done in a simple secure way if all user accounts were moved from SYSUAF to an application-level database (which is an indexed RMS file in our case). This also means that only five people (developers and sys-admins) would ever have SYSUAF accounts. Very cool!
- So I copied the password hashes of our user accounts from SYSUAF to our user profile database but still needed to keep ~100 accounts in SYSUAF to be authenticated via mod_auth_openvms. This means doing password changes in two places -AND- not allowing full self-serve account maintenance on these. (well I could but then how would I sleep at night knowing someone might stumble on a way to hack the system?)
- So we need to replace mod_auth_openvms with something more useful; something that supports session cookies
- I started with mod_auth_openvms.c as my template but first needed to learn how to build it or there would be no point in continuing
- I downloaded a UNIX distribution of httpd-2.0.63 from Apache with the intent of only building part of it (the authorization plugins) on OpenVMS
- Then I realized APR (Apache Portable Runtime) was a prerequisite which needed to be downloaded then built separately. I wondered if this would be best done with a MAKE file under GNV or by some other method
- So I bounced the MAKE/GNV questions off people at comp.os.vms who suggested downloading CSWS sources from HP then linking against the Apache run-time which is already present (who knew?)
- CSWS source kits can be downloaded from HP's website via anonymous FTP
- ftp://ftp.hp.com/pub/openvms/apache/ (works from IE-11)
- note that CSWS-V20-2 is newer and larger while CSWS-V21-1 is older and smaller (???)
- my examples use CSWS-V21-1 (older and smaller)
- note: the official build instructions in chapter 5 of manual csws_install_22.pdf are broken; see my scripts below for one possible solution
- caveat: my original programs were pure hacks based upon peeking into the ".h files" included in the kit. Since then, I have learned my time could have been reduced by 80% had I first purchased a copy of Apache: The Definitive Guide published by O'Reilly. A complete list of API routines (like ap_get_basic_auth_pw) can be found in the appendix. Professional module writers may also want to check out Writing Apache Modules with Perl and C
- this module was published by Compaq before the merger with HP (now HPE)
- This 2015 production file hasn't changed since 2003-01-15 (presented here for reference purposes)
- Comments: with the benefit of 12 years of hindsight
- Parts appear to be derived from a Solaris program written before y2k (keep in mind that Solaris usernames are case sensitive)
- as written, this plugin is very inefficient. It contains two hooks:
- hook-1 will fire for every transaction "requiring Apache authentication" in the whole system whether this plugin is enabled in that directory or not. This means the hook is always fired but has been defaulted to DECLINE. When enabled, will access SYSUAF
- hook-2 will fire for every transaction "requiring an Apache authorization" in the whole system whether this plugin is enabled in that directory or not. This means the hook is always fired but has been defaulted to DECLINE. When enabled, will search for the current USERNAME in an external GROUP FILE.
- It might have been a better idea for Compaq to operate both routines under a single hook which would make the plugin slightly less parasitic. Alternatively, they could have published two plugin modules: "authentication only" and "authentication and authorization"
- Another problem with the code under hook-2 is that the username test is case-sensitive. OpenVMS usernames are up-cased before going through SYSUAF authentication. This means that the GROUP FILE associated with hook-2 needs to contain both upper and lower versions of every username but someone logging in with a mixed-case username will still be bounced.
- the first two "c language" tests near the top of hook-1 are in the wrong order. This plugin could be made a tiny bit less parasitic if the very first line of code was the test of fUserEnable rather than a call to ap_get_basic_auth_pw (why test for a password in places where the module is disabled?)
- my DCL build script: build_mod_auth_openvms.com
- it works with cookies, and/or username+passwords
- it can (optionally) engage an indexed RMS-based "session cache" so the external authenticator is called less often (I do not think it would be too difficult to move this into a "global section" -- also called a shared memory object)
- it spawns an external authenticator (which can connect to any kind of database)
- it can command a browser to raise the username-password dialog
- it can (optionally) command the browser to store a COOKIE (cookie data must be provided by your authenticator)
- it can (optionally) do GROUP access restriction tests (group data must be provided by your authenticator)
- DCL build script: build_mod_auth_vms_ext.com
- opens an indexed RMS file to perform application-level authentication (can use characters not allowed in SYSUAF)
- essentially identical to mod_auth_vms_ext.c (above) except that lib$spawn was replaced by directly accessing an RMS-file
- DCL build script: build_mod_auth_vms_rms.com
gSOAP Specific (c)
When you check out the sample programs delivered with the gSOAP stack they seem fairly straight forward (right now I'm thinking about "calc") but most were created using the WSDL-last method. When someone hands you a WSDL then says "build client and server apps which will be part of a two way messaging system" things get complicated fast. You will be forced to use the WSDL-first method, and the tool wsdl2h (read as: wsdl-to-h) will stuff your SOAP variables deep inside complicated structures and unions then pass the whole mess to you with a pointer. If the WSDL contains more than 100 fields like the one I saw a few months ago, you might want to tackle a smaller project first just to develop skills.
Last year I used the WSDL-last method to create a Java-based web service under Tomcat/AXIS2 called "SimpleService". This service only had three methods: ssADD, ssSubtract, and ssEcho (I added the "ss" prefix to my variables so I would see them in various places in the code). Then I used AXIS2 to generate a WSDL which would be used to build this gSOAP service with the same functionality.
gSOAP Specific (c++)
Brett Cameron and John Apps have provided a really good set of gSOAP sample programs written in HP-C but nothing (as of 2012) written in HP-C++. So here are my unofficial offerings of sample programs tweaked for use with HP-C++ for OpenVMS.Notes:
- Large gSOAP programs written in C++ seem clearer than the same ones written in C. Consider using C++ if you you want your gSOAP code to me more easily maintainable by other programmers.
- When the gSOAP preprocessor "soapcpp2" generates C++ code it will use a ".CPP" extension
- HP-C++ defaults to a file extension of ".CXX" while competing compilers default to ".CPP" so be sure to include a fully qualified file name.
|gSOAP Calc Samples|
gSOAP sample Client program from //gsoap/samples/calc++/calcClient.cpp
gSOAP sample Server program from //gsoap/samples/calc++/calcServer.cpp
DCL script to build these two gSOAP sample programs on OpenVMS
gSOAP sample Client program (w/OpenSSL) from //gsoap/samples/calc++/calcClient.cpp
(this last demo has only been tested on Windows-7)
Porting OpenSSL to OpenVMS
While it is possible to build OpenSSL from public sources, the libraries will be okay but at least one of the demonstration applications will not work without modifications. The main reason for this involves use of the UNIX I/O select() statement to test for I/O readiness. While this is perfectly acceptable in the UNIX/Linux world, select() can only be used to wait for network I/O in the VMS and Windows worlds. There are two possible fixes for this:
- use a different technique to detect the start of keyboard I/O
- see other platform examples in latest versions of s_client.c where a home-brewed function called kbhit (keyboard hit) is used to detect when the "C" program should begin a keyboard read operation in Windows
- this is the el-cheapo solution which should be okay for s_client.c since it is not production quality code.
- use a VMS-recommended technique to use a socket to connect to SYS$COMMAND
- this solution requires a socket interface at the SSL end and a SYS$QIO at the keyboard end
- this gold-plated solution can be seen in the program term_sock.c (terminal socket) shown just below
|s_client.c||This program was found in kit CPQ-OpenSSL-1.0A (OpenSSL 0.9.6b) published July 2001 by Compaq|
It shows how to create a socket (TerminalSocket) to communicate with stdin (your keyboard)
No Compaq/HP copyright was included with this package.
While this program is copyrighted by Eric Young, the VMS changes are not copyrighted by anyone.
"I think" it was modified by someone at Compaq Computer Corporation
|term_sock.c||This program was found in kit CPQ-OpenSSL-1.0A and is used to create a socket (TerminalSocket) to communicate with stdin (your keyboard).|
No Compaq/HP copyright was included with this package or this file.
"I think" it was written by someone at Compaq Computer Corporation.
It allows the programmer to get around the "bad select 38" problem
|term_sock.h||This c header supports term_sock.c|
No Compaq/HP copyright was included with this package or this file.
"I think" it was written by someone at Compaq Computer Corporation
|select_demo.c||A demo program (derived from HP's UNIX to OpenVMS porting guide) to show how to use select() |
Webpage Hit CountersWe have always been told to never use hit-counters because they consume precious resources. But many rinky-dink sites, especially some on corporate intranets sitting behind a firewall, need them for good P.R. with other departments.
These seem to be the gold-standard in counters. Between 1995 and 2000 (when web servers were powerful platforms serving up mostly static webpages to underpowered desktop PCs sporting Pentium-2 or Pentium-3 processors) most sites used a bit of freeware written by Muhammad Muquit named "count". Needless to say I am envious of his programming skills. But this solution places a more-than-trivial computational burden on the server. Why? Updating a counter file is the easy part. Using the digits to reference a graphics library of individual digits then assemble the binary pieces scan-line by scan-line into a resultant GIF is the hard part.Text-based Counters
I'm running an overworked decade-old server (AlphaServer DS20e installed in 2002) and I think the time has come to shift the computational burden from the server to the client. I have a little "C" program ready which increments the counter file then sends back the plain-text result. The calling webpage uses a small amount of AJAX (~20 lines) to accomplish the task then the browser is responsible for displaying the count. The browser may not have access to all the cool image sets seen in Muhammad's offering but that may not be as big a deal as you might think. I will publish it soon.
- here is the c-based hit counter code. It requires polishing. it also requires the addition of calls to the OpenVMS distributed lock manager (DLM) to provide better file locking than is possible with vanilla C routines for UNIX. This should be easy to do and I will publish that as soon as time permits.
Web Page Hit Counter (2.4 + 2.6)
- Muhammad Muquit is the author of a very cool multi-platform 'Hit Counter' which is currently run by many large ISPs in the world (if you don't believe me, just intercept a generated GIF then look inside where you'll see his name and URL in the comments area; his name continually flies around the internet but this is a small prices to pay for the efforts of such a gifted programmer)
- Phil Ottewell's code to work with Process Software Corp's Purveyor 2.x Encrypt Web Server and the code is available at here).
BTW, I also inserted calls to the OpenVMS Distributed Lock Manager to prevent the counter data file from becoming locked out due to multiple process requiring exclusive write access.
- If you don't believe this can happen, set up a demo web page with 20 image references to the same counter like so:
<img src="https://www.your-site.org/cgi-bin/Count.cgi?dd=D&df=your-counter.dat&sh=1&incr=1" >
- now publish the web page to your ISP so you've got maximum CPU power and network bandwidth
- now access the web page and watch the counters get displayed in your browser
- now add another 10 images references and repeat the whole process. I've discovered that the counter data file will become permanently locked out by the ISP when the number of images exceeds 32 (according to www.netcraft.com one site was running Netscape-Enterprise/4.1 over Solaris while another was Microsoft-IIS/4.0 over Windows-NT4). BTW, before my patch, you would never see a permanent lockout on an OpenVMS system; just the odd counter GIF not being displayed. My Distributed Lock Manager routines make sure that every request gets honored so the odd GIF will just pop in a few milliseconds late. No big deal but that is what is so neat about OpenVMS.
- 2003-05-23: I've got a pre-release of Hit Counter v2.6 for Compaq Secure Web Server version 1.3.26 (a.k.a. CSWS, or Apache for OpenVMS) working on my AlphaServer-DS20
Technical Observations (2013-11-xx)
- As currently implemented on OpenVMS under Apache, count.exe will produce two files. File-1 contains "the MIME header text" while file-2 contains "the binary graphic data"
- In 2013 I shifted my webserver to Ruthless Browser Caching which caused the web-counter image to stop updating (and we still need it for departmental P.R.)
- Since I hadn't worked on this code for ten years, I started by rewriting the DCL script to make it smaller and more efficient
- Next, I replaced the copying of file-1 (the MIME header) with output from DCL. To do this I replaced one image execution (APACHE$DCL_BIN) with another (APACHE$FLIP_CCL) which is still less expensive since we do not need to locate, then transfer, file-1.
- After that I added a second MIME line ("Cache-Control: no-cache, no-store") to deal with servers implementing Ruthless Browser Caching.
- It should be possible to modify the C sources to do away with creating intermediate files. The result would be a two-line DCL script. I publish this solution as soon as possible
Updated DCL script for use with CSWS (aka Apache on OpenVMS)
$ on error then $ continue
$ t = f$cvtime(,,"TIME") -":"-":"-"."
$ p = f$getjpi("","pid")
$ f = f$trnlnm("csws_server_root") + "count_" + p +"_" + t +".gif"
$ set prot=(S:rwed,o:rwed,g:rwed,W:rwed)/default
$ define/user/nolog sys$output 'f'
$ mcr www_root:[scripts]count.exe
$ m=1 ! DCL
$ if (m.eq.0)
$ then ! MIME from a file
$ APACHE$DCL_BIN 'f';1 ! MIME header
$ else ! MIME from DCL
$! write sys$output f$fao("!AS!/!/","Content-type: image/gif")
$ write sys$output f$fao("!AS!/!AS!/!/","Content-type: image/gif","Cache-Control: no-cache, no-store")
$ APACHE$DCL_BIN 'f';2 ! GIF data
$ del/nolog 'f';*
OpenVMS DCL scripts
|DECnet communication|| || |
| ||decnet_demo_c.com||client (start a remote server then send it some messages) |
| ||decnet_demo_s.com||server (echo messages back to client)|
| || |
|Example DCL script for properly setting up FTP/SFTP|
DCL script to intercept an FTP/SFTP login then submit a login helper job
DCL script which will wait for the network partner to exit then process received file(s)
(similar to what happens at the end of an XCOM transaction)
More notes can be found here: OpenVMS Notes: TCPware
|FTP Scripts|| || |
| ||ftp_multi_stack_scripting.com||a script which can FTP from three different stacks: MultNet, TCPware, and TCPIP Services |
|uppercase - lowercase|
| || |
| ||sense_nonsense.com||a tool to easily toggle your account between two modes: case-sensitive and normal|
| ||downcase_filenames.com||a tool to easily down-case filenames. Intended for SSH key files.|
|Apache Support|| || |
| ||rotate_apache_logs.com||a little batch job to rotate apache client log files every 24 hours|
also will (optionally) do monthly maintenance of apache process logs
Monitoring (AXP only)
tweaked for calling from your LOGIN.COM
OpenVMS Macro32 source code
Introduction to OpenVMS programming
It was a lot tougher before y2k
It was a lot tougher learning how to write VMS applications in the 1980s. Why? DEC charged big-bucks for everything including hardware, software, training and documentation (including pocket programming cards!) Many IT people wondered if that was why DEC used a dollar sign ($) for the VMS/DCL command-line prompt.
First off, you needed the 32-binder Orange Wall of VMS documentation which I remember being around $5k for the initial purchase. Then, even if you already knew how to program in one of the many DEC-supported programming languages, you still needed to attend DEC language-specific courses to learn how to do system calls. This meant you needed to work for a company with deep pockets, or needed to get VMS-specific training in college or university.
In the early 1990s, DEC renamed the product line from VMS to OpenVMS at the same time they introduced the 64-bit Alpha hardware line as a successor to 32-bit VAX. During this time the Orange Wall became the Gray Wall
Then Things Changed
In 1998, DEC was purchased by "PC manufacturer" Compaq.
- The first thing Compaq did was to convert the 32-binder Gray Wall (with a single CD-ROM. GOOD This allowed them to shut down their document publishing business)
- Next, they put all their OpenVMS documents online so that anyone could access them for free. GOOD
- Next, they shut down their educational services division to save money. BAD Technically speaking, they didn't just shut it down cold, they attempted to pass the business to third-party educators like they did with their PC products. But the scheme flopped. (hey, what would you expect from a shoot-from-the-hip PC company?)
In 2002, HP purchased (err, merged with) Compaq. HP restored some in-house training but now OpenVMS was just one of many software offerings from HP's, and I think OpenVMS has been lost in the mix.
Although my employer did provide us with the 32-binder Gray Wall, we had to learn how to interface VMS BASIC to VMS the good old-fashioned hard way: by hacking. In 1993 I fortunately stumbled across a copy of Writing VAX/VMS Applications Using Pascal (Author: Theo De Klerk, Published: 1991 by Digital Press) in DEC's student retail store in Bedford, Massachusetts. I had learned Apple Pascal in late 1970s so found it relativity easy to convert many of these programs into DEC-BASIC and DEC-C. This book contained many descriptions of VMS subsystems and was full of "system calls" on how to access them. These books pop up every so often on www.bookfinder.com and I think every serious OpenVMS programmer should own a copy.
Okay so that was then and this is now. To the best of my knowledge, you can only get the OpenVMS Documentation CD-ROM by purchasing it from HP, buying a used copy on eBay, or begging a friend to make you a copy. But if you use HP's online documentation along with examples from this book, you will learn how to become a VMS BASIC developer much faster than me.
Online HP Manuals + Documents
Advanced hacking (system services)Many novice programmers do not know about this cool feature to trace OpenVMS system calls originating from your process (DCL or otherwise). So if you have the necessary privs, then try this experiment:
<sr> = system response
<ur> = user response
<ur> set process /sslog=(state=on,count=4,flag=arg) ! enable tracing on YOUR process
<ur> run your-program.exe
<sr> ...program runs then exits...
<ur> set process /sslog=(state=off) ! disable tracing
<ur> set process /sslog=(state=unload) ! this closes the file
<ur> ANALYZE/SSLOG/FULL/WIDE/OUTPUT=ss_trace.txt ! human readable data goes to this file
<ur> type/page ss_trace.txt
Noteworthy C/C++ demos at other sites
Noteworthy C/C++ articles and manuals
How to recover from FTP-induced saveset corruption
$set file /attr=(RFM:FIX,MRS:512,LRL=512,ORG=SEQ,RAT=NONE) yourfile.bck
$set file /attr=(RFM:FIX,MRS:512,LRL=512,ORG=SEQ,RAT=NONE) yourfile.sav
$set file /attr=(RFM:FIX,MRS:512,LRL=512,ORG=SEQ,RAT=NONE) 'p1'
(where P1 is parameter #1 in a DCL script)
Back to OpenVMS
Back to Home
Kitchener - Waterloo - Cambridge, Ontario, Canada.