Metrum Institute

NMQual 6: User's Guide

Contents

NMQual is a tool for automating and documenting the installation of NONMEM®. This guide describes the normal use of NMQual, version 5. NMQual is available under the GNU General Public License from Metrum Institute. NONMEM® is available under license from Globomax, LLC. Instruction is given here on how to obtain NMQual, but we assume you already have a Globomax CD with NONMEM® version V, level 1.1 (15 April 2004 or 18 March 2005) or version VI, level 1.0 (27 October 2006).

System Requirements

NMQual requires the presence of

NMQual has been tested with the following operating system(s) and Fortran compiler(s):

If g77 for Windows is used, make the two recommended permanent changes to envrironment variables (PATH, LIBRARY_PATH). See http://members.tripod.com/kkourakis/ for details. We provide a value-added redistribution at http://metruminstitute.org/downloads/g77. On Linux, the 'f77' executable is probably already in the 'PATH' environment variable. For Mac, there are several ports of Fortran77 compilers, like GNU Fortran from fink.sourceforge.net and hpc.sourceforge.net.

This version of NMQual was written to support the installation and qualification of NONMEM® version V, level 1.1, as it appears on the Globomax distribution CD-ROM dated 15 April 2004 or 18 March 2005; or version VI (6.1.0) as it appears on the CD-ROM dated 27 October 2006. Modifications to NMQual may be necessary if you are using NONMEM® distribution media generated on a different date. Customized installation is available as a fee-based service (contact nmqual-services@metrumrg.com).

Obtaining NMQual

NMQual is available at Metrum Institute's website. Download the current zip file from the appropriate link at http://metruminstitute.org/downloads/nmqual/.

Verifying the Distribution

You may want to verify the checksum for NMQual. The procedure is system dependent. If you have an MD5 utility, calculate a checksum for nmqual.zip, and verify that it matches the value linked to http://metruminstitute.org/downloads/nmqual/. If you don't have an MD5 utility, you can download a free one. Search with a major web index for something like "free md5 windows", or try Metrum Institute's multi-platform utility: http://metruminstitute.org/downloads/md5.

Installing NMQual

Make sure you have Perl 5.8 or greater installed. Type 'perl -V' at a command prompt to diagnose.

Make sure you have XML-XPath installed. Type 'perl -e "use XML::XPath". Silence indicates success. Consult your system administrator if necessary. For ActivePerl's PPM utility, type 'ppm' to get a prompt (full path may be necessary); then type 'install XML-XPath'. Consult PPM's documentation for further help as necessary, or try Nick Holford's instructions. Newer versions of ActivePerl have a GUI version of PPM that is much harder (!) to use: try View | All packages; then right-click on XML-XPath and check the box for install, then click on the little green arrow in the tool bar to install marked packages.

Make sure you have a Fortran77 compiler installed. Type, e.g., 'g77 -v' at the comand prompt.

Make sure you have appropriate account permissions for your intended install locations (NMQual and NONMEM). Consult your system administrator if necessary.

  1. 'Unzip' the nmqual zip file with appropriate decompression software. Your operating system may have a built-in decompression utility. Extract the directory 'nmqual' to an appropriate directory, such as 'C:\nmqual\' (on Windows). Note: the full path can be nested arbitrarily deep, such as 'C:\metrum\downloads\utilities\'. In our experience, g77 2.95 does not work with paths that contain spaces.

  2. Customize the configuration file (config.template.xml). At very least, you will probably need to edit the two directory specifications ('source', 'target'). Instructions and examples are provided. By default, config.template.xml is set up for WindowsXP using the g77 compiler, with NONMEM5in the d:\ drive, installing to c:\nmv.

  3. Did you break something when you edited the config file? Go to a validator and check your configuration file (XML Instance) against 'config.xsd' (XML Schema). If validation is successful, you have a better chance of getting good results when you run NMQual. If validation fails, it is likely that your configuration file is corrupted. The validator should give some hints.

  4. You may want to customize and rename 'changes.options.nm5.std.xml' (or equivalent) if it's one of the change files mentioned in your configuration file. Pre-written customizations may be available, e.g. changes.options.nm5.big.xml. These files control the sizes of NONMEM® items. changes.options.nm5.std.xml gives Metrum Institute's 'standard' sizes, and '.big.' gives Metrum Institute's 'big' sizes. Both give NONMEM installations with sizes different from the defaults. changes.options.nm6.std.xml contains non-default settings (bigger buffer sizes) for NONMEM6.

  5. You may want to edit the shebang line (#!/usr/bin/env perl) in each of nmqual/*.pl, if this matters for your system. Often it won't, especially if you are on Windows or use the recommended convention of invoking perl scripts as arguments to perl itself, e.g., 'perl nmqual.pl [...]'.

Running NMQual

  1. Make sure your NONMEM® files are available at the location specified in your configuration file.
  2. Open a command window (e.g. start->run->cmd on Windows).
  3. Navigate to the directory 'nmqual'.
  4. Execute nmqual.pl, specifying a configuration file, e.g., at the prompt, ' perl nmqual.pl config.template.xml' . If you omit the config file, nmqual.pl will display usage advice.
  5. You can run NMQual repeatedly with different configuration files, to create different versions of NONMEM® on your system. (You can also tweak a config file and re-run it immediately.) Each completed NONMEM® installation is independent of other NONMEM installations and independent of the NMQual installation. The last subdirectory in each target directory must be unique, since NONMEM is invoked by a script bearing the same name (created for you by NMQual).
  6. After a successful run, you may want to add the relevant path to your system path variable, and alert other potential users to do the same. On Linux and Mac, you will probably need to add a line to your shell's profile script, such as 'export PATH=/opt/nmv:$PATH'. Consult your system administrator if necessary. On Windows, right-click on My Computer, then choose Properties> Advanced>Environment Variables>User (or System) variables>PATH>Edit. Prepend e.g. 'C:\nmv;' to whatever is listed in the edit box. Close windows appropriately. On most systems, you will need to open a new command window, maybe even reboot (Windows), for path changes to take effect.

Prompts and Errors

Many prompts occur in supervised mode, fewer in unsupervised mode. Only the more noteworthy prompts are mentioned here. Errors should give some helpful details on command line when they occur.

Almost all errors are fatal: nmqual.pl should 'die' if it (or a secondary script) can't do what it was supposed to do.

Editing Change Logs

Editing the change logs is not terribly difficult, but requires precision, and some appreciation of the XML file format. You can add a change by cloning and editing an existing change, i.e., everything from a 'change' tag to its corresponding 'end change' tag, inclusive. Change other fields as appropriate, e.g. identify the file you want changed. In the 'from' field specify some text to find, and in the 'to' field, specify some replacement text. Preserve the line 'C ATTRIBUTION' exactly as in other instances: NMQual will replace this with an installation-specific comment ('attribution' in the config file) and timestamp.

There is a bit of an art to selecting text for the 'from' field. First, you'll get better results if you copy text directly from a Fortran file. The match must be exact, and hand-typing can result in subtle errors. Second, it is useful to begin the selection with a return character (start highlighting at the end of the previous line) so that the Fortran code lines up nicely in the XML file. We generally don't include the final return character before the 'end from' tag. Third, be sure to include some static 'bounding' text before AND after a desired point of insertion. If you do, the 'from' text won't exist anymore as such, after the change is complete. If you don't, scripts may have a hard time later trying to tell if the change was made or not (scripts are not as smart as you).

After you've mangled one of the standard changes*.xml files, or written one of your own, you may want to verify that your file format is still what NMQual is expecting. First, the name of the change file must be listed in your config file, e.g.'<file type='change'>my.changes.xml</file>. Second, the file must still be valid XML: matching tags and all that. Many web browsers will try to parse your XML file, and will show you any format errors. Third, the file must meet the content model that NMQual expects. For instance, the root element must be 'changes'. The file changes.nmqual-5.xsd (a 'schema'), completely specifies the content model for changes*.xml files. Submit changes.xsd and any homemade changes*.xml file to an online validator. The validator will tell you whether your XML file meets NMQual's content expectations, and will probably show you where the problems are, if any.

Some bug fixes will require no editing on your part. Metrum Institute may periodically release new changes files to reflect bug fixes recently identified by Globomax. Watch for updates at http://metruminstitute.org/downloads/nmqual/. Save it in your nmqual directory and list it in your config file(s).

Running Control Tests

NMQual makes it easier to qualify a NONMEM® installation. It provides a tool called runtest.pl which compares the output of control tests to expected results. Several control tests and associated reference files are provided by NMQual. You can extend the usability of runtest.pl (i.e., add more control tests) by carefully imitating existing file structure. More on that later.

Currently, NMQual will implement 6 test cases. Tests 3 - 7 are the example control streams (CONTROL3 - CONTROL7) provided in the NONMEM® distribution media. Test 8 is a user-defined test case, based on simulated data from a published population pharmacokinetic model describing the disposition of parent and metabolite plasma and urine concentration data. It is provided as an example.Users who wish to contribute test cases to the NMQual project may send suggestions to nmqual-comment@metruminstitute.org.

After a successful install, you should run the control tests to see if NONMEM® is operating as expected. Navigate to the test directory of the NONMEM® installation , e.g. c:/nmv/test. Execute ' perl runtest.pl 3 4 5 6 7 8'. Expect 7 and 8 to take a while. runtest.pl uses the argument list as file name stems for control tests, and generates/guesses the file names as necessary. For instance, in the example given, it will look for 3.ctl, 4.ctl, etc. (control streams) and pass them to the NONMEM® executable. After the run, it will look at the results and compare them to control3.ref.xml, control4.ref.xml, etc., and log the results.

Try this to make a new control test. Windows code is given, but the Linux equivalent should be obvious.

  1. Navigate to: c:/nmv/test/ .
  2. Execute: 'copy 3.ctl 9.ctl'.
  3. Execute: 'copy PARAMS3.for PARAMS9.for'
  4. Edit 9.ctl: change 'PARAMS3.for' to 'PARAMS9.for'.
  5. Edit PARAMS9.for: change 'control3.test.xml' to 'control9.test.xml'.
  6. Customize modeling code in 9.ctl and PARAMS9.for (or do nothing, for the sake of demonstration only).
  7. Execute: 'perl runtest.pl 9'. runtest.pl will complain that it cannot find 'control9.ref.xml'. We haven't made control9.ref.xml yet, because we don't necessarily know the expected result of your code modifications above. However, if NONMEM® were functioning properly, newly created control9.test.xml should look pretty much like our ideal control9.ref.xml. So...
  8. Execute: 'rename control9.test.xml control9.ref.xml'.
  9. Execute: 'perl runtest.pl 9'. Now runtest.p finds what it wants. Of course, this example is circular. Ideally you would study 9.lst to decide if the content of control9.test.xml really does merit use as a reference file.

Running NONMEM®

If you tried the control tests above after installation, NONMEM® was run for you by runtest.pl. To run NONMEM® yourself, simply invoke the script (same name as your install directory), supplying the name of a control stream and a name for an output file. Navigate to the 'test' subdirectory and...

  1. Execute: 'perl -S nmv.pl 3.ctl 3.lst'. Your results are in 3.lst. nmv.pl (or whatever yours is called) appends the installation log to the end of the output file; then it appends any runtest logs it finds. '-S' tells perl to search for nmv.pl in directories listed in the path variable. If you didn't change the path variable on installation, you may have to specify the location of nmv.pl on the command line.
  2. Optionally execute: 'perl -S nmv.pl 3.ctl 3.lst nochecksum'. Same result, but nmv.pl doesn't bother to run the checksums. This is handy if you've tinkered with the Fortran code, because a bad checksum would cause nmv.pl to fail. You can run checksum.pl at anytime: 'perl checksum.pl c:/nmv' ('c:/nmv' is your install directory). checksum.pl will calculate checksums (for items listed in the configuration file) and either create a checksum log or compare to a log if one exists. Caution: don't use the 'nochecksum' option for qualified, production NONMEM® runs unless your qualification procedures specifically call for it.

Directory and File Structure

The directory structure of NONMEM® installed by NMQual is similar to that created by a manual installation. The major difference is the addition of a test directory. The test directory contains the NMQual utilities, and is where the NONMEM® executable actually gets created. Some files are present on a configuration-dependent basis. Most are copied from the 'nmqual' directory.

Similarity

The script runtest.pl compares NONMEM output to copies of accepted output for appropriately formatted control tests. The control tests consist of a control stream, a PARAMS file that creates test output, and a reference file that contains accepted output. The script runs the control stream, and then makes comparisons for each test/reference value pair. 'Status' and 'similarity' are calculated for each comparison. Results are stored as runtest.[timestamp].xml.

Behavior depends on whether the reference value is text, a nonzero number, or zero. Text is evaluated by string comparison: status is either "match" or "mismatch" and similarity is either 99 or -99. A zero invokes a numeric comparison: status is either "match" or "mismatch" and similarity is either 99 or -99. A non-zero numeric reference value invokes a numeric comparison: similarity is calculated as (-1)* log10(abs((ref-test)/ref)). Status will be set to "mismatch" for similarity <=-99; "match" for similarity >=99; and "partial match" for all other values. Very informally, similarity of "3" means that the reference value and the test value are the same to 3 significant digits.

At the end of runtest.pl, a summary table is given on the command line where rows correspond to analyses in a batch. Column "test" names the analysis; "mis" gives the number of mismatches for that analysis; "exact" gives the number of exact matches; "part" gives the number of partial matches; and "min" gives the minimum similarity across all partial matches.

NMQual and XML

XML is a presentation-neutral data storage format that promotes interoperability. XML is designed to be machine readable as well as somewhat human readable. All logs created by NMQual scripts are encoded as valid XML: nmqual*.xml, checksum*.xml, and runtest*.xml. Additionally, bug fixes and options are encoded as XML for use by NMQual during installation: changes*.xml. The configuration file is also XML. Even the NONMEM® output is repackaged as XML. An XML-aware browser like Mozilla Firefox is a convenient tool for viewing native XML files. Online web apps are convenient ways to test an XML file against a W3C Schema content model (usually *.xsd). Here's some we've tried. You can also do an internet search on something like 'XML schema validator'.

Administrative Acceptance

The utility 'accept.pl' provides a mechanism to document administrative acceptance of an installation. It accepts two arguments: the path to an installation and (optionally) a string of text (presumably an administrative statement). The executor must have 'write' permission for the installation log. The user login (as defined by Perl), a time stamp, and the text string (if present) are embedded in the installation log as items "acceptor", "time", and "statement" (respectively) under item "acceptance". For installations that have already been accepted, no changes are made and the conditions of acceptance are reported on the command line. Since the NONMEM invocation script attaches the installation log to all run output, acceptance information is propagated.

Additional Help

See http://metruminstitute.org/downloads/nmqual/faq.html.

NONMEM® is a registered trademark of the University of California and is licensed exclusively to GloboMax LLC.

home | contact | ©2008 Metrum Institute