EQS statistical software under UNIX is installed on specific machines in the Research Computing Facility (RCF) and Student Computing Facility (SCF) clusters of ITS machines. Users must have an account on a machine in one of these clusters in order to use EQS or any of the other statistics packages. Such accounts are available from the ITS Computer Accounts Administration Office in LVL Lower Commons, 213/740-5555.

You may wish to work from a "personal subdirectory", which means you would routinely cd to a subdirectory other than your home directory. EQS, however, can be invoked from any subdirectory. The examples below will assume that files are stored and/or written to the current subdirectory (i.e., the directory from which you invoked EQS). (Files that are not stored in the current directory may be used, but must be specified with the complete home directory pathname or the tilde-slash home directory convention. If you have questions about file specification in UNIX, please contact the UNIX consultants at the number given at the end of this document.)

You can receive additional online help by typing,

eqs -h     or      eqs ?

Invoking EQS

To invoke EQS under UNIX, type eqs at the UNIX prompt. In order for this command to function, your environment needs to be set up to find the command and any other files or libraries the software needs. In the RCF, this is done automatically when you login, allowing you to access all the statistics software by simply typing the invocation command.

In the SCF, users of this (or any) statistics software need to add the following lines to their .login file:

if  (-e  /usr/usc/stats/setup.csh)  then
source  /usr/usc/stats/setup.csh
endif

SCF and RCF users whose .login or .cshrc files are non-standard or who once had setup files for one or more specific statistics packages sourced in .login or .cshrc may need to fix their .login or .cshrc files to eliminate redundancy and/or to allow the system's automatic setup to take effect. For example, users on the RCF may have modified their .login or .cshrc to such an extent that the system's automatic setup is bypassed. These users can either restore the system's automatic setups (recommended) or add the lines shown above (which are usually only necessary for SCF users). Please call consulting (see last page) for help with these files.

The EQS Package

EQS is designed to perform structural modeling, including regression, latent variable, and structural equation models such as Multiple Regression, Multivariate Regression, Path Analysis, Simultaneous Equations, Latent Variable Models, Confirmatory Factor Analysis, Structured Means Analysis, Multiple Population Comparisons, and Monte-Carlo Simulations.

EQS runs in batch and semi-interactive modes in most terminal emulations. (Semi-interactive means that you can run the commands in an EQS program and see the results at the screen, but you cannot stay in interactive mode to enter more commands or data.) EQS output files are character data, so that they can be sent to a line printer.

In terms of command and subcommand syntax, EQS Under UNIX is similar to versions running on other platforms (e.g., PC-DOS). Most of the differences, of course, relate to differences in the way external files are specified (if and when they are used), a convention which is operating-system-specific.

EQS operates either in batch mode or semi-interactive mode, though the only difference between batch and interactive seems to be that in interactive mode you see the results scrolling by on the screen as the program executes your EQS commands, and in batch mode the results are available only in the .log file. It is not possible, at present, to stay in interactive EQS to enter program statements or data. All programs must be run from UNIX files.

EQS uses a series of "sections", each of which begins with a slash (/) and a keyword. The standard EQS paragraphs for statistical analysis are /SPECIFICATIONS, which describes the problem to be analyzed, /EQUATIONS, which describes the equations involved in the model under consideration, and /VARIANCES, which specifies the variance for each independent variable in the model. The /END section signals the end of program statements. Several optional sections allow the user to specify tests, constraints, and other special operations.

It is possible to enter data in the same program that is being written to analyze them. This is called "inline" or "instream" data entry, but apparently can only be used when the data are in the form of correlation or covariance matrices. (The appearance of data in the program is introduced by the /MATRIX keyword.) When raw data are used, they are accessed from an external file.

The output from EQS jobs contains the original input statements, results of the calculations and plots requested, and any error messages or other notes that might be generated by your program. Output is always saved into a file. If you do not specify the file name for output, EQS will use "eqs.log", but note that any previous output in a file called "eqs.log" will be overwritten by subsequent programs that do not specify a file name other than "eqs.log". As noted above, interactive processing allows the results of your EQS program to be shown at the screen (as well as placing them in the "*.log" file), while batch processing writes to the file only.

The elements of EQS programs are fully documented in the EQS manual, which also contains a fair amount of statistical background text relating to Structural Modeling, including references and suggestions for further reading on the subject.

To invoke EQS under UNIX, the following command is used:
eqs

This displays the EQS "banner", and begins the prompts which ask for the input program file name (required), external data file name (optional: default is none), and output file name (optional: default is "eqs.log"). Note that external data files must be in the same subdirectory from which eqs is invoked. The developers of the software will be removing this limitation in future versions.

If you prefer to specify these in the command line, you can do so with any or all of the following switches after the command:

-i

<input file name, required: no default>

-d

<external data file name: no default> -- (data file must be in same directory where eqs is invoked)

-o

<output file name: default is "eqs.log">

-l

<integer words of working space: default is 25,000>

-m

<execution mode: b=batch, i=interactive, a=abort:default is b>

-h

(Help for eqs)

Any of these switches that are used may be entered in any order. For example, you may want to run EQS programs held in the file "mytest.eqs", direct the output to a file called "mytest.log", ask the system to allocate half a million words of working space, and run interactively. The commands to make this happen are

eqs  -o  mytest.log  -l  500000  -i  mytest.eqs

The same choices can be made by simply typing eqs and then answering the prompts that follow.

Please note that 'interactive' here does not mean what it might in some other statistics software. See the explanation of EQS semi-interactive processing in "The EQS Package" above.

Using Raw Data from External Files

During a EQS program (either batch or semi-interactive) you may specify an external file containing raw data, instead of typing it "instream" with the other EQS commands. (If you are using "raw" data, as contrasted with correlation or covariance matrix data, an external file is your only option.) The external file must be encoded in ascii and must not have variable names or labels, only data values. All data values must be numeric. Alphanumeric values (character data) are not allowed.

If you want EQS to read data from an external file, you can either specify it with a "-d" switch when you invoke EQS, or you can name it in the /SPECIFICATION section of the EQS program itself. NOTE that if you name the raw data file in the /SPECIFICATION section of the EQS program, the file name must have only CAPITAL letters, since UNIX is case-sensitive, and the EQS program converts all characters in the EQS program to upper case. If you specify the input file with the "-d" switch when invoking EQS, you can use any arrangement of upper and lower case for the file name.

To view EQS output, use the UNIX command more against the file you specified for the output, or "eqs.log" (which is the default). Note that if your default screen width is less than 82, you may not be able to view the output without changing that width. If you type 'more eqs.out' (or whatever the name of the output file is), and you get no output (or garbage under your command line), type the following to fix your screen to view the output:

stty  columns  82    (or larger)

The effect of this screen width on other operations in UNIX is unpredictable, so you may want to change back to 80 after viewing EQS output, but it probably won't matter.

The default width of EQS output files is 81 characters (to accomodate carriage controls in column 1), but column 81 is rarely used, so you can print output on virtually all types of printers without losing any characters.

To edit out portions of the output you don't need or don't want to print, use any UNIX editor (e.g., emacs, vi). Some users also like to remove the carriage controls ("1" in column 1), so that paper isn't wasted on printers that recognize carriage controls. In emacs, it is relatively easy to remove carriage control characters. Removing these has the effect of filling up printed pages (rather than ejecting after each "eqs" page), and also allows you to use emacs to view the file without extra lines appearing between each line of output (which happens because each line by default is 81 characters long, and emacs has to "wrap" the last character to the next line). To remove the first character of each line using emacs, 1) open the file (emacs <filename>); 2) position the cursor at the top left of the document, and type CTRL-@ to set the "mark"; 3) position your cursor on the line after the last output text line, and move the cursor to column 2; 4) type ESC-X to get the "M-x" prompt at the bottom of the screen; 5) at the "M-x" prompt, type kill-rectangle. Column 1 will be deleted, and the document will now fit within an 80-column screen.

Once the output from your program is stored in a file, you can print that file in any way you print other text files. You can download it to a Macintosh or PC and print it there, or you can submit it to a printer known to UNIX and print it here on campus. If the printer is controlled by ITS in a Public User Room there will be a per-page charge for picking up the printout.

To see a list of printers known to UNIX, type printers at your UNIX prompt. The printer "code" will be shown in the leftmost column.

To print a file on one of the UNIX printers, use the lpr command with the -P switch, as in the following example, which prints "my.file" to the printer in UCC 101 (the UCC Distribution Area):

lpr -Pps_ucc101 my.file

NOTE: ps_ucc101 is used only as an example; this probably is not the most convenient printer for most users.

Some printers are configured for PostScript only, and will not accept text-file input from the lpr command. In this case the enscript command (which converts text to PostScript) should be used instead, as shown here:

enscript -Pps_ucc101 my.file

To print output that is more than 80 characters wide, many printers will require that you rotate the orientation of the page to "landscape" (the default usually is "portrait") and often you will also need to reduce the font size to get all 132 columns on the page without wrapping. This can be accomplished with the enscript command followed by the -r (rotate) option and the -f (font specification) option. One font that has been tested for wide output is Courier9, though others may work as well. The following will print wide output (132 characters per line) without any wrapping or truncation:

enscript -Pps_ucc101 -r -fCourier9 file.name

You can also download your output (or any file) to a PC or Mac, and print on a local printer, if you prefer. The consultants have documentation on how to download files.

Since UNIX users have quotas, it is possible to run out of disk space when processing large data sets. To avoid running out of disk space, users may request quota increases when needs arise. If you believe your circumstances require more disk space, contact the authorized signer in your department for information and permission. If you don't know who your authorized signer is, the ITS Computer Accounts office in CAL 323-4 (213.740.5555) can help you.

/tmp Space

Quite often, large data sets are needed only for short periods of time while analyses are done or subsets are extracted. For these situations, the best solution often is to make use of the temporary space in RCF and SCF machines that is set aside for all users. SCF users will have quotas on /tmp space, similar to their own disk file quotas, so this may not solve SCF users' problems completely. RCF users also have /tmp space quotas, and can request increases from ITS as necessary. The space is in a subdirectory called /tmp (and, on some systems, /tmp1, and so forth; enter ls -ld /tm* to see them all). For descriptions and policies related to shared /tmp space, scroll to Section 7 in the following page:
http://www.usc.edu/its/policies/procedures/

Anyone can write in the shared /tmp spaces, create subdirectories in them, and in general manipulate files as you would in your home directory. You should be aware that files stored in /tmp are not permanent, and will be deleted by the system on the regular /tmp cleanup schedule. This schedule is different for different machines and /tmp areas, but the bottom line is that your files will go away.

Compressing Large Files to Save Space

Users with large amounts of data can make use of compression techniques when data are stored for periods of more than a few days without being used. UNIX allows users to compress their files, then uncompress them when they need to be used, allowing a substantial savings of space both in the user's own subdirectory structure and in the common areas (such as /tmp) if they are used. Users are urged to compress any files larger than 20 Mb if they are not being used for long periods of time. The amount of time it takes to compress or uncompress files is minuscule, and much space will be saved by this operation.

To compress a file in UNIX, use 'compress'; to uncompress a compressed file, use 'uncompress' or 'zcat'. In the following examples we compress a file called test1.bigfile, which results in a compressed file called test1.bigfile.Z:

compress test1.bigfile

Either of these commands results in an uncompressed file:

uncompress test1.bigfile.Z
zcat test1.bigfile.Z > test1.bigfile

The 'zcat' command, shown above, normally is used for viewing files, but with the use of the redirect, can re-create files from their compressed versions. While the 'compress' and 'uncompress' commands use the previous names (with the addition or deletion of the .Z extension), 'zcat' can redirect the compressed file into an uncompressed file of any name.

When large numbers of users are running jobs on RCF or SCF machines, it may be advisable not to try large jobs that are I/O or CPU intensive, since the portions of time allocated to each user are, obviously, much smaller than at times of low usage. The longer a job takes, the lower priority it often receives. This can result in jobs that may take hours of real time in the middle of the day, when they might only take minutes at a less busy time.

There are several useful techniques for dealing with large jobs that take a lot of time. You can run your jobs in batch mode, rather than interactive. This avoids using any memory that would be necessary for interactive mode. You can run your batch job in the background (by placing an ampersand [&] at the end of the command-line invocation), which allows you to logout while the job finishes, if you wish. (Running a job in the background also allows you to avoid timeout problems associated with long periods of terminal inactivity when you are connected via a micom line or modem.) And you can queue your job to run at a later time, when the system is likely not to be so busy.

Batch Processing

To run EQS in batch mode, enter your EQS program into a file, then issue the appropriate command followed by the name of that file. For example,
eqs -i myinput.file -m b

To run this batch program in the background, simply put an ampersand (&) after the file specification:

eqs -i myinput.file -m b &

When you have done this, you can perform other tasks in the foreground, and if you wish, you can logout and your job will continue to completion.

Delayed Batch Job Processing -- the  at  and  qsub  Commands

Any user can queue a job to run at a later time. Users running particularly large, compute-intensive, and/or long-running jobs are required to queue their jobs using the Portable Batch System (PBS) that makes use of rcf-01. For more information, see below, as well as our online document "Portable Batch System (PBS)":
www.usc.edu/hpcc/systems/use-l-4.php

To queue your job to run at a later time, you can use the  at  UNIX command. This requires that you place your invocation command and all other specifications (such as the filename of the program and any options or switches) into an executable shellscript file, and then specify the executable in an  at  command, such as

at 2:30a < execthis

where  execthis  is an executable shellscript you have created. (Of course, the name  execthis  is arbitrary, and can be any filename you choose.)

Let's say you have a statistics package called THYSTAT (for those who take documentation literally, note that this is a mythical package name, for illustration only), and your THYSTAT program is contained in  myprog.thystat1  which you keep in your home directory (~/) in a subdirectory called  programs. Part of the peculiar syntax of THYSTAT is that before the batch file name, you use a switch spelled  -in. For whatever reason, you wish to run your program late at night. To make an executable shellscript to run your batch program, enter the following into a file called  execthis :

#!/bin/sh -f
thystat -in ~/programs/myprog.thystat1

Then mark this file as an executable shellscript by typing the following at your UNIX prompt:
chmod u+x execthis

You can then run the  at  command, as shown above, and specify any time you feel would be best to run your job. For more details on the  at  and  chmod  commands, see the man pages or call the consultants.

The  qsub  Command and the PBS System

Multiple batch programs can be entered into a queue and run at a later time, allowing the user to 'submit' any number of jobs and still meet the system's limitation of one compute-bound job running at any given time. This method is required for jobs that are compute-intensive and/or will need multiple processors or extensive system resources. This type of queueing is done with the  qsub  command, which sends a request to the Portable Batch System (PBS).

RCF users normally can use the  qsub  command automatically. If it is not immediately available, run

source /usr/usc/pbs/default/setup.csh

Once this is set up, you can type  man pbs  and  man qsub  for complete online help information. Another helpful man page is  man pbs_resources_linux  which details the resource options (e.g., walltime, cput [cpu time], etc.) available for the  qsub  command.

The  qsub  command itself runs a script file, and inside the script file is the batch invocation of the software you wish to run (along with any other appropriate UNIX commands.  qsub  also writes to a LOG file that you specify, so you can see the STDIN and STDOUT messages associated with the running of your queued job. The general form of the  qsub  command is

qsub <flags> SCRIPTFILE

Once again, as in the example above, let's say you have a statistics package called THYSTAT (this is just a mythical example, to be substituted with whatever real command name and syntax you want to queue) and your THYSTAT program is contained in  myprog.thystat1  which you keep in your home directory (~/) in a subdirectory called  programs. Part of the peculiar syntax of THYSTAT, let's say, is that before the batch file name, you use a switch spelled  -in. Normally, you would run it in batch as follows:

thystat -in ~/programs/myprog.thystat1

In order to queue it with  qsub  and have the system run it later, you would create a qsub-appropriate SCRIPTFILE containing at least one line with a pound symbol ( # ) followed by the invocation of your software program, as shown above. Let's say you decide to put this into a file called  my.qsub.script. The contents of that file would be

#!/bin/csh -- a line beginning with # must begin the script file 
thystat  -in  ~/programs/myprog.thystat1

(Once again, remember that thystat is a bogus name, for example purposes only, and needs to be substituted with a real command name and syntax of your choice.)

The following command would queue your THYSTAT program for execution at a later time chosen by the system.

qsub -m be -k eo my.qsub.script

where  -m be  asks NQS to broadcast a message (i.e., to tell you) when the request ends execution;  -k eo  tells qsub to keep the STDERR and STDOUT output. A number of other flags are available, as listed in  man qsub  (remember to  source  the pbs setup file as shown above).

Many flags can be incorporated into the script file itself. For example, if you want your program always to report the results to your home directory, the -k eo flag may be entered into the file. Also, directions as to how much time to allow (walltime), how many nodes to use (nodes), and how many processors to use (ppn). If you want your job to run on the main rcf-01 processors, you can specify that in a separate nodes instruction (as shown in the example). Following is a sample (and simple!) script file that includes these requests. (The ls command is used as an example of a simple program that returns a detailed listing of the files in your home directory.)

#!/bin/csh -- any text here

#PBS -k eo
#PBS -l walltime=2:00:00
#PBS -l nodes=rcf-01
#PBS -l nodes=1:ppn=1

# run the program
ls -la ~/
exit

IMPORTANT NOTES:

• The syntax for the walltime value is hh:mm:ss (hours, minutes, seconds)
• The value for nodes= must always be 1 when submitting jobs to rcf-01, as the facility is configured as a single node.
• If explicit paths are used for commands or file specifications in the script file, /export and /auto must not be used. Instead, use the form /home/rcf-nn/username, where nn is the number of the file system partition where the files are stored, and username is the username of the home directory where the files are stored.

Often the  qsub  command will execute within a few minutes, though in some cases it could be delayed until the system is relatively unbusy at night.

Multivariate Software (the company that distributes EQS) has provided several sample programs for anyone's use. They can be useful as demonstration jobs, to allow you to see how EQS runs and what the output looks like. The sample programs are in /usr/usc/eqs, and are named as follows:

  
manul1.eqs   manul5.eqs    manul9.eqs   manul13.eqs
manul2.eqs   manul6.eqs   manul10.eqs 
manul3.eqs   manul7.eqs   manul11.eqs 
manul4.eqs   manul8.eqs   manul12.eqs

There are also two external data files (MANUL4.DAT and MANUL7.DAT) which are called by their respective programs (manul4.eqs and manul7.eqs). To use these programs, copy them (and their DAT files, if applicable) into a directory of your own and run them there.

For example, to run manul7.eqs from your ~/myeqs subdirectory, type

cp  /usr/usc/eqs/manul7.eqs  ~/myeqs/manul7.eqs

and

cp  /usr/usc/eqs/MANUL7.DAT  ~/myeqs/MANUL7.DAT

then go to ~/myeqs and run the test by typing

eqs  -i  manul7.eqs  -m  b  -o  manul7.out

This will run the test in batch mode and direct the output to manul7.out, where you can more it to see the results. (Note that eqs.log is the default name that would be used for the output file. If this is acceptable, you can leave out the "-o" switch completely.)

Interface With UNIX

Whenever EQS needs an external file, or whenever one is created by the program or the user, such files are stored in the UNIX directory system. EQS requires that program statements be stored in a UNIX file before being run by the eqs command. In addition, data may be stored in (and accessed from) a separate file outside the EQS program, and when it runs, the program creates a file to contain the output.

EQS regards the directory in which the eqs command is entered as the "current directory". Within EQS, if a file is called from, or directed to, the current directory, it is only necessary to use the file name. In all other situations, or if the user is unsure or wants to be unambiguous in specifying a filename, the entire UNIX path must be specified, including the filename and extension itself, as well as the complete path for your home directory. The EQS command recognizes the UNIX tilde-slash (~/) convention for "home directory".

Problems with EQS Processes under UNIX

It is possible, though infrequent, that you will leave a EQS process running after you leave your EQS session or even after you logout of UNIX. This can happen especially when you get forced out of EQS abruptly, with the system returning inexplicably to the UNIX prompt. If you have any questions as to whether or not you have old processes running, you can easily find out with the following command:
ps -ugx

(The ps command on various systems may need options different from -ugx to get the desired results; check with the consultants if you have questions.) This shows a list of your processes, like the one below (some columns have been left out to save space):

USER       PID %CPU STAT  START    TIME   COMMAND
janedoe   6393  7.9  R    18:05    0:00   ps -ugx
janedoe   6256  0.0  IW   Sep  2   0:02   -csh (csh)
janedoe   4659  0.4  S    Jan 3  210:00   /usr/usc/EQS/EQS

If you see something under COMMAND that has EQS in it, this means a process involving EQS has been running for the amount of time under TIME. If you don't want any EQS processes to be running, you should kill the process by issuing the following command:
kill -9 4659

which "kills" process number 4659. Please check your processes daily to guard against overuse of the machine by unintended processes. This is a courtesy to other users, besides being in your best interests.

References

EQS Structural Equations Program Manual, by Peter M. Bentler.
Multivariate Software, Inc.
15720 Ventura Boulevard, Suite 306
Encino CA 91436

800/301-4456
818/906-0740
818/906-8205 (fax)

<sales@mvsoft.com> === http://www.mvsoft.com/

This reference describes the EQS language and procedures for UNIX and PC applications, although specific usage in the UNIX environment is not addressed. The manual is available in the manual racks in the Public User Area in the basement of Waite Phillips Hall and in the Leavey Library Information Commons on the Ground Floor.

EQS manuals are not usually available at the USC bookstore. References must be obtained directly from the vendor.

USC Students, Faculty and Staff may use computers and documentation in the Public User Areas in Waite Phillips Hall, King Olympic Hall and Salvatori Computer Science Center, as well as the Leavey Library Information Commons (Ground Floor).

Consulting. The ITS Consultants may be familiar with the language and general operations of this software, but it may be necessary to make an appointment with a member of the full time staff in order to receive detailed help. Support of statistics software is the responsibility of the ITS Statistics Consultant with the participation of other full-time ITS staff. These people may be contacted through the ITS Customer Support Center as shown here.

Customer Support. USC students, staff or faculty who would like information about ITS Workshops or about obtaining site-licensed software or other computing-related questions should visit the Customer Support Center in Leavey Library Lower Commons, or call 213.740.5555, or send eMail to <consult@usc.edu>

Documentation. This document, and many others on a variety of topics, are available in the ITS Customer Support Help System, available on the World Wide Web at:

http://www.usc.edu/its/

You can find Statistics Software Help Documents through the search engine at this same URL, or go directly to them at:

http://www.usc.edu/its/doc/statistics/help/

Newsgroups. Another source of information, this one providing the opportunity to exchange thoughts with other users, is the newsgroup 'usc.comp.all.stat.users'. For more information about reading news, subscribing to newsgroups, and related topics, visit

  
http://www.usc.edu/its/doc/internet/news/