LIMDEP software under UNIX is installed on specific machines in the Research Computing Facility (RCF) and Student Computing Facility (SCF) clusters of ITS machines. Users must have an account on a machine in these clusters in order to use LIMDEP or any of the other statistics packages. Such accounts are available from the ITS Computer Accounts Office in LVL Lower Commons (213/740-5555).

To invoke LIMDEP under UNIX, type limdep at the UNIX prompt. In order for this command to function, your environment needs to be set up to find the command and any other files or libraries the software needs. In the RCF, this is done automatically when you login, allowing you to access all the statistics software by simply typing the invocation command.

In the SCF, users of this (or any) statistics software need to add the following lines to their .login file:

if  (-e  /usr/usc/stats/setup.csh)  then
      source  /usr/usc/stats/setup.csh
endif

SCF and RCF users whose .login or .cshrc files are non-standard or who once had setup files for one or more specific statistics packages sourced in .login or .cshrc may need to fix their .login or .cshrc files to eliminate redundancy and/or to allow the system's automatic setup to take effect. For example, users on the RCF may have modified their .login or .cshrc to such an extent that the system's automatic setup is bypassed. These users can either restore the system's automatic setups (recommended) or add the lines shown above (which are usually only necessary for SCF users). Please call UNIX consulting (see last page of this document) for help with these files.

At the end of this document, there is a list of LIMDEP manuals that are available from Econometric Software, Inc.

There is no online help within LIMDEP.

LIMDEP uses commands, which start with a "verb" and often require further "specifications". Each command and specification sequence ends with a "$". LIMDEP programs may typed in upper or lower case, as all words are converted to upper case except filenames. Since UNIX is case-sensitive, filenames must be typed exactly as they appear in the UNIX directory listing.

Many commands have what LIMDEP calls "specifications". The delimiter between commands and specifications is a semicolon (;). Specifications may be typed on the same line as commands, or the line may be broken at any point and continued on the next line. If you forget to end the last subcommand with a '$', simply type the '$' on its own line and continue. If you are working interactively and you wish to abort a particular command and start it over, type !! and you will be returned to the Cmnd> prompt.

Comments may be added to LIMDEP verbs by typing beyond the minimum four characters before you type a semicolon. Hence, using the STAT command, for example, you can add a comment by typing STATUS OF THE PROGRAM before typing the semicolon delimiter. You can also add comments by typing after the $ on any line that has one. The $ ends a command/specification sequence, and anything typed on a line after a dollar sign is ignored by the LIMDEP processor.

To exit LIMDEP when in interactive mode, type the command STOP$ at the Cmnd> prompt. In batch mode (i.e., when running LIMDEP commands from a file), it is not necessary to type STOP at the end of the commands.

LIMDEP sometimes in the past has had trouble reading raw data. Some or all of these problems may be fixed in the current versions; however, if you have tried reading your data and you get errors, check to see if one or both of the following techniques can save you. First, truncate (or round off) all numbers to three decimal places or less. For example, 3.98452 might become 3.984, or 4253.8844323 might become 4253.88.

Another possible cause of data reading errors in LIMDEP is data that are not arranged in columns, with Fortran formats specified. If you suspect this might be the cause of your problems, edit your raw data file so that all numbers line up in columns, then tell LIMDEP what the format of those columns is. For example, the following arrangement

78.42 8.5
943.674 43.1
8.4 883.784


might cause problems, but rearranging the data as follows

 78.42      8.5
943.674    43.1
  8.4     883.784

and adding the specification

format=(2F8.3)

to the READ statement might fix the problem.

Note that the techniques noted here might not be necessary, so don't think of them as limitations unless you run into problems with LIMDEP not reading your data properly. Also, if you have a large amount of data, you might want to modify only a few observations then run a test on that small subset, to see if the technique you are using is going to work before you spend the time and energy to modify everything.

The LIMDEP interactive session is begun by entering the command

limdep

which gives you some startup messages followed by the prompt:

Cmnd>

LIMDEP commands are entered at this prompt, as shown in the LIMDEP manual, and results are shown on the screen as commands are entered. If you are entering data at your terminal (as opposed to reading data from an external file, as discussed below), after you enter the DATA command at the Cmnd> prompt (which invokes the Data Editor), LIMDEP will prompt you for entry of the data.

The following dialogue illustrates a sample LIMDEP interactive session in which data are entered at the terminal.

you:   limdep

crt:   <type start to begin INTERACTIVE execution>

you:   start

crt:   L I M D E P logo, including:


       (C) Econometric Software, Inc.   Version 7.0
           1985-1995                    Mainframe  

       Cmnd>

LIMDEP is now ready to accept your data. In this example you will enter your data case by case. LIMDEP will store the cases as rows and variables as columns. The data from an experiment are listed below:

         VARIABLE #1      VARIABLE #2
CASE #1     2.3               4.5
CASE #2     3.1               2.2
CASE #3      .96             34.5
CASE #4     1                 4

Enter the above data as illustrated below. When you type data at the Cmnd> prompt, the Data Editor is entered. When you are finished with the Data Editor, type a period in column one. "crt" shows what you will see on your screen.

crt:   Cmnd>

you:   data

crt:   You may enter up to 5 variables at a time.
       Variables may be new or existing. (Existing
       variables will be replaced by this command.)
       Enter the names, separated by commas.
       Variables? (.=Exit)>

you:   var1,var2

crt:   Enter each row on a single line. Separate numbers on
       the line with commas.  For missing data, enter the
       value -999.  When finished, enter a line containing
       only a period in the first column.

           VAR1         VAR2
       1  >

you:   2.3,4.5

crt:       VAR1         VAR2
       2  >

you:   3.1,2.2

crt:       VAR1         VAR2
       3  >

you:   .96,34.5

crt:       VAR1         VAR2
       4  >

you:   1,4

crt:       VAR1         VAR2
       5  >

you:   .

Commands for statistical procedures are entered at the Cmnd> prompt. For example, to see descriptive statistics on the data entered above, type:

  crt:   Cmnd>

  you:   dstat;rhs=var1,var2 $

  crt:              Descriptive Statistics

  Variable Mean Std. Dev. Skewness Kurtosis Minimum Maximum Cases

  VAR1   1.8400  1.0455    0.240     1.02    0.9600  3.100    4
  VAR2   11.300  15.498    0.988     1.74    2.200   34.50    4

  Press a key then ENTER to continue.>

  crt:   Cmnd>

  you:   stop$

The LIMDEP author has provided a number of sample programs which illustrate various operations of LIMDEP. These are found in

/usr/usc/limdep/default/examples

Some of the sample programs use other files in the examples directory, so you may need to browse the example file you wish to run. Others require modification before running, as shown in comments at the top of each sample file. When you browse the file, if you find any references to external files, copy the example file and all the other relevant files to a directory of your own, and run the sample from there. If modifications are indicated by instructions at the top of the file, copy it to a directory of your own and make the necessary changes.

For example, if you wish to run "sample.lim" (found in the abovementioned "examples" subdirectory), type

cp  /usr/usc/limdep/default/examples/sample.lim  ~/sample.lim

No modifications or external files are necessary to run this sample. Once it is copied, you can run it as follows:
1. Invoke LIMDEP by typing limdep at your UNIX prompt.
2. At the Cmnd> prompt, type open; you will see a menu.
3. Choose "2" to open an output file. (The output will scroll by your screen too fast to read, so you will want the output in a file.) Enter a file name, for example "sample.output".
4. When you see the OPEN menu again, choose "1" to open the input file. Enter "sample.lim". Your output will appear on the screen, and will be saved into "sample.output" for your inspection.

In interactive LIMDEP sessions, it is possible to use information saved in UNIX files, or to direct information to UNIX files. In general, (except for raw data) this is done with the OPEN command. Type OPEN at the Cmnd> prompt, and you will see a menu such as,
*******************************************************
* Enter the number of the type of file to be opened
* 0 <---> None, return to command level
* 1 <---> Input (command) file
* 2 <---> Output file
* 3 <---> Save file for intermediate results
*******************************************************

When you make your choice (by number) from the menu, you will be prompted to "enter the file name or unit number", at which point you should write the name of the file (as it should be listed in the UNIX directory), without a $ at the end. (Unit numbers, as mentioned in the prompt, are more appropriate for IBM Mainframe LIMDEP, and should not be used in UNIX.)

Apparently, the current version of LIMDEP has fixed some previous limitations that restricted the flexibility with which files could be named. For example, names (with paths) longer than 40 characters were not recognized in previous versions, and the tilde-slash (~/) convention for "home directory" also was not recognized. These things should no longer be a problem; however, if you notice errors that may be related to these issues, it is recommended that you locate all the files used and created in a LIMDEP session or program in the same directory from which LIMDEP has been invoked. This will keep file names under 40 characters, and avoid the necessity to use the tilde-slash (~/) characters.

You can open external files without the menus by typing the OPEN command followed by the appropriate specification, as shown here:

OPEN; OUTPUT = filename $
OPEN; INPUT = filename $
OPEN; SAVE = filename $


For example, if you have a program already in a UNIX file called OPENing the file as follows:

OPEN; INPUT = myfirst.prog $

The output will appear on your screen and will not be saved anywhere, unless you have entered a separate OPEN command for OUTPUT, such as

OPEN; OUTPUT = myfirst.output $

(Note that the OPEN commands must be separate; you cannot have both an OUTPUT and an INPUT specification on the same OPEN command.)

If an "OPEN; OUTPUT=" command is in effect, the output will be sent to the specified file and will also appear on the screen. Once an OUTPUT file has been OPENed, it will continue to receive the output from your session until you type STOP$ and end the session, or until you type CLOSE$ at the Cmnd> prompt.

At any point in any session, you can permanently store all of LIMDEP's active memory, tables, data matrices, etc., into a file with the SAVE command, such as:

SAVE; FILE = session5.limdep $

This is the same as the older LIMDEP's DUMP command, which is an alias. (Notice, however, that this is NOT the same as "OPEN; SAVE=" which was mentioned above. See the LIMDEP manual for details.) Previously SAVEd (DUMPed) "sessions" can be recalled by typing

LOAD; FILE= session5.limdep $

which allows you to continue where you left off in your previous session.

During a LIMDEP program (either batch or interactive) you may specify an external file containing raw data, instead of typing the data "instream" with the other LIMDEP commands, or entering the data in the interactive Data Editor. The external file must be encoded in ascii and must not have variable names or labels, only data values (either numeric or alphanumeric). Also, as noted above in "Miscellaneous Information Regarding LIMDEP", it is best to truncate raw data values to three decimal points, and some versions of LIMDEP require the data to be in columns with Fortran formats specified.

If you want LIMDEP to read an external file containing raw data, add the name of the file to the READ command, and specify the number of variables (nvar=), the number of observations (nobs=), and the names of the variables (names=). Say you had the data from the examples above stored in a file called 'my.rawdata'; your READ command would be:

read; nvar=2; nobs=4; names=var1,var2; file=my.rawdata$

If files are in the "current" directory (i.e., the directory where you have invoked LIMDEP) there is no need to specify any path or subdirectory names.

Once the output from your program is stored in a file, you can print that file in any way you print other text files. You can download it to a Macintosh or PC and print it there, or you can submit it to a printer known to UNIX and print it here on campus. If the printer is controlled by ITS in a Public User Room there will be a per-page charge for picking up the printout.

To see a list of printers known to UNIX, type printers at your UNIX prompt. The printer "code" will be shown in the leftmost column.

To print a file on one of the UNIX printers, use the lpr command with the -P switch, as in the following example, which prints "my.file" to the printer in UCC 101 (the UCC Distribution Area):

lpr -Pps_ucc101 my.file

NOTE: ps_ucc101 is used only as an example; this probably is not the most convenient printer for most users.

Some printers are configured for PostScript only, and will not accept text-file input from the lpr command. In this case the enscript command (which converts text to PostScript) should be used instead, as shown here:

enscript -Pps_ucc101 my.file

To print output that is more than 80 characters wide, many printers will require that you rotate the orientation of the page to "landscape" (the default usually is "portrait") and often you will also need to reduce the font size to get all 132 columns on the page without wrapping. This can be accomplished with the enscript command followed by the -r (rotate) option and the -f (font specification) option. One font that has been tested for wide output is Courier9, though others may work as well. The following will print wide output (132 characters per line) without any wrapping or truncation:

enscript -Pps_ucc101 -r -fCourier9 file.name

You can also download your output (or any file) to a PC or Mac, and print on a local printer, if you prefer. The consultants have documentation on how to download files.

Since UNIX users have quotas, it is possible to run out of disk space when processing large data sets. To avoid running out of disk space, users may request quota increases when needs arise. If you believe your circumstances require more disk space, contact the authorized signer in your department for information and permission. If you don't know who your authorized signer is, the ITS Computer Accounts office in CAL 323-4 (213.740.5555) can help you.

/tmp Space

Quite often, large data sets are needed only for short periods of time while analyses are done or subsets are extracted. For these situations, the best solution often is to make use of the temporary space in RCF and SCF machines that is set aside for all users. SCF users will have quotas on /tmp space, similar to their own disk file quotas, so this may not solve SCF users' problems completely. RCF users also have /tmp space quotas, and can request increases from ITS as necessary. The space is in a subdirectory called /tmp (and, on some systems, /tmp1, and so forth; enter ls -ld /tm* to see them all). For descriptions and policies related to shared /tmp space, scroll to Section 7 in the following page:
http://www.usc.edu/its/policies/procedures/

Anyone can write in the shared /tmp spaces, create subdirectories in them, and in general manipulate files as you would in your home directory. You should be aware that files stored in /tmp are not permanent, and will be deleted by the system on the regular /tmp cleanup schedule. This schedule is different for different machines and /tmp areas, but the bottom line is that your files will go away.

Compressing Large Files to Save Space

Users with large amounts of data can make use of compression techniques when data are stored for periods of more than a few days without being used. UNIX allows users to compress their files, then uncompress them when they need to be used, allowing a substantial savings of space both in the user's own subdirectory structure and in the common areas (such as /tmp) if they are used. Users are urged to compress any files larger than 20 Mb if they are not being used for long periods of time. The amount of time it takes to compress or uncompress files is minuscule, and much space will be saved by this operation.

To compress a file in UNIX, use 'compress'; to uncompress a compressed file, use 'uncompress' or 'zcat'. In the following examples we compress a file called test1.bigfile, which results in a compressed file called test1.bigfile.Z:

compress test1.bigfile

Either of these commands results in an uncompressed file:

uncompress test1.bigfile.Z
zcat test1.bigfile.Z > test1.bigfile

The 'zcat' command, shown above, normally is used for viewing files, but with the use of the redirect, can re-create files from their compressed versions. While the 'compress' and 'uncompress' commands use the previous names (with the addition or deletion of the .Z extension), 'zcat' can redirect the compressed file into an uncompressed file of any name.

When large numbers of users are running jobs on RCF or SCF machines, it may be advisable not to try large jobs that are I/O or CPU intensive, since the portions of time allocated to each user are, obviously, much smaller than at times of low usage. The longer a job takes, the lower priority it often receives. This can result in jobs that may take hours of real time in the middle of the day, when they might only take minutes at a less busy time.

There are several useful techniques for dealing with large jobs that take a lot of time. You can run your jobs in batch mode, rather than interactive. This avoids using any memory that would be necessary for interactive mode. You can run your batch job in the background (by placing an ampersand [&] at the end of the command-line invocation), which allows you to logout while the job finishes, if you wish. (Running a job in the background also allows you to avoid timeout problems associated with long periods of terminal inactivity when you are connected via a micom line or modem.) And you can queue your job to run at a later time, when the system is likely not to be so busy.

Batch Processing

To run LIMDEP in batch mode, enter your LIMDEP program into a file, make sure the first line of the file reads "BATCH$", then issue the appropriate command followed by a redirect symbol (<), the name of the batch file, another redirect (>) and the name of a file to contain the results. For example,

limdep <  myinput.file  >  myoutput.file

To run this batch program in the background, simply put an ampersand (&) after the file specification:

limdep <  myinput.file  >  myoutput.file &

When you have done this, you can perform other tasks in the foreground, and if you wish, you can logout and your job will continue to completion.

If you prefer, you may use the LIMDEP program itself to open an output file. This is done with the OUTPUT command. See below, and consult the LIMDEP documentation for details.

Following is a sample you can use to test batch LIMDEP:

BATCH$
read;  nvar=2
  ;  nrec=4 $
2.3 4.5
3.1 2.2
.96 34.5
1 4
dstat;rhs=x1,x2 $
stop$

Delayed Batch Job Processing -- the  at  and  qsub  Commands

Any user can queue a job to run at a later time. Users running particularly large, compute-intensive, and/or long-running jobs are required to queue their jobs using the Portable Batch System (PBS) that makes use of rcf-01. For more information, see below, as well as our online document "Portable Batch System (PBS)":
www.usc.edu/hpcc/systems/use-l-4.php

To queue your job to run at a later time, you can use the  at  UNIX command. This requires that you place your invocation command and all other specifications (such as the filename of the program and any options or switches) into an executable shellscript file, and then specify the executable in an  at  command, such as

at 2:30a < execthis

where  execthis  is an executable shellscript you have created. (Of course, the name  execthis  is arbitrary, and can be any filename you choose.)

Let's say you have a statistics package called THYSTAT (for those who take documentation literally, note that this is a mythical package name, for illustration only), and your THYSTAT program is contained in  myprog.thystat1  which you keep in your home directory (~/) in a subdirectory called  programs. Part of the peculiar syntax of THYSTAT is that before the batch file name, you use a switch spelled  -in. For whatever reason, you wish to run your program late at night. To make an executable shellscript to run your batch program, enter the following into a file called  execthis :

#!/bin/sh -f
thystat -in ~/programs/myprog.thystat1

Then mark this file as an executable shellscript by typing the following at your UNIX prompt:
chmod u+x execthis

You can then run the  at  command, as shown above, and specify any time you feel would be best to run your job. For more details on the  at  and  chmod  commands, see the man pages or call the consultants.

The  qsub  Command and the PBS System

Multiple batch programs can be entered into a queue and run at a later time, allowing the user to 'submit' any number of jobs and still meet the system's limitation of one compute-bound job running at any given time. This method is required for jobs that are compute-intensive and/or will need multiple processors or extensive system resources. This type of queueing is done with the  qsub  command, which sends a request to the Portable Batch System (PBS).

RCF users normally can use the  qsub  command automatically. If it is not immediately available, run

source /usr/usc/pbs/default/setup.csh

Once this is set up, you can type  man pbs  and  man qsub  for complete online help information. Another helpful man page is  man pbs_resources_linux  which details the resource options (e.g., walltime, cput [cpu time], etc.) available for the  qsub  command.

The  qsub  command itself runs a script file, and inside the script file is the batch invocation of the software you wish to run (along with any other appropriate UNIX commands.  qsub  also writes to a LOG file that you specify, so you can see the STDIN and STDOUT messages associated with the running of your queued job. The general form of the  qsub  command is

qsub <flags> SCRIPTFILE

Once again, as in the example above, let's say you have a statistics package called THYSTAT (this is just a mythical example, to be substituted with whatever real command name and syntax you want to queue) and your THYSTAT program is contained in  myprog.thystat1  which you keep in your home directory (~/) in a subdirectory called  programs. Part of the peculiar syntax of THYSTAT, let's say, is that before the batch file name, you use a switch spelled  -in. Normally, you would run it in batch as follows:

thystat -in ~/programs/myprog.thystat1

In order to queue it with  qsub  and have the system run it later, you would create a qsub-appropriate SCRIPTFILE containing at least one line with a pound symbol ( # ) followed by the invocation of your software program, as shown above. Let's say you decide to put this into a file called  my.qsub.script. The contents of that file would be

#!/bin/csh -- a line beginning with # must begin the script file 
thystat  -in  ~/programs/myprog.thystat1

(Once again, remember that thystat is a bogus name, for example purposes only, and needs to be substituted with a real command name and syntax of your choice.)

The following command would queue your THYSTAT program for execution at a later time chosen by the system.

qsub -m be -k eo my.qsub.script

where  -m be  asks NQS to broadcast a message (i.e., to tell you) when the request ends execution;  -k eo  tells qsub to keep the STDERR and STDOUT output. A number of other flags are available, as listed in  man qsub  (remember to  source  the pbs setup file as shown above).

Many flags can be incorporated into the script file itself. For example, if you want your program always to report the results to your home directory, the -k eo flag may be entered into the file. Also, directions as to how much time to allow (walltime), how many nodes to use (nodes), and how many processors to use (ppn). If you want your job to run on the main rcf-01 processors, you can specify that in a separate nodes instruction (as shown in the example). Following is a sample (and simple!) script file that includes these requests. (The ls command is used as an example of a simple program that returns a detailed listing of the files in your home directory.)

#!/bin/csh -- any text here

#PBS -k eo
#PBS -l walltime=2:00:00
#PBS -l nodes=rcf-01
#PBS -l nodes=1:ppn=1

# run the program
ls -la ~/
exit

IMPORTANT NOTES:

• The syntax for the walltime value is hh:mm:ss (hours, minutes, seconds)
• The value for nodes= must always be 1 when submitting jobs to rcf-01, as the facility is configured as a single node.
• If explicit paths are used for commands or file specifications in the script file, /export and /auto must not be used. Instead, use the form /home/rcf-nn/username, where nn is the number of the file system partition where the files are stored, and username is the username of the home directory where the files are stored.

Often the  qsub  command will execute within a few minutes, though in some cases it could be delayed until the system is relatively unbusy at night.

1. LIMDEP User's Manual, Version 7.0 (revised and expanded), Greene, 1998
2. LIMDEP Abridged User's Manual, Version 6.0, Greene, 1992 (not published for Version 7)
3. LIMDEP Reference Guide, Version 7.0 (quick reference), Greene, 1995
   Econometric Software / William H. Greene
   15 Gloria Place
   Plainview, NY 11803
   
   516/938-5254 (voice)
   516/938-2441 (fax)
   
   <support@limdep.com> === http://www.limdep.com/

The References listed above can be found in the manual racks in the Public User Area in the basement of Waite Phillips Hall and on the Reference shelves in the Lower Level Information Commons in Leavey Library.

These References are not normally available at the USC Bookstore. All references must be obtained directly from the vendor.

USC Students, Faculty and Staff may use computers and documentation in the Public User Areas in Waite Phillips Hall, King Olympic Hall and Salvatori Computer Science Center, as well as the Leavey Library Information Commons (Ground Floor).

Consulting. The ITS Consultants may be familiar with the language and general operations of this software, but it may be necessary to make an appointment with a member of the full time staff in order to receive detailed help. Support of statistics software is the responsibility of the ITS Statistics Consultant with the participation of other full-time ITS staff. These people may be contacted through the ITS Customer Support Center as shown here.

Customer Support. USC students, staff or faculty who would like information about ITS Workshops or about obtaining site-licensed software or other computing-related questions should visit the Customer Support Center in Leavey Library Lower Commons, or call 213.740.5555, or send eMail to <consult@usc.edu>

Documentation. This document, and many others on a variety of topics, are available in the ITS Customer Support Help System, available on the World Wide Web at:

http://www.usc.edu/its/

You can find Statistics Software Help Documents through the search engine at this same URL, or go directly to them at:

http://www.usc.edu/its/doc/statistics/help/

Newsgroups. Another source of information, this one providing the opportunity to exchange thoughts with other users, is the newsgroup 'usc.comp.all.stat.users'. For more information about reading news, subscribing to newsgroups, and related topics, visit

  
http://www.usc.edu/its/doc/internet/news/