Skip to main content

Latest Tweets

Short tutorial on ECCO and SDS usage

Print Friendly

A short tutorial on how to use GNOME, NX, etc. on ECCO and SDS. It is assumed that you've already configured access to ECCO and SDS, respectively. If for some reason, the embedded videos below don't work, please view it via the Youtube playlist.

Useful first step

You will often use the Terminal window. Why don't we start by putting it in easy reach? This shows you how you can customize the GNOME interface to your liking.

Launching an application

First, for the most direct way of developing programs, start a statistical application from the menu, and run a program. This menu is specific to the VirtualRDC systems, and you won't find it on other systems, at the Census Bureau RDC or your own Ubuntu laptop.

Behind the scenes

We'll show you first how to launch the same application, but from an interactive terminal shell. This will start to show you how to put all the components together for a later custom qsub job.

 Limits

There are imposed limits to the number of interactive jobs you can run at any one time - in essence, one job only. But in general, how to you know how many jobs of your own and of other users of the  system are running at present? The command 'qstat' shows you in this example that you have the interactive job running, and the attempt to launch a second interactive job (initiated by the command 'iqsub', which you already ran from the menu) will wait (not fail) until the first job has finished. By the way, the command 'iqsub' is a custom command, used here and in some other places, that uses a specific call to 'qsub' behind the scenes. 'iqsub -h' will print out its short list of options.

Control

In the previous step, we used the command 'qstat' to see jobs. You can also use qstat to identify a job you would like to halt ("kill"), using the command 'qdel'

Longer running jobs

In order to balance the potential load on the system, interactive jobs are limited to a certain length (check the cluster-specific queue page, typically 1-2 hours). and may otherwise also be limited (memory, number of CPUs). You may want to run longer jobs that use more CPUs and memory. You will do this using batch submissions via qsub. The following video shows you how to edit a Stata job, and then submit it to a non-interactive queue from the command line.

In the example, the only difference with an interactive job is the default length of the job being submitted - it is not subject to the same limitations as the interactive queue. But suppose your job is expected to run for 24 hours, using 4 CPUs. Two alternatives are available:

  1. Use the 'qstata' command options. 'qstata' and its equivalent SAS, Matlab, R, etc. functions, will print out there usage when used without arguments. They will highlight the use of "chunks" - you can request additional chunks by adding a number to the end of the command line. For instance, to request 4 CPUs, you would request 2 chunks: qstata test.do 2 (at the time of writing this, it is not possible to modify the number of hours for the job with qstata)
  2. Write a custom qsub command (see the qsub page here).This allows you to change a large number of parameters, including the length of time a job reserves. You then submit a custom qsub job with qsub test.qsub.