Skip to main content

Latest Tweets

Running a Jupyter notebook (ipython kernel) on a compute node

On the typical desktop system, you will simply run a Jupyter notebook by typing

jupyter notebook

in your shell, and your browser will pop up. This doesn't work in a compute cluster, where the browser will be on the head node, and the computation will occur on a compute node. This quick tutorial will show you how to get it to work. Much of this information is drawn from http://jupyter-notebook.readthedocs.io/en/latest/public_server.html

Creating a jupyter notebook config

In order to get this to work, you will need a non-default jupyter config file:

jupyter notebook --generate-config

Now edit the config file at   ~/.jupyter/jupyter_notebook_config.py (note: the ".jupyter" directory is hidden by default, but you'll figure it out...), and uncomment/change the following lines:

c.NotebookApp.ip = '*'
c.NotebookApp.open_browser = False 
c.NotebookApp.port = NNNNN

where NNN should be a number that only you know (larger than 8888, which would be the default port). Note: this is INSECURE, if you want to prevent other folks from connecting to your instance of Jupyter, follow ALL the guidelines at http://jupyter-notebook.readthedocs.io/en/latest/public_server.html, in particular the part about setting a password.

Launching the jupyter notebook compute job

Use the following qsub file, adjusted to your needs (in particular the duration and number of CPUs). Take note of the job ID.

Connect to your notebook instance

You can now use the browser (Menu -> Applications -> Internet -> Firefox) to connect to your instance, but where? Run the following command:

>qstat -f JJJJJ | grep exec_host
   exec_host = compute-0-K/0

where JJJJJ is the job ID you took note of (if not, check with qstat, and find your job). Of course, your exec_host will vary from run to run. You should now connect to something like

http://compute-0-K:NNNN

where NNNN is the number you configured above. Success is yours:

selection_300