TTU Home IT Division HPCC Globus

Connecting via globus

Globus at http://www.globus.org, a grid computing software system, is running on the HPCC compute hosts. Currently Globus is only fully available for Unix/Linux computers, and the Globus software must be installed on both client and server computers. The initial installation is somewhat involved.

The Globus toolkit installation official home page is http://www.globus.org/.
Another good page is the NSF Middleware Initiative (a repackaged Globus) http://www.nsf-middleware.org.

Linux (Rocks cluster) installation notes are at http://www.sdsc.edu/~tkaiser/globus/build.

Irix installation notes are at http://www.lrz-muenchen.de/services/compute/globus/globus_install_20/globus_install_20-2.html.

Those installation notes are generally for Globus toolkit 2.0 rather than the current 2.2; refer to the www.globus.org home page in case of conflicts. The major installation steps are
(1) Download and install the software. If you want the Globus MPI software, MPICH-G2, you will need to download as source and compile. Otherwise you may download binaries. There is an automated installer; follow the directions exactly.
(2) Decide on a certificate authority. Since TTU does not have a local authority yet, you will need to request from Globus a user certificate for yourself. Some supercomputer centers such as SDSC also have certificate authorities; a file is installed in /etc/grid-security/ to specify whose authority the host will recognize. To generate your user certificate, do after setting environment variables (see (9) below)
grid-cert-request "George W. Bush"
You will supply a pass phrase to be used each time you run Globus. The Globus certificate request program saves files userkey.pem (don't mail) and usercert_request.pem , which should be mailed to the certificate authority. The Globus authority requires that you send the file from a email account traceable to the name you used; i.e. George.W.Bush@ttu.edu , not hotmail. They should reply with certificates in 1-2 working days. The user certificates are installed in your home under ~/.globus/ . The request usercert_request.pem , and the answer usercert.pem should have permissions 644. The key userkey.pem should have permission 600. Copy these files to same directory on each machine on which you will run Globus.

(3) You will also need a host certificate for each of your hosts running Globus. Run the host certificate request as root on each machine, i.e.
grid-cert-request -gatekeeper mycomp.alchemy.ttu.edu
with more arguments, see the web pages. Mail the file hostcert.req to Globus as above. The host certificates and keys are installed in /etc/grid-security/ owned by root with the permissions as above. Since some hosts do not have email clients, the email need not be traceable to the host. It is easiest to use the same email account at the same time as for the user certificates.

(4) Install the file /etc/grid-security/grid-mapfile which maps Globus certificates to local users. Each user is one line; an example line is
/O=Grid/O=Globus/OU=alchemy.ttu.edu/CN=George Bush gwbush
The string should reflect your domain as shown by grep OU usercert_request.pem . If you have an HPCC account and want to use Globus on HPCC computers, email the string above to mailto:David.Chaffin@ttu.edu.

(5) The Globus services also need to be added to /etc/services so that the host can start them from inetd/xinetd:
globus-gatekeeper 2119/tcp # Globus Gatekeeper
globus-gatekeeper 2119/udp # Globus Gatekeeper
gsiftp 2811/tcp

(6) On Linux systems running xinet, startup scripts need to be added to /etc/xinetd.d/ for globus-gatekeeper and gsiftp, and restart xinetd. See the linux cluster install notes for details.

(7) On systems with IP filtering, you will need to open at least the ports 2119 and 2811 shown above to all Globus hosts on your grid, and restart the filter (iptables, ipchains, ipfilterd). Probably other ports will be needed as well, to be determined by network monitoring while using Globus.

(8) You should now be ready to test. Try on your Globus computer:
telnet localhost 2119
If you get " connected to localhost ", your globus-gatekeeper is running. If you get
telnet: Unable to connect to remote host: Connection refused , either globus-gatekeeper is not running or IP filtering is not allowing the connection. If you can start globus-gatekeeper manually,
/export/home/local/globus/sbin/globus-gatekeeper -conf /etc/xinetd.d/globus-gatekeeper.conf
replacing with the correct directories, and the tests below work, you have a problem with inetd/xinetd. Manual startup is not recommended since there is no restart facility without inetd/xinetd..

(9) Once globus-gatekeeper is running, you may try to actually run a job. Try as yourself:
setenv GLOBUS_LOCATION /export/home/local/globus or export GLOBUS_LOCATION=/export/home/local/globus ,
depending on shell and replacing with the correct directory. Then
source ${GLOBUS_LOCATION}/etc/globus-user-env.csh or source ${GLOBUS_LOCATION}/etc/globus-user-env.sh , and
globusrun -o -r localhost '&(executable=/bin/date)'
If it returns the date, you have run a Globus job. Otherwise it should return an error code which may be helpful. If the localhost job works, try from another computer, say bigcomp.alchemy.ttu.edu running Globus back to your computer:
globusrun -o -r mycomp.alchemy.ttu.edu '&(executable=/bin/date)'
If it returns the date from mycomp.alchemy.ttu.edu , you have run a remote Globus job. If you are installed as a Globus user on both computers, you should also be able to run the same job in reverse from mycomp :
globusrun -o -r bigcomp.alchemy.ttu.edu '&(executable=/bin/date)'


(10) With two machines, you may similarly try to run a gsi-ftp file copy. Try as yourself:
globus-url-copy gsiftp://localhost:2811/home/gwbush/testfile gsiftp://localhost:2811/home/gwbush/testfile2

If there are problems, try telnet localhost 2811 as above. If the localhost copy runs correctly, try to copy to/from another machine:
globus-url-copy gsiftp://localhost:2811/home/gwbush/testfile gsiftp://bigcomp.alchemy.ttu.edu:2811/home/gwbush/testfile2
globus-url-copy gsiftp://bigcomp.alchemy.ttu.edu:2811/home/gwbush/testfile2 gsiftp://localhost:2811/home/gwbush/testfile3