previous episode

Intro to Linux

next episode

Remote Access to Other Systems

Overview

Teaching: 15 min
Exercises: 0 min
Questions

  • How can I interact with remote Linux servers?

Objectives

  • Explain how to log in to a remote Linux server, and how to copy data back and forth.

  • Explain hostnames as identifiers for computers on a network.

  • Access the cluster from the command line.

  • Copy data to and from a login node.

Now that we have learned how to use some of the basics of the shell, let’s take a look at how we can access MSI’s Linux environment remotely. There are two main commands that are used for remote access, which are used for logging in to a remote server and for copying data to and from a remote server, respectively.

You can even access our systems from your own personal or lab computer. Linux and Mac OS X computers can use the built-in terminal applications to connect, while Windows users will need to install either a terminal emulator application like Cygwin https://www.cygwin.com, or a purpose-built connection tool like putty https://www.chiark.greenend.org.uk/~sgtatham/putty/. Both of these solutions on Windows have some difficulty with graphical applications, so you may prefer using MSI’s NICE desktop environment instead https://www.msi.umn.edu/support/faq/how-do-i-obtain-graphical-connection-using-nice-system.

Linux systems are identified on a network by their hostname and/or their IP address. You need to know either the hostname, web address, or the IP address of the remote system that you want to connect to. At MSI, you will mostly be working with hostnames and shortcuts for hostnames that we have created. Let’s consider the hostname agate.msi.umn.edu. We can check if a host by that name exists and is configured to communicate with us using the ping command:

$ ping agate.msi.umn.edu

$ ping agate.msi.umn.edu
PING agate.msi.umn.edu (128.101.189.225): 56 data bytes
64 bytes from 128.101.189.225: icmp_seq=0 ttl=56 time=1.737 ms
64 bytes from 128.101.189.225: icmp_seq=1 ttl=56 time=4.592 ms
64 bytes from 128.101.189.225: icmp_seq=2 ttl=56 time=17.480 ms

If the hostname we are pinging exists, then you should see lines of output being printed to your screen. You can stop ping using Ctrl-C to cancel the command. The output specifies how large the test transmission was in bytes, the IP address of agate.msi.umn.edu, some specifics of the connection, and how long it took for agate.msi.umn.edu to respond. This is the output we expect to see if a host exists and is configured to communicate with us. If we try a hostname that doesn’t exist, we will see an error:

$ ping fake.msi.umn.edu

ping: cannot resolve fake.msi.umn.edu: Unknown host

Seeing this error means that either we have the hostname wrong, or that we are unable to connect to the hostname for some reason. You might see this error if you are trying to connect to MSI from outside of the UMN VPN, for instance. For this reason, ping can be a useful tool for orienting ourselves if we want to know why a connection fails.

Once we know that we have a valid hostname that we are able to connect to, we can log in to that server using ssh, which stands for ‘secure shell’. This tool makes an encrypted connection to the remote server that you can use to run commands there.

$ ssh dunn0404@agate.msi.umn.edu

dunn0404@agate.msi.umn.edu's password:

If you have entered a valid hostname you will then be prompted for your password. When you type in this prompt, the password is not visible on the screen and is not logged in the bash history. If you have made a typo in your username or are trying to connect to a server where you don’t have an account, your password will be rejected. Otherwise, you will be dropped into a shell that is running on the remote host:

Unauthorized access to the system is prohibited and subject to University
and/or Federal or State legal actions.

By proceeding you acknowledge, consent, and agree to the following:
  * Acknowledge: You are authorized to access this system.
  * Consent: Your use of the system may be monitored, recorded, and is
    subject to audit.
  * Agree: By using this system you agree to adhere to MSI and UMN
    Acceptable Use Policies - see https://policy.umn.edu/it/itresources

IF YOU DO NOT MEET THE CRITERIA STATED ABOVE, DISCONNECT IMMEDIATELY.
-------------------------------------------------------------------------------
            University of Minnesota Supercomputing Institute
                                 Agate
                          AMD EPYC Linux Cluster
-------------------------------------------------------------------------------
For assistance please contact us at https://msi.umn.edu/helpdesk
help@msi.umn.edu, or (612)626-0802.
-------------------------------------------------------------------------------
Home directories are snapshot protected. If you accidentally delete a file in
your home directory, type "cd .snapshot" then "ls -lt" to list the snapshots
available in order from most recent to oldest.
-------------------------------------------------------------------------------
[16:14:17 PM] [msistaff:dunn0404@ahl04:~]$

Many of MSI’s systems will present a message similar to the above when you log in, but some systems may not. In those cases, you can verify that you have been logged into the expected system with the hostname command:

$ hostname

ahl04

Once you have successfully logged in, you have access to your data and settings on the remote server, and you can use the shell to interact with it as if you were working directly on that machine.

One exception to this is if you want to run an application with a graphical user interface (GUI). If we try to run xclock, an application that opens a window with a graphical clock, we will get an error message:

$ xclock

$ Error: Can't open display:

We are seeing this error because ssh is not configured to work with graphical applications by default. Historically, it has been used primarily for purely command-line-driven applications. However, we can tell ssh that we would like to enable graphical applications over the current connection. First, let’s exit from the current session:

$ exit

This closes the ssh session and drops us back into our local machine. Now, we can use a flag with the ssh command to enable graphical applications:

$ ssh -Y dunn0404@agate.msi.umn.edu

We will again be prompted to enter our passwords, and we should see the same welcome message as before. Now, however, we can run graphical applications like xclock:

$ xclock

Now we should see a new window open with an animated clock. This method of using graphics over ssh is known as ‘X-forwarding’, named after the X Window Server that Linux uses to draw windows, and how we are ‘forwarding’ windows from the remote machine to the local one over the ssh connection.

You can also use ssh to run commands on the remote machine without entering the remote shell session in your terminal. This can be useful in cases where you need some information from the remote machine. For instance, I could use this command to query my home directory on the cluster:

$ ssh dunn0404@agate.msi.umn.edu 'pwd'

/home/msistaff/dunn0404

You can also string together multiple commands using semicolons:

$ ssh dunn0404@agate.msi.umn.edu 'pwd; hostname'

/users/3/dunn0404
ahl04

In the event that we need to transfer files rather than the output of basic commands, we will want to use the scp command, which stands for ‘secure copy’. It works similarly to the cp tool we are already familiar with, but requires us to specify a username and hostname as part of the destination. Let’s create a file in a local directory:

$ touch /tmp/dunn0404_new_file.txt

This will create an empty file named my_new_file.txt in the local directory /tmp. At MSI most of your storage is available from any system you log into, including your home directory, your group’s home directory, and /scratch.global. Other directories, like /tmp and /scratch.local are only available on the current host, so you will want to copy data you may produce in those locations back to a globally accessible location to avoid losing data. To copy the file we just created into our home directory, we can use:

$ scp /tmp/dunn0404_new_file.txt dunn0404@agate.msi.umn.edu:~/

Similar to cp, we must pass -r in order to copy a directory and its contents:

$ mkdir /tmp/dunn0404_dir
$ touch /tmp/dunn0404_dir/file.txt
$ scp -r /tmp/dunn0404_dir dunn0404@agate.msi.umn.edu:/home/msistaff/dunn0404

Using scp to move data can take a long time to run depending on your connection speed and on the size and number of files you are moving. For especially large transfers, you may want to consider using Globus https://www.msi.umn.edu/support/faq/how-do-i-use-globus-transfer-data-msi-0. Globus is a file transfer service with a graphical interface that you can access from the web that specializes in large file transfers.

Key Points

  • ssh starts a secure shell session on a remote server.

  • ssh -Y starts a secure shell session with graphics.

  • ‘ssh’ can also be used to directly send commands.

  • scp copies files and directories to and from remote servers

  • You can use ‘scp’ and ‘ssh’ from the Terminal in Mac OS X

  • Tools like cygwin and putty allow remote connections from Windows

previous episode

next episode