Accessing Files
From Molecular Modeling Wiki
(Created page with "= Idea =") |
(→Notes) |
||
| (47 intermediate revisions not shown) | |||
| Line 1: | Line 1: | ||
| - | = | + | == Problem == |
| + | |||
| + | Frequently, some visualization of data (inputs, results) in user's $HOME directory is required and programs like '''molden''', '''VMD''', and others are often run on cluster servers to display images on the user's workstation through the X-Window system. However, this method has '''major disadvantages''' - it is slow, as a huge amount of graphical data is transferred through the network (especially when the object consisting of many atoms moves and rotates), and it often does not work at all (mostly because of incompatible versions of OpenGL libraries on the cluster server and user's workstation). The solution is to run visualization programs locally (on user's workstation) and bring data to the program, instead of bringing the program to data. For this, you do not need to '''copy''' data from server to your workstation - there is a way to '''embed''' (''mount'') your remote $HOME directory to your local file system. | ||
| + | |||
| + | == Sshfs == | ||
| + | |||
| + | One of the systems which allow to mount the remote directory to local filesystem is called ''sshfs''. To use it, follow these instructions (specific for Ubuntu, see also this [https://help.ubuntu.com/community/SSHFS howto]): | ||
| + | |||
| + | Install the sshfs package and add yourself to the ''fuse'' group (this has to be done only once) | ||
| + | |||
| + | <pre> | ||
| + | sudo apt-get install sshfs | ||
| + | sudo gpasswd -a $USER fuse | ||
| + | </pre> | ||
| + | |||
| + | Once you have added yourself to the ''fuse'' group, you should log out and log back in again for the change to take effect. | ||
| + | |||
| + | For every filesystem you want to mount do | ||
| + | |||
| + | <pre> | ||
| + | mkdir -p ~/mnt/SERVERNAME | ||
| + | sshfs -o idmap=user USERNAME@SERVERNAME:/home/USERNAME ~/mnt/SERVERNAME | ||
| + | </pre> | ||
| + | |||
| + | replacing SERVERNAME with the name of remote server (cluster) and USERNAME with your login name on this server (cluster). For example | ||
| + | |||
| + | <pre> | ||
| + | mkdir -p ~/mnt/zinc | ||
| + | sshfs -o idmap=user novak@zinc:/home/novak ~/mnt/zinc | ||
| + | </pre> | ||
| + | |||
| + | {{Note|This process can be repeated for as many cluster servers as needed.}} | ||
| + | {{Note|The directories (mountpoints) need to be created only once for every remote directory.}} | ||
| + | |||
| + | Once the remote directory is mounted, you can use it as if it was local, so, for example, you can call your local installation of '''VMD''' or '''molden''' and open a (remote) file in it browsing to ''~/mnt/SERVERNAME/...'' | ||
| + | |||
| + | To unmount (disconnect) the remote directory from your local filesystem call | ||
| + | |||
| + | <pre>fusermount -u ~/mnt/SERVERNAME</pre> | ||
| + | |||
| + | == Notes == | ||
| + | |||
| + | This method can also be used to mount cluster $HOME directory to your home computer. For this, however, opening the firewall with [[Access_from_the_Internet#Port_knocking_method|knock]] is needed before calling ''sshfs''. | ||
| + | |||
| + | The same ''sshfs'' functionality in Windows XP/7/8 can be achieved by open source software called [http://dokan-dev.net/en/download/ Dokan SSHFS] (credits to Lukasz Cwiklik). | ||
| + | |||
| + | [[Category:Clusters]] | ||
Latest revision as of 10:44, 29 January 2014
Problem
Frequently, some visualization of data (inputs, results) in user's $HOME directory is required and programs like molden, VMD, and others are often run on cluster servers to display images on the user's workstation through the X-Window system. However, this method has major disadvantages - it is slow, as a huge amount of graphical data is transferred through the network (especially when the object consisting of many atoms moves and rotates), and it often does not work at all (mostly because of incompatible versions of OpenGL libraries on the cluster server and user's workstation). The solution is to run visualization programs locally (on user's workstation) and bring data to the program, instead of bringing the program to data. For this, you do not need to copy data from server to your workstation - there is a way to embed (mount) your remote $HOME directory to your local file system.
Sshfs
One of the systems which allow to mount the remote directory to local filesystem is called sshfs. To use it, follow these instructions (specific for Ubuntu, see also this howto):
Install the sshfs package and add yourself to the fuse group (this has to be done only once)
sudo apt-get install sshfs sudo gpasswd -a $USER fuse
Once you have added yourself to the fuse group, you should log out and log back in again for the change to take effect.
For every filesystem you want to mount do
mkdir -p ~/mnt/SERVERNAME sshfs -o idmap=user USERNAME@SERVERNAME:/home/USERNAME ~/mnt/SERVERNAME
replacing SERVERNAME with the name of remote server (cluster) and USERNAME with your login name on this server (cluster). For example
mkdir -p ~/mnt/zinc sshfs -o idmap=user novak@zinc:/home/novak ~/mnt/zinc
 | Note: | This process can be repeated for as many cluster servers as needed. |
| Note: | The directories (mountpoints) need to be created only once for every remote directory. |
Once the remote directory is mounted, you can use it as if it was local, so, for example, you can call your local installation of VMD or molden and open a (remote) file in it browsing to ~/mnt/SERVERNAME/...
To unmount (disconnect) the remote directory from your local filesystem call
fusermount -u ~/mnt/SERVERNAME
Notes
This method can also be used to mount cluster $HOME directory to your home computer. For this, however, opening the firewall with knock is needed before calling sshfs.
The same sshfs functionality in Windows XP/7/8 can be achieved by open source software called Dokan SSHFS (credits to Lukasz Cwiklik).
