VI-SEEM iRODS installation and operation

From VI-SEEM Wiki
Jump to: navigation, search

This short guide will cover the installation and operation of iRODS with GridFTP support.

Installation

The installation at IPB consists of an iCAT server at irods.ipb.ac.rs and a GridFTP server with iRODS DSI, and a resource server at gftp.ipb.ac.rs

In this setup iRODS systems are used for actual storage, and GridFTP is just used as an access interface for clients that are for some reason limited to GridFTP.

B2STAGE DSI is installed into the GridFTP to provide access to iRODS.

The following packages are required for this installation:

  • on the iCAT host machine:
    • cmake
    • globus-gridft-server-progs globus-gass-copy-progs
    • libglobus-common-dev libglobus-gridftp-server-dev libglobus-gridmap-callout-error-dev
    • libcurl4-openssl-dev
    • git
    • g++
    • dpkg-dev
    • cdbs
    • irods-dev
    • irods-runtime
  • on the client machine:
    • globus-gass-copy-progs


iRODS installation

The iRODS installation consists of the iCAT server (which uses postgresql database to store metadata) and a resource server that provides file storage. The installation follows the standard procedure described in the iRODS documentation.

Resource server is optionally installed on a different machine, and added as a resource to the iCAT server:

iadmin mkresc ipbResc passthru
iadmin mkresc ipbres1 unixfilesystem resource1.ipb.ac.rs:/var/lib/irods/iRODS/Vault
iadmin addchildresc ipbResc ipbres1


GridFTP installation

GridFTP is installed on the same machine as the iCAT server. It is installed from the standard repositories, which on debian based system would look like the following:

sudo apt-get install globus-gridftp-server-progs \
                        globus-gass-copy-progs \
                        libglobus-common-dev \
                        libglobus-gridftp-server-dev \
                        libglobus-gridmap-callout-error-dev

The valid host certificate and key should be installed at /etc/grid-security directory and owned by root if the server is run by root.

If you are using a local grid map file, create file /etc/grid-security/grid-mapfileand add the mapping of DN to the iRODS user.

"/DC=org/DC=example/DC=grid/CN=Some User" username


B2STAGE installation

B2STAGE dsi has to be installed from the source. To clone the repository, use the following command:

git clone https://github.com/EUDAT-B2STAGE/B2STAGE-GridFTP.git

After that, create the target directories for the installation. In our case, /opt/b2stage/iRODS_DSI was used.

mkdir -p /opt/b2stage/iRODS_DSI
chmod 777 /opt/b2stage/iRODS_DSI

Then change into B2STAGE-GridFTP project directory and copy the setup.sh.template into setup.sh and make the following changes:

export GLOBUS_LOCATION="/usr"
export IRODS_PATH="/usr"
export DEST_LIB_DIR="/opt/b2stage/iRODS_DSI"
export DEST_BIN_DIR="/opt/b2stage/iRODS_DSI"
export DEST_ETC_DIR="/opt/b2stage/iRODS_DSI"

The DSI build is then executed:

source setup.sh
cmake CMakeLists.txt
C_INCLUDE_PATH=/usr/include/x86_64-linux-gnu/globus make install

After the installation, /etc/gridftp.conf needs to have the following lines appended:

$LD_LIBRARY_PATH "$LD_LIBRARY_PATH:/opt/b2stage/iRODS_DSI"
$irodsConnectAsAdmin "rods"
load_dsi_module iRODS
auth_level 4
$GSI_AUTHZ_CONF /opt/b2stage/iRODS_DSI/gridmap_iRODS_callout.conf
$HOME /root

The last two lines are optional. The GSI_AUTHZ_CONF sets the automatic mapping between DNs and iRODS user names and configuring this feature eliminates the need for a local grid map file. The HOME setting is included because the GridFTP is configured to run as a system service.

GridFTP server library and DSI library need to be preloaded. Following two lines should be added to /etc/init.d/globus-gridftp-server file:

LD_PRELOAD="$LD_PRELOAD:/usr/lib/x86_64-linux-gnu/libglobus_gridftp_server.so:\
/opt/b2stage/iRODS_DSI/libglobus_gridftp_server_iRODS.so"
export LD_PRELOAD

As user root will be running the FTP server, file /root/.irods/irods_environment.json should be added, containing following lines:

{
"irods_host": "<your_icat_server>",
"irods_zone_name": "<your_Zone>",
"irods_port": 1247,
"irods_user_name": "rods",
"irods_default_resource": "<your_default_Resource>",
}

In case you have configured your iRODS server with SSL, additional lines regarding SSL should be added to json file.

After that perform an iinit to authenticate the iRODS user.


Working with files using iCommands

Here we will give a short overview of the basic commands, most commonly used. The iRODS manual available here offers a more detailed overview of all the available commands.

To start the iRODS session, make sure you have iRODS environment in ~/.irods/irods_environment.json configured for the iRODS server you wish to connect to. In case of the installation at IPB, the environment on a client machine looks like the following:

{
    "irods_host": "irods.ipb.ac.rs",
    "irods_port": 1247,
    "irods_user_name": <your_user_name>,
    "irods_zone_name": "IPB",
    "irods_default_resource_name": "rootResc"
}

Once the environment is set up, you can initialize the session with:

iinit

The following table lists the basic commands for file operations.

command description
ils List the contents of the current directory in the active iRODS session.
icd Change the current directory in the iRODS session.
iput Uploads a given file to the current directory on iRODS or a different destination on iRODS if specified.
iget Downloads a file from iRODS.
imkdir Creates a new directory on iRODS


Working with files using GridFTP

For access from places which only have GridFTP installed, the iRODS installation provides the standard GridFTP interface.

GridFTP session is initiated using grid-proxy-init command. From there on, the standard globus-url-copy commands are available.

To put file into iRODS:

$ globus-url-copy test_file.txt gsiftp://<your_icat_server>:2811/<your_Zone>/home/<your_user_name>/

To get file from iRODS:

$ globus-url-copy gsiftp://<your_icat_server>:2811/<your_Zone>/home/<your_user_name>/test_file.txt test_file2.txt

To list the content of a directory:

$ globus-url-copy --list gsiftp://<your_icat_server>:2811/<your_Zone>/home/<your_user_name>/


Adding resources

To add a resource or perform any other administrative level operation on iRODS, one must first initialize iRODS session with administrator account (default username is ‘rods’). This is done by configuring irods_environment.json file appropriately and calling the iinit command.

After the session is initialized, iadmin mkresc command is used to create resources. The command syntax to add a basic unix file system resource is:

iadmin mkresc <newrescname> unixfilesystem <fully.qualified.domain.name>:</full/path/to/new/vault>

More options can be found by calling iadmin mkresc help. Other resource types and more information can be found on in the [iRODS documentation] (https://docs.irods.org/4.1.8/icommands/administrator/).


Adding users

Adding users is also accomplished through the iadmin command, so it requires iRODS administrator session to be active. The command to add a new user is:

iadmin mkuser <newusername> rodsuser

To see the registered users, type:

iadmin lu

Typing iadmin mkuser help will show more information about this command.



This page was written by Petar Jovanovic, petar at ipb dot ac dot rs, on 29 Sep 2016.