next up previous contents
Next: Test Suite Up: Quick Start and Simple Previous: First Steps   Contents

Replication Examples using Optimization Functions

This example runs through a scenario how the edg-replica-manager commands might be used to achieve standard data management tasks. In the scenario the user knows that there is a file available at CERN, that has been put on a host accessible through GridFTP. It is not a grid-aware store, so first the user has to copy the file to a Storage Element and register it in the Grid. Say that for some reason the user cannot copy it to the local CERN Storage Element but has to copy it to the one at IN2P3.

In the example the file is called `higgs0' and resides at testbed008.cern.ch/tmp/.

The copy and registration is an atomic operation. In the example we assign also a Logical File Name alias to it in the process, lfn:higgs, which is easier to remember than the GUID that is returned by the call:

edg-rm --vo=wpsix copyAndRegisterFile gsiftp://testbed008.cern.ch/tmp/higgs0
-l lfn:higgs -d ccgridli02.in2p3.fr

A GUID is created and returned to the screen:

guid:7c29f32b-4964-11d7-a86c-9ee9a33b1f19

To verify whether the operation was successfully executed, we can issue listReplicas:

edg-rm --vo=wpsix listReplicas lfn:higgs

which yields:

srm://ccgridli02.in2p3.fr/edg/StorageElement/dev2/wpsix/higgs

In order to retrieve the GUID based on the LFN, we can issue

edg-rm --vo=wpsix listGUID lfn:higgs

As a second step, the user might want to have a replica of this data file available at NIKHEF, because he intends to share it or to submit jobs that require resources at NIKHEF. A replica can be created using the replicateFile command:

edg-rm --vo=wpsix replicateFile lfn:higgs -d se01.nikhef.nl

The command confirms its execution by returning the actual SURL used. If the -d option only specifies a hostname and not a full path, an automatic SURL is created. Here the output is:

srm://se01.nikhef.nl/flatfiles/wpsix/higgs

To list all replicas now in the system, we can issue listReplicas again:

edg-rm --vo=wpsix listReplicas lfn:higgs

which yields:

srm://ccgridli02.in2p3.fr/edg/StorageElement/dev2/wpsix/higgs
srm://se01.nikhef.nl/flatfiles/wpsix/higgs

To see which replica has the best network connection to CERN, we can use listBestFile:

edg-rm --vo=wpsix listBestFile lfn:higgs -d pcrd24.cern.ch

The output is, for example:

srm://se01.nikhef.nl/flatfiles/wpsix/higgs

which means that the file at NIKHEF can be made available at CERN faster than the one from LYON. We now want to see the file access costs of the best replica with respect to CERN, NIKHEF and Lyon using a CE hostname from each site:

edg-rm --vo=wpsix getAccessCost -l lfn:higgs \
       -d lxshare0313.cern.ch:2119/jobmanager-pbs-short\
       ce01.nikhef.nl:2119/jobmanager-pbs-short\
       ccgridli01.in2p3.fr:2119/jobmanager-pbs-short

The output is:

Access Cost 0 : 
CE = lxshare0313.cern.ch:2119/jobmanager-pbs-short
0:srm://se01.nikhef.nl/flatfiles/wpsix/higgs
TotalTime = 0.13

Access Cost 1 : 
CE = ce01.nikhef.nl:2119/jobmanager-pbs-short
0:srm://se01.nikhef.nl/flatfiles/wpsix/higgs
TotalTime = 0.0

Access Cost 2 : 
CE = ccgridli01.in2p3.fr:2119/jobmanager-pbs-short
0:srm://ccgridli02.in2p3.fr/edg/StorageElement/dev2/wpsix/higgs
TotalTime = 0.0

The list is grouped by the CEs given on the command line. For each CE the 'best' replica is listed and the time it would take to make it available locally. We can see that the expected access cost to transfer the file from NIKHEF to CERN is 0.13 sec. For the other sites the access cost is 0 since the file is already locally available and no network transfer is required.

To actually make the best file available at CERN, we can issue getBestFile:

edg-rm --vo=wpsix getBestFile lfn:higgs -d pcrd24.cern.ch

The output is something like:

srm://pcrd24.cern.ch/data/temp/a6289c7c-4966-11d7-bc63-d91230733e2d

We should now have three replicas:

edg-rm --vo=wpsix listReplicas lfn:higgs

The output is:

srm://pcrd24.cern.ch/data/temp/aaa64014-4967-11d7-a6cc-f7a1ff1899b0
srm://se01.nikhef.nl/flatfiles/wpsix/higgs
srm://ccgridli02.in2p3.fr/edg/StorageElement/dev2/wpsix/higgs

To delete a replica on a specific SE we can use the deleteFile command with the -s option:

edg-rm deleteFile lfn:higgs -s ccgridli02.in2p3.fr


next up previous contents
Next: Test Suite Up: Quick Start and Simple Previous: First Steps   Contents
2004-06-14