Difference between revisions of "Access files on JLab XrootD server"

From epsciwiki
Jump to navigation Jump to search
(Created page with " Here is an example of copying files to a location on the JLab EIC work disk that can then be accessed from anywhere using xrootd. On a JLab ifarm machine, create a director...")
 
 
Line 10: Line 10:
 
     scp /data/tridas/EIC_3x3_R00000235_F00000004.pt davidl@ifarm1901:/work/eic3/users/davidl/2023.02.16.TriDAS_data
 
     scp /data/tridas/EIC_3x3_R00000235_F00000004.pt davidl@ifarm1901:/work/eic3/users/davidl/2023.02.16.TriDAS_data
  
You should now be able to access the files using an xroot client on your local computer. Here, I do it with a docker container. The key info is to use '''''dtn-eic.jlab.org''''' as the host and the path will be the same as on the CUE.
+
You should now be able to access the files using an ''xrootd'' client on your local computer.  
 +
 
 +
If you have '''MacOS'' and have ''homebrew'' installed, then just install the ''xrootd'' package:
 +
 
 +
    brew install xrootd
 +
 
 +
This will allow you to run the ''xrdfs'', ''xrdcp'', ... programs directly. For example, you can list the contents of a directory with something like the following. The key info is to use '''''dtn-eic.jlab.org''''' as the host and the path will be the same as on the CUE.
 +
 
 +
    xrdfs dtn-eic.jlab.org ls /work/eic3/users/davidl/2023.02.16.TriDAS_data
 +
      /work/eic3/users/davidl/2023.02.16.TriDAS_data/EIC_3x3_R00000235_F00000004.pt
 +
 
 +
If you have issues with that or want to use another platform, then you can always use docker.
 +
Here is how I do it with a docker container:
  
 
     docker run --rm -it lukasheinrich/xrootdclient xrdfs dtn-eic.jlab.org ls /work/eic3/users/davidl/2023.02.16.TriDAS_data
 
     docker run --rm -it lukasheinrich/xrootdclient xrdfs dtn-eic.jlab.org ls /work/eic3/users/davidl/2023.02.16.TriDAS_data

Latest revision as of 20:39, 10 November 2023


Here is an example of copying files to a location on the JLab EIC work disk that can then be accessed from anywhere using xrootd.

On a JLab ifarm machine, create a directory on the EIC work disk.

   ssh ifarm1901
   mkdir /work/eic3/users/davidl/2023.02.16.TriDAS_data

Copy files to the directory. Here, I do this from the group account streamdaq on clondaq5

   scp /data/tridas/EIC_3x3_R00000235_F00000004.pt davidl@ifarm1901:/work/eic3/users/davidl/2023.02.16.TriDAS_data

You should now be able to access the files using an xrootd client on your local computer.

If you have 'MacOS and have homebrew installed, then just install the xrootd package:

   brew install xrootd

This will allow you to run the xrdfs, xrdcp, ... programs directly. For example, you can list the contents of a directory with something like the following. The key info is to use dtn-eic.jlab.org as the host and the path will be the same as on the CUE.

   xrdfs dtn-eic.jlab.org ls /work/eic3/users/davidl/2023.02.16.TriDAS_data
     /work/eic3/users/davidl/2023.02.16.TriDAS_data/EIC_3x3_R00000235_F00000004.pt

If you have issues with that or want to use another platform, then you can always use docker. Here is how I do it with a docker container:

   docker run --rm -it lukasheinrich/xrootdclient xrdfs dtn-eic.jlab.org ls /work/eic3/users/davidl/2023.02.16.TriDAS_data
     /work/eic3/users/davidl/2023.02.16.TriDAS_data/EIC_3x3_R00000235_F00000004.pt

To copy a file to your local machine, use the xrdcp command. Again, using docker

   # Use the "-v ${PWD}:/work" option to bind the current directory
   # on your local machine to a directory named "/work" inside the
   # container. This lets the you copy to the host file system rather
   # than the temporary one docker creates for the container.
   docker run --rm -it -v ${PWD}:/work lukasheinrich/xrootdclient xrdcp root://dtn-eic.jlab.org//work/eic3/users/davidl/2023.02.16.TriDAS_data/EIC_3x3_R00000235_F00000004.pt /work