Adding non-standard ways to use EESSI

[hvelab]

As per this issue: https://gitlab.com/eessi/support/-/issues/119

First batch regarding the cvmfsexec usage. A couple of thoughts:

• Should we add the cvmfsexec_eessi.sh and the orted wrapper in some repo so people can directly clone and not have to copy-paste it?
• Worked fine in my local Ubuntu 22.04 and a remote Rocky Linux test environment,. For some colleagues I asked to test in their locals (Ubuntu 23), this error msg occured:

 $ ./test.sh ls /cvmfs/software.essi.io
/tmp/rzarco/tmp.sdsrKCrQH7/cvmfsexec/dist should be rpm2cpio of cvmfs rpm
mountrepo software.eessi.io failed

so seems only works for RHEL :/

[ocaisa]

cvmfsexec only works for RHEL-like systems 😢

[hvelab]

cvmfsexec only works for RHEL-like systems 😢

True, in my Ubuntu works but seems that its because I have CVMFS already available, I tested it in other systems without CVMFS and the same error message shows :(

[ocaisa]

Should link to the CVMFS docs on this early on.

[ocaisa]

cvmfs-2.13.0 was released a few minutes ago, so the approach with shrinkwrap should work out-of-the-box if you have that version available. This also means I can start working on a script to manage this once our client containers are updated

[casparvl]

I think the spec file requires extra explanation. E.g. we should probably state that the compat layer is always needed, as is the dir that allows initializing the environment. Then, we should probably give some examples: e.g. how to export the full x86_64 tree (with the warning that it requires a massive amount of memory due to a known issue, see https://gitlab.com/eessi/support/-/issues/118), how to export for a single (or a handful) of micro-architectures, and maybe even an approach how to export for a single piece of software (i.e. load the GROMACS module, check all EBROOT variables, and export those - that should be enough...)

[hvelab]

added new info, how do you see it now?

[hvelab]

ping to check

[casparvl]

/init/*

[hvelab]

what am I not seeing? D:

[ocaisa]

There is another folder /init alongside /versions

[casparvl]

Yes, at the root of the software.eessi.io repo there is an(other) /init/* directory which contains the EESSI modules. The instructions for initializing the EESSI environment actually require these: you either add this dir to MODULEPATH and load the modules (if you already have a module command on your system), or you source the /versions/2023.06/init/lmod/<shell> script - which then sets the MODULEPATH and loads the mdoule.

I.e. your instructions seem to be originally made for the 'old' method of sourcing /versions/2023.06/init/bash, but that approach deprecated since we wanted to stop trying to keep two init methods in sync (the module, and that bash script).

[casparvl]

It actually sounds quite nice to have a separate squashfs for the compat layer. I never tried it, but: how would I then mount both? Does a straightforward

apptainer shell -B software.eessi.io.sqsh:/cvmfs:image-src=/ -B software.eessi.io_compat.sqsh:/cvmfs:image-src=/ docker://ubuntu

work? It seems weird to me that you'd mount two different squasfs filesystems to the same mountpoint - how would it know how to integrate them if there are overlapping files/directory names?

[hvelab]

it definetly collides:

hvela@ThinkPad-hvela:~/HPCNow/EESSI/tests$ apptainer shell -B software.eessi.io.sqsh:/cvmfs:image-src=/ -B software.eessi.io_compat.sqsh:/cvmfs:image-src=/ docker://ubuntu
INFO:    Using cached SIF image
INFO:    Cleanup error: while unmounting /var/lib/apptainer/mnt/session/data-images/1 directory: no such file or directory, while unmounting /var/lib/apptainer/mnt/session/data-images/0 directory: no such file or directory, while unmounting /var/lib/apptainer/mnt/session/final directory: no such file or directory, while unmounting /var/lib/apptainer/mnt/session/rootfs directory: no such file or directory
FATAL:   container creation failed: while adding data bind /var/lib/apptainer/mnt/session/data-images/1 -> /cvmfs: destination is already in the mount point list

But changing the mount points seems fine:

hvela@ThinkPad-hvela:~/HPCNow/EESSI/tests$ apptainer shell   -B software.eessi.io.sqsh:/cvmfs/software.eessi.io:image-src=/   -B software.eessi.io_compat.sqsh:/cvmfs/versions/2023.06/compat:image-src=/   docker://ubuntu
INFO:    Using cached SIF image
Apptainer> ls /cvmfs/
software.eessi.io  versions

The contents of the directory seem a bit odd to me... But that's because the content of the spec file is:

hvela@ThinkPad-hvela:~/HPCNow/EESSI/tests$ cat software.eessi.io_compat.spec
/versions/2023.06/compat/linux/x86_64/*
# Exclude the Gentoo ebuild repo and cache files
!/versions/2023.06/compat/linux/x86_64/var/db/repos/gentoo
!/versions/2023.06/compat/linux/x86_64/var/cach

Doubting if this is the best approach now

[casparvl]

See comments :)

[boegel]

@hvelab Can you look into the last review, so we can get this across the finish line?