Skip to content

OCI container creation at the LPC#18

Merged
aperloff merged 6 commits into
FNALLPC:masterfrom
aperloff:feature-containerize
Jun 28, 2021
Merged

OCI container creation at the LPC#18
aperloff merged 6 commits into
FNALLPC:masterfrom
aperloff:feature-containerize

Conversation

@aperloff

Copy link
Copy Markdown

Given the presence of buildah, the containerize.sh script will create an OCI style image containing the current CMSSW release. The only prerequisites are (which are checked before running):

  1. Buildah is installed on the system
  2. The users id is given a set of subuids and subgids in /etc/subuid and /etc/subgid
  3. /cvmfs/cms.cern.ch is mounted on the host machine
  4. There is sufficient space to store the images

The basic workflow would be:

ssh -Y <username>@cmslpcheavy-sl7.fnal.gov
cd <path to cmssw>
cmsenv
<path to lpc_scripts>/containerize.sh -t <name:tag> -b <base_image>
podman run --rm -it <name>:<tag>

Alexx Perloff added 2 commits June 25, 2021 18:12
@aperloff aperloff requested review from kpedro88 and mtonjes June 25, 2021 23:27
Comment thread containerize/Dockerfile Outdated
Comment thread containerize/Dockerfile Outdated
Comment thread containerize/Dockerfile Outdated
Comment thread containerize/Dockerfile Outdated
Comment thread containerize/containerize.sh Outdated
Comment thread containerize/containerize.sh Outdated
Comment thread containerize/containerize.sh Outdated
Comment thread containerize/containerize.sh
Comment thread containerize/containerize.sh
Comment thread containerize/containerize.sh Outdated
@aperloff

Copy link
Copy Markdown
Author

@kpedro88 I believe I have addressed all of your review comments. Please let me know if you're happy with the results.

Comment thread containerize/Dockerfile Outdated
Comment thread containerize/Dockerfile Outdated
ARG CMSSW_VERSION
ARG TAR

COPY ${TAR} /home/cmsusr/${CMSSW_VERSION}.tar.gz

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to previous comment, could this be:

COPY ${TAR} ${WORKDIR}/${CMSSW_VERSION}.tar.gz

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed it to /home/${USER} as the ${WORKDIR} and ${HOME} variables are not available and misleading, respectively.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is ${WORKDIR} not available? In fact, it's defined in this file but then never used anywhere...

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not an argument or environment variable. The line:

WORKDIR /home/${USER}

Is the location the user will be dropped into when they open the container, as opposed to '/'.

@kpedro88 kpedro88 Jun 28, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah okay, it's a special Docker directive. In that case, I propose:

ARG USERDIR /home/${USER}
WORKDIR ${USERDIR}

then ${USERDIR} can be referenced where useful.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, take a look. I think I've implemented what you asked for.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is better.

Comment thread containerize/Dockerfile Outdated
Alexx Perloff added 2 commits June 28, 2021 14:47
…option to override default username in container and some changes to the Dockerfile for sustainability.
Comment thread containerize/Dockerfile Outdated
ARG NONPRIVILEGED_USER=cmsusr
ARG USERDIR=/home/${NONPRIVILEGED_USER}
WORKDIR ${USERDIR}
COPY --from=builder --chown=cmsusr:cmsusr /home/cmsusr/${CMSSW_VERSION} ${USERDIR}/${CMSSW_VERSION}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If ${USERDIR} works here, can't you define ARG USER_BUILD cmsusr and then USER ${USER_BUILD}, etc. to work around the #19 issue?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can certainly do what you're suggesting, and it does partially get around the problem. However, it won't get around #19 because I still won't be able to evaluate arguments in the parameters (chown) of the COPY command. Let me know what you think now.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is an improvement. Can you update #19 to clarify the point about using arguments specifically in the parameters of the COPY command?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, #19 is updated.

@aperloff aperloff force-pushed the feature-containerize branch from 2be2c3e to 9431c84 Compare June 28, 2021 21:27
@kpedro88

Copy link
Copy Markdown
Contributor

+1

@aperloff aperloff merged commit a225f1f into FNALLPC:master Jun 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants