Skip to content

Add support for Centos7/ppc64 #1763

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mhdawson merged 3 commits into from
Apr 24, 2019
Merged

Add support for Centos7/ppc64 #1763

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e440a2e
Use devtoolset-6 on node > 11 for centos7-ppc64le
sam-github Apr 16, 2019
ab432f5
Exclude centos7-ppcle for node < 12.x
sam-github Apr 16, 2019
c6e9433
Setup centos7-ppc64le jenkins workers
sam-github Apr 15, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions ansible/inventory.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ hosts:
aix61-ppc64_be-1: {ip: 140.211.9.99}
ubuntu1404-ppc64_be-1: {ip: 140.211.168.76}
ubuntu1404-ppc64_le-1: {ip: 140.211.168.66}
centos7-ppc64_le-1: {ip: 140.211.168.61, user: centos}

- scaleway:
ubuntu1604-armv7l-1: {ip: 212.47.245.242}
Expand Down Expand Up @@ -150,6 +151,8 @@ hosts:
ubuntu1404-ppc64_le-1: {ip: 140.211.168.106}
ubuntu1404-ppc64_le-2: {ip: 140.211.168.94}
ubuntu1404-ppc64_le-4: {ip: 140.211.168.140}
centos7-ppc64_le-1: {ip: 140.211.168.193, user: centos}
centos7-ppc64_le-2: {ip: 140.211.168.244, user: centos}

- rackspace:
centos7-x64-1: {ip: 119.9.27.82}
Expand Down
11 changes: 10 additions & 1 deletion ansible/roles/baselayout/tasks/partials/repo/centos7.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,26 @@
state: present

- name: centos7 | install ius
when: "arch != 'arm64'"
when: "arch != 'arm64' and arch != 'ppc64'"
yum:
name: "https://centos{{ ansible_distribution_major_version }}.iuscommunity.org/ius-release.rpm"
state: present


- name: centos7 | install scl for devtoolset-6
when: "arch != 'arm64' and arch != 'ppc64'"
yum:
name: centos-release-scl
state: present

- name: centos7 | ppc64 | install sclo7 devtoolset-6
when: "arch == 'ppc64'"
yum_repository:
name: sclo7-devtoolset-6
description: cbs.centos.org sclo7-devtoolset-6-rh-release
baseurl: https://cbs.centos.org/repos/sclo7-devtoolset-6-rh-release/$basearch/os/
gpgcheck: no

- name: centos7 | aarch64 | install sclo repo
when: "arch == 'arm64'"
copy:
Expand Down
2 changes: 1 addition & 1 deletion ansible/roles/baselayout/vars/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ packages: {
centos7_x64: ['git2u','centos-release-scl',], # centos-release-scl is required to enable SCLo
# but we do it manually in partials/repo/centos7.yml for arm64
centos7: [
'ccache,gcc-c++,devtoolset-6,sudo,git',
'ccache,gcc-c++,devtoolset-6,sudo,git,devtoolset-6-libatomic-devel',

@rvagg rvagg Apr 18, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added git in #1765, if it gets merged first then you'll just have to resolve that conflict

],

aix: [
Expand Down
21 changes: 18 additions & 3 deletions ansible/roles/java-base/tasks/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,20 +32,35 @@
include_role:
name: package-upgrade

- name: download java
when: os in "ubuntu1404" and arch == "ppc64"
- name: download java ppc64
when: os in ("ubuntu1404", "centos7") and arch == "ppc64"
get_url:
url: https://github.com/AdoptOpenJDK/openjdk8-releases/releases/download/jdk8u172-b11/OpenJDK8_ppc64le_Linux_jdk8u172-b11.tar.gz
dest: /tmp/

- name: unarchive java
- name: unarchive java ubuntu1404 ppc64
when: os in "ubuntu1404" and arch == "ppc64"
unarchive:
src: /tmp/OpenJDK8_ppc64le_Linux_jdk8u172-b11.tar.gz
remote_src: yes
dest: /home/iojs
tags: java

- name: unarchive java centos7 ppc64

@sam-github sam-github Apr 18, 2019

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these two stanzas have the effect of putting java in /usr/bin/java, where the systemd jenkins unit file (and anyone else) would expect it to be.

when: os in "centos7" and arch == "ppc64"
unarchive:
src: /tmp/OpenJDK8_ppc64le_Linux_jdk8u172-b11.tar.gz
remote_src: yes
dest: /opt
tags: java

- name: symlink java centos7 ppc64
when: os in "centos7" and arch == "ppc64"
file:
src: "/opt/jdk8u172-b11/bin/java"
dest: "/usr/bin/java"
state: link

# if this fails you want to check in vars/main.yml and add package name
# as appropriate -- try to use generic os family if available.

Expand Down
1 change: 1 addition & 0 deletions jenkins/scripts/VersionSelectorScript.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ def buildExclusions = [
[ /centos[67]-(arm)?(64|32)-gcc6/, anyType, lt(10) ],
[ /centos6-32-gcc6/, releaseType, gte(10) ], // 32-bit linux for <10 only
[ /^centos7-64/, releaseType, lt(12) ],
[ /^centos7-ppcle/, anyType, lt(12) ],

@sam-github sam-github Apr 18, 2019

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, the new machines are labelled centos7-ppcle, this excludes them when node lt 12.

[ /debian8-x86/, anyType, gte(10) ], // 32-bit linux for <10 only
[ /^ubuntu1804/, anyType, lt(10) ], // probably temporary
[ /^ubuntu1204/, anyType, gte(10) ],
Expand Down
13 changes: 12 additions & 1 deletion jenkins/scripts/select-compiler.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ if [ "$DONTSELECT_COMPILER" != "DONT" ]; then
NODE_NAME=${NODE_NAME:-$HOSTNAME}
echo "Selecting compiler based on $NODE_NAME"
case $NODE_NAME in
*ppc64_le* ) SELECT_ARCH=PPC64LE ;;
*ppc64*le* ) SELECT_ARCH=PPC64LE ;;

@mhdawson mhdawson Apr 17, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It this because the node names are inconsistent?

@sam-github sam-github Apr 18, 2019

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because of

build/ansible/roles/baselayout/tasks/main.yml

Line 16 in 6d12c28

safe_hostname: "{{ inventory_hostname | regex_replace('_', '--') }}"
, it means that HOSTNAME := NODE_NAME.replace(_, --), so select compiler's attempt to use the HOSTNAME as a default, which usually works, fails here.

@rvagg rvagg Apr 18, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change should be OK, but again it's an oddity because NODE_NAME should come from Jenkins and the only situation where it's not set is when this is run independently, I guess if this is run as an independent test?

@sam-github sam-github Apr 18, 2019

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, anyone attempting to trouble shoot build issues or bootstrap a ci machine needs to read select-compiler by hand, after reading it to figure out what they should set NODE_NAME to. I added HOSTNAME as the default. If we don't want that, I can back it out. Given that ansible sets the HOSTNAME to the NODE_NAME (modulo the - and _), I'm not sure why this script depends on ci env vars. If this change is out of scope, I'll back it out. Its more of a public service for future users.

@mhdawson mhdawson Apr 22, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, understood now and makes sense. The result is that you can call select-compiler.sh on the host by hand and get the right behavior without having to export NODE_NAME.

@mhdawson mhdawson Apr 22, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

*s390x* ) SELECT_ARCH=S390X ;;
*aix* ) SELECT_ARCH=AIXPPC ;;
esac
Expand All @@ -22,6 +22,17 @@ if [ "$SELECT_ARCH" = "PPC64LE" ]; then

echo "Setting compiler for Node version $NODEJS_MAJOR_VERSION on ppc64le"

case $NODE_NAME in
Comment thread
sam-github marked this conversation as resolved.
*centos7* )
if [ "$NODEJS_MAJOR_VERSION" -gt "11" ]; then
# Setup devtoolset-6, sets LD_LIBRARY_PATH, PATH, etc.
. /opt/rh/devtoolset-6/enable
echo "Compiler set to devtoolset-6"
return
fi
;;
esac

if [ "$NODEJS_MAJOR_VERSION" -gt "11" ]; then
# See: https://github.com/nodejs/build/pull/1723#discussion_r265740122
export PATH=/usr/lib/binutils-2.26/bin/:$PATH
Expand Down