Skip to content

Install gcc 11.2 in docker images - #18477

Merged
keithc-ca merged 1 commit into
eclipse-openj9:masterfrom
pshipton:docker11
Nov 20, 2023
Merged

Install gcc 11.2 in docker images#18477
keithc-ca merged 1 commit into
eclipse-openj9:masterfrom
pshipton:docker11

Conversation

@pshipton

@pshipton pshipton commented Nov 17, 2023

Copy link
Copy Markdown
Member

No description provided.

@pshipton
Install gcc 11.2 in docker images
aaa93ec 
Signed-off-by: Peter Shipton <Peter_Shipton@ca.ibm.com>
@pshipton
pshipton requested a review from keithc-ca November 17, 2023 18:10
keithc-ca
keithc-ca reviewed Nov 17, 2023
View reviewed changes
Comment thread buildenv/docker/mkdocker.sh
Comment on lines +425 to +430
echo "# Install gcc-11.2."
echo "RUN cd /usr/local \\"
echo " && $wget_O gcc-11.tar.xz 'https://ci.adoptopenjdk.net/userContent/gcc/gcc112.$arch.tar.xz' \\"
echo " && tar -xJf gcc-11.tar.xz \\"
echo " && rm -f gcc-11.tar.xz"
echo ""

@keithc-ca keithc-ca Nov 17, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems reasonable. Have you tested the result?

@pshipton pshipton Nov 17, 2023

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, looks good. After this is merged I'll build a new container on dockerhub via https://openj9-jenkins.osuosl.org/view/Infrastructure/job/Build-Jenkins-Agent-Container/ and then modify the jdk8,11 builds to use it with gcc 11.2.
The change will be similar to master...pshipton:openj9:xadopt but using eclipseopenj9/jenkins-agent-x86-centos6 rather than adoptopenjdk/centos6_build_image.

docker run -it localhost/openj9/cent6 /bin/bash
[jenkins@7703735b8486 /]$ /usr/local/gcc11/bin/gcc-11.2 --version
gcc-11.2 (Adoptium/sxa) 11.2.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[jenkins@7703735b8486 /]$ /usr/local/gcc11/bin/g++-11.2 --version
g++-11.2 (Adoptium/sxa) 11.2.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

@AdamBrousseau

AdamBrousseau commented Nov 17, 2023

Copy link
Copy Markdown
Contributor

jenkins build docker x86 centos6

@pshipton

pshipton commented Nov 17, 2023

Copy link
Copy Markdown
Member Author

@AdamBrousseau the docker build is waiting on https://openj9-jenkins.osuosl.org/computer/ub20%2Dx86%2D1/ but you've taken this node offline earlier today for the reason "docker test"

@AdamBrousseau

AdamBrousseau commented Nov 17, 2023

Copy link
Copy Markdown
Contributor

Re-enabled. I had pulled the adopt container to it so I had to remove it in order to free up space.

@pshipton

pshipton commented Nov 17, 2023

Copy link
Copy Markdown
Member Author

It still doesn't have enough space to build the container.
Cannot contact ub20-x86-1: java.io.IOException: No space left on device

@AdamBrousseau

AdamBrousseau commented Nov 17, 2023

Copy link
Copy Markdown
Contributor

I'm not surprised. I'll have to try and recall why we're limiting it to that machine.

@pshipton

pshipton commented Nov 17, 2023 •
edited
Loading

Copy link
Copy Markdown
Member Author

There doesn't seem to be much we could clean to get more space. There is 26G now, I'll try again, but we'll probably get the same result.

jenkins build docker x86 centos6

@pshipton

pshipton commented Nov 17, 2023

Copy link
Copy Markdown
Member Author

Did I break it?

First time build. Skipping changelog.
[Pipeline] Start of Pipeline
[Pipeline] error
[Pipeline] End of Pipeline
ERROR: Invalid Parameters. Either ARCH:'unknown' or OS:'unknown' were not declared properly

jenkins build docker x86 centos6

@pshipton

pshipton commented Nov 17, 2023

Copy link
Copy Markdown
Member Author

I don't know what system that was running on, it didn't even get to ub20-x86-1 yet.
https://openj9-jenkins.osuosl.org/job/PullRequest-Build-Jenkins-Agent-Container/12/

@pshipton

pshipton commented Nov 17, 2023

Copy link
Copy Markdown
Member Author

jenkins build docker x86 centos6

@pshipton

pshipton commented Nov 17, 2023

Copy link
Copy Markdown
Member Author

The PR build needs to be launched in a comment by itself.

@pshipton

pshipton commented Nov 17, 2023

Copy link
Copy Markdown
Member Author

I'll have to try and recall why we're limiting it to that machine.

The criteria is sw.tool.docker && hw.arch.x86 && sw.os.ubuntu.20 and ub20-x86-1 is the only ub20 machine.

@pshipton

pshipton commented Nov 18, 2023 •
edited
Loading

Copy link
Copy Markdown
Member Author

Not sure why it was able to succeed this time. I deleted a little bit of stuff but not much. I did delete the container and all the images. There seemed to be plenty of free space while it was running.

Now that it's built and at dockerhub with the tag PR18477, do we need to built it again to get a better tag?
https://hub.docker.com/r/eclipseopenj9/jenkins-agent-x86-centos6/tags

@pshipton

pshipton commented Nov 18, 2023

Copy link
Copy Markdown
Member Author

Trying a jvm build with this image using gcc 11.2 so we can see it's working.
https://openj9-jenkins.osuosl.org/job/Pipeline-Build-Test-Personal/455/

@pshipton

pshipton commented Nov 18, 2023 •
edited
Loading

Copy link
Copy Markdown
Member Author

By default we get cuda 12 in the image, but the builds want cuda 9. It seems only 11, 12 are available from dockerhub. I recall there was some discussion about cuda versions earlier, but I'm not sure of the outcome.

configure: error: The path of cuda_home, which resolves as "/usr/local/cuda-9.0", is not found

@pshipton

pshipton commented Nov 18, 2023 •
edited
Loading

Copy link
Copy Markdown
Member Author

Trying again using /usr/local/cuda, which is what was added to the image.
https://openj9-jenkins.osuosl.org/job/Pipeline-Build-Test-Personal/456/

@pshipton

pshipton commented Nov 18, 2023

Copy link
Copy Markdown
Member Author

ub16x64j95 ran out of space building jdk11.
I deleted an older jenkins-agent-x86-centos6:24 image, and an older openj9-docs image. There is only 9.4G free now with the two remaining centos6 images. Once we're happy with the new one we can delete the older one(s).

The jdk8 build on cent7-x64-1 was successful. It has plenty of space but I cleaned up some old images.

keithc-ca
keithc-ca approved these changes Nov 20, 2023
View reviewed changes

@keithc-ca keithc-ca left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Java 11 build failed

[2023-11-18T04:27:12.960Z] /home/jenkins/workspace/Build_JDK11_x86-64_linux_Personal/openj9/runtime/compiler/env/CHTable.cpp:1297:4: fatal error: error writing to /tmp/ccshUMls.s: No space left on device

which is not (likely) a problem due to this change.

The Java 8 build succeeded: approving.

@keithc-ca

keithc-ca commented Nov 20, 2023

Copy link
Copy Markdown
Contributor

Should we remove gcc 7.5 once defaults.yml is updated to use 11.2 on all (docker) platforms?

@keithc-ca
keithc-ca merged commit aa5cf33 into eclipse-openj9:master Nov 20, 2023
@pshipton

pshipton commented Nov 20, 2023 •
edited
Loading

Copy link
Copy Markdown
Member Author

Should we remove gcc 7.5 once defaults.yml is updated to use 11.2 on all (docker) platforms?

gcc 7.5 is still required to build the docker image. There are some components built from source, and at least one of them fails to compile with gcc 11.2. I noticed this because I accidentally replaced the default compiler, but I didn't record what actually failed.

@keithc-ca

keithc-ca commented Nov 20, 2023

Copy link
Copy Markdown
Contributor

We could remove it after building those other things, but the space savings might not be worth worrying about.

@pshipton pshipton mentioned this pull request Nov 20, 2023
Merged
@AdamBrousseau

AdamBrousseau commented Nov 20, 2023

Copy link
Copy Markdown
Contributor

https://openj9-jenkins.osuosl.org/job/Build-Jenkins-Agent-Container/29/

pshipton added a commit to pshipton/openj9 that referenced this pull request Nov 21, 2023
@pshipton
Use the new centos6 image to build xlinux with gcc 11.2
b1523cd 
See eclipse-openj9#18477

Signed-off-by: Peter Shipton <Peter_Shipton@ca.ibm.com>
pshipton added a commit to pshipton/openj9 that referenced this pull request Nov 30, 2023
@pshipton
Use the new centos6 image to build xlinux with gcc 11.2
6a11808 
See eclipse-openj9#18477

Signed-off-by: Peter Shipton <Peter_Shipton@ca.ibm.com>
@pshipton pshipton mentioned this pull request Nov 30, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@keithc-ca keithc-ca keithc-ca approved these changes

Assignees

No one assigned

Labels

comp:build

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@pshipton @AdamBrousseau @keithc-ca