Exactly a year ago today, Tim Ellison sent me a note. He had just watched a presentation I had recorded, talking about the work my team had started to vastly ‘simplify Java testing’.
He mentioned that there was this project he was involved with, “AdoptOpenJDK”, where they were talking about some of the same concepts that we were implementing. He wondered if what we had started implementing could be used at this project. I replied, “sure, by when”. His answer, “last week”.
Here we are, 1 year later, diligently improving the way we test Java. I am witnessing the vision that we had laid out over a year ago of “make test… better” become reality. It is a collaborative and fun effort! Running all kinds of testing, and very notably this week, AdoptOpenJDK project is able to claim its first JCK certified builds, starting with openjdk8-openj9 builds on 3 Linux platforms (x64/ppc64le/s390x). See the check marks in the openjdk8-openj9 build archive (also available in Docker images).
I really do feel lucky to be part of this project, and to work with the small but dedicated team of folks who make it fly. A big thank you and congratulations to the team on this anniversary of sorts, and oh how you capped it off with the JCK compliant icing! I can only imagine what it will look like a year from now, as we continue to innovate, refine and deliver on our goals.
I recently joined the AdoptOpenJDK community, and chose to focus on helping to maintain the build and test machines and the automation used to set up and maintain them.
It’s something I’ve been meaning to do for a while, but for one reason or another I’ve not got round to it. I think one of those reasons was that I thought it would be difficult, and involve me having to talk to lots of people, all giving me slightly different advice.
I found that it wasn’t difficult at all, and the people I spoke with were all very helpful.
What I did
I’ve described the steps I took to get onboard in the AdoptOpenJDK Infrastructure wiki here. I put them there to make it easier for anyone to update should any processes or tools change.
Many of the steps in my instructions are common to all of the AdoptOpenJDK projects. If you go to https://github.com/AdoptOpenJDK you can see all 49 repositories. Please get involved if you think you’ve got something to offer any of them.
Adding third party application tests into the automated testing at AdoptOpenJDK is easier now than it has ever been!
There exists a template project in the openjdk-test repository that can be customized to fit any third party application and run as part of the External Tests builds at Adopt.
The thirdparty_containers README file outlines the steps involved in this process.
For a practical example, you may also want to go over the following demo. It shows how we have added the Lucene-Solr nightly smoke test to the list of third party application tests at Adopt.
Please feel free to grow the list of third party application tests we run in AdoptOpenJDK.
For any questions or queries, reach out to us on our testing slack channel.
Wow! Hard to believe how much progress has been made since I first posted a mission statement of sorts… (in Part 1: Testing Java: Let Me Count the Ways). As I look back over 2017, and assess where we are at with testing the OpenJDK binaries being produced at adoptopenjdk.net, I am prompted to write this “Part 2” blog post. The intent is to share the status and some of the accomplishments of the talented and dedicated group of individuals that are contributing their time, skills and effort towards the goal of fully open-source, tested and compliant binaries.
We have added 4 test categories to date (which constitute tens of thousands of tests, running on several platforms, with more to come as machines become available):
- OpenJDK regression tests (“make openjdk“) – from OpenJDK
- system (and stress) tests (“make system“) – from AdoptOpenJDK/openjdk-systemtest
- 3rd party application tests (“make external“) – the unit tests from each application’s github repo, such as Scala, Derby, Cassandra, etc.
- compliance (JCK) tests (“make jck“) – under OCTLA License (OpenJDK Community TCK License Agreement)
With 2 more test categories on the way:
- functional tests (“make functional“) – from Eclipse/openj9
- performance benchmarks (“make perf“) – from various open-source github repos
To make it easy to run these tests, we’ve added an intentionally thin wrapper that allows us to call some logical make targets to execute these tests. So the ways that we can tag and categorize the test material include by:
- Test group (as listed above, openjdk, system, external, jck, functional, perf)
- Java version (we currently test Java8, Java9 and Java10 builds)
- VM implementation (we currently test OpenJDK with Hotspot, and OpenJDK with OpenJ9)
- Test level (for example, a quick check for pull request builds, we can tag a subset of tests from any group with “sanity”, to run the entire sanity set, “make sanity“, to run just the subset of openjdk or system tests that have been tagged then “make sanity.openjdk” and “make sanity.system” respectively)
For more details on this, please check out the presentation Test Categorization and Test Pipelines at AdoptOpenJDK.
There is also an open discussion on setting the test criteria for marking a build “good”. With the most stringent bar applied to the release builds, nightly builds will have less testing (with constraints such as machine resources and time as factors), and PR builds as mentioned being a targeted sanity check.
We are still getting some of these test builds up and running. And this is where the call for assistance comes in… We would love to have extra hands and eyes on the tests at AdoptOpenJDK. While there are too many tasks to list in this post, here are some meaty work items to pique your interest:
- Triaging any of the open test issues, we know some are likely due to test or machine configuration irregularities, while others are known OpenJDK issues. Properly annotating the current set of failures and rerunning/re-including fixed issues are top of the TODO list.
- Enabling more 3rd party application tests (currently Scala, and shortly Derby and Solr/Lucene tests are running, with the opportunity to include many more).
- A large percentage of the JCKs are automated. There is however a set of these compliance tests that are manual/interactive. We are looking for some dedicated volunteers to help step through the interactive tests on different platforms.
- We have automated these tests in Jenkins Pipeline builds, and want to continue adding builds for the various platforms that the binaries are built on, extra hands here would also be very helpful
Seeing all this come together has been very rewarding. It has been wonderful to work with the capable and dedicated folks working on the AdoptOpenJDK project. There is still a long way to go, but we have a great base to start from, and it seems we can make a big difference by offering a truly open and flexible approach to testing Java. If you really want to learn more about Java, join us in testing it!
We are pleased to announce the availability of the Adopt OpenJDK multi-arch docker images !
The docker images are available for both Hotspot and Eclipse OpenJ9. These images are built and published nightly and are based on the nightly builds from here. For more information on the Dockerfiles and related scripts, see the github repo.
To get the latest version 9 images on any architecture
(Hotspot on Ubuntu)
$ docker pull adoptopenjdk/openjdk9:latest
$ docker run --rm -it adoptopenjdk/openjdk9 java -version
openjdk version "9-internal"
OpenJDK Runtime Environment (build 9-internal+0-adhoc.jenkins.openjdk)
OpenJDK 64-Bit Server VM (build 9-internal+0-adhoc.jenkins.openjdk, mixed mode)
(Latest Version 8 Eclipse OpenJ9 on Ubuntu)
$ docker run --rm -it adoptopenjdk/openjdk8-openj9 java -version
openjdk version "1.8.0-internal"
OpenJDK Runtime Environment (build 1.8.0-internal-jenkins_2017_11_22_15_06-b00)
Eclipse OpenJ9 VM (build 2.9, JRE 1.8.0 Linux amd64-64 Compressed References 20171122_6 (JIT enabled, AOT enabled)
OpenJ9 - 41b7b9b
OMR - 76b44ef
OpenJDK - 84153c7 based on jdk8u152-b16)
For latest Alpine Linux images
$ docker pull adoptopenjdk/openjdk9:alpine
$ docker pull adoptopenjdk/openjdk9-openj9:alpine
You can get a specific release. Eg jdk8u152-b16
$ docker pull adoptopenjdk/openjdk8:jdk8u152-b16
$ docker pull adoptopenjdk/openjdk8:jdk8u152-b16-alpine
$ docker pull adoptopenjdk/openjdk8-openj9:jdk8u152-b16
$ docker pull adoptopenjdk/openjdk8-openj9:jdk8u152-b16-alpine
If you want a specific architecture
Eg. OpenJDK for aarch
$ docker pull adoptopenjdk/openjdk8:aarch64-ubuntu-jdk8u144-b01
Eclipse OpenJ9 and s390x
$ docker pull adoptopenjdk/openjdk8-openj9:s390x-ubuntu-jdk8u152-b16
To include the latest Eclipse OpenJ9 Alpine Linux image in a Dockerfile
RUN mkdir /opt/app
COPY japp.jar /opt/app
CMD ["java", "-jar", "/opt/app/japp.jar"]
Recently, I had the good fortune to speak (fondly!) about some excellent open-source projects that I participate in… AdoptOpenJDK, Eclipse OpenJ9 and Eclipse OMR. Across those three projects, we are trying to simplify the activity of testing. One of the great side-effects of simpler, easier testing is that it frees up time to do some fun work around building microservices for test. We hope that we can eventually bring the best of these microservices into the open also, to help all open-source endeavours be even better at the activities of test.
If you are interested in hearing more about microservices for test, please give a listen to “Cloud-based Test Microservices“, a presentation which I gave at the Eurostar conference in Copenhagen this year.
250,000 hits in 24 hours!
What a crazy last week it has been for everyone at AdoptOpenJDK. We are very excited to have begun adding OpenJ9 builds to our website! The interest has been overwhelming and it was incredible to see our website reach well over 250,000 hits after a small thread on Reddit quickly became much more! The picture below from our Cloudflare analytics tool shows very clearly when the post was added to Reddit and the views kept pouring in for the rest of the day.
We currently only have builds for x86, s390x and ppc64le Linux but we plan to add Windows, macOS and many more as soon as the OpenJ9 team is ready. The easiest way to download them is at adoptopenjdk.net.
To read more about the advantages of using OpenJ9 over Hotspot read the OpenJ9 FAQ here.
We have contributed additional tests and a test framework the tests depend on, to the AdoptOpenJDK project in these repositories: openjdk-systemtest and stf.
The tests are longer running tests which cannot be automated using standard unit test frameworks (such as JUnit or TestNG). They fall under the broad category of ‘system tests’ – tests which attempt to simulate running production workloads and specific user scenarios.
The tests are mostly load tests which run a collection of java methods (typically discrete test case methods) for either a set number of invocations, or a period of time, using one or more threads. Although the tests can (and do) find functional issues executing the individual methods in the load, that is not the main goal of the testing. Running a workload for a period of time is often the most effective way to identify defects in the java virtual machine such as the garbage collector and the dynamic java compiler. See the load test tool documentation for more details of these tests.
Other tests are more multi-process, multi-step in nature. For example, tests which use the Java Debugging Interface (JDI) to examine a test program running in a second JVM, and tests which perform remote operations on a second JVM running a workload, such as attaching a Java transformer agent. See here for more details of these tests.
You can download and execute the tests locally – follow the instructions at openjdk-systemtest. You will also find additional documentation about the tests and the test tooling in the repositories.
AdoptOpenJDK has certainly come a long way over the last few months!
We have a clean, bright website to distribute binaries of OpenJDK, with exciting plans for how it can be improved in the future.
Behind the website “shop front” there is a large continuous integration build and test farm, covering multiple operating system and hardware combinations. The farm is put to good use compiling the OpenJDK code and running it through a suite of tests before publishing it.
All of our code is out there in Github for you to study, so you know exactly what those binaries contain and how they were built. We believe in open build and open testing of the open source code!
We started with OpenJDK version 8 as the latest stable code stream. Now that the OpenJDK project is close to declaring the OpenJDK version 9 final, we have started building and testing that too, and expect to have tested binaries available from the AdoptOpenJDK website and via our API simultaneously.
We are also happy to see the proposal for Eclipse OpenJ9 as another main stream open source Java Virtual Machine. Once that code is available we will take a look with a view to building and testing OpenJDK with Eclipse OpenJ9 binaries too.
As an open build, test and distribution project we aim to be the “go to” location for high quality OpenJDK-based runtimes; and we are working hard to earn that reputation. If you are maintaining an OpenJDK derived runtime, drop us a note.
There are many interesting and challenging tasks ahead. Everyone is welcome to participate at whatever level suits you best. Take a look at some of the work we know we need to do, or join us on Slack to talk about your new ideas.
We are grateful to the sponsors who recognize the value of this project and have generously provided services and resources to make it successful.
The next few months are going to be equally exciting! Join in.
Adding zOS to a Jenkins server as an ssh agent can be a difficult task, that’s why I’ve documented the steps that we were required follow to add our zOS machines to Jenkins.
Adding Java to zOS
All Jenkins build agents require Java to be on the machine to run the Slave agent. Normally an OpenJDK binary would be recommended to run the slave agent but there is no official zOS OpenJDK binary so I would recommend fetching IBM’s Java from here.
Adding the Machine to Jenkins
Head over to
https://<jenkins-server>/computer/new, specify a node name and tick permanent agent.
Once you have reached the machine configuration page specify the Launch method as Launch slave agents via SSH. Set the host IP Address, Credentials and Host Key Verification Strategy and then click advanced.
In the advanced page, set the JavaPath as the path to the Java binary that you downloaded earlier and then add the following to JVM Options:
This will allow Jenkins to understand the Ebcdic output from Java on the machine.
You should now be able to successfully launch the Jenkins Java agent on your zOS machines.