Death by Jenkins - Really difficult bits to find out about Declarative Pipeline DSL (Part 2)

As I said in my last post, I'm not a build engineer and I don't know groovy. I am currently working in a team that is creating a cross-platform system that integrates iOS and Android SDKs to be used as a shared library in C++. But somehow I've become the Jenkins Lady of the team...

So first thing I want to say is that I think there is a serious problem of not error-ing with Pipeline.

Say you have the following piece of code:

You have separate methods to load separate groovy files and run them with different parameters.
This seems fairly straightforward and valid to me.

But not to Jenkins Pipeline!

This will result in a successful Pipeline build but your external script, in this case neither build.groovy nor test.groovy will load. And it won't give you any output either.

So then I spend hours Googling and trying things out.
And well, it turns out, you cannot have the same method name in your external script.
Because the Jenkinsfile had two method calls to "run()", it could not figure out which it needed to call and failed silently.

So if I changed my code to the following, the external scripts are loaded as expected.


That was the first problem.

A few days later, I ran into the same exact problem. AGAIN! But now my scripts all had differently named methods so I had to go Googling and trying things all over again.

Now, my Jenkinsfile had some other methods that needed to go through a map and add the key to a separate array if the value was true. Like this:

While making this change, I first got an error saying:
"Calling public static java.util.Map org.codehaus.groovy.runtime.DefaultGroovyMethods.each(java.util.Map,groovy.lang.Closure) on a CPS-transformed closure is not yet supported (JENKINS-26481); encapsulate in a @NonCPS method, or use Java-style loops"

After confirming on various sites (like here) that you need to do this if you want to use .each in Pipeline, I put @NonCPS on prepareTests() and everything was all green (y)

But then of course I realised it wasn't loading the script.

Turns out you can't load scripts in a method that has or calls other methods with @NonCPS.

I have since found this document Pipeline Best Practices and it does say:
"Beware for (Foo f: foos) loops and Groovy closure-style operators like .each and the like. They will not work right in normal Scripted Pipeline contexts where Pipeline steps are involved directly."

So I wish I had known that earlier but the solution here is to use a for loop and give up on nicer .each syntax.
(also another link I have since found has other people complaining about Pipeline! yay friends! :P )

Well of course my misery doesn't end there and I could go on about how much Pipeline's been more of a headache than help but now our build process looks pretty so yay! x)

I feel like I've been rather grumpy writing these blogs and complaining about Pipeline but obviously there are benefits and I've been told by my team that it's been useful for them so all is good :) 

Death by Jenkins - Really difficult bits to find out about Declarative Pipeline DSL (Part 1)

I haven't written in a long while, but I've been working with Jenkins for the past 4 weeks (yes, I've been working for over 2 years now! no, I am not a build engineer), and I found it so excruciatingly painful to find answers about Jenkins Pipeline syntax on the Internet, that I thought I needed to update my blog and save others from a similar fate x)

Some context:
I've been working on converting our myriad of Jenkins jobs into the new Jenkins Pipeline job.
Jenkins Pipeline, if you don't already know what it is, is a new way of setting up CI. Every time you push code, it goes through different stages (Build, Unit Test, Integration Test, etc) so that by the end of the pipeline, if it all passes, you should be super confident about the code you committed, that it can be released automatically.

Commit => Build => Test => Deploy

(https://jenkins.io/doc/book/pipeline/)

It had been on my mind for a while that our Jenkins jobs were getting out of hand and we needed to migrate to using the DSL. Once you have two sets of 10 jobs, it starts to become unmanageable...
But wasn't sure if I should use a Pipeline or standard DSL linked to a job. In the end, I decided to go for the Pipeline because Jenkinsfile scanning using github enterprise and workspace management came with existing Pipeline plugins and it would tidy up all the jobs we had into one job per branch. I still don't know if this was the correct decision because I find that having only one job makes it more difficult to look through workspaces because the pipeline UI is a bit clunky.
Anyway, I'm rambling, let's get to the code.

The most annoying thing I could not find on the web was to re-assign/reset a global variable set in the environment block.

Here is a simple boolean environment variable:

Now I want to change this value when the Step succeeds within a try-catch block:

This doesn't even error but if I do an echo isSuccess before and after, I get false both times.

So after hours of Googling, I finally found out you need to override the environment variable. 

But how? 

The documentation merely says the below, and I couldn't find any examples anywhere!

(https://jenkins.io/doc/pipeline/steps/workflow-basic-steps/#code-withenv-code-set-environment-variables)

So I after a few trial and error, I finally got it. The correct syntax is:

But this didn't work either. Now I was getting a type mismatch. Because when you defined isSuccess = false in the environment, it was kindly casting it to a string "false"!

Soooooo:

Finally, it was setting the correct value and now in my next stage, I could check if(isSuccess=="true")

Maybe you can properly define a boolean variable using: def  isSuccess = true

but I didn't have the patience to try this out so let me know if you try and it works!

After all of this, turns out I didn't want a try catch and so I didn't need to set a global variable but hey at least I know something now and I've shared it here so I hope this is helpful to someone out there, so my efforts weren't for nothing! 

Part 2 coming up with my finding about:

- loaded groovy script method cannot have the same name

- @NonCPS doesn't let you load scripts - problem with using .each