I ran into this myself today and through some bruteforce I've figured out both how to resolve it and potentially why.

Probably best to start with the why:

Jenkins has a paradigm where all jobs can be interrupted, paused and resumable through server reboots. To achieve this the pipeline and its data must be fully serializable - IE it needs to be able to save the state of everything. Similarly, it needs to be able to serialize the state of global variables between nodes and sub-jobs in the build, which is what I think is happening for you and I and why it only occurs if you add that additional build step.

For whatever reason JSONObject's aren't serializable by default. I'm not a Java dev so I cannot say much more on the topic sadly. There are plenty of answers out there about how one may fix this properly though I do not know how applicable they are to Groovy and Jenkins. See this post for a little more info.

How you fix it:

If you know how, you can possibly make the JSONObject serializable somehow. Otherwise you can resolve it by ensuring no global variables are of that type.

Try unsetting your object var or wrapping it in a method so its scope isn't node global.

Use JsonSlurperClassic instead.

Since Groovy 2.3 (note: Jenkins 2.7.1 uses Groovy 2.4.7) JsonSlurper returns LazyMap instead of HashMap. This makes new implementation of JsonSlurper not thread safe and not serializable. This makes it unusable outside of @NonDSL functions in pipeline DSL scripts.

However you can fall-back to groovy.json.JsonSlurperClassic which supports old behavior and could be safely used within pipeline scripts.

Example

import groovy.json.JsonSlurperClassic




@NonCPS
def jsonParse(def json) {
new groovy.json.JsonSlurperClassic().parseText(json)
}


node('master') {
def config =  jsonParse(readFile("config.json"))


def db = config["database"]["address"]
...
}

ps. You still will need to approve JsonSlurperClassic before it could be called.

EDIT: As pointed out by @Sunvic in the comments, the below solution does not work as-is for JSON Arrays.

I dealt with this by using JsonSlurper and then creating a new HashMap from the lazy results. HashMap is Serializable.

I believe that this required whitelisting of both the new HashMap(Map) and the JsonSlurper.

@NonCPS
def parseJsonText(String jsonText) {
final slurper = new JsonSlurper()
return new HashMap<>(slurper.parseText(jsonText))
}

Overall, I would recommend just using the Pipeline Utility Steps plugin, as it has a readJSON step that can support either files in the workspace or text.

The way pipeline plugin has been implemented has quite serious implications for non-trivial Groovy code. This link explains how to avoid possible problems: https://github.com/jenkinsci/pipeline-plugin/blob/master/TUTORIAL.md#serializing-local-variables

In your specific case I'd consider adding @NonCPS annotation to slurpJSON and returning map-of-maps instead of JSON object. Not only the code looks cleaner, but it's also more efficient, especially if that JSON is complex.

A slightly more generalized form of the answer from @mkobit which would allow decoding of arrays as well as maps would be:

import groovy.json.JsonSlurper


@NonCPS
def parseJsonText(String json) {
def object = new JsonSlurper().parseText(json)
if(object instanceof groovy.json.internal.LazyMap) {
return new HashMap<>(object)
}
return object
}

NOTE: Be aware that this will only convert the top level LazyMap object to a HashMap. Any nested LazyMap objects will still be there and continue to cause issues with Jenkins.

I found more easy way in off docs for Jenkins pipeline

Work example

import groovy.json.JsonSlurperClassic




@NonCPS
def jsonParse(def json) {
new groovy.json.JsonSlurperClassic().parseText(json)
}


@NonCPS
def jobs(list) {
list
.grep { it.value == true  }
.collect { [ name : it.key.toString(),
branch : it.value.toString() ] }


}


node {
def params = jsonParse(env.choice_app)
def forBuild = jobs(params)
}

Due to limitations in Workflow - i.e., JENKINS-26481 - it's not really possible to use Groovy closures or syntax that depends on closures, so you can't > do the Groovy standard of using .collectEntries on a list and generating the steps as values for the resulting entries. You also can't use the standard > Java syntax for For loops - i.e., "for (String s: strings)" - and instead have to use old school counter-based for loops.

The other ideas in this post were helpful, but not quite all I was looking for - so I extracted the parts that fit my need and added some of my own magix...

def jsonSlurpLaxWithoutSerializationTroubles(String jsonText)
{
return new JsonSlurperClassic().parseText(
new JsonBuilder(
new JsonSlurper()
.setType(JsonParserType.LAX)
.parseText(jsonText)
)
.toString()
)
}

Yes, as I noted in my own git commit of the code, "Wildly-ineffecient, but tiny coefficient: JSON slurp solution" (which I'm okay with for this purpose). The aspects I needed to solve:

1. Completely get away from the java.io.NotSerializableException problem, even when the JSON text defines nested containers
2. Work for both map and array containers
3. Support LAX parsing (the most important part, for my situation)
4. Easy to implement (even with the awkward nested constructors that obviate @NonCPS)

Noob mistake on my part. Moved someones code from a old pipeline plugin, jenkins 1.6? to a server running the latest 2.x jenkins.

Failed for this reason: "java.io.NotSerializableException: groovy.lang.IntRange" I kept reading and reading this post multiple times for the above error. Realized: for (num in 1..numSlaves) { IntRange - non-serializable object type.

Rewrote in simple form: for (num = 1; num <= numSlaves; num++)

All is good with the world.

I do not use java or groovy very often.

Thanks guys.

This is the detailed answer that was asked for.

The unset worked for me:

String res = sh(script: "curl --header 'X-Vault-Token: ${token}' --request POST --data '${payload}' ${url}", returnStdout: true)
def response = new JsonSlurper().parseText(res)
String value1 = response.data.value1
String value2 = response.data.value2


// unset response because it's not serializable and Jenkins throws NotSerializableException.
response = null

I read the values from the parsed response and when I don't need the object anymore I unset it.

I want to upvote one of the answer: I would recommend just using the Pipeline Utility Steps plugin, as it has a readJSON step that can support either files in the workspace or text: https://jenkins.io/doc/pipeline/steps/pipeline-utility-steps/#readjson-read-json-from-files-in-the-workspace

script{
def foo_json = sh(returnStdout:true, script: "aws --output json XXX").trim()
def foo = readJSON text: foo_json
}

This does NOT require any whitelisting or additional stuff.

According to best practices posted on Jenkins blog (Pipeline scalability best practice), it is strongly recommended to use command-line tools or scripts for this kind of work :

Gotcha: especially avoid Pipeline XML or JSON parsing using Groovy’s XmlSlurper and JsonSlurper! Strongly prefer command-line tools or scripts.

i. The Groovy implementations are complex and as a result more brittle in Pipeline use.

ii. XmlSlurper and JsonSlurper can carry a high memory and CPU cost in pipelines

iii. xmllint and xmlstartlet are command-line tools offering XML extraction via xpath

iv. jq offers the same functionality for JSON

v. These extraction tools may be coupled to curl or wget for fetching information from an HTTP API

Thus, it explains why most solutions proposed on this page are blocked by default by Jenkins security script plugin's sandbox.

The language philosophy of Groovy is closer to Bash than Python or Java. Also, it means it's not natural to do complex and heavy work in native Groovy.

Given that, I personally decided to use the following :

sh('jq <filters_and_options> file.json')

See jq Manual and Select objects with jq stackoverflow post for more help.

This is a bit counter intuitive because Groovy provides many generic methods that are not in the default whitelist.

If you decide to use Groovy language anyway for most of your work, with sandbox enabled and clean (which is not easy because not natural), I recommend you to check the whitelists for your security script plugin's version to know what are your possibilities : Script security plugin whitelists

You can use the following function to convert LazyMap to a regular LinkedHashMap (it will keep the order of original data):

LinkedHashMap nonLazyMap (Map lazyMap) {
LinkedHashMap res = new LinkedHashMap()
lazyMap.each { key, value ->
if (value instanceof Map) {
res.put (key, nonLazyMap(value))
} else if (value instanceof List) {
res.put (key, value.stream().map { it instanceof Map ? nonLazyMap(it) : it }.collect(Collectors.toList()))
} else {
res.put (key, value)
}
}
return res
}


...


LazyMap lazyMap = new JsonSlurper().parseText (jsonText)
Map serializableMap = nonLazyMap(lazyMap);

or better use a readJSON step as noticed in earlier comments:

Map serializableMap = readJSON text: jsonText

If you can't use JsonSlupurClassic, here a simple way to transform LazyMap to LinkedHashMap.

def deserialize(String jsonStr) {
def obj = new JsonSlurper().parseText(jsonStr)
if (obj instanceof Map) {
return cloneMap(obj)
} else if (obj instanceof List) {
return cloneList(obj)
} else {
return obj
}
return
}


def cloneMap(Map map) {
def res = [:]
map.each {e ->
def key = e.getKey()
def value = e.getValue()
if (value instanceof Map) {
res.put(key, cloneMap(value))
} else if (value instanceof List) {
res.put(key, cloneList(value))
} else {
res.put(key, value)
}
}
return res
}


def cloneList(List list) {
def res = []
list.each {i ->
if (i instanceof List) {
res.add(cloneList(i))
} else if (i instanceof Map) {
res.add(cloneMap(i))
} else {
res.add(i)
}
}
return res
}