Can Script runner cause JIRA to run out of memory?

[Vishy Subramanyam]

We have scripted fields that we are using in our application and sometimes the JIRA application crashes with an out of memory condition. When we submitted the logs to JIRA, they came back saying it is the Script runner and for us to contact the plugin company. Can you let us know your thoughts on this?

Thanks

[Vishy Subramanyam]

Thanks to Jamie for helping us get to the bottom of the issue.

The team was able to re-create the issue using the issueFieldMatch function - for some reason invoking this function causes JIRA to use up 100% of the memory and crash - it is something to do with the interaction between JIRA and Script Runner.  We are not really sure where the issue is.  I am sure this issue will be fixed in an upcoming release either by Atlassian or Jamie. The following functions within Script Runner uses the same class - ComponentMatch, IssueFieldExactMatch, IssueFiedMatch, ProjectMatch.  We disabled these 4 functions and have not had the crash since then.

Thanks everyone for your support and help.

[Vishy Subramanyam]

Hi Jamie We sent you some data to your email. Please could you verify it there and let us know what you see via email so we can close this ticket or update it once the issue is resolved. Thanks for your help in advance.

[Vishy Subramanyam]

Do you have any ftp site or something similar where i can upload this file of 6 GB size? Please let me know. The other option is if you can update the JIRA ticket on Atlassian with your contact info then I can send an email to that account with some ways to get the data to you for your review.

[JamieA]

Meh. It's like saying "can a plugin cause an out of memory error?". Answer: of course it can.

Script Runner on its own doesn't degrade performance, unless you can prove to me it does.

What do your scripted fields do? Are they leaking database connections or filehandles or keeping references to stuff somehow?

Have you tried increasing the memory?

[Nic Brough -Adaptavist-]

I've made the script runner cause out of memory errors loads of times.

It's the code I put in it. Not the script runner. Take my code out, it's fine.

[Bob Swift OSS (Bob Swift Atlassian Apps)]

Make sure you understand if the problem you have is with permgen space or overall memory. It will make a difference on how you increase memory. Groovy can use permgen space.

[Nic Brough -Adaptavist-]

That's one of my nightmares too. Ok, so I've been paid well to sort it out a couple of times, but it still causes the nightmares...

[JamieA]

Yes, groovy will generate classes from scripts which will be in permgen. Also you may need to increase -XX:ReservedCodeCacheSize.

> I've made the script runner cause out of memory errors loads of times

Me too... also created infinite chains of comments by writing a comment in a comment listener, infinite numbers of subtasks too, and loads of other disastrous things.

I have this recurring nightmare that people are writing code directly in their production instances, without testing on a staging server. It's just a bad dream though, I'm sure that doesn't really happen.

[Vishy Subramanyam]

LoL - OK folks - thanks for the input. We have a dev, test, train and prod environments and everything goes through testing. Further analysis shows that it is not really the script runner but more of the email handler that is causing the issue.

Either ways, more work is going on to determine the exact cause. All I was trying to get at was to see if there is a known issue with script runner.... looks like there are no known issues. We have a couple of scripted fields something simple based on the value of one - the other field is set on the issue.

We will see about increasing the ReserverCodeCacheSize but I think it is a matter of finding out what is going on with the mail handler... Thanks folks

[JamieA]

Cheers for that. You only need increase ReserverCodeCacheSize if your heap or permgen is low and you get an error "out of space in code cache for adapters". If you don't, don't worry about it.

[Vishy Subramanyam]

Thanks we will look at that. Appreciate the help. Is there a recommended value for the Cache Size? We have about 1000 users, around 100K in terms of issues. Please let me know.

[Pritish Sinha]

Hi @Jamie Echlin [Adaptavist] , Our production instance went down again. We raised an issue with Atlassian and they are saying that its the script runner plugin, specifically the JQL search that is causing the memory to max out. They asked us to create a Heap Dump and pointed to com.onresolve.jira.groovy.jql.AbstractScriptedJqlFunction.getIssues. We are seeing memory spikes to 100% and then the system crashes.

[JamieA]

AbstractScriptedJqlFunction is involved in all script runner jql functions, even the ones that were not written by me. I think you should find which function is causing it. Maybe you can look in the access log, or the jira slow queries log.

[JamieA]

Is the heap dump huge? Maybe there is some way of getting it to me, like add me to your SAC support ticket or something. If it's an attachment on your ticket, add me "cc users", my id is jamie.echlin.

[Vishy Subramanyam]

The heap dump is around 6GB. Let me see if I can add you to the ticket that may be easier than sending the large dump files.

[Pritish Sinha]

Files and details sent through email. Thx!