Enabling pagination over 100 issues in the Jira Lookup Issues Action

[Benjamin Baker]

I figured out you can get around the 100-issue limit in the Jira automation Lookup Issues action by leveraging the {{lookupIssues.last}} value, specifically in this example lookupIssues.last.key but I imagine this could be adapted for other fields.

1. Do your JQL query in the Lookup Issues action.

2. Do whatever you want with the first 100 results.

3. Create variable action, call it myKey, set it to: {{if(equals(lookupIssues.size, 0), "outOfScopeKey", lookupIssues.last.key)}}.  The placeholder outOfScopeKey is something you know won't return anything from the next JQL lookup. For example, if you're filtering for issues created in the last day, use a key you know is older than a day. If the lookup issues size is 0, the conditional logic will return the out-of-scope key, if anything else, will return the last key. 

4. Repeat the JQL search, but add "and key < {{myKey}}".  This will get up to the next 100 issues if there are over 100 issues in the first call and will return nothing otherwise.

This process can then be repeated as many times as you want!

[Bill Sheboy]

Hi @Benjamin Baker 

I am glad to learn you found a workaround for your scenario, and...

That approach will work when the following conditions apply:

• The JQL statement has an ORDER BY statement, allowing the use of the last issue's data in the next query.  In your case, the sort would be Key; probably ORDER BY Key DESC based on what you describe.
• The actions performed by the branch on the issues must:
  • not impact the results of the JQL query
  • not be order dependent, as the branch(es) will execute in parallel and asynchronously
• There is a maximum, known number of "chunks" of 100 issues needed, and it fits within the 65 component rule limit

There are other workarounds for this scenario, such as self-triggering rules with webhooks.  All workaround approaches for this limit have pros / cons, and so please confirm which works for you.  Thanks!

Kind regards,
Bill

[Sidharth verma]

Hi Bill,

can you tell me what the workaround is using webhooks if i want to fetch more than 100 issues ?

Regards,

Sid

[Bill Sheboy]

Hi @Sidharth verma 

I have not used that approach, although I have seen community threads describing examples of it.  The basic approach is:

• Create a webhook triggered rule, with only a log action, and save it.  This is the rule which will eventually loop.
  • Disable the rule.
  • Copy the webhook from that rule and save it somewhere
• Create your starting scenario rule, which determines the number of issues / things using the Send Web Request action and the REST API issue search function with JQL
  • In the response, it will contain the number of issues (and so pages) to process in maxResults
  • Add a Send Web Request action to trigger the webhook saved earlier, sending custom data with the JQL and the startAt (i.e., paging) to access.  The default will be 0 for the first record.
• Edit your webhook triggered rule...
  • to accept the custom data,
  • parse it for you needs (e.g. calling the issue search again),
  • update the custom data to advance to the next page, and
  • finally use Send Web Request to trigger this rule again...without waiting for a response / success
• Enable both rules and test thoroughly

Please note well: self-triggering and looping rules can quickly exceed the automation limit (of 10 loops).  When that happens, it will shut down all of your automation rule execution.  If you have any other solution approaches, I recommend trying those first.

Kind regards,
Bill