Jigit - Performance considerations
Performance considering for Jigit. Includes information about frequency of indexing and requests to the 3rd party system.
Jigit App uses GitLab/GitHub REST APIs to index commit messages against Jira issues. Those APIs can be throttled, as a result, the indexing should be spread over time to smooth the load, otherwise Git systems will reject requests of Jigit.
It is done in the following way:
There is a scheduled task that runs every 2 minutes (can be changed with
moveworkforward.jigit.index.interval)It checks the head commit and indexes it. By indexing meant:
extracts Jira issue keys
stores those in the db together with the commit information
adds to the internal queue the next commit (if not was not indexed before)
sleeps every X requests to the GitLab (configurable)
If commit had been indexed, the whole index process for that branch/repo combination stops
The scheduled task is run from 1 Data Center node
Throttling related parameters
Commit Request Timeout - tell for how long the connection should be open to call the GitHub/GitLab api before it times out. Default and recommended - 10 seconds.
Sleep - the time Jigit will sleep between request batches (controlled by parameter each X requests). Default and recommended value is 10.
Each X requests - is the amount of requests in each request branch before the sleep. Default and recommended value is 10.
The logic behind those values are: GitHub has limit of 5000 requests per hour per agent (Jigit is an Agent in this case), which is around 83 per minute. With the default settings Jigit may execute 60 requests per minute, which is close, but still below 83.
A couple of other performance considerations:
when the App runs first time it may take a while to index an old repository with multiple commits
there is a scheduled task per each configuration rule.
We will publish the Data Center performance testing results.
Updated: