How to utilize Jigit for enterprises

Some customers have thousands of repositories that need to be scanned in Jira, but using one configuration makes it slow and can take several days to consume the data.

Tips to improve the indexation process

1. Use multiple configurations.

Try to use multiple configurations that use different users' tokens and cover a specific set of repositories.

GitLab

For GitLab, it can be configurations where every configuration handles some group of repositories.

GitHub

Try to use “GitHub App” for integration. GitHub App usually has a bigger request limit and GitHub App can be used for some subset of repositories which can be extended or narrowed.

See:
https://docs.github.com/en/apps/creating-github-apps/registering-a-github-app/registering-a-github-app
https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/managing-private-keys-for-github-apps

For instance, for 50K repositories, there can be created 5 GitHub Apps, each of which is linked to the 10K repositories.
Repositories can be added manually via UI:

Or adding repositories can be automated via GitHub REST API: https://docs.github.com/en/rest/apps/installations?apiVersion=2022-11-28#add-a-repository-to-an-app-installation

CODE

curl -L \
  -X PUT \
  -H "Accept: application/vnd.github+json" \
  -H "Authorization: Bearer <YOUR-TOKEN>" \
  -H "X-GitHub-Api-Version: 2022-11-28" \
  https://api.github.com/user/installations/1/repositories/REPOSITORY_ID

To get all repositories in the organization the following API can be used: https://docs.github.com/en/rest/repos/repos?apiVersion=2022-11-28#list-organization-repositories

CODE

curl -L \
  -H "Accept: application/vnd.github+json" \
  -H "Authorization: Bearer <YOUR-TOKEN>" \
  -H "X-GitHub-Api-Version: 2022-11-28" \
  https://api.github.com/orgs/ORG/repos

2. Group repositories by name.

If grouping by GitLab groups or GitHub App repositories is not a solution try to use a repositories pattern to restrict the number of repositories per configuration.
Regexp is a Java-based syntax https://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html. To combine multiple patterns use “|“

3. Use “selected branches” branch indexation mode.

Try to avoid the “all branches” branch indexation mode. Due to the development process, sooner or later, feature and bug-fixing branches are merged into the main branch (main/master) via pull requests. During the merge process changes can be squashed and the branch is deleted or the changes can be rejected and the branch is abandoned. Indexing all branches brings more extra work for the plugin and irrelevant data which does not exist in the Git anymore. Try to index only long-living branches.

Use:

In case long-living branches are varied in the repositories, a branch mask can be used:

“master|dev.*|release.*|.*feature.*” - indexes “master“, branches start with “dev”, “release” and ones which contain “feature“.

4. Avoid mixing configurations for the Development panel and commit indexation.

If it is possible try to create different configurations that use different users' tokens (GitHub Apps) for the commit indexation and Development panel. Because commit and branch indexation are different tasks that run on their own. There can be a race condition between who takes the allowed amount of requests - commit indexation might start first and when branch indexation starts no more requests can be allowed.

Commits only indexation configuration:

Branch and pull-request indexation configuration:

If different configurations for commits and the Development panel are not a solution for you, then try first to create a “Use only for Development panel“ configuration and allow it to run a few indexation iterations and then enable it for the commit indexation.

Branch indexation can be tracked on a main configuration page:

154/154 means “processed repositories count“/”total repositories count”.

5. Use Webhooks

Only works when the Git system can reach the Jira instance.

Using webhooks reduces the pressure on the plugin and brings the changes into the Jira as soon as they happen on the GitLab/GitHub side. Webhooks help to get new commits, pull requests and branches.
Use the URL displayed on the global configuration page or substitute it with the one which is reachable from the internet:

Webhook secret can be any value but this value should be used on the Git side as well.

GitLab webhooks:

GitHub webhooks can be configured on a repository, organization or GitHub App level:

The plugin handles only events for the repositories handled by the configurations. If no configuration is found for the repository event then this event is skipped.

6. Use project-level configurations instead of Global configurations.

Because global level configurations affect all the Jira projects and it might not be required to have integration in all Jira projects, it’s strongly advised to create Jigit configurations by Jira project admins and include only those Git repositories which are required for the particular Jira project:

It also solves the issue with request throttling because each Jira project admin uses their own PAT in the configurations.

7. Using Git user permissions.

Usually, it’s not desirable that Jira users can see all the data from the repositories they do not have access to. For this case, the plugin can require that each user maps their token with some configuration.

Jigit Development under Settings tab:

User profile page:

This solution has big disadvantages in performance because each data from the repository should be checked for user Git permissions via REST call to the Git system.

Each branch and the pull-request repository are checked to ensure whether the user has access to the repository using their PAT. Also when the user tries to create a branch or pull request user permissions should be checked for the repository dropdown list:

Updated: 29 Jul 2024