Migrating repositories from Gitblit to Stash

an efficient conversion of ~200 repositories, by Joel Kleier

While we loved using Gitblit the last few years, we wanted a more integrated toolset.

Wildcard decided to move a whole lot of repositories from Gitblit into Stash – almost 200. We wanted to do this in a single quick conversion, rather than trying to run two systems and slowly migrating repos.

A little bit of bash, curl, and the Stash REST API made the whole process take about 12 hours in all – including development, testing, execution, and then cleanup (e.g., making sure team members were up to speed and including documentation fixes).

First, I'd recommend taking a gander at the Stash API documentation; it has a lot of useful information and can manipulate most aspects of the system!

It can, for example, automatically create a project:

curl -u api_user:pass -X POST -H "Content-Type: application/json" -d @project.json "https://example.org/rest/api/1.0/projects"

where `@project.json` is an `@` followed by a path to a file that contains JSON formatted data for creating the project:

"key": "EX",
"name": "Example Project",
"description": "just an example project"

It can also automatically create a repo for a project:

curl -u api_user:pass -X POST -H "Content-Type: application/json" -d @repo.json "https://example.org/rest/api/1.0/projects/EX/repos"

Like the one for creating a project, here's the `repo.json` file:

"name": "example repo",
"scmId": "git"

Our overall process was to create a new project and repository in Stash (since the organization in our old system was largely incompatible with Stash), clone all branches and tags to a local copy, reset the remote 'origin' url on each repo, then push the repo with all of its tags and branches up to its new location in Stash.

The structure we decided to go with in Stash is fairly light on the "projects" Stash defines, so we just created those by hand. Our number of projects, however, is much larger and we needed some automation.

Combining a little bash with some API calls, we came up with a little script similar to the following:

#! /usr/env/bin bash
# user credentials for calling into the api
# loop through each line
while IFS='' read -r line || [[ -n "$line" ]]; do
# each line is a list of values split by ';' IFS=';' read -ra parts <<< "$line"
# store each value in a line in a more readable variable name projkey=${parts[0]}
# print some info about what's being done currently echo $projkey/$reposlug
# create the repository
curl -X POST -u $username:$pass -H "Content-Type: application/json" https://example.org/rest/keys/1.0/projects/$projkey/repos -d '{"name":"'$reponame'","scmId":"git"}'
# pipe the file passed as the first parameter into the while loop's `read` done < "$1"

You'd use the script something like this:

$ generate_repos.sh repolist.txt

And `repolist.txt` would look something like:


(it contains project key - project name mappings)

Then, we had a script run through the process of cloning our repos (including all branches and tags), and resetting the origin URLs:

#!/usr/bin/env bash
while IFS='' read -r line || [[ -n "$line" ]]; do
    IFS=';' read -ra parts <<< "$line"

echo "changing ${oldurl}" cd /path/to/temporary/repository/storage/base/
# get a clone of the repository, including branches and tags git clone $oldurl ./$name cd ./$name git branch -r | grep -v HEAD | grep –v master | awk -F'origin/' '{print $2 " " $1"origin/"$2}' | xargs -L 1 git branch -f --track git fetch --all git pull --all
# change the origin to the new url git remote set-url origin $newurl
# push the repo into the new location (including branches and tags git push --all git push --tags done < "$1"

The script would be called like:

$ clone_repos.sh repolist.txt

And `repolist.txt` would look something like:


And then it's a matter of going through the code base and deployment configuration to find any references to your old repositories and make sure they all work in the way you expect!

While we loved using Gitblit the last few years, we wanted a more integrated toolset.