When looking up information on how to migrate from Bitbucket to GitHub, it is easy to find a lot of information about how to clone one repository from Bitbucket to GitHub or how to prepare a specific pipeline to replicate the changes, but… what happens when you want to migrate an entire organisation’s codebase?
In this post, we’re going to share with you the details about what you should consider if this challenge appears in your company.
First of all, you need to consider the differences between Bitbucket and GitHub:
- Default reviewers vs CODEOWNERS
- Snippets vs Gists
- Huge file management
First step: Analysis
You might think at the very beginning that migrating your codebase is pretty straightforward. However, we strongly recommend considering the number of repositories to migrate before taking any action because this will increase the complexity of the project considerably. In our case, more than four hundred should be migrated and for that reason, we decided to create two mechanisms, a Jenkins pipeline to sync all the repositories and webhooks in the repositories to sync specific ones that receive some change on it.
Apart from that, we need to use other processes to create the repositories and also migrate the metadata. For that, we recommend using this Terraform tool that, combined with the Bitbucket API, gives us all the necessary information to generate your repositories with the proper data, branch restrictions, user access and it is also important to consider the difference between Code-Owners, in GitHub, that are defined in a repository file (CODEOWNERS), and the default reviewers, in Bitbucket, that are manually configured in each repository.
The migration plan will also be influenced by dependencies between repositories. Identifying the order that repositories are migrated is an important part of the overall strategy to avoid numerous pain points later on.
Second step: Pipelines implementations
As we mentioned, we used Jenkins pipelines to sync the repositories from Bitbucket to GitHub. This is the flow;
If you have snippets, you’ll require a separate one to sync them because the snippets don’t exist in GitHub in the same way as Bitbucket (Gists), for that reason they need to cloned and a temporal project created to include all of them This results in a project called “snippets” in GitHub that contains a folder with the data for each snippet.
Another important point to consider is the management of huge files in your repositories. In Bitbucket you “don’t have per-file limits” but in GitHub hard restrictions exist relating to files with more than 100MB. In these scenarios it’s necessary to use the Git LFS tool and manage those files with it.
Third step: Adapt references
Some of our repositories have references and we divided them in three types:
- Doc references. References that are related to Bitbucket repositories URLs, included in README files.
- Read references. References that are related to Bitbucket repositories URLs as dependencies to clone them.
- Write references. References that are related to Bitbucket repositories URLs with hard Bitbucket dependencies like credentials, API endpoints, etc.
Fourth step: Pilot team(s)
It is important to have a team familiar with GitHub instead of Bitbucket so that they can hit the ground running and identify possible issues. This step could be skipped if your organisation isn’t huge but is recommended to help prevent issues in a full migration and avoid rollback process.
Fifth step: GitHub as source of true
The final step sees all teams start to work with GitHub. However, you should have prepared or adapted the previous process to synchronise GitHub with Bitbucket just in case there is a major issue and it becomes necessary to revert the process BUT we have complete confidence that with the careful planning and steps we’ve described that you’ll certainly SUCCEED!
As a final tip, this is a project that requires help from all the tech departments, not just one team! So don’t hesitate to request help from your colleagues as we did.
Special thanks to Fernando López, Victor Cañadas, Alberto Carrique, Luis Piedra, Adrián Matellanes and all the people that collaborated to help achieve this goal!