brainsteam.co.uk/brainsteam/content/posts/2024/08/03/Migrating from Linear to Jira.md

categories	date	draft	tags	title	type	url
Uncategorised	2024-08-03 10:11:37	true		Migrating from Linear to Jira	posts	/

Our company recently decided to move from Linear to Jira because we wanted a bunch of the fancier stuff that Jira offers around workflow management and resource burndown. The migration process has been a little painful but not too overwhelming. I wrote up some of my notes and built some Python scripts to facilitate the process so I thought I'd share them here.

Our company uses something like an agile Scrum methodology with a bi-monthly release cycle.

Mapping Linear to Jira Data Structures

Linear has Teams, each team can have multiple projects and then within projects we have stories, tasks and bugs. A project in Linear is typically a time-bound initiative over a number of releases that contains a set of stories and bugs. Stories contain tasks and sub-tasks

Jira's top level data structure is a Project, within a project you can have an Epic which is a time-bound initiative over a number of releases and can contain a number of stories and bugs. Stories contain tasks and sub-tasks.

So with this in mind, we mapped the data structures into something like the following:

Linear Data Structure	Jira Data Structure
Team	[Jira] Project
[Linear] Project	Epic
Story	Story
Bug	Bug
Task	Task
Sub-Task	Sub-Task

A mapping of Linear data structures to Jira data structures

There's a pretty straight forward 1:1 mapping of data from one system to the other. Not bad.

Linear has the concept of Cycles for timekeeping whereas Jira calls them Sprints. We actually decided not to try to map cycles and instead manually sorted that out after the migration.

Getting Data Out of Linear into Jira

Linear offers a CSV export function which is easy enough to use. However, not everything gets exported by this tool - this includes project metadata, comments on issues and attachments. I have provided some support tools that we can use to get those files out.

Getting the Main Tickets Out of Linear

In the Workspace Settings menu, go to Import/Export and scroll down to the Export CSV button. Linear will generate a CSV of your workspace in the background and email it to you.

Upon downloading the CSV file, we can see all of the tickets from all teams in your workspace.

Getting Project Metadata out of Linear

The projects are not represented by rows in the CSV but instead each item has a Project [Name] and Project ID associated with them. What if we want more metadata about each project? For example, the name, description, start date and status. Well I wrote a small script called export_projects.py that fetches all of that information from Linear's GraphQL API and stores it in a separate CSV. You can find it in the git repo.

Use it by running:

export LINEAR_API_KEY=
python export_projects.py projects.csv

You can find out more about how to get a linear API key here. The generated CSV file projects.csv should contain a set of all of your linear projects and associated metadata.

Getting Comments out of Linear

Another shortcoming of the main linear CSV export is that it does not include comments from tickets, only the main ticket body. I've provided a script called export_comments.py which, like the projects script above, will grab all comments directly from the API. Use it like this:

export LINEAR_API_KEY=
python export_comments.py comments.csv

The generated comments.csv should contain all the comments from your projects and the IDs of the parent thread that they address.

Getting Attachments out of Linear

I've also provided a script for exporting attachments - specifically images - from linear. This script goes through the CSV file provided by linear looking for URLS that appear to be attachments. It will then download each of them in turn and place them in a local directory so that we can re-upload them to Jira. This step is necessary because the JIRA import worker cannot directly access images on the linear server.

With the csv file you exported from Linear you should be able to run:

export LINEAR_API_KEY=
python export_images.py --issues-file ./linear_export.csv --output-dir ./images

This script will produce a new directory called images which contains all the attachments and a file called index.csv which essentially holds a catalogue of all the images, what their old URL was in Linear and what their local filename is.

So now you should have 2 CSV files. The one you exported directly from Linear and the projects file created by this script.

Converting Markdown to Atlassian Wiki Markup

Linear uses Markdown for rich descriptions but Atlassian uses their own wiki markup (because y'know, standards). We need to auto-convert the item descriptions from Linear to Atlassian format. We need to do this for the main linear export, the projects and the comments.

When I did a bit of googling for how to go about this I found an example that uses mistletoe markdown to parse markdown and emit jira. I've created a lightweight command-line script called markdown2jira.py that will do this job for you.

python markdown2jira.py -i projects.csv -o projects_converted.csv -c description 
python markdown2jira.py -i linear_export.csv -o linear_export_converted.csv -c Description 
python markdown2jira.py -i comments.csv -o comments_converted.csv -c node.body 

You should now have converted csv files for comments, projects and linear items.

Hosting the Attachments

The JIRA import process does not allow you to directly upload image attachments. Instead, it expects them to be available in some public location or website. My solution to this was to use ngrok to provide a temporary public tunnel to the images directory on my laptop.

You will need an ngrok api key which you can get for free by signing up here, then you can run the following:

export NGROK_AUTHTOKEN=
python attachment_host.py host-ngrok --path ./images --csv-outfile ./attachments.csv

This will run a server which we will need to leave running while the JIRA import takes place. Ngrok's free tunnels time out after 8 hours so bear in mind that if you choose now to take a break and go and do something else, you might need to re-run this step (also if you have a really massive project).

nb: I was also toying with the idea of writing a version of this that uploads all the images to S3 or Google storage but I haven't had time. Pull Requests welcome.

You should now also have a new file called attachments.csv - this is a map generated by the server which converts from the old image url in the JIRA ticket to the new one in your ngrok server.

Merge, Link and Tidy up Timestamps

Ok so the next step (script) is actually doing three things at once:

1. We need to merge the two converted CSV files together so that we have a single file that we can upload using JIRA's import system.
2. We need to link together issues with their parents properly and make sure that issues which specify a Linear Project ID but no parent ID (because in Linear they were top level issues in the project) get mapped to the relevant Epic in the new world.
3. Linear uses some very strange date strings when it exports by default so we will need to tidy those up and write them back out in a simple format that is easy for Jira to parse.

The script merge_link.py handles all of these operations.