214 lines
12 KiB
Markdown
214 lines
12 KiB
Markdown
---
|
|
categories:
|
|
- Uncategorised
|
|
date: '2024-08-03 10:11:37'
|
|
draft: true
|
|
tags: []
|
|
title: Migrating from Linear to Jira
|
|
type: posts
|
|
url: /
|
|
---
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>Our company recently decided to move from Linear to Jira because we wanted a bunch of the fancier stuff that Jira offers around workflow management and resource burndown. The migration process has been a little painful but not too overwhelming. I wrote up some of my notes and built some Python scripts to facilitate the process so I thought I'd share them here.</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>Our company uses something like an agile Scrum methodology with a bi-monthly release cycle.</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:heading -->
|
|
<h2 class="wp-block-heading">Mapping Linear to Jira Data Structures</h2>
|
|
<!-- /wp:heading -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>Linear has Teams, each team can have multiple projects and then within projects we have stories, tasks and bugs. A project in Linear is typically a time-bound initiative over a number of releases that contains a set of stories and bugs. Stories contain tasks and sub-tasks</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>Jira's top level data structure is a Project, within a project you can have an Epic which is a time-bound initiative over a number of releases and can contain a number of stories and bugs. Stories contain tasks and sub-tasks.</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>So with this in mind, we mapped the data structures into something like the following:</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:table -->
|
|
<figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th>Linear Data Structure</th><th>Jira Data Structure</th></tr></thead><tbody><tr><td>Team</td><td>[Jira] Project</td></tr><tr><td>[Linear] Project</td><td>Epic</td></tr><tr><td>Story</td><td>Story</td></tr><tr><td>Bug</td><td>Bug</td></tr><tr><td>Task</td><td>Task</td></tr><tr><td>Sub-Task</td><td>Sub-Task</td></tr></tbody></table><figcaption class="wp-element-caption">A mapping of Linear data structures to Jira data structures</figcaption></figure>
|
|
<!-- /wp:table -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>There's a pretty straight forward 1:1 mapping of data from one system to the other. Not bad.</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>Linear has the concept of Cycles for timekeeping whereas Jira calls them Sprints. We actually decided not to try to map cycles and instead manually sorted that out after the migration.</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:heading -->
|
|
<h2 class="wp-block-heading">Getting Data Out of Linear into Jira</h2>
|
|
<!-- /wp:heading -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>Linear offers a CSV export function which is easy enough to use. However, not everything gets exported by this tool - this includes project metadata, comments on issues and attachments. I have provided some support tools that we can use to get those files out.</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:heading {"level":3} -->
|
|
<h3 class="wp-block-heading">Getting the Main Tickets Out of Linear</h3>
|
|
<!-- /wp:heading -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>In the Workspace Settings menu, go to<strong><em> Import/Export</em></strong> and scroll down to the Export CSV button. Linear will generate a CSV of your workspace in the background and email it to you.</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>Upon downloading the CSV file, we can see all of the tickets from all teams in your workspace. </p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:heading {"level":3} -->
|
|
<h3 class="wp-block-heading">Getting Project Metadata out of Linear</h3>
|
|
<!-- /wp:heading -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>The projects are not represented by rows in the CSV but instead each item has a Project [Name] and Project ID associated with them. What if we want more metadata about each project? For example, the name, description, start date and status. Well I wrote a small script called <code>export_projects.py</code> that fetches all of that information from Linear's GraphQL API and stores it in a separate CSV. You can find it in the git repo.</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>Use it by running:</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:enlighter/codeblock {"language":"bash"} -->
|
|
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">export LINEAR_API_KEY=<your-api-key>
|
|
python export_projects.py projects.csv</pre>
|
|
<!-- /wp:enlighter/codeblock -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>You can find out more about how to get a linear API key <a href="https://developers.linear.app/docs/graphql/working-with-the-graphql-api">here</a>. The generated CSV file <code>projects.csv</code> should contain a set of all of your linear projects and associated metadata.</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:heading {"level":3} -->
|
|
<h3 class="wp-block-heading">Getting Comments out of Linear</h3>
|
|
<!-- /wp:heading -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>Another shortcoming of the main linear CSV export is that it does not include comments from tickets, only the main ticket body. I've provided a script called <code>export_comments.py</code> which, like the projects script above, will grab all comments directly from the API. Use it like this:</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:enlighter/codeblock {"language":"bash"} -->
|
|
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">export LINEAR_API_KEY=<your-api-key>
|
|
python export_comments.py comments.csv</pre>
|
|
<!-- /wp:enlighter/codeblock -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>The generated <code>comments.csv</code> should contain all the comments from your projects and the IDs of the parent thread that they address.</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p></p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:heading {"level":3} -->
|
|
<h3 class="wp-block-heading">Getting Attachments out of Linear</h3>
|
|
<!-- /wp:heading -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>I've also provided a script for exporting attachments - specifically images - from linear. This script goes through the CSV file provided by linear looking for URLS that appear to be attachments. It will then download each of them in turn and place them in a local directory so that we can re-upload them to Jira. This step is necessary because the JIRA import worker cannot directly access images on the linear server.</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>With the csv file you exported from Linear you should be able to run:</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:enlighter/codeblock -->
|
|
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">export LINEAR_API_KEY=<your-api-key>
|
|
python export_images.py --issues-file ./linear_export.csv --output-dir ./images</pre>
|
|
<!-- /wp:enlighter/codeblock -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>This script will produce a new directory called images which contains all the attachments and a file called index.csv which essentially holds a catalogue of all the images, what their old URL was in Linear and what their local filename is.</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>So now you should have 2 CSV files. The one you exported directly from Linear and the projects file created by this script.</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:heading -->
|
|
<h2 class="wp-block-heading">Converting Markdown to Atlassian Wiki Markup</h2>
|
|
<!-- /wp:heading -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>Linear uses Markdown for rich descriptions but Atlassian uses their own wiki markup (because y'know, <a href="https://xkcd.com/927/">standards</a>). We need to auto-convert the item descriptions from Linear to Atlassian format. We need to do this for the main linear export, the projects and the comments.</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>When I did a bit of googling for how to go about this I found an example that uses <a href="https://github.com/miyuchina/mistletoe">mistletoe markdown</a> to parse markdown and emit jira. I've created a lightweight command-line script called <code>markdown2jira.py</code> that will do this job for you.</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:enlighter/codeblock {"language":"bash"} -->
|
|
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">python markdown2jira.py -i projects.csv -o projects_converted.csv -c description
|
|
python markdown2jira.py -i linear_export.csv -o linear_export_converted.csv -c Description
|
|
python markdown2jira.py -i comments.csv -o comments_converted.csv -c node.body </pre>
|
|
<!-- /wp:enlighter/codeblock -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>You should now have converted csv files for comments, projects and linear items.</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:heading {"level":3} -->
|
|
<h3 class="wp-block-heading">Hosting the Attachments</h3>
|
|
<!-- /wp:heading -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>The JIRA import process does not allow you to directly upload image attachments. Instead, it expects them to be available in some public location or website. My solution to this was to use <a href="https://ngrok.com/">ngrok</a> to provide a temporary public tunnel to the images directory on my laptop. </p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>You will need an ngrok api key which you can get for free by signing up <a href="https://dashboard.ngrok.com/signup">here</a>, then you can run the following:</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:enlighter/codeblock {"language":"bash"} -->
|
|
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">export NGROK_AUTHTOKEN=<your-ngrok-api-key>
|
|
python attachment_host.py host-ngrok --path ./images --csv-outfile ./attachments.csv</pre>
|
|
<!-- /wp:enlighter/codeblock -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>This will run a server which we will need to leave running while the JIRA import takes place. Ngrok's free tunnels time out after 8 hours so bear in mind that if you choose now to take a break and go and do something else, you might need to re-run this step (also if you have a really massive project). </p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p><strong><em>nb: I was also toying with the idea of writing a version of this that uploads all the images to S3 or Google storage but I haven't had time. Pull Requests welcome.</em></strong></p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>You should now also have a new file called <code>attachments.csv</code> - this is a map generated by the server which converts from the old image url in the JIRA ticket to the new one in your ngrok server. </p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:heading {"level":3} -->
|
|
<h3 class="wp-block-heading">Merge, Link and Tidy up Timestamps</h3>
|
|
<!-- /wp:heading -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>Ok so the next step (script) is actually doing three things at once:</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:list {"ordered":true} -->
|
|
<ol class="wp-block-list"><!-- wp:list-item -->
|
|
<li> We need to merge the two converted CSV files together so that we have a single file that we can upload using JIRA's import system. </li>
|
|
<!-- /wp:list-item -->
|
|
|
|
<!-- wp:list-item -->
|
|
<li>We need to link together issues with their parents properly and make sure that issues which specify a Linear Project ID but no parent ID (because in Linear they were top level issues in the project) get mapped to the relevant Epic in the new world.</li>
|
|
<!-- /wp:list-item -->
|
|
|
|
<!-- wp:list-item -->
|
|
<li>Linear uses some very strange date strings when it exports by default so we will need to tidy those up and write them back out in a simple format that is easy for Jira to parse.</li>
|
|
<!-- /wp:list-item --></ol>
|
|
<!-- /wp:list -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p>The script <code>merge_link.py</code> handles all of these operations.</p>
|
|
<!-- /wp:paragraph -->
|
|
|
|
<!-- wp:paragraph -->
|
|
<p></p>
|
|
<!-- /wp:paragraph --> |