148 lines
8.4 KiB
HTML
148 lines
8.4 KiB
HTML
<!DOCTYPE html>
|
||
<html>
|
||
<head>
|
||
<meta charset="utf-8" />
|
||
<meta http-equiv="X-UA-Compatible" content="IE=edge"><title>Uploading HUGE files to Gitea - Brainsteam</title><meta name="viewport" content="width=device-width, initial-scale=1">
|
||
<meta itemprop="name" content="Uploading HUGE files to Gitea">
|
||
<meta itemprop="description" content="I recently stumbled upon and fell in love with Gitea – a lightweight self-hosted Github and Gitlab alternative written in the Go programming language. One of my favourite things about it – other than the speed and efficiency that mean you can even run it on a raspberry pi – is the built in LFS support. For the unfamiliar, LFS is a protocol initially introduced by GitHub that allows users to version control large binary files – something that Git is traditionally pretty poor at."><meta itemprop="datePublished" content="2018-10-20T10:09:41+00:00" />
|
||
<meta itemprop="dateModified" content="2018-10-20T10:09:41+00:00" />
|
||
<meta itemprop="wordCount" content="337">
|
||
<meta itemprop="keywords" content="devops,docker,git,lfs," /><meta property="og:title" content="Uploading HUGE files to Gitea" />
|
||
<meta property="og:description" content="I recently stumbled upon and fell in love with Gitea – a lightweight self-hosted Github and Gitlab alternative written in the Go programming language. One of my favourite things about it – other than the speed and efficiency that mean you can even run it on a raspberry pi – is the built in LFS support. For the unfamiliar, LFS is a protocol initially introduced by GitHub that allows users to version control large binary files – something that Git is traditionally pretty poor at." />
|
||
<meta property="og:type" content="article" />
|
||
<meta property="og:url" content="https://brainsteam.co.uk/2018/10/20/uploading-huge-files-to-gitea/" /><meta property="article:section" content="posts" />
|
||
<meta property="article:published_time" content="2018-10-20T10:09:41+00:00" />
|
||
<meta property="article:modified_time" content="2018-10-20T10:09:41+00:00" />
|
||
|
||
<meta name="twitter:card" content="summary"/>
|
||
<meta name="twitter:title" content="Uploading HUGE files to Gitea"/>
|
||
<meta name="twitter:description" content="I recently stumbled upon and fell in love with Gitea – a lightweight self-hosted Github and Gitlab alternative written in the Go programming language. One of my favourite things about it – other than the speed and efficiency that mean you can even run it on a raspberry pi – is the built in LFS support. For the unfamiliar, LFS is a protocol initially introduced by GitHub that allows users to version control large binary files – something that Git is traditionally pretty poor at."/>
|
||
<link href='https://fonts.googleapis.com/css?family=Playfair+Display:700' rel='stylesheet' type='text/css'>
|
||
<link rel="stylesheet" type="text/css" media="screen" href="https://brainsteam.co.uk/css/normalize.css" />
|
||
<link rel="stylesheet" type="text/css" media="screen" href="https://brainsteam.co.uk/css/main.css" />
|
||
|
||
<link id="dark-scheme" rel="stylesheet" type="text/css" href="https://brainsteam.co.uk/css/dark.css" />
|
||
|
||
<script src="https://brainsteam.co.uk/js/feather.min.js"></script>
|
||
|
||
<script src="https://brainsteam.co.uk/js/main.js"></script>
|
||
</head>
|
||
|
||
<body>
|
||
<div class="container wrapper">
|
||
<div class="header">
|
||
|
||
<div class="avatar">
|
||
<a href="https://brainsteam.co.uk/">
|
||
<img src="/images/avatar.png" alt="Brainsteam" />
|
||
</a>
|
||
</div>
|
||
|
||
<h1 class="site-title"><a href="https://brainsteam.co.uk/">Brainsteam</a></h1>
|
||
<div class="site-description"><p>The irregular mental expulsions of a PhD student and CTO of Filament, my views are my own and do not represent my employers in any way.</p><nav class="nav social">
|
||
<ul class="flat"><li><a href="https://twitter.com/jamesravey/" title="Twitter" rel="me"><i data-feather="twitter"></i></a></li><li><a href="https://github.com/ravenscroftj" title="Github" rel="me"><i data-feather="github"></i></a></li><li><a href="/index.xml" title="RSS" rel="me"><i data-feather="rss"></i></a></li></ul>
|
||
</nav></div>
|
||
|
||
<nav class="nav">
|
||
<ul class="flat">
|
||
|
||
<li>
|
||
<a href="/">Home</a>
|
||
</li>
|
||
|
||
<li>
|
||
<a href="/tags">Tags</a>
|
||
</li>
|
||
|
||
<li>
|
||
<a href="https://jamesravey.me">About Me</a>
|
||
</li>
|
||
|
||
</ul>
|
||
</nav>
|
||
</div>
|
||
|
||
<div class="post">
|
||
<div class="post-header">
|
||
|
||
<div class="meta">
|
||
<div class="date">
|
||
<span class="day">20</span>
|
||
<span class="rest">Oct 2018</span>
|
||
</div>
|
||
</div>
|
||
|
||
<div class="matter">
|
||
<h1 class="title">Uploading HUGE files to Gitea</h1>
|
||
</div>
|
||
</div>
|
||
|
||
<div class="markdown">
|
||
<p>I recently stumbled upon and fell in love with <a href="https://gitea.io/en-us/">Gitea</a> – a lightweight self-hosted Github and Gitlab alternative written in the Go programming language. One of my favourite things about it – other than the speed and efficiency that mean <a href="https://pimylifeup.com/raspberry-pi-gitea/">you can even run it on a raspberry pi</a> – is the built in LFS support. For the unfamiliar, <a href="https://git-lfs.github.com/">LFS is a protocol initially introduced by GitHub</a> that allows users to version control large binary files – something that Git is traditionally pretty poor at.</p>
|
||
<p>Some of my projects have huge datasets that I want to store somewhere safe and keep under version control. LFS is not perfect but it is a reasonable solution for this particular problem.</p>
|
||
<p>When I installed Gitea I was initially disappointed that uploading large files to LFS seemed to result in errors. I was getting:</p>
|
||
<pre>api error: Authentication required: Authorization error: <REPO_URL>/info/lfs/objects/batch
|
||
Check that you have proper access to the repository
|
||
batch response: Authentication required: Authorization error: <REPO_URL>/info/lfs/objects/batch
|
||
Check that you have proper access to the repository</pre>
|
||
<p>Irritatingly I couldn’t find any references to this particular error message or documentation. But, I had a hunch that the authentication on the LFS upload was timing out because I was able to upload smaller files that don’t take as long to send.</p>
|
||
<p>It turns out that this is exactly what was happening. When you push to a Gitea SSH repository, the server gives your local machine an authorization token that it can use to upload the LFS files. This token has an expiry time which defaults to 20 minutes in the future. If you’re uploading 5GB of data over 100Mb/down 10Mb/up DSL line then you’re gonna have a bad time…</p>
|
||
<p>I had a dig through the Gitea github repository and came across an example <a href="https://github.com/go-gitea/gitea/blob/master/custom/conf/app.ini.sample">config file</a> which includes a variable called LFS_HTTP_AUTH_EXPIRY with a default value of 20m. In your gitea config file you can set this to 120m then you have 2 hours to get that file uploaded. Adjust as you see fit/require.</p>
|
||
|
||
</div>
|
||
|
||
<div class="tags">
|
||
|
||
|
||
<ul class="flat">
|
||
|
||
<li><a href="/tags/devops">devops</a></li>
|
||
|
||
<li><a href="/tags/docker">docker</a></li>
|
||
|
||
<li><a href="/tags/git">git</a></li>
|
||
|
||
<li><a href="/tags/lfs">lfs</a></li>
|
||
|
||
</ul>
|
||
|
||
|
||
</div><div id="disqus_thread"></div>
|
||
<script type="text/javascript">
|
||
(function () {
|
||
|
||
|
||
if (window.location.hostname == "localhost")
|
||
return;
|
||
|
||
var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;
|
||
var disqus_shortname = 'brainsteam';
|
||
dsq.src = '//' + disqus_shortname + '.disqus.com/embed.js';
|
||
(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
|
||
})();
|
||
</script>
|
||
<noscript>Please enable JavaScript to view the </a></noscript>
|
||
<a href="http://disqus.com/" class="dsq-brlink">comments powered by <span class="logo-disqus">Disqus</span></a>
|
||
</div>
|
||
</div>
|
||
<div class="footer wrapper">
|
||
<nav class="nav">
|
||
<div>2021 © James Ravenscroft 2020 | <a href="https://github.com/knadh/hugo-ink">Ink</a> theme on <a href="https://gohugo.io">Hugo</a></div>
|
||
</nav>
|
||
</div>
|
||
|
||
|
||
<script type="application/javascript">
|
||
var doNotTrack = false;
|
||
if (!doNotTrack) {
|
||
window.ga=window.ga||function(){(ga.q=ga.q||[]).push(arguments)};ga.l=+new Date;
|
||
ga('create', 'UA-186263385-1', 'auto');
|
||
|
||
ga('send', 'pageview');
|
||
}
|
||
</script>
|
||
<script async src='https://www.google-analytics.com/analytics.js'></script>
|
||
<script>feather.replace()</script>
|
||
</body>
|
||
</html>
|