ravenscroftj
/
brainsteam.co.uk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update all content
Deploy Website / build (push) Successful in 19s Details

This commit is contained in:
James Ravenscroft 2024-09-08 15:00:57 +01:00
parent 9d279b3edc
commit d59575f8b2
87 changed files with 6872 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
brainsteam/content/bookmarks/2024/08/Bookmarked https___phrack.org_issues_71_17.html_article. A primer on how hacking mentality ....md Normal file
View File

@ -0,0 +1,19 @@
---
categories:
- Personal
date: "2024-08-27 20:11:00"
draft: true
tags: [indiehacking, entrepeneurship]
title:
Bookmarked https://phrack.org/issues/71/17.html#article. A primer on how hacking
mentality ...
type: bookmarks
---
<!-- wp:indieblocks/context -->
<div class="wp-block-indieblocks-context"><i>Bookmarked <a class="u-bookmark-of" href="https://phrack.org/issues/71/17.html#article">https://phrack.org/issues/71/17.html#article</a>.</i></div>
<!-- /wp:indieblocks/context -->
<!-- wp:group {"className":"e-content"} -->
<div class="wp-block-group e-content"><!-- wp:freeform --><p>A primer on how hacking mentality can be applied to business</p>
<!-- /wp:freeform --></div>
<!-- /wp:group -->

16
brainsteam/content/bookmarks/2024/09/04/Bookmarked The Art of Finishing _ ByteDrum by ....md Normal file
View File

@ -0,0 +1,16 @@
---
categories:
- Engineering Leadership
- Software Development
date: "2024-09-04 12:06:18"
draft: false
tags: []
title: Bookmarked The Art of Finishing | ByteDrum by ...
type: bookmarks
---
<!-- wp:indieblocks/bookmark {"empty":false} -->
<div class="wp-block-indieblocks-bookmark"><div class="u-bookmark-of h-cite"><p><i>Bookmarked <a class="u-url p-name" href="https://www.bytedrum.com/posts/art-of-finishing/">The Art of Finishing | ByteDrum</a> by <span class="p-author">Tomas Stropus</span>.</i></p></div><div class="e-content"><!-- wp:paragraph -->
<p>An incredibly relatable essay with a lot of sensible advice and suggestions for issues I struggle with. I think I'm getting better at shipping MVPs but the hard bit is not getting distracted by new shiny ideas when you get stuck with something else. This philosophy is in direct opposition to <a href="https://tilde.town/~dozens/sofa/">the SOFA principle</a>.</p>
<!-- /wp:paragraph --></div></div>
<!-- /wp:indieblocks/bookmark -->

28
brainsteam/content/bookmarks/2024/09/04/Bookmarked Where does Postgres fit in a world ....md Normal file
View File

@ -0,0 +1,28 @@
---
categories:
- AI and Machine Learning
- Software Development
date: "2024-09-04 16:01:51"
draft: false
tags:
- AI
- database
- postgresql
- vectors
title: Bookmarked Where does Postgres fit in a world ...
type: bookmarks
---
<!-- wp:indieblocks/bookmark {"empty":false} -->
<div class="wp-block-indieblocks-bookmark"><div class="u-bookmark-of h-cite"><p><i>Bookmarked <a class="u-url p-name" href="https://stackoverflow.blog/2024/08/27/postgres-genai-vector-databases-timescale-open-source-db/">Where does Postgres fit in a world of GenAI and vector databases? - Stack Overflow</a>.</i></p></div><div class="e-content"><!-- wp:paragraph -->
<p>The title and framing of this talk are weird and it's bugging me</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The question could be paraphrased as "why would we need to efficiently store and retrieve data in a deterministic way when we have GenAI?" This is like asking "why do we need cars when we have speedboats?" or "Why do we need butter knives now that we've invented the chainsaw?".</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The actual subject matter is "PostgreSQL with a couple of plugins can do pretty good nearest neighbour search". I've long been a big fan of Postgres. You probably don't need separate vector database engines, you can just <a href="https://www.amazingcto.com/postgres-for-everything/">use postgres for everything</a>.</p>
<!-- /wp:paragraph --></div></div>
<!-- /wp:indieblocks/bookmark -->

13
brainsteam/content/likes/2023/12/04/Likes https___snarfed.org_2023-12-03_51578 by Ryan Barrett..md Normal file
View File

@ -0,0 +1,13 @@
---
categories: []
date: '2023-12-04 14:29:00'
draft: false
tags: []
title: Likes https://snarfed.org/2023-12-03_51578 by Ryan Barrett.
type: likes
---
<!-- wp:indieblocks/like -->
<div class="wp-block-indieblocks-like"><div class="u-like-of h-cite"><p><i>Likes <a class="u-url" href="https://snarfed.org/2023-12-03_51578">https://snarfed.org/2023-12-03_51578</a> by <span class="p-author">Ryan Barrett</span>.</i></p></div>
</div>
<!-- /wp:indieblocks/like -->

12
brainsteam/content/likes/2023/12/26/Likes Indiewebifying a WordPress Site _ 2023 Edition ....md Normal file
View File

@ -0,0 +1,12 @@
---
categories: []
date: '2023-12-26 19:24:00'
draft: false
tags: []
title: "Likes Indiewebifying a WordPress Site \u2013 2023 Edition ..."
type: likes
---
<!-- wp:indieblocks/like -->
<div class="wp-block-indieblocks-like"><div class="u-like-of h-cite"><p><i>Likes <a class="u-url p-name" href="https://david.shanske.com/2023/12/19/indiewebifying-a-wordpress-site-2023-edition/">Indiewebifying a WordPress Site – 2023 Edition</a> by <span class="p-author">David Shanske</span>.</i></p></div></div>
<!-- /wp:indieblocks/like -->

12
brainsteam/content/likes/2023/12/29/Likes The Web is Fantastic _ Robb Knight..md Normal file
View File

@ -0,0 +1,12 @@
---
categories: []
date: '2023-12-29 10:05:00'
draft: true
tags: []
title: "Likes The Web is Fantastic \u2022 Robb Knight."
type: likes
---
<!-- wp:indieblocks/like -->
<div class="wp-block-indieblocks-like"><div class="u-like-of h-cite"><p><i>Likes <a class="u-url p-name" href="https://rknight.me/blog/the-web-is-fantastic/">The Web is Fantastic • Robb Knight</a>.</i></p></div></div>
<!-- /wp:indieblocks/like -->

13
brainsteam/content/likes/2024/08/27/Likes Productivity gains in Software Development through AI ....md Normal file
View File

@ -0,0 +1,13 @@
---
categories: []
date: '2024-08-27 22:31:00'
draft: false
tags: []
title: Likes Productivity gains in Software Development through AI ...
type: likes
---
<!-- wp:indieblocks/like -->
<div class="wp-block-indieblocks-like"><div class="u-like-of h-cite"><p><i>Likes <a class="u-url p-name" href="https://tante.cc/2024/08/27/5312/">Productivity gains in Software Development through AI</a> by <span class="p-author">tante</span>.</i></p></div>
</div>
<!-- /wp:indieblocks/like -->

15
brainsteam/content/notes/2023/12/04/It was lovely to visit my parents over ....md Normal file
View File

@ -0,0 +1,15 @@
---
categories:
- Personal
date: "2023-12-04 12:39:00"
draft: false
photo:
- url: /media/1701693562222_d752939e.jpg
alt: a british shorthair cat in a bin
tags:
- caturday
title: It was lovely to visit my parents over ...
type: notes
---
<div class="e-content">It was lovely to visit my parents over the weekend and say hi to my dad's naughty cat Bertie who decided to climb into the kitchen bin for a photo op</div>

12
brainsteam/content/notes/2023/12/20/Feeling handy after fixing an airlock in our ....md Normal file
View File

@ -0,0 +1,12 @@
---
categories:
- Personal
date: '2023-12-20 17:55:00'
draft: false
tags: []
title: Feeling handy after fixing an airlock in our ...
type: notes
---
<!-- wp:freeform --><p>Feeling handy after fixing an airlock in our heating system by painstakingly standing with a bucket and draining water out of a radiator until it started hissing and cursing like a scene out of The Exorcist. Phew, no need to call out a plumber after all! #SmallWin</p>
<!-- /wp:freeform -->

18
brainsteam/content/notes/2023/12/20/I spent some time updating my digital garden ....md Normal file
View File

@ -0,0 +1,18 @@
---
categories:
- Personal
date: '2023-12-20 11:46:37'
draft: false
tags:
- music
title: I spent some time updating my digital garden ...
type: notes
---
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I spent some time updating <a href="https://wiki.jamesravey.me/books/seed-propagator/page/music">my digital garden page on music</a> with some bands that I recommend. My taste is somewhat eclectic so you have been warned!</p>
<!-- /wp:paragraph -->

35
brainsteam/content/notes/2023/12/21/Took a trip out to Koh Thai at ....md Normal file
View File

@ -0,0 +1,35 @@
---
categories:
- Personal
date: "2023-12-21 15:26:00"
tags:
- trips
title: Took a trip out to Koh Thai at ...
type: notes
---
<!-- wp:paragraph -->
<p>Took a trip out to Koh Thai at Port Solent for lunch for Mrs R's birthday</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Restaurant was lovely and the food was top notch. I especially enjoyed the spice ratings on the menu from a little tingle up to life changing</p>
<!-- /wp:paragraph -->
<!-- wp:gallery {"linkTo":"none"} -->
<figure class="wp-block-gallery has-nested-images columns-default is-cropped"><!-- wp:image {"id":1082,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/1703172369782-e1703173142457-771x1024_1eede797.jpg" alt="an excerpt from a menu with a spice rating scale from slightly tingly to life changing" class="wp-image-1082"/></figure>
<!-- /wp:image -->
<!-- wp:image {"id":1083,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/1703172369910-771x1024_96af34a6.jpg" alt="a photo of a green thai curry" class="wp-image-1083"/></figure>
<!-- /wp:image -->
<!-- wp:image {"id":1081,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/1703172369661-1024x771_4de5a766.jpg" alt="a tastefully decorated restaurant" class="wp-image-1081"/></figure>
<!-- /wp:image -->
<!-- wp:image {"id":1080,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/1703172369518-771x1024_83558535.jpg" alt="James sat in a restaurant smiling" class="wp-image-1080"/></figure>
<!-- /wp:image --></figure>
<!-- /wp:gallery -->

17
brainsteam/content/notes/2023/12/22/I was pretty horrified to be greeted by ....md Normal file
View File

@ -0,0 +1,17 @@
---
categories:
- Software Development
date: '2023-12-22 13:50:34'
draft: false
tags: []
title: I was pretty horrified to be greeted by ...
type: notes
---
<!-- wp:image {"id":1125,"width":"645px","height":"auto","sizeSlug":"full","linkDestination":"none"} -->
<figure class="wp-block-image size-full is-resized"><img src="/media/image-6_ee0053ab.png" alt="" class="wp-image-1125" style="width:645px;height:auto"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>I was pretty horrified to be greeted by this message whilst running #ubuntu today. I've always enjoyed Ubuntu as a distro that "just works" but I'm starting to get pretty frustrated with Canonical #enshittifying the experience over time. Thinking about jumping back to Mint or possibly #fedora - I think #arch is too high maintenance.</p>
<!-- /wp:paragraph -->

15
brainsteam/content/notes/2023/12/24/Drinking Irish coffee with Jonnie Walker whisky and ....md Normal file
View File

@ -0,0 +1,15 @@
---
categories: []
date: "2023-12-24 15:13:00"
draft: false
photo:
- url: /media/1703430793977_a7199d9c.jpg
alt: "An Irish coffee in a festive mug in front of a gaggia coffee machine"
tags:
- coffee
- drinks
title: Drinking Irish coffee with Jonnie Walker whisky and ...
type: notes
---
<p>Drinking Irish coffee with Jonnie Walker whisky and HICS coffee beans</p>

18
brainsteam/content/notes/2023/12/26/Drinking HICS El Salvador blend coffee Smooth medium ....md Normal file
View File

@ -0,0 +1,18 @@
---
categories: []
date: "2023-12-26 08:45:00"
draft: false
tags:
- coffee
- drinks
title: Drinking HICS El Salvador blend coffee Smooth medium ...
photo:
- url: /media/1703580025838_8188525b.jpg
alt: a bag of coffee in a person's hand
type: notes
---
<!-- wp:freeform --><p>Drinking HICS El Salvador blend coffee</p>
<p>Smooth medium roast</p>
<!-- /wp:freeform -->

14
brainsteam/content/notes/2023/12/31/New year_ new desk.md Normal file
View File

@ -0,0 +1,14 @@
---
categories:
- Personal
date: "2023-12-31 12:55:00"
draft: false
photo:
- url: /media/1704027302456_a403ee18.jpg
alt: A standing desk with a couple of monitors and a light on it in an office room with blue walls
tags: []
title: New year, new desk
type: notes
---
New year, new desk

24
brainsteam/content/notes/2024/01/01/Today we saw Anyone But You_ a romcom ....md Normal file
View File

@ -0,0 +1,24 @@
---
categories:
- Personal
date: "2024-01-01 21:15:00"
draft: false
tags:
- movies
title: Today we saw Anyone But You, a romcom ...
type: notes
---
{{< youtube gbjdSlTHFts >}}
<!-- wp:paragraph -->
<p>Today we saw Anyone But You, a romcom starring Glen Powell and Sydney Sweet which was a retelling of much ado about nothing. The character names were a giveaway: Ben as in Benedict and Bea as in Beatrice. Overall it was a fun, silly film set in very scenic parts of Australia. The Aussie setting also meant that there was quite a lot of strong language used. It didn't take itself too seriously as the gag reel at the end showed.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I enjoyed it but it wasn't a masterpiece.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>3.5/5</p>
<!-- /wp:paragraph -->

14
brainsteam/content/notes/2024/01/11/Drinking Volcano Island blend coffee from hics.md Normal file
View File

@ -0,0 +1,14 @@
---
categories: []
date: "2024-01-11 07:03:00"
draft: false
tags:
- coffee
title: Drinking Volcano Island blend coffee from hics
type: notes
photo:
- url: /media/1704956520871_b891e6bc.jpg
alt: "a bag of coffee on a white counter top"
---
Drinking Volcano Island blend coffee from [hics](https://hics.biz)

13
brainsteam/content/notes/2024/03/16/Edinburgh castle with a touch of lens flare ....md Normal file
View File

@ -0,0 +1,13 @@
---
categories: []
date: "2024-03-16 08:57:31"
draft: false
tags: []
title: Edinburgh castle with a touch of lens flare ...
type: notes
photo:
- url: /media/g5GRr39PRCR27dQd7R3OkgSCJqF2PRedQ0EWZXDU_98514577.jpg
alt: Edinburgh castle with a touch of lens flare from the street lighting
---
Edinburgh castle with a touch of lens flare #edinburgh #castle #streetlights

13
brainsteam/content/notes/2024/05/04/home for the weekend_ Passford House Hotel _newforest ....md Normal file
View File

@ -0,0 +1,13 @@
---
categories: []
date: "2024-05-04 10:30:20"
draft: false
tags: []
title: "home for the weekend, Passford House Hotel #newforest ..."
photo:
- url: /media/NTYJmS9XZBj16uCVrtdwGL38JE8Y6pOKs8NEmcFS_2129e733.jpg
alt: an old looking English country house with purple wisteria growing around it and the sun shining
type: notes
---
home for the weekend, Passford House Hotel #newforest #sunshine #countryhouse

21
brainsteam/content/notes/2024/08/10/It was lovely to head up to London ....md Normal file
View File

@ -0,0 +1,21 @@
---
categories:
- Personal
date: "2024-08-10 20:40:48"
draft: false
photo:
- url: /media/img-20240810-wa00012515327513561028989-768x1024_50a1ef11.jpg
alt: James and Daniel selfie looking at camera sat in the national library
tags:
- personal
title: It was lovely to head up to London ...
type: notes
---
<!-- wp:paragraph -->
<p>It was lovely to head up to London today to meet up with my friend and fellow NLP nerd Daniel. We spent some time discussing some ideas we had for side projects and talked in depth about how web development has become too complex and our desire to build new software with simple stacks with html templates and limited frontend code.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We spent quite a lot of time hanging out in the National Library where there's plenty of space to study and conspire over ideas and schemes. Daniel is something of a digital nomad so it's always nice to get some in-person time when possible. Unfortunately none of our contacts at the Alan Turning Institute were available to let us in today (I mean who can blame them, it's the weekend after all).</p>
<!-- /wp:paragraph -->

13
brainsteam/content/notes/2024/08/18/Had the pleasure of visiting Bletchley Park yesterday ....md Normal file
View File

@ -0,0 +1,13 @@
---
categories: []
date: "2024-08-18 08:08:12"
draft: false
tags: []
title: Had the pleasure of visiting Bletchley Park yesterday ...
type: notes
photo:
- url: /media/WxKAbJ8jLRAwF3iE1LmP3ISzYt5mIEyXgI3wiB2M_cc70c64a.jpg
alt: A statue of Alan Turing fashioned from slate set against a black and white photo of an office in Bletchley park
---
<p style="padding:10px;">Had the pleasure of visiting Bletchley Park yesterday to see where Alan Turing and many others did amazing work on early computers and cryptanalysis. As a computer scientist it has been on my bucket list for a really long time so I'm glad to have finally gotten around to it. The bombe machines were fascinating. Mechanical machines (not computers as they were not general purpose/programmable) that brute forced enigma code cyphers by trying different combinations of circuits. They could try 37k combinations in 12 minutes. <a href="https://pixelfed.social/discover/tags/BletchleyPark?src=hash" title="#BletchleyPark" class="u-url hashtag" rel="external nofollow noopener">#BletchleyPark</a> <a href="https://pixelfed.social/discover/tags/AlanTuring?src=hash" title="#AlanTuring" class="u-url hashtag" rel="external nofollow noopener">#AlanTuring</a> <a href="https://pixelfed.social/discover/tags/computers?src=hash" title="#computers" class="u-url hashtag" rel="external nofollow noopener">#computers</a></p>

14
brainsteam/content/notes/2024/08/27/Breakfast with a view during our stay in ....md Normal file
View File

@ -0,0 +1,14 @@
---
categories:
- Personal
date: "2024-08-27 20:09:00"
draft: false
tags: []
title: Breakfast with a view during our stay in ...
type: notes
photo:
- url: "/media/1724785714404_26e2b97f.jpg"
alt: "a bowl of cereal and coffee in the foreground and a view out over a park and sunrise"
---
Breakfast with a view during our stay in Milton Keynes after visiting Bletchley Park

16
brainsteam/content/notes/2024/08/27/Productivity gains in Software Development through AI.md Normal file
View File

@ -0,0 +1,16 @@
---
categories: []
date: '2024-08-27 22:30:00'
draft: false
tags:
- AI
- hype
title: Productivity gains in Software Development through AI
type: notes
---
<!-- wp:indieblocks/bookmark {"empty":false} -->
<div class="wp-block-indieblocks-bookmark"><div class="u-bookmark-of h-cite"><p><i>Bookmarked <a class="u-url p-name" href="https://tante.cc/2024/08/27/5312/">Productivity gains in Software Development through AI</a> by <span class="p-author">tante</span>.</i></p></div>
<div class="e-content"><!-- wp:freeform --><p>A fairly good rebuttal of the 4500 hours saved piece by Amazon - I had some similar thoughts.</p>
<!-- /wp:freeform --></div></div>
<!-- /wp:indieblocks/bookmark -->

14
brainsteam/content/notes/2024/08/30/The sunset was beautiful today as we drove ....md Normal file
View File

@ -0,0 +1,14 @@
---
categories:
- Personal
date: "2024-08-30 23:27:00"
draft: false
tags: [personal]
photo:
- url: /media/1725056765290_632449d8.jpg
alt: A picture of a beautiful golden sunset over the motorway as photographed from inside a car.
title: The sunset was beautiful today as we drove ...
type: notes
---
<p>The sunset was beautiful today as we drove up north to visit my family for the weekend. </p>

14
brainsteam/content/notes/2024/09/01/Long cat is long.md Normal file
View File

@ -0,0 +1,14 @@
---
categories:
- Personal
date: '2024-09-01 20:30:00'
draft: true
photo:
- url: /media/1725219019267_009ab453.jpg
tags: []
title: Long cat is long
type: notes
---
<!-- wp:freeform --><p>Long cat is long</p>
<!-- /wp:freeform -->

13
brainsteam/content/notes/2024/09/01/Test Hidden_ Snowdon Drive_ Catisfield_ HAM 19 _C ....md Normal file
View File

@ -0,0 +1,13 @@
---
categories: []
date: '2024-09-01 20:45:00'
draft: true
photo:
- url: /media/1725219912483_55765ba2.jpg
tags: []
title: "Test Hidden: Snowdon Drive, Catisfield, HAM 19 \xB0C ..."
type: notes
---
<!-- wp:freeform --><p>Test</p>
<!-- /wp:freeform -->

14
brainsteam/content/notes/2024/09/08/Moody sky this morning on a walk along ....md Normal file
View File

@ -0,0 +1,14 @@
---
categories:
- Personal
date: '2024-09-08 12:52:00'
draft: false
photo:
- url: /media/1725796327250_c8c555b4.jpg
tags: []
title: Moody sky this morning on a walk along ...
type: notes
---
<!-- wp:freeform --><p>Moody sky this morning on a walk along the seafront in Lee on the Solent.</p>
<!-- /wp:freeform -->

72
brainsteam/content/posts/2023/08/08/We moved offices_.md Normal file
View File

@ -0,0 +1,72 @@
---
categories:
- Personal
date: '2023-08-08 11:33:16'
draft: false
tags: []
title: We moved offices!
type: posts
---
<p>
I've been meaning to write about this for a while - we actually moved offices at the end of May!
<br />
</p>
<p>
My company <a href="https://filament.ai/">Filament</a> has, since 2017, had two UK offices. One up in London and one on the South Coast in the <a href="https://en.wikipedia.org/wiki/The_Solent">solent</a> area. I live on the south coast and I spend most of my time at the latter office. I've always been a huge advocate of hybrid or even full remote working - since long before COVID made remote working more "socially acceptable." All of our staff have always had the option to work from home when they want or come in and spend time on site if they prefer. We have a small number of fully remote staff who live on the other side of the country from our office space. People who live near one of our hub offices tend to balance their time between home and the office according to what works for them.
<br />
</p>
<h2>
Our Pre-COVID Setup
<br />
</h2>
<figure>
<img src="/media/1693229548__46828320" height="500px" alt="an large and airy office space with a few desks, some potted plants in the foreground" />
<figcaption>Our old USSP was large and airy and had breakout areas and a kitchenette.</figcaption>
</figure>
<p>Most of our tech team are located on the south coast and I was given the mandate to
make our Solent office as much of a "developers' paradise" as possible - a place where people would want to come and spend time solving problems together and socialising.</p>
<p>From 2017 to May we were based at the University of Southampton's Science Park (USSP) nestled away near a small village, Chilworth, just north of Southampton and surrounded by woodland.</p>
<p>
The science park itself is a pretty nice spot, you can do loads of walks and there's a really lovely on site cafe. If you want a bit of variation, there's a pub within walking distance with a large beer garden and the team and I spent many extended lunch breaks  sat out there on a sunny afternoon.
<br />
</p>
<p>Pre-covid we had a really nice big office space on the science park with a pool table and a little kitchen area. People like being there and most of the time, local-based folks would come in a few days a week and we'd hang out, play some games of pool, get lunch together, that kind of stuff.</p>
<h2>
During and Post-COVID
<br />
</h2>
<figure>
<img src="/media/1693230404__114f116a" height="500px" alt="a cluttered office space full of desks and too many chairs" />
<figcaption>Our post-covid USSP office was small and full of furniture from our rush to downsize</figcaption>
</figure>
<p>Unfortunately, we ended up downsizing during lockdown and that was sub-optimal for a few reasons. There wasn't anything wrong with the space per-se, but it was small and uninspiring and when it got busy, it was really hard to concentrate. It probably didn't help that it was half full with flat-packed furniture from our older larger space making it feel cluttered and scruffy. Also, post-COVID working patterns have changed a lot. </p>
<p>Now people want to spend more time at home but tend to congregate in the office when there's a big in-person meeting, a birthday meal or a well-liked full-remoter is in town for a few days. This leads to a bit of a weird pattern where the office is at maybe 50-60% capacity most of the time and then suddenly experiences big bursts where everyone wants to be in at the same time. In the "small" USSP office these bursts probably meant you weren't going to get any work done that day and there wasn't really anywhere to go and take calls unless you paced in the corridor, booked a meeting room, or as I started doing towards the end of our stay there, lurk in the mail room. </p>
<p>When we raised some money in April, we decided to start looking for a new space. We figured that we probably wanted some kind of new "hybrid" deal to facilitate these new ways of working. Ideally, a smallish office with access to a communal breakout space/lounge area and ideally some small phone pods where we can take calls. We spoke to the USSP and unfortunately they were not able to offer a deal like this although they did tell us that they're working on revamping the building that we were in to provide some of these amenities. So we widened our search.</p>
<h2>
Our new Space and Whiteley Village
<br />
</h2>
<p>We looked at a few local offices before settling on Spaces Whiteley. It's about 20 minutes down the motorway from the old office location and it's a modern co-working setup managed by <a href="https://work.iwgplc.com/">IWG</a> (the guys who also own Regus). </p>
<figure>
<img src="/media/1693231831__806bb1ea" height="500px" alt="James standing with his back to the camera in a light breezy office space" />
</figure>
<figure>
<img src="/media/1693231832__ef6b5c26" height="500px" alt="A sofa and chairs in a large open breakout area" />
</figure>
<figure>
<img src="/media/1693231833__a2439be8" height="500px" alt="A breakout space with a coffee bar and a pool table" />
</figure>
<p>There's a pool table (ah we missed you pool), loads of communal break-out space with comfy seating and a couple of sound-proofed phone booths for us to take calls from when the office gets busy.</p>
<p>Like USSP it has loads of really nice scenery and walks nearby but it definitely wins in terms of amenities. It's a 15 minute walk from an outdoor mall with a bunch of different restaurants and coffee shops.</p>
<br />
<figure>
<img src="/media/1693228881__64006a58" height="500px" alt="A square with some cafes and restaurants" />
</figure>
<figure>
<img src="/media/1693228880__cfda05ca" height="500px" alt="A lake with some ducks and swans in the foreground" />
</figure>
<p>
I think the team have generally been enjoying the new space more too. People tend to want to come in more frequently and make use of the new space a few days a week. I've been enjoying going in more regularly too. The new office location means that I no longer need to drive on the motorway to get to work so I'm considering starting to cycle in a couple of times a week instead.
<br />
</p>

309
brainsteam/content/posts/2023/08/14/Prod-Ready Airbyte Sync.md Normal file
View File

@ -0,0 +1,309 @@
---
categories:
- AI and Machine Learning
- Data Science
date: '2023-08-14 11:57:25'
draft: false
tags: []
title: Prod-Ready Airbyte Sync
type: posts
---
<!-- wp:paragraph -->
<p>Airbyte is a tool that allows you to periodically extract data from one database and then load and transform it into another. It provides a performant way to clone data between databases and gives us the flexibility to dictate what gets shared at field level (for example we can copy the users table but we can omit name, address, phone number etc). There are a bunch of use cases where this kind of thing might be useful. For example, say you have a data science team who want to generate reports on how many sales your e-shop made this month and train predictive models for next month’s revenue. You wouldn’t want to give your data team direct access to your e-shop database because:</p>
<!-- /wp:paragraph -->
<!-- wp:list {"ordered":true} -->
<ol><!-- wp:list-item -->
<li>there might be sensitive personal information (SPI) in there (think names, addresses, bank details, links to what individual users ordered)</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>running this kind of analysis might impact the performance of your shop database and customers might start churning.</li>
<!-- /wp:list-item --></ol>
<!-- /wp:list -->
<!-- wp:paragraph -->
<p>Instead, we can use a tool, such as airbyte, regularly make copies of a subset of the production database minus the SPI and load it into an external analytics database. The data team can then use this external database to make complex queries all day long without affecting application performance. This pattern is called Extract Load Transform or ELT.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>In this post I’ll summarise some strategies for using airbyte in a production environment and share some tips for navigating some of it’s “rough edges” based on my own experience of setting up this tool as a pilot project for some of my clients in my day job.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading" id="general-requirements">General Requirements</h2>
<!-- /wp:heading -->
<!-- wp:list -->
<ul><!-- wp:list-item -->
<li>We <strong>need</strong> to <em>securely</em> copy data from our “live” application database to our “analytics” database</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>We <strong>need</strong> to do this without impacting the performance of the application</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>We <strong>need</strong> to be able to do this daily so that the analytics db contains the latest information</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>We <strong>need</strong> to be able to control which tables and columns get sent</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>We <strong>need</strong> to limit who is able to configure airbyte and incorporate checks to prevent accidental data leakage or loss</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>We <strong>want</strong> to be able to do this relatively cheaply</li>
<!-- /wp:list-item --></ul>
<!-- /wp:list -->
<!-- wp:heading -->
<h2 class="wp-block-heading" id="database-technologies">Database Technologies</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Airbyte has a large number of mature data “Source” plugins for a bunch of standard SQL databases like MySQL/MariaDB, PostgreSQL, MSSQL and so on. It’s Source plugins for data warehouse systems like BigQuery are still in alpha. The reverse appears to be true for “Destination” plugins. Airbyte provides mature destination implementations for data warehouse products like Google BigQuery and Snowflake but immature alpha and beta support for syncing to a traditional RDBMS.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The full list of supportect source and destination connectors is available <a href="https://airbyte.com/connectors">here</a>. We are able to use the MySQL source to pull data from the core application and store our data in Google BigQuery.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading" id="picking-an-edition-of-airbyte">Picking an Edition of Airbyte</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Airbyte comes in two “flavours”:</p>
<!-- /wp:paragraph -->
<!-- wp:list -->
<ul><!-- wp:list-item -->
<li>A hosted cloud version where you pay Airbyte to host an instance of their application on your behalf and they copy all your data around</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>A self-hosted version where you install Airbyte yourself in your own infrastructure and you debug it yourself when you inevitably mess up installation instructions.</li>
<!-- /wp:list-item --></ul>
<!-- /wp:list -->
<!-- wp:paragraph -->
<p>I have a risk averse client-base who have strong data protection requirements so we opted for self-hosted instances that sit in the same region and virtual network in our Google Cloud instance as our application. The target database is Google BigQuery in the same data centre region and the involved clients were ok with that.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading" id="picking-a-virtual-machine-spec">Picking a Virtual Machine Spec</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>If you opt for the self-hosted version like we did you’ll need to pick a VM that has enough resources to run Airbyte. We went for google’s <code>n2-standard-4</code> machine spec which has 4 CPU cores and 16GB of RAM. This was actually our second choice after picking an <code>e2-standard-2</code> which only had 8GB of RAM which was not enough to run Airbyte optimally and caused thrashing/spiking issues.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Although all the data does pass through the VM, it’s done in buffered chunks so your VM doesn’t need a lot of storage space - 50GiB was sufficient for our setup.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading" id="setting-up-airbyte">Setting up Airbyte</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>We deployed Airbyte using docker-compose by following their <a href="https://docs.airbyte.com/quickstart/deploy-airbyte/">quick start guide</a>. We locked down access to the machine over HTTPS so that it only accepts requests from inside our corporate VPN.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We were able to create a database connection to the MySQL database via it’s internal Google Cloud IP address which meant that no production data is routed outside of the virtual network during the first leg of the extraction (everything is encrypted with TLS anyway).</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading" id="cdc-versus-full-refresh">CDC versus Full Refresh</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>When you configure a MySQL source (or indeed a bunch of other compatible sources), you can turn on either full refresh or incremental sync via change data capture. The latter makes use of logs from the SQL server to play back all of the SQL queries that have been run since the last sync and reduce the amount of data that is transferred. If you have a large database (of the order of 10s to 100s of GiB), this may be worthwhile as it is likely to accelerate the sync process significantly after the first run.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>However, the current implementation of CDC/incremental sync for MySQL appears to get stuck and continue to run if you have ongoing changes being made to the system. For example, if you have a high availability application that is always in use or if you have automated processes making changes to the data around the clock, the sync driver will continue to pick up these new changes and append them on an ongoing basis unless you’re able to turn off the workers or create an isolated copy of the source database (as described below).</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We opted to stick with basic “full sync” as we’re only copying around 10GiB of data on a daily basis and our Airbyte setup is able to do this in about 30-45 mins under normal circumstances.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading" id="when-to-sync">When to Sync</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Ideally you want to run the sync when the system is not in use or when it is least likely to impact the ongoing operation of your primary business application. Since most of our clients operate between 9am-6pm GMT we have a nice big “out of hours” window during which we can run this sync.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>If you don’t have that luxury because you have a high-availability application, Google Cloud SQL has the ability to take disk snapshots that don’t appear to significantly impact the database performance. We did sketch out a potential workflow that would involve using the Google Cloud SQL Admin API to:</p>
<!-- /wp:paragraph -->
<!-- wp:list {"ordered":true} -->
<ol><!-- wp:list-item -->
<li>Create a new disk dump of the client db</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>“Restore” the disk dump to a secondary database replica</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>Run the airbyte sync, targetting the secondary replica as the “sync”</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>Turn off the replica db</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>Rinse & repeat</li>
<!-- /wp:list-item --></ol>
<!-- /wp:list -->
<!-- wp:paragraph -->
<p>However, we were able to get full sync (without incremental CDC) working on a nightly basis without adding these extra complications. This would have required us to use an external orchestrator like <a href="https://airflow.apache.org/">Airflow</a> which has <a href="https://airflow.apache.org/docs/apache-airflow-providers-airbyte/stable/operators/airbyte.html">Airbyte operators</a> to execute our process. Thanksfully, we were able to use the built in cron scheduler to have Airbyte run the sync nightly before our application maintenance window.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading" id="securing-the-process">Securing the Process</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>As outlined, it can be important to make sure that certain fields are not shared via airbyte. Airbyte does provide a UI for toggling fields and indeed whole tables on and off as part of sync but for schemas with large numbers of tables and columns this is time-consuming and unwieldy and is likely to lead to human error.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":53,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/image-1-1024x472_1d752f1d.png" alt="" class="wp-image-53"/><figcaption class="wp-element-caption">The UI in airbyte allows us to turn syncing on or off at field level granularity a given table but gets unwieldy at scale</figcaption></figure>
<!-- /wp:image -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading" id="octavia-cli">Octavia CLI</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Airbyte’s answer to this problem is a command line tool named <a href="https://github.com/airbytehq/airbyte/tree/master/octavia-cli">Octavia</a>. Octavia communicates with Airbyte via a REST API and generates YAML configuration files that contain all tables, fields and configurations.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>This is a game changer as it means we can script the fields that get synchronized and we can version control and quality control both the script and the config file itself. We can also gate changes to the sync config via CI tooling and “lint” the config for potential errors (e.g. by providing a script that compares the config against a blacklist of columns that are never allowed).</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Below is an excerpt from one such script that might give readers some inspiration for implementing a similar process themselves:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"python"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import yaml
import click
@cli.command()
@click.option("--connection-file", "-c", type=click.Path(exists=True, file_okay=True, dir_okay=False, readable=True, writable=True), required=True)
@click.option("--fix/--no-fix", type=bool, default=False)
def lint_connection(connection_file, fix):
with open(connection_file,'r') as f:
config = yaml.safe_load(f)
print(f"Reading config file for {config['resource_name']}")
print(f"Checking Connection Resource: {config['resource_name']}")
streams = config['configuration']['sync_catalog']['streams']
print(f"Found {len(streams)} configured streams (tables) from source")
errors = []
for i, stream in enumerate(streams):
name = stream['stream']['name']
if ("admin" in name) or (name in BLOCKED):
print(f"Checking {name} is not enabled for sync...")
if stream['config']['selected']:
err_string = f"ERROR: {name} must not be synced but is being synced"
errors.append(err_string)
if fix:
streams[i]['config']['selected'] = False
print(f"Found {len(errors)} problems")
for error in errors:
print(f"\t - {error}")
if len(errors) &gt; 0:
if fix:
config['configuration']['sync_catalog']['streams'] = streams
print("Fix=True, writing updated config")
with open(connection_file, 'w') as f:
yaml.safe_dump(config, f)
else:
print("Fix=False, please manually fix any problems or rerun with --fix to resolve automatically")
else:
print("No problems found! 😊")
</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>The idea here is that we run the script as part of CI and have the pipeline fail to deploy if it catches any errors and have a <code>--fix</code> mode that attempts to automatically rectify any problems which our maintainer can run locally.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading" id="octavia-bug">Octavia “Bug”</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>When I started using Octavia, I noticed that it would bug out and print error messages when I started to use change which fields are selected for sync. I found <a href="https://github.com/airbytehq/airbyte/issues/27473">a bug ticket</a> about this issue and then eventually realised that the Airbyte documentation for Octavia is quite out of date and by default it installs an old version of Octavia that is not compatible with the current version of the Airbyte server itself. In order to make it work, I simply changed my <code>.zshrc</code> file (or <code>.bashrc</code> file for some) to use the latest version of the tool - which at time of writing is <code>0.50.7</code>:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"bash"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">OCTAVIA_ENV_FILE=/home/james/.octavia
export OCTAVIA_ENABLE_TELEMETRY=True
alias octavia="docker run -i --rm -v \$(pwd):/home/octavia-project --network host --env-file \${OCTAVIA_ENV_FILE} --user \$(id -u):\$(id -g) airbyte/octavia-cli:0.50.7"
</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading" id="octavia-and-ssl">Octavia and SSL</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Unfortunately I couldn’t find an easy way to get Octavia to play nicely with self-signed SSL certificates which meant we had to load in an externally “signed” SSL cert. Octavia is written in Python and uses <a href="https://pypi.org/project/requests/">requests</a> to interact with Airbyte so you could theoretically configure it to trust a self-signing certificate authority (as per <a href="https://stackoverflow.com/questions/30405867/how-to-get-python-requests-to-trust-a-self-signed-ssl-certificate">this stackoverflow post</a>).</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading" id="keeping-an-eye-on-things">Keeping an Eye on Things</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Once you have your sync up and running you probably want to make sure it keeps running regularly. Airflow has slack webhook integration which means that it’s easy enough to have it automatically notify you when sync has passed or failed.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":54,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/image-2-1024x131_5277606b.png" alt="" class="wp-image-54"/></figure>
<!-- /wp:image -->
<!-- wp:heading -->
<h2 class="wp-block-heading" id="conclusion">Conclusion</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>There are a lot of variables and moving parts to consider when using Airbyte, especially when data security and privacy are so important. In this post I outlined some hints and tips for using airbyte successfully based on my own experience of setting up the tool. Hopefully some of my observations were useful or interesting to any readers who are thinking about picking up Airbyte.</p>
<!-- /wp:paragraph -->

35
brainsteam/content/posts/2023/08/20/Weeknote 13-19 August 2023.md Normal file
View File

@ -0,0 +1,35 @@
---
categories:
- Personal
date: '2023-08-20 21:24:31'
draft: false
tags:
- weeknotes
title: Weeknote 13-19 August 2023
type: posts
---
<figure>
<img width="600px" src="/media/202308201808678_237e3f3d.jpeg" alt="James looking pretty happy sat with a tray of cake in front of him" /><figcaption>High Tea taking on a double meaning at the top of the Spinaker Tower</figcaption></figure>
Stuff that happened this week:
<ul>
<li>
I worked on some internal projects and wrote about my experiences of setting up <a href="https://brainsteam.co.uk/2023/8/14/stable-airbyte-sync/">Airbyte</a> for some client use cases.</li>
<li>Mrs R and I celebrated our 2nd wedding anniversary with an afternoon tea at the top of Portsmouth's <a href="https://www.spinnakertower.co.uk/">Spinnaker tower</a>. The food was delicious and the view was spectacular. Thankfully it was a lovely sunny day for it.</li>
<li>
I spent some time in our Whiteley office and went up to our London office on Friday for a couple of  meetings.</li>
<li>
On the way home from London I made the very expensive mistake of losing my car keys which meant walking 30 minutes home to get my spare key and walking back to the station to get the car. I then had to shell out £400 for a new key from the car dealership after being advised by an independent auto locksmith that my car's lock is "too fancy" for them to be able to reliably break into. I won't be doing that again in a hurry.</li>
<li>Mrs R and I finished Season 2 in our Justified rewatch as we prepare for Justified: City Primeval. Margo Martindale certainly deserved <a href="https://youtube.com/watch?v=HAyO0QNFifE">her award</a> for her portrayal of a bone-chillingly cold villain.</li>
<li>
I found some cool new feeds to follow:
<ul>
<li>
<a href="https://tracydurnell.com/mind-garden/">Tracy Durnell</a> - a freelance sustainability consultant who writes about all sorts of interesting stuff in her digital garden</li>
<li><a href="https://elizabethtai.com/">Elizabeth Tai </a>- an essayist, sci-fi writer and digital garden</li>
<li>Shout out to <a href="https://thoughtshrapnel.com/">Doug Belshaw's Thought Shrapnel </a>which I've been following for a while but which has recently come out of hiatus.</li>
</ul>
</li>
<li>I've just finished the 2nd book in William Gibson's Sprawl series</li>
</ul>
This week we have a few office days planned and a few house chores to do. I'll be up in London attending the <a href="https://www.meetup.com/london-apache-airflow-meetup/events/295297770">Airflow Meetup</a> on Thursday evening.

52
brainsteam/content/posts/2023/09/30/TIL_ Unlocking Ubuntu Remotely.md Normal file
View File

@ -0,0 +1,52 @@
---
categories:
- Software Development
date: '2023-09-30 09:34:37'
draft: false
tags:
- til
title: 'TIL: Unlocking Ubuntu Remotely'
type: posts
---
<!-- wp:paragraph -->
<p>My use case is starting a game stream from my steamdeck after my pc has locked itself. The game will start but the desktop lock screen/screensaver is visible and you can't play or do anything.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>You need to SSH in as the active user (from another device like your phone or another laptop) and run:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"bash"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">loginctl list-sessions
</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>You will get a table output with a list of sessions:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"bash"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">SESSION UID USER SEAT TTY
38 1000 james pts/1
44 1000 james pts/2
5 1000 james seat0 tty2
</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>You want to take the one that has a seat allocated and run unlock against it:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"bash"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">loginctl unlock-session 5
</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>Hopefully this will unlock the screen and your steamdeck screen will show the application you're trying to use.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><a href="https://askubuntu.com/questions/341014/unlock-login-screen-using-command-line">https://askubuntu.com/questions/341014/unlock-login-screen-using-command-line</a></p>
<!-- /wp:paragraph -->

165
brainsteam/content/posts/2023/09/30/Turbopilot - a Retrospective.md Normal file
View File

@ -0,0 +1,165 @@
---
categories:
- AI and Machine Learning
date: '2023-09-30 16:42:58'
draft: false
tags: []
title: Turbopilot - a Retrospective
type: posts
---
<!-- wp:paragraph -->
<p>As of today, I am deprecating/archiving <a href="https://github.com/ravenscroftj/turbopilot">turbopilot</a>, my experimental LLM runtime for code assistant type models. In this post I'm going to dive a little bit into why I built it, why I'm stopping work on it and what you can do now.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>If you just want a TL;DR of alternatives then <a href="#what-setup-do-you-recommend">just read this bit</a>.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Why did I build Turbopilot?</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>In April I got COVID over the easter break and I had to stay home for a bit. After the first couple of days I started to get restless. I needed a project to dive into while I was cooped up at home. It just so happened that people were starting to get excited about running large language models on their home computers after <a href="https://github.com/ggerganov">ggerganov</a> published <a href="https://github.com/ggerganov/llama.cpp">[llama.cpp]</a>. Lots of people were experimenting with asking llama to generate funny stories but I wanted to do something more practical and useful to me.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I started to play around with a project called <a href="https://github.com/fauxpilot/fauxpilot">fauxpilot</a>. This touted itself as an open source alternative to <a href="https://github.com/features/copilot">Github Copilot</a> that could run the <a href="https://huggingface.co/Salesforce/codegen-2B-multi">Salesforce Codegen models</a> locally on your machine. However, I found it a bit tricky to get running, and it didn't do any kind of quantization or optimization which meant that you could only run models on your GPU if you have enough VRAM and also if you have a recent enough GPU. At the time I had an Nvidia Titan X from 2015 and it didn't support new enough versions of CUDA to allow me to run the models I wanted to run. I also found the brilliant <a href="https://github.com/Venthe/vscode-fauxpilot">vscode-fauxpilot</a> which is an experimental vscode plugin for getting autocomplete suggestions from fauxpilot into the IDE.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>This gave me an itch to scratch and a relatively narrow scope within which to build a proof-of-concept. Could I quantize a code generation model and run it using ggerganov's runtime? Could I open up local code-completion to people who don't have the latest and greatest nvidia chips? I set out to find out...</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">What were the main challenges during the PoC?</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I was able to whip up a proof of concept over the course of a couple of days, and I was pretty pleased with that. The most difficult thing for me was finding my way around the GGML library and how I could use it to build a computation graph for a transformer model built in <a href="https://pytorch.org/">PyTorch</a>. This is absolutely not a criticism of ggerganov's work but more a statement about how coddled we are these days as developers who use these high-level Python libraries to abstract away all the work that's going on whenever we build out these complex models. Eventually, I found a way to cheat by using a <a href="https://gist.github.com/moyix/7896575befbe1b99162ccfec8d135566">script written by moyix</a> to convert the Codegen models to run in a model architecture already supported by the ggml example code. This meant that I didn't need to spend several days figuring out how to code up the compute graph and helped me get my POC together quickly.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Once I'd figured out the model, it was just a case of quantizing it and running the example code, then I made use of <a href="https://github.com/CrowCpp/Crow">CrowCPP</a> to provide a lightweight HTTP server over the top. I reverse engineered the fauxpilot code to figure out what the REST interface needed to look like and started crafting.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>When I typed in my IDE and got those first code suggestions back, I got that magical tingly feeling from making something work.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">How successful was the PoC?</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Once I had my PoC I added a readme, some badges and some CI pipelines for docker images, Mac pkgs and so on. Then I shared my project on Twitter, Reddit and Mastodon. I was surprised at how much attention it got, I accumulated about 2.5k stars on github in the first couple of days, then it slowed down to about 100 stars a day for the rest of the week. I think it helped a lot that Clem from Huggingface retweeted and replied to my tweet:</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":237} -->
<figure class="wp-block-image"><img src="/media/image-2_81cf7b69.png" alt="A screenshot from twitter: @clem very cool, would be awesome to have the models on hf.co/models" class="wp-image-237"/><figcaption class="wp-element-caption">The project got a lot of attention once Clem from huggingface took an interest.</figcaption></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>In the weeks that followed I got emails and linkedin notifications from all over the place. I got invited for coffee by a bunch of investors and I got asked to demo my tool at a large industry event (unfortunately the demo didn't go ahead due to last minute agenda changes. I will leave this <a href="https://techwireasia.com/2023/09/how-is-intel-launching-the-ai-pc-era/">totally, completely unrelated link here</a>). I would say that as a proof-of-concept my work was very successful and demonstrated pretty clearly that it's feasible to have a local code assistant that stacks up pretty well against big, centralised models LLMs.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">How did the project go after the PoC stage?</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>After the initial buzz, my progress slowed down quite a bit. Partly because <a href="https://www.uktech.news/ai/filament-funding-20230509">we'd just raised some investment at my day job</a> and things had really taken off and that meant I had less time to spend on the project. Part of it is because this is an area that is moving so fast right now and over the summer there seemed to be new code assistant model releases every week that someone would raise a ticket against. I felt guilty for not being able to keep up with the pace and that made me start to resent the project.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I did have a few exciting highlights though. Once I refactored the code and integrated Nvidia GPU offloading, I was getting blazingly fast responses to prompts from very complex models. I also hired a virtual Mac Mini from <a href="https://www.scaleway.com/en/">Scaleway</a> to fix some MacOS specific bugs and seeing how quickly the inference server ran on an M2 chip was great fun too. I also enjoyed working with the guys who wanted to run the conference demo to add more improvements to the tool.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Why are you downing tools?</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>There have been a number of recent independent developments that provide a feature-rich and stable way to run state-of-the-art models locally and have VSCode communicate them, effectively deprecating Turbopilot. I just don't have the time to commit to competing with these alternatives and now that I've got a decent understanding of how GGML + quantization work, I'd like to spend my extra-curricular time on some other challenges.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The main reason for downing tools is that llama.cpp has feature parity with turbopilot. In fact, it's overtaken turbopilot at a rate of knots. It's not surprising, the project is amazing and has garnered a lot of interest and the community are making loads of contributions to make it better all the time. The new GGUF model format allows you to store metadata alongside the model itself so now you can run the llama server and automatically load Starcoder format models (including santacoder and wizardcoder), GPT-NEOX format models (i.e. StableCode) and LlamaCode models. Llama also provide a proxy script that converts their server API to be compatible with OpenAI which means that you can use llama's server directly with vscode-fauxcode without the need for turbopilot.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Secondly, there is a new open source VSCode plugin called <a href="https://continue.dev/">Continue</a> which provides both code autocomplete and that conversational style chat experience that <a href="https://github.com/features/preview/copilot-x">CoPilot now supports</a>. Continue can communicate directly with Llama.cpp's vanilla server without the need for the conversion proxy script. This is now my preferred way to code.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">What setup do you recommend?</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>My recommendation is the vanilla llama.cpp server with <a href="https://huggingface.co/TheBloke/CodeLlama-7B-GGUF">CodeLlama 7B</a> and <a href="https://continue.dev/">Continue</a>.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Firstly, download and build llama.cpp, I have an Nvidia 4070 so I build it with CUDA:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"bash"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
mkdir build
cd build
cmake .. -DLLAMA_CUBLAS=ON -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc
make -j6
</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>Next I <a href="https://huggingface.co/TheBloke/CodeLlama-7B-GGUF">download the gguf model from thebloke's repo</a> and I run the server, offloading the whole model to GPU:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"bash"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">wget https://huggingface.co/TheBloke/CodeLlama-7B-Instruct-GGUF/resolve/main/codellama-7b-instruct.Q5_K_S.gguf
./bin/server -m ./codellama-7b-instruct.Q5_K_S.gguf -ngl 35
</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>With that in place, I installed the <a href="https://continue.dev/">Continue plugin</a> in VSCode and open the sidebar. Select the '+' button to add an LLM config.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":236} -->
<figure class="wp-block-image"><img src="/media/image-1_8748ca79.png" alt="A screenshot of the continue sidebar with the add button highlighted in red" class="wp-image-236"/><figcaption class="wp-element-caption">Open the continue sidebar and click the + button to add a model config</figcaption></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>Then select llama.cpp and "Configure Model in config.py".</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":235} -->
<figure class="wp-block-image"><img src="/media/image_0528701a.png" alt="A screenshot of the continue sidebar with llama config view" class="wp-image-235"/><figcaption class="wp-element-caption">Click Configure Model in config.py</figcaption></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>This will open up a python <code>config.py</code> file, append the server url to the default model and click save:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"python"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""> default=LlamaCpp(
model="llama2",
server_url="http://localhost:8080"
),
</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>And now you're ready to do some coding with the new tool!</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Conclusion</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I've had fun building Turbopilot. It's been a great learning opportunity and helped me to expand my network and meet some cool people. However, given how much the landscape has changed since I started the project and my lack of time to devote to this project, it's right to move on and pass the torch to others in the community.</p>
<!-- /wp:paragraph -->

51
brainsteam/content/posts/2023/10/01/Weeknote Week 39 2023.md Normal file
View File

@ -0,0 +1,51 @@
---
categories:
- Personal
date: '2023-10-01 15:48:08'
draft: false
tags:
- weeknotes
title: Weeknote Week 39 2023
type: posts
---
Another busy week here as "Q3" gives way for "Q4" in the business world.
Early this week we had a company retreat. On Monday we invited middle and senior management teams and on Tuesday we invited the whole tech team including fully remote staff. We rented a huge AirBnB with a games room and a projector and even a hot tub (although only our CEO and one of the braver account managers went in the tub, the rest of the team stood around awkwardly).
The retreat was an opportunity to try to build some good working relationships between different parts of the business since we've had so many new starters in recent days. It was kind of cool seeing different people, who wouldn't normally spend time together, hanging out and chatting over beers.
On Wednesday I gave a CTO update at my first board meeting since we raised funding. The segment went well and we had a discussion about where machine learning and AI are useful and where they are not.
The rest of the week was quieter and I got to work from home for a couple of days which was great because I was absolutely shattered after all of that socialising.
<hr />
Outside of work this week I published a few short blog posts and short comments about some of the articles that I read.
I wrote some philosophical stuff about AI:
<ul>
<li><a href="https://brainsteam.co.uk/bookmarks/2023/09/25/elegant-and-powerful-new-result-that-seriously-undermines-large-language-models1695627930/">Elegant and powerful new result that seriously undermines large language models - Brainsteam</a></li>
<li><a href="https://brainsteam.co.uk/posts/2023/09/27/crappy-printers-and-crappy-content1695806638/">Crappy Printers and Crappy Content - Brainsteam</a></li>
</ul>
I also wrote about <a href="/posts/2023/09/30/turbopilot-obit/">my decision to stop developing TurboPilot</a> which I still have mixed feelings about. However, it does free up some mental bandwidth for some other side-projects and concentrate attention into fewer places as far as open source AI code assistants are concerned.
<hr />
Earlier in the week I was playing Mass Effect 2 as part of the Legendary re-release. I definitely preferred the gunplay in the first game but, the story in the 2nd game is kind of fun. It leans more into a cheesy/stereotype storyline.
I also spotted that Nvidia shipped some <a href="https://developer.nvidia.com/vulkan-driver">new beta drivers (535.43.10)</a> that allow my RTX 4070 to finally run Starfield reasonably well via Proton. I've been streaming it to my Steamdeck and getting 50-60 FPS on 'High' settings, dropping to ~30FPS for densely populated areas. I suspect that there will be more performance gains to be had over the coming weeks.
I wrote a <a href="https://memos.jamesravey.me/m/221">TIL</a> about automatically unlocking your Gnome session remotely with a shell comand which is very useful if I want to launch Starfield on my SteamDeck without going upstairs to physically unlock my desktop PC which has locked me out.
<hr />
I put off my trip up to visit my parents in Shropshire since my Mum and her partner both have COVID. I will try to visit later in the month. Instead, we had a pretty quiet weekend which was appreciated since I had a pretty active week and I needed some introvert energy recharging time. Instead, I've been playing Starfield, reading, writing and snoozing. We've also tackled a few house chores that needed sorting out.
I've been making some small changes to my habits when it comes to writing and journaling that probably deserve their own full blog post. The most interesting thing has been trying to write more of my thoughts down either on paper or in Memos. Keeping track of more of the things that I do reminds me of just how much stuff I achieve in a given day. If I'm beating myself up for being unproductive, it's useful to reflect on some of this stuff.
<hr />
Next week will be another busy one again. I'm in London for some work events during the middle of the week and I've got a few calls starting to fill up my diary.
We also need to make our quarterly migration to CostCo as we're starting to run low on some of our bulk items and we're planning to go to the cinema and make use of our Cineworld memberships at least once this week.

48
brainsteam/content/posts/2023/10/15/Weeknote CW 41 2023.md Normal file
View File

@ -0,0 +1,48 @@
---
categories:
- Personal
date: '2023-10-15 21:18:18'
draft: false
tags:
- weeknotes
title: Weeknote CW 41 2023
type: posts
---
This week has been jam packed with traveling, meetings, events and all sorts! For an introvert like me, it's been pretty hard going pretending to be extroverted and interacting with lots of folks.
The biggest news this week was that my company won another award. A few weeks ago in September we won the CogX award for Best Fintech Company 2023. On Thursday I attended an awards ceremony with my colleague in order to accept the award for<a href="https://thebusinessmagazine.co.uk/business_events/south-coast-tech-awards-2023/"> Best Emerging Tech Company (on the south coast) 2023</a>.
<figure>
<img alt="two men wearing tuxedos smiling and holding a glass trophy" width="600" src="/media/202310151754184_b3b90a50.png" /><figcaption>
Doing my very best Chandler Bing photo smile face whilst holding our award with my colleague Alex (left).
</figcaption></figure>
<hr/>Earlier in the week I did a lot of traveling. On Monday I was up in our London office for a bunch of meetings and face to face catch ups with some new members of the team. Then, on Monday night I made my way over to a hotel near the London Excel centre and prepared for the <a href="https://www.snowflake.com/events/data-cloud-world-tour-london/">Snowflake World Tour</a> on Tuesday.
<figure>
<img src="/media/1697386875__8a3c3ac7" width="600" alt="a huge projector screen showing some snowflake graphics in a dimly lit conference hall" />
<figcaption>The Snowflake World Tour was pretty glitzty and very high energy. Some of the content gave me real "Pied Piper" / "Silicon Valley" vibes</figcaption></figure>
We're not using Snowflake that heavily at work so it was an interesting opportunity for me to learn a little bit more about other features they offer and what we might want to start looking into. The new streaming features that allow you to build views that combine "streamed" data with "static" data looked pretty cool. One theme that kept coming up again and again was Large Language Models and Generative AI. However, very few of the speakers actually said what it is they thought you should be doing with these tools, merely that the platform "now supports them". I wrote in my notes that
<blockquote>
Every time llms come up it is always very vague and hand wavey</blockquote>
There was one demo which was actually pretty cool where they describe a database schema to a model and ask it to generate SQL queries based on natural language questions and then execute these queries and display the answers. This is actually a pretty reasonable use case for this tech because:
<ul>
<li>the risks of inconspicuous hallucination are low the (query either works or doesn't work)</li>
<li>if the SQL is invalid we can feed the error message back to the model and attempt to fix the query).</li>
<li>Sure the user can attempt to jailbreak the model or do prompt injection but we can use RBAC controls on the model's database access to make it read only and limit it's access to tables that we're happy the end user can see.</li>
</ul>
I  had a play with <a href="https://github.com/Snowflake-Labs/sfguide-frosty-llm-chatbot-on-streamlit-snowflake/blob/main/src/prompts.py">the demo </a>that the salesforce team shared afterwards and I was able to get it working with a local MySQL database (as opposed to a Snowflake instance) pretty damn quickly which was cool
<hr/>I stayed on in London on Tuesday night and on Wednesday morning I took a train down to Chelmsford to give a talk to some young professionals attending a networking event by the University of Essex as part of their KTP programme. I really enjoy public speaking these days so I had a great time and afterwards I got lunch with the University team and we chatted TV shows.
<hr/>A pattern I tried out this week, particularly at the conference on Tuesday was using my <a href="https://github.com/usememos/memos">memos</a> instance in combination with <a href="https://github.com/mudkipme/MoeMemosAndroid/">MoeMemos</a> a bit like twitter to "live tweet" interesting observations from the conference. This was a really nice way to do low friction note taking while enjoying the conference and adding photos is pretty seamless. I've got the <a href="https://github.com/EINDEX/logseq-memos-sync">LogSeq Memos Sync</a> plugin which pulls in all the memos that I make and attaches them to my daily journal so that I can process and summarise my notes later. The other really cool thing about this is that tags in memos just transparently sync with tags in Logseq so if I tag something #snowflake in memos it will show up as a mention on my snowflake page in Logseq later on. Very groovy.
<hr/>This week I've not had a whole lot of recreational time and when I have had a bit of time I've felt like just snoozing or watching TV. I did watch the latest episode of Great British Bake Off on tuesday and Mrs R and I have started watching the latest series of Only Murders In The Building on Disney+.
On Saturday I went with Mrs R to visit my sister-in-law and we started watching Fall of the House of Usher on Netflix. This series is apparently loosely based on the works of Edgar Allan Poe and it's by Mike Flanagan who directed a bunch of shows I've previously enjoyed including <a href="https://www.imdb.com/title/tt6763664/">Haunting of Hill House</a>, <a href="https://www.imdb.com/title/tt10970552/">Haunting of Bly Manor</a> and <a href="https://www.imdb.com/title/tt10574558/">Midnight Mass</a>. There is a lot I enjoyed about House of Usher except there is a particularly graphic/gory scene in one of the early episodes which is still etched permanently into my memory and left me feeling pretty panicked and prevented me from sleeping. The last thing that impacted me so deeply was <a href="https://www.imdb.com/title/tt0914798/">The Boy in Striped Pajamas</a>. I don't know if I want to keep watching the show as whilstt I'm a big fan of psychological horror, I don't do gore and normally Flanagan excels in the former rather than the later. However, I'm kind of captivated by the storyline so I'm wondering if I should just keep pushing through and hope that it won't cause any more lost sleep.
<hr/>I've not read any more of Ringworld by Larry Niven this week, I'm kind of put off by the in-your-face misogyny at this point. I may DNF it and move on to something else.
I did pick up <a href="https://www.amazon.co.uk/Think-Like-CTO-Alan-Williamson/dp/1617298859">Think Like a CTO </a>(which is probably a good idea since I am one) this week which I'm enjoying so far.
<hr/>This week should be a lot less hectic and busy. We are going to the theatre to see 2:22 on Tuesday, I'm only in London for one day this week for some meetings on Wednesday and I'm taking a half day on Friday to visit my parents and some friends up in the midlands whilst Mrs R stays at home and celebrates her sister's birthday with a girly spa afternoon.
Here we go again!

101
brainsteam/content/posts/2023/10/18/Dealing with death-by-a-thousand questions in the workplace.md Normal file
View File

@ -0,0 +1,101 @@
---
categories:
- Engineering Leadership
date: '2023-10-18 09:04:00'
draft: false
tags: []
title: Dealing with death-by-a-thousand questions in the workplace
type: posts
---
Inundated by 'quick questions' that prevent you from carrying out your main role? Slack has made it very easy to invade your focus time and before slack people would turn up at your desk in the office and say "can I borrow you for a sec...?"
<!--more-->
<!-- wp:paragraph -->
<p><em>Featured Image by <a href="https://unsplash.com/@towfiqu999999?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Towfiqu barbhuiya</a> on <a href="https://unsplash.com/photos/a-blue-question-mark-on-a-pink-background-oZuBNC-6E2s?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a></em></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Inundated by 'quick questions' that prevent you from carrying out your main role? Slack has made it very easy to invade your focus time and before slack people would turn up at your desk in the office and say "can I borrow you for a sec...?"</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The big problem, especially for software engineers and deep knowledge workers, is that these "quick questions" and just being borrowed comes with context switching cost. The person who asked it may not realise that their 2 minute question just cost you half an hour of work time because you need to get back into the zone (or a nerdier analogy - you need to page the contents of your brain out to disk and loading it back into memory is slow work...)</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Here are a few suggestions and anecdotes for dealing with these quick questions from my personal experience.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Document Frequently Asked Questions</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>If you are an individual contributor and you're good at your job you likely have a reputation as an expert at X and people will want to ask you about that thing. In one way that's flattering but when it takes up a lot of time it obviously becomes disruptive. Have a think about what you're being asked on a regular basis and consider writing up answers to common questions and either blogging about them or putting them somewhere appropriate within your workplace (a notion or confluence page or a Google doc or something). Then next time someone asks you that question send them a link to you FAQ. Rather than just sending the link without any context (which could potentially come across passive aggressive) have a pre-canned "polite" response about how you get asked that a lot and how you took the time to write up a detailed answer and hope that it's helpful. If you put all your answers in the same place people will start to check there first and share the answer amongst themselves.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Set Some Boundaries for Yourself</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I hope this next one doesn't come across too patronising because it might sound obvious but many thirty-somethings or even forty-somethings that I work with hadn't really thought about it.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We all want to be helpful and please our colleagues but does answering their quick question the moment it arrives in slack actually unblock them there and then? Some people, particularly those that juggle a lot of to-dos, like to dump questions straight into slack as a way to offload - once its in your inbox it no longer has to be in theirs. What's more, IM apps like Slack are designed to make us alert and stressed and want to get rid of that little red indicator and drive the number of unread messages down to zero.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>So what can you do? Set yourself some boundaries and be disciplined about them. Block out an hour of "focus time" in your calendar to just get on with your main day job and turn off slack. Just right click -> exit! It's ok! You can do it! The world is not going to stop spinning and if it really is "that urgent" they probably have your phone number. Make a habit of "batch replying" to all your quick questions at once - this reduces the context switching required.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Set Some Boundaries for Others</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>If you already had an inkling about the above or you took my advice and people switched from slacking to calling it's time to lay down some boundaries.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>My favourite approach here is to schedule "AMA sessions" or "office hours" - put in a recurring meeting at a time when you're not at your most productive (I'm a morning person so mid-afternoon works well for me). Tell everyone that you will no longer be responding to ad-hoc questions and to bring the questions to your office hours session.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>You can assume that people will bring urgent questions to you as soon as they pop up so don't say the quiet part out loud. If you do, people will likely assume everything they care about is urgent and bombard you anyway. A tiny bit of friction here goes a long way</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Ideally your line manager will be on board with this but regardless, give yourself permission to do it and ask for forgiveness rather than permission. After all, this is easily defensible to the wider business (the question askers) as a necessary process change to improve your productivity.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Normally this has the effect of condensing down all your ad-hoc question answering (good for context switching) and often has another pleasant side effect. Two or more people might join with the same question and they both get it answered simultaneously. Not quite as good as having an FAQ written up but certainly more efficient than parroting the same thing to different people)</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>An important step is to follow through and actually ignore people who message you outside of office hours (again -probably using your professional judgement to decide whether it really is urgent).</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Conclusion</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>In the modern work environment, juggling tasks is particularly painful without being bombarded with questions constantly. Whilst you might feel the need to be helpful, if you're not setting boundaries you could end up burning out trying to find time to do everything. Start by setting yourself some intrinsic boundaries and habits and then consider going radical and using tools like "office hours" to cut down wasted time. Give yourself permission to do your job without distractions - your manager will almost certainly be onboard!</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Appendix: Getting Your Manager On Board</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>If you're an individual contributor you, hopefully, have a pretty well defined day job and core set of deliverables and goals. If answering questions all day is getting in the way of that and your manager is actually decent at their job, they are going to care and want to help you find ways to make things work. If the office hours/AMA idea sounds a bit radical to them, gather some data. Start to time how long you are spending answering questions over time and show your manager in your next 1:1. Good managers should be horrified and want to help to facilitate you getting more time to do your core job.</p>
<!-- /wp:paragraph -->

55
brainsteam/content/posts/2023/10/29/Comment_ Once Interrupted_ Give It Your Full Attention.md Normal file
View File

@ -0,0 +1,55 @@
---
categories:
- Engineering Leadership
date: '2023-10-29 22:48:23'
draft: false
tags: []
title: 'Comment: Once Interrupted, Give It Your Full Attention'
type: posts
---
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:quote -->
<blockquote class="wp-block-quote"><!-- wp:paragraph -->
<p>Good leaders do their best to prevent distractions and avoid interruptions. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>...</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>But no matter how much planning goes into eliminating distractions and arranging the physical work environment to enhance focus, people (and pets) interrupt the flow. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>...</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Once interrupted, it often makes the most sense to give the source of the disturbance your full attention. If it’s going to take a few minutes to redeploy your focus anyway, why not achieve the equally important goal of doing what leaders are meant to do: focus on the problems and issues of others. </p>
<!-- /wp:paragraph --><cite><a href="https://admiredleadership.substack.com/p/once-interrupted-give-it-your-full" target="_blank" rel="noreferrer noopener">Admired Field Notes</a></cite></blockquote>
<!-- /wp:quote -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>This really makes a lot of sense even though our brains will actively try to pull us back into whatever we were doing before. By going back to what you were working on you have to go through the pain of another context switch whilst the interruption presumably remains unresolved.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->

40
brainsteam/content/posts/2023/10/29/Weeknote CW43 2023.md Normal file
View File

@ -0,0 +1,40 @@
---
categories:
- Personal
date: '2023-10-29 19:26:14'
draft: false
tags:
- weeknotes
title: Weeknote CW43 2023
type: posts
---
<!-- wp:paragraph -->
<p>We had a great time at Peter Pan goes wrong - lots of silly slapstick</p>
<!-- /wp:paragraph -->
<!-- wp:list -->
<ul><!-- wp:list-item -->
<li>This week the weather has taken a turn for the worst across the UK.</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>Work has been relatively quiet and I managed to avoid going up to London all week. I did get the chance to spend some f2f time with my CEO on Thursday and do some whiteboarding about our product strategy for the next few months which was cool. I also finished a tech due dilligence project that I've been running on another software startup which is always really interesting and insightful. I love to see what other (non-competing) businesses are doing, process-wise, and see if I can learn anything.<br></li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>On Tuesday Mrs R and I went to see Peter Pan goes Wrong at the Mayflower Theatre in Southampton. We really enjoyed the show, there was lots of silly slapstick comedy.</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>I started spending a bit more time looking at substack because <a href="https://newsletter.thoughtshrapnel.com/">Doug</a> and <a href="https://freshlybrewed.substack.com/">Laura</a> started posting more over there, there are some other interesting things going on over there too. Amongst others, I've been following <a href="https://garymarcus.substack.com/">Gary Marcus </a>who is a fellow AI specialist that is pretty scathing about the current hype, <a href="https://wheresyoured.at/">Ed Zitron</a> who has some spicy takes on silicon valley and <a href="https://www.garbageday.email/">Garbage Day</a> which is a bit of a random collection of online "stuff. I've noticed Substack has started to pick up a fair bit of interest and traffic recently despite some <a href="https://www.techdirt.com/2023/04/14/substack-ceo-chris-best-doesnt-realize-hes-just-become-the-nazi-bar/">big controversies earlier in the year</a>. Also, regardless of political connotations there's still the whole "you don't own your stuff" argument for not publishing there (or at least syndicating there from an independently owned site). I enjoyed <a href="https://elizabethtai.com/2023/10/22/what-i-learned-from-one-year-of-substack/">Elizabeth Tai's</a> recent reflections on the platform.<br></li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>Yesterday I published about <a href="https://brainsteam.co.uk/posts/2023/10/28/a-musical-interlude-last-dinner-party-and-dream-theater1698481925/">some cool happenings in music </a>and updated my <a href="https://jamesravey.me/">professional landing page </a>and my <a href="https://brainsteam.co.uk/now/">now</a> page. I'm forever wanting to fiddle and tweak my website and I'm never quite happy with the layout and the way it's all set up. I'm intrigued by the new Wordpress ActivityPub integrations and forever interested in whether I should be separating my blog outputs based on content type (for example keeping brainsteam "personal" and posting tech/programming stuff somewhere else).</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>This coming week I'm in London a couple of days and then we have some time off and a mini-break in Bournemouth towards the end of the week.</li>
<!-- /wp:list-item --></ul>
<!-- /wp:list -->

33
brainsteam/content/posts/2023/11/01/Going Mainstream with Wordpress.md Normal file
View File

@ -0,0 +1,33 @@
---
categories:
- Personal
date: '2023-11-01 20:12:49'
draft: false
tags: []
title: Going Mainstream with Wordpress
type: posts
---
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Hello folks - I've taken the decision to migrate my blog back to wordpress after a few years of "dogfooding" my own publishing stack with Hugo and a little python script I wrote to integrate with indieweb and webmention stuff.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Why have I done this? Well frankly I realised that my blog setup was annoying me and holding me back from actually publishing. I couldn't find a workflow that I was totally comfortable with and that friction was making me less likely to want to write. During this period of reflection I also noticed <a href="https://kevquirk.com/other-peoples-enthusiasm">Kev's</a> comment about moving between CMS systems and had a little chat with him about it over email. He also advised me that I should do what I need to in order to reduce friction so that I actually want to write.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I know that Wordpress is the boring, mainstream, vanilla option when it comes to web publishing but it's also very convenient to use and I'm very comfortable drafting and publishing pages from this interface. It also makes updating the layout and design of my page a lot easier and I am very much a backend/data person rather than a web designer so that is pretty appealing too. I was also pretty keen to try out the new WP activitypub integration for directly sharing my content on the fediverse.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I've been migrating my post across from my Hugo site using a little conversion script that I wrote up to convert the markdown files and add them as drafts in wordpress. This way I can curate my archive of posts and bring it with me over to Wordpress over time. I've brought some of my more popular articles over with me already and I'll slowly make my way through my back catalogue for posts I want to retain.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I'm hoping that this will help inspire me to write a little more regularly along side some <a href="https://notes.jamesravey.me/writing-practice">deliberate daily writing practice</a> that I'm trying out at the moment. Apologies for breaking peoples' RSS feeds and whatnot, hopefully it won't happen too regularly from here on out.</p>
<!-- /wp:paragraph -->

58
brainsteam/content/posts/2023/11/03/Living with Aphantasia.md Normal file
View File

@ -0,0 +1,58 @@
---
categories:
- Personal
date: '2023-11-03 11:20:35'
draft: false
tags:
- aphantasia
title: Living with Aphantasia
type: posts
---
<!-- wp:paragraph -->
<p>Aphantasia is what they call it when you can't see stuff in your "mind's eye", it's latin for "no imagination". It was a term originally coined by Dr Adam Zeman <a href="https://www.bbc.co.uk/news/health-34039054">who did a study in 2015 </a>that I responded to. I am an aphant and I spent most of my life believing that this is the normal state of affairs. When my secondary school art teacher got frustrated with me during still life drawing, they said "just visualise it in your mind's eye" which completely threw me - I assumed that this was a turn of phrase or a metaphor or something. I didn't realise that most people genuinely do have mental imagery.</p>
<!-- /wp:paragraph -->
<!-- wp:embed {"url":"https://www.youtube.com/embed/vbVlte9hSrE","type":"rich","providerNameSlug":"embed-handler","responsive":true,"className":"wp-embed-aspect-16-9 wp-has-aspect-ratio"} -->
<figure class="wp-block-embed is-type-rich is-provider-embed-handler wp-block-embed-embed-handler wp-embed-aspect-16-9 wp-has-aspect-ratio"><div class="wp-block-embed__wrapper">
https://www.youtube.com/embed/vbVlte9hSrE
</div><figcaption class="wp-element-caption">In this video, Dr Adam Zeman summarises Aphantasia - the inability to imagine things</figcaption></figure>
<!-- /wp:embed -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I think in terms of "facts" and "statements". If I ask you to picture a sunset in your mind you are probably able to see a lovely orange glowing sun setting over a beach with waves gently lapping across pink coral sand. I don't see any of that stuff but I know, theoretically what sunsets look like and I can pick language that describes how that sunset might interact with other stuff (like a sandy beach). What I can't do is "picture" anything in my head. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Living with Aphantasia is welll... all I've ever known. It's my lived experience. I don't believe it's ever held me back in any major ways. I'm not very good at visual art but I never set out to be an artist (perhaps there was alway an unconscious bias there?). There are a few things about being an aphant that annoy me.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I am very bad at facial recognition. I'm not face-blind, I know my wife and family when they are there in front of me. However, I'm awful at recognising people "out and about" or actors in movies. I am often reminded by my mother of occasion from my childhood where, upon seeing a photo of the famously beautiful model <a href="https://en.wikipedia.org/wiki/Heidi_Klum">Heidi Klum</a>, I remarked "she looks just like <a href="https://en.wikipedia.org/wiki/Gollum">Gollum</a> from Lord of the Rings". I'm not consciously aware of how my facial recognition faculties work (or don't work) but I'm sort of vaguely aware that it's quite brittle and "fact based". For example, <em>person in question</em> has have big eyes and black hair and a particular shape to their nose or mouth and so does <em>other person</em>. This can make for some very embarrassing interactions.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>When it comes to aesthetics, I know what good looks like but I find it hard to replicate. I often get excited about DIY projects and dream of replicating styles that I've seen in showrooms or other peoples' houses or even fancy hotels. However, when I get home, I see how the room is now and I can't picture how I want it to look. This is a struggle when I'm buying a house - they always tell you to "look past" the current appearance and imagine how it could look. Pahahahaha - I'd love to be able to do that! I have the same frustrations when it comes to all visual creative endeavours - web design, drawing, even designing the invites we sent out for our wedding. This is a major reason that I specialise in backend/server-side and machine learning code rather than frontend as part of my day job.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>A third and final frustration I'll talk about here is mindfulness and stress relief techniques. Many meditation and stress relief exercises involve statements like "imagine a warm beam of sunlight that beams up and down your body and relaxes your muscles" or "imagine going to your happy place" or "imagine you're holding all your stress in a balloon and you're going to let it go and it will float away". None of these work for aphants like me. What does work for me is breath work and actual videos (for example, the headspace "thoughts as traffic" analogy which I couldn't picture 'in my head' but I can appreciate as a cute cartoon)</p>
<!-- /wp:paragraph -->
<!-- wp:embed {"url":"https://www.youtube.com/embed/iN6g2mr0p3Q","type":"rich","providerNameSlug":"embed-handler","responsive":true,"className":"wp-embed-aspect-16-9 wp-has-aspect-ratio"} -->
<figure class="wp-block-embed is-type-rich is-provider-embed-handler wp-block-embed-embed-handler wp-embed-aspect-16-9 wp-has-aspect-ratio"><div class="wp-block-embed__wrapper">
https://www.youtube.com/embed/iN6g2mr0p3Q
</div><figcaption class="wp-element-caption">Headspace's "thoughts as traffic" animation is cute and I don't have to try and visualise it.</figcaption></figure>
<!-- /wp:embed -->
<!-- wp:paragraph -->
<p>Overall though, I can't say that Aphantasia is particularly problematic for me, it's all I've known for 30-something years and, as such, I've just generally "got on with it". However, learning about Aphantasia, interacting with Dr Zeman and being able to put two-and-two together later in life ("oh that's why my art teacher said that stuff and that's why I'm not great at painting") has been very positive for me.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><strong><em>I'm planning to write a series on aphantasia and how I "work around" it. If you have questions or are curious about it, I'd love to hear from you. Or, if you are a fellow aphant, drop me a message and say hi!</em></strong></p>
<!-- /wp:paragraph -->

41
brainsteam/content/posts/2023/11/05/Weeknote CW 44 2023.md Normal file
View File

@ -0,0 +1,41 @@
---
categories:
- Personal
date: '2023-11-05 17:43:31'
draft: false
tags: []
title: Weeknote CW 44 2023
type: posts
---
<!-- wp:paragraph -->
<p>I spent two days up in London this week, attending various in-person meetings and I got to have lunch with my friends and former co-founders of Filament who span out to run <a href="https://ebm.ai/"><span style="text-decoration: underline;">EBM</span></a>.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>On Wednesday we were "locked down" for Storm Ciaran - our local council declared a Major Incident and encouraged people to stay at home. In the end, the storm wasn't that bad for us although it caused a lot of damage on the channel islands.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I ended up also catching a cold/flu from London so with the storm and feeling pretty bad, we decided not to visit Bournemouth to see Dawn French.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>On our planned day off, we went out for lunch instead. We visited <a href="https://www.garsons.co.uk/"><span style="text-decoration: underline;">a local garden and craft centre</span></a> with an amazing cafe and deli attached to it. They had christmas displays up and Mrs R talked me into spending a fortune on decorations. I also lost <a href="https://en.wikipedia.org/wiki/Whamageddon"><span style="text-decoration: underline;">Whammagedon</span></a> - on the 2nd of November!</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I finally decided to make the move from Hugo to Wordpress -<span style="text-decoration: underline;"> <a href="https://brainsteam.co.uk/2023/11/01/going-mainstream-with-wordpress/">I wrote about it a bit here.</a></span></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I also wrote about my <a href="https://brainsteam.co.uk/2023/11/03/living-with-aphantasia/"><span style="text-decoration: underline;">experience of Aphantasia</span></a> and had a few chats with people over on <a href="https://substack.com/home"><span style="text-decoration: underline;">Substack Notes</span></a>.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I finally finished Larry Niven's Ringworld. The sci-fi concepts were cool but it was really let down by a whole bunch of problematic treatment of women. I've moved on to Dredd: Year Three from a fiction POV and I've started reading Herbert Lui Creative Doing and MAKE by Pieter Levels on the non-fic pile.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>This week my Dad and his wife are coming to visit us for a couple of days so I've got a little longer off work. I suspect I'll be up in London for some f2f meetings too. Then we've got a few weeks reprieve before christmas stuff starts kicking off.</p>
<!-- /wp:paragraph -->

62
brainsteam/content/posts/2023/11/12/Weeknote CW45 2023.md Normal file
View File

@ -0,0 +1,62 @@
---
categories:
- Personal
date: '2023-11-12 19:58:30'
draft: false
tags:
- weeknotes
title: Weeknote CW45 2023
type: posts
---
<!-- wp:paragraph -->
<p><img crossorigin="anonymous" draggable="true" tabindex="0" class="gndfcl4n p357zi0d ppled2lx ac2vgrno gfz4du6o r7fjleex g0rxnol2 ln8gz9je b9fczbqn _11JPr" style="visibility: visible; opacity: 1; transform: translateX(0px) translateY(0px) scale(1);" src="blob:https://web.whatsapp.com/6a06638e-010b-46eb-bac1-a298a3a6c115" alt=""><img crossorigin="anonymous" draggable="true" tabindex="0" class="gndfcl4n p357zi0d ppled2lx ac2vgrno gfz4du6o r7fjleex g0rxnol2 ln8gz9je b9fczbqn _11JPr" style="visibility: visible; opacity: 1; transform: translateX(0px) translateY(0px) scale(1);" src="blob:https://web.whatsapp.com/6a06638e-010b-46eb-bac1-a298a3a6c115" alt="">This week was another busy one both at work and at home.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>My Dad and his wife came to stay with us at the start of the week. My dad is reeaaally into coffee and I got him a voucher to go on a barista training course at Winchester Coffee School for his birthday earlier in the year. While he was doing his course, I took my evil stepmother for a walk along the beach at Lee-on-the-Solent and we had lunch in one of the seafront cafes. I snapped a photo of some brave kite surfer, out in the sea on a very cold and windy day.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Since I had Monday afternoon, Tuesday and Wednesday morning off from work, I mainly worked from home and the Whiteley office this week and managing to avoid London. I'm currently looking at our internal tooling to support our software development lifecycle. We currently use Linear and even though the UX is crisp and polished, there are a few things that aren't working well for us right now. In particular, I'm looking for something that gives us better transparency and visibility of team velocity. We're looking at whether a move back to Jira might be the right thing for us (to butcher an old phrase, nobody got fired for buying Jira).</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Nourishing Side Project</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>During my downtime I've been playing with an idea for building a federated recipe app (kind of like if Whisk.com - which was recently acquired and enshittified by Samsung had open interoperable recipe formats). I've been playing with local AI tools like Stable Diffusion and CodeLlama to accelerate the development. The idea seems to attract a few interested folks <a href="https://fosstodon.org/@jamesravey/111392287153062085">when I tooted</a> about some of the "default avatars" I had Stable Diffusion create for me.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":373,"width":"377px","height":"auto","sizeSlug":"full","linkDestination":"none"} -->
<figure class="wp-block-image size-full is-resized"><img src="/media/image-5_29bbf2b6.png" alt="Screengrab of a mastodon toot where I mention that I am creating an activitypub recipe app and share a screenshot of some of the cute avatars I got SD to create." class="wp-image-373" style="width:377px;height:auto"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>I'm really excited about this idea and it sounds like some other people are too. I'm planning to follow up with a blog post about my plans for what it will do and a name reveal in the next few days. I'm planning to build it "with the garage door up" so hopefully it will help others too.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Watching and Reading</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>On Saturday we went to see <a href="https://www.imdb.com/title/tt21942866/fullcredits/">Dream Scenario </a>- a film about an awkward, introverted man, played by Nicolas Cage, who ends up appearing in a large number of peoples' dreams - sort of like a sleep meme. It's a really good movie with some dark moments, some cringy moments and some funny moments and it spends some time examining some important societal tropes. Although I sat and watched the film through gritted teeth and occasionally with my head in my hands, I genuinely enjoyed it. I did see myself (an awkward introverted man) in Cages character although Mrs R assures me that I'm nicer and have more common sense.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I am still reading Make by Pieter Levels which so far has focused on finding interesting ideas and idea generation and Dredd: Year Three which provides me with some pulpy sci-fi entertainment. Today I saw <a href="https://www.smbc-comics.com/comic/golden-age">this SMBC comic</a> which really resonated after <a href="https://brainsteam.co.uk/2023/11/05/weeknote-cw-44-2023/">my recent reading of Ringworld</a>.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Coming up Next Week</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>This coming week I will mainly be working from home again. It's cold, wet and dark and there aren't many hours of sunlight so I'm aiming to get out and do some walks to try and get some endorphins and some exercise. I'll be writing more about my recipe app.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->

120
brainsteam/content/posts/2023/11/13/Gastronaut - Fediverse Recipe App.md Normal file
View File

@ -0,0 +1,120 @@
---
categories:
- Software Development
date: '2023-11-13 20:56:06'
draft: false
tags:
- gastronaut
title: Gastronaut - Fediverse Recipe App
type: posts
---
<!-- wp:paragraph -->
<p>Over the last few days I've been starting to build out a webapp. I was inspired by a personal "itch" I want to "scratch". Although I'm a long way from ready to share my app, I thought I'd take the time to write up some of my initial ideas and choices. I'm a big fan of building with the garage door up and as Pieter Levels says - it's better to share ideas early and get out of your own head.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">My Inspiration? Whisk.com</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I was, until recently, a big fan of <a href="https://whisk.com">whisk.com</a>, a proprietary recipe app that allowed you to create and share recipes, create a food plan for the week and then use that food plan to put together a shopping list. I love the fact that it normalises units across recipes and estimates conversions between volumes and weights for common foodstuffs (e.g. it will guess at how many grams of sugar you need in total if one recipe calls for 200g and another calls for 2 tablespoons). I also love the recipe importer tool that allows you to paste in URLs to your favourite recipes around the web and automatically pull in the ingredients and quantities for your grocery list.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Whisk was recently acquired by Samsung and is now Samsung Food. Most of the functionality is still there but there are some obnoxious signs of early enshittification, a big "Get App" button if you're using the web form. It got me thinking.... what would happen to all my recipes and food plans if this app went away?</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Name Ideas</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I spent a little time coming up with some name ideas. Initially I went with nurishify but I decided it was too many syllables. The next one, which I was very pleased with was GastroPub which is a portmanteu of Gastronomy and ActivityPub but also a play on the british Gastro Pub - a fancy pub/bar that also serves nice food. I decided that GastroPub might be too British and also might confuse people who are genuinely looking for GastroPub recommendations. The name I've settled on for now is Gastronaut - a portmanteau of Gastronomy and Astronaut. I like the idea that it's like food space exploration and I imagine I could have a cute little logo with an Astronaut holding a knife and fork or something.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Intended Functionality</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>The idea is to offer many of <a href="https://whisk.com">Whisk.com</a>'s features but in an open, interoperable way. For example:</p>
<!-- /wp:paragraph -->
<!-- wp:list {"ordered":true} -->
<ol><!-- wp:list-item -->
<li>Import recipes from websites or books</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>Share recipes with friends (and plug into ActivityPub so that they can be tooted/reposted etc)</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>Manage your food plan and grocery list</li>
<!-- /wp:list-item --></ol>
<!-- /wp:list -->
<!-- wp:paragraph -->
<p>In the short term, my intention would be to make a neat import/export tool that uses common formats like CSV and Markdown so that someone could download all their data and keep it safe if their instance started to creak but perhaps longer term there could be some kind of profile migration tool like Mastodon offers.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I am a machine learning and natural language processing specialist so I have some ideas about how the system could auto-import recipes from websites but maybe also from photos that have been OCRed. LLMs are probably a bit overkill for this kind of use case, I suspect open source NLP pipelines like <a href="https://spacy.io/">SpaCy</a> would do pretty well at some of these tasks.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Tech Stack</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Rather than using this as an excuse to learn some new trendy language or tool, I want to get something up and running quickly. Therefore, I'm going to rely on well established, and probably 'boring' tools and frameworks to get the job done. The core web app is <a href="https://www.djangoproject.com/">Python Django</a>. I want to keep the frontend simple and minimalist and avoid javascript hell so I'm using the <a href="https://milligram.io/">Milligram</a> lightweight CSS framework along side jQuery and <a href="https://htmx.org/">HTMX</a> for doing some lightweight UX stuff. I'm going to use Python+<a href="https://docs.celeryq.dev/en/stable/getting-started/introduction.html">Celery</a>+<a href="https://redis.io/">Redis</a> for the worker queue system (for doing things like importing recipes).</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Funding and Sustainability</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>It may be a bit cheeky but<a href="https://ko-fi.com/jamesravey#"> I already have a Ko-fi page</a>. If this thing takes off I'd be interested in turning it into a side-gig. I would also ask people using the flagship instance of this software to make a regular donation if they can. I may also provide some "plus" features such as OCRing recipes from phone camera pics and charge a small fee for convenience. People would be welcome to self-host stuff and set up and pay for all the infrastructure for the premium features but likewise, they would probably find it cheaper and easier to use the central instance. </p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Early Screenshots/Mockups</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I've already started building out the app. Below are some photos of the current state of the system with the old Nurishify name/brand that I came up with.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I started with a simple landing page explaining what the app does and encouraging the user to sign up</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":382,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/image-7-1024x1021_f2a36b74.png" alt="A landing page. There is a photo of some food on a chopping board and a summary of the app and what it does. There is a button inviting users to sign up to use the app." class="wp-image-382"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>I made a mock up of the home screen where you'd be able to see your friends and what they'd posted about and manage your own recipes</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":381,"sizeSlug":"full","linkDestination":"none"} -->
<figure class="wp-block-image size-full"><img src="/media/image-6_5084b797.png" alt="A mockup of the app. On the left there is a full size picture of the user's avatar and underneath it summaries and links to the user's recipes, food plans and shopping list. On the right, a news feed showing people liking the user's recipe and adding their own." class="wp-image-381"/><figcaption class="wp-element-caption">A screenshot of the home screen with the old Nurishify name</figcaption></figure>
<!-- /wp:image -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">What's Next?</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I'm quite excited about building this thing out but I'd love to get your feedback! Any killer feature ideas? Any thoughts about how things should work? Any names that are better than Gastronaut?</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>In the next few days I will probably share a Github repo for my project but for now please follow me on my blog or on fosstodon for updates if you're interested!</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Thanks for listening to my ted talk!</p>
<!-- /wp:paragraph -->

150
brainsteam/content/posts/2023/11/18/Gitea Actions and PDM.md Normal file
View File

@ -0,0 +1,150 @@
---
categories:
- Software Development
date: '2023-11-18 22:44:24'
draft: false
tags:
- gitea
- python
title: Gitea Actions and PDM
type: posts
---
<!-- wp:paragraph -->
<p><a href="https://about.gitea.com/">Gitea</a> actions is the new Github-compatible CI/automation pipeline feature that ships with Gitea and <a href="https://forgejo.org/">Forgejo</a>. In theory it is interoperable with <a href="https://docs.github.com/en/actions">Github actions </a>but there are still a few rough edges and for that reason, the feature is still disabled by default.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I have been trying to get <a href="https://brainsteam.co.uk/2023/11/13/gastronaut-fediverse-recipe-app/" data-type="post" data-id="380">a django project</a> that uses <a href="https://brainsteam.co.uk/2021/04/01/opinionated-guide-to-virtualenvs/" data-type="post" data-id="193">PDM</a> for Python dependency management to to install itself and run some tests in a Gitea CI environment.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>In theory, my workflow is simple:</p>
<!-- /wp:paragraph -->
<!-- wp:list -->
<ul><!-- wp:list-item -->
<li>Check out the code with actions/checkout@v4</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>Install pdm with with<code> pdm-project/setup-pdm@v3</code></li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>Use a run: block to get pdm to install dependencies with <code>pdm install</code></li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>Use another run block to have PDM run my tests: <code>pdm run manage.py test</code></li>
<!-- /wp:list-item --></ul>
<!-- /wp:list -->
<!-- wp:paragraph -->
<p>Unfortunately there were a couple of odd quirks that I had to resolve first before I could get it working.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Add a Github Key</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I was initially getting an error about being Unauthorized from the <code>pdm-project/setup-pdm@v3</code> step. This is because the action attempts to download a list of pre-built python distributions from github and since we're running outside of github it is initially unable to get this list without an additional API key. All we need to do is <a href="https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-personal-access-token-classic">create a Github token</a> and then set up a new project secret and paste in the token that we just created. I use the name <code>GH_TOKEN</code> because Gitea does not allow you to use any secret prefixed <code>GITHUB</code>.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":410,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/image-10-1024x404_c1d64f51.png" alt="A screenshot of the project settings in gitea. Navigate to Actions, Secrets, Add Secret and reate a new secret called GH_TOKEN." class="wp-image-410"/><figcaption class="wp-element-caption">Create a secret called GH_TOKEN which we can pass to the action.</figcaption></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Now, we can pass the token into the setup pdm step of the yaml like so:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"yaml"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="yaml" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""> - uses: pdm-project/setup-pdm@v3
with:
python-version: 3.10
token: ${{ secrets.GH_TOKEN }}</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Change the Container Image</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Once I resolved the authorization error above, I started getting error messages about how no Python releases were available for the given OS and architecture. I thought that was weird because in theory we're running Ubuntu on x64. <a href="https://forum.gitea.com/t/gitea-actions-with-python/7605/4">This forum post</a> suggested that changing the docker image that the runner uses to execute the pipeline might work. I'm not 100% sure what the default Gitea action runner uses as its base image but Gitea Actions are based on <a href="https://github.com/nektos/act">Act</a>, a local runner for Github actions and the official Act project recommends images by <a href="https://github.com/catthehacker/docker_images">catthehacker</a> for use as runners. By specifying one of these images, we seem to be able to 'fix' whatever metadata is missing from the default image.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We can pass a container in via the job <code>container</code> directive like so:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"yaml"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="yaml" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">jobs:
run_tests:
runs-on: ubuntu-latest
container: catthehacker/ubuntu:act-latest
steps:
...
- uses: pdm-project/setup-pdm@v3
with:
python-version: 3.10
token: ${{ secrets.GH_TOKEN }}</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>With this change in place, the rest of my pipeline seems to have burst into life. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Here is the full yaml file for my CI:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"yaml"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="yaml" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">name: Run Tests
run-name: ${{ gitea.actor }} is testing out Gitea Actions 🚀
on: [push]
jobs:
run_tests:
runs-on: ubuntu-latest
container: catthehacker/ubuntu:act-latest
steps:
- name: Checkout Codebase
uses: actions/checkout@v3
- name: Set up python
run: |
apt-get update && apt-get install -y python3-venv
pip install --upgrade pdm
- uses: pdm-project/setup-pdm@v3
with:
python-version: 3.10
token: ${{ secrets.GH_TOKEN }}
- name: Install dependencies
run: cd ${{ gitea.workspace }} && pdm install
- name: Run tests
run: |
cd ${{ gitea.workspace }} && pdm run manage.py test
</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->

271
brainsteam/content/posts/2023/11/19/Parsing Ingredient Strings with SpaCy PhraseMatcher.md Normal file
View File

@ -0,0 +1,271 @@
---
categories:
- Data Science
- Software Development
date: '2023-11-19 18:13:40'
draft: false
tags:
- django
- gastronaut
- python
title: Parsing Ingredient Strings with SpaCy PhraseMatcher
type: posts
---
<!-- wp:paragraph -->
<p>As part of my work on <a href="https://brainsteam.co.uk/2023/11/13/gastronaut-fediverse-recipe-app/" data-type="post" data-id="380">Gastronaut</a>, I'm building a form that allows users to create recipes and which will attempt to parse ingredients lists and find a suitable stock photo for each item the user adds to their recipe. As well as being cute and decorative, this step is important for later when we want to normalise ingredient quantities.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>What we're looking to do is take a string like "2lbs flour" and turn it into structured data - a json representation might look like this:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"json"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="json" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">{
"ingredient":"flour",
"unit":"lbs",
"quantity":"2"
}</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>We can then do whatever we need with this structured data - like using the <code>ingredient</code> to look up a thumbnail in another system or generate a link or reference to the new recipe so that people looking for "banana" can find all the recipes that use them.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Building a Parser</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>There are a few options for parsing these strings. If you're feeling frivolous and want to crack a walnut with a sledgehammer, you could probably get OpenAI's GPT to parse these strings with a single API call and a prompt. However, I wanted to approach this problem with a more proportional technique.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I'm using Spacy along with Spacy's <a href="https://spacy.io/api/phrasematcher">PhraseMatcher</a> functionality which basically looks for a list of possible words and phrases. Once we've <a href="https://spacy.io/usage">installed Spacy</a>, we make a long list of units and we tell Spacy about them:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"python"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import spacy
from spacy.matcher import PhraseMatcher
# Load the English language model
nlp = spacy.load("en_core_web_sm")
# Define a list of known units
known_units = [
"grams",
"g", "kg",
"kilos",
"kilograms",
# ... many more missing for brevity
"lbs",
"cup",
"cups",
"tablespoons",
"teaspoons"]
# Initialize the pattern matcher
matcher = PhraseMatcher(nlp.vocab)
# Add the unit patterns to the matcher
matcher.add("UNITS", [nlp(x) for x in known_units])</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>Now we can write a function to use this matcher along with a bit of logic to figure out what's what and structure it into the json format I outlined above.</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"python"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Function to parse ingredient strings
def parse_ingredient(ingredient):
doc = nlp(ingredient)
matches = matcher(doc)
quantity = None
unit = None
ingredient_name = []
for token in doc:
if token.text in ['(',')']:
continue
if not quantity and token.pos_ == 'NUM':
quantity = token.text
elif unit is None and any(match_id == token.i for _, match_id, _ in matches):
unit = token.text
else:
ingredient_name.append(token.text)
return {
"quantity": quantity,
"unit": unit,
"ingredient": " ".join(ingredient_name)
}
</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:list -->
<ul><!-- wp:list-item -->
<li>First, we allow spacy to parse the document and then use our matcher to try and find unit matches. </li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>Then we iterate through each token in the document, ignoring brackets - we might want to expand this to other punctuation.</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>If the token is a number and we don't have a quantity yet, assume it's the quantity.</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>We assume <code>quantity</code> can only defined once per ingredient string and therefore once we've found it, we don't consider any more tokens for that role. </li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>Otherwise, We check to see whether the current was matched to a unit.</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>Last but not least, if the token can't be matched to a quantity or a unit then we assume it's part of the ingredient name.</li>
<!-- /wp:list-item --></ul>
<!-- /wp:list -->
<!-- wp:paragraph -->
<p>This approach works if the items are in a different order too (which would completely throw a regular expression off) e.g. Milk (1 cup) rather than 1 cup milk.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Making it More Robust</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>This logic is not perfect but it should cover most reasonable use cases where someone enters an ingredient following <quantity> <unit> <ingredient>. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>However, if I want to improve the parsing performance and variety of things that we want to be able to understand in the future, I could train a custom NER model inside spacy. I will likely write about doing exactly that at some point in the future.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Adding Some Tests</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Since this is going to operate an API endpoint in my recipe app, I want to be relatively sure it will work reliably for a few different examples. I'm building a Django app so I've set up a test case using the Django testing framework</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"python"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">from django.test import TestCase
import spacy
from spacy.matcher import PhraseMatcher
from recipe_app.nlp.ingredients import parse_ingredient
class ParseIngredientTestCase(TestCase):
def test_parse_ingredient(self):
test_cases = [
("4 bananas", {"quantity": "4", "unit": None, "ingredient": "bananas"}),
("200g sugar", {"quantity": "200", "unit": "g", "ingredient": "sugar"}),
("1 stock cube", {"quantity": "1", "unit": None, "ingredient": "stock cube"}),
("1/2 tbsp flour", {"quantity": "1/2", "unit": "tbsp", "ingredient": "flour"}),
("3 lbs ground beef", {"quantity": "3", "unit": "lbs", "ingredient": "ground beef"}),
("2.5 oz chocolate chips", {"quantity": "2.5", "unit": "oz", "ingredient": "chocolate chips"}),
("5 kg potatoes", {"quantity": "5", "unit": "kg", "ingredient": "potatoes"}),
("1 cup milk", {"quantity": "1", "unit": "cup", "ingredient": "milk"}),
("2 tablespoons olive oil", {"quantity": "2", "unit": "tablespoons", "ingredient": "olive oil"}),
("1/4 pound sliced ham", {"quantity": "1/4", "unit": "pound", "ingredient": "sliced ham"}),
("2 liters water", {"quantity": "2", "unit": "liters", "ingredient": "water"}),
("750 ml orange juice", {"quantity": "750", "unit": "ml", "ingredient": "orange juice"}),
("3 teaspoons salt", {"quantity": "3", "unit": "teaspoons", "ingredient": "salt"}),
("milk (1 cup)", {"quantity": "1", "unit": "cup", "ingredient": "milk"}),
("tomatoes (3 pieces)", {"quantity": "3", "unit": "pieces", "ingredient": "tomatoes"}),
("pasta (200g)", {"quantity": "200", "unit": "g", "ingredient": "pasta"}),
]
for ingredient, expected in test_cases:
parsed = parse_ingredient(ingredient)
self.assertEqual(parsed, expected)
</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>This code tests a few different variations and options and includes some examples where there is no unit. I also used American english spellings of some of the units for variety.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Building an Endpoint</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I'm using HTMX for doing asynchronous interaction between the web page and the form. In my Form class, I set <code>hx-get</code> on the ingredient form field so that whenever the value changes it makes a request to an <code>ingredient_parser</code> endpoint:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"python"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">from django import forms, urls
from recipe_app.models import Recipe, RecipeIngredient, Ingredient
class IngredientForm(forms.Form):
ingredient = forms.CharField(widget=forms.TextInput(attrs={
'hx-get': urls.reverse_lazy('ingredient_parser'),
'hx-trigger': "change",
"hx-target":"this",
"hx-swap": "outerHTML"
}))
</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>Then I have a view class defined which grabs the value from the ingredient form, parses it and responds with a little HTML snippet which HTMX swaps in on the frontend. In the future I will look up a stock image per ingredient but for now I've got a mystery food picture:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"python"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">class IngredientAutocomplete(View):
def get(self, request, *args, **kwargs):
ingredient = None
form_key = None
for key in request.GET:
if key.startswith("recipeingredient"):
ingredient = request.GET[key]
form_key = key
break
if ingredient is None:
return JsonResponse({}) # TODO: make this respond in a better way
else:
ing = parse_ingredient(ingredient)
ing['raw_text'] = ingredient
ing['form_key'] = form_key
ing['thumbnail'] = static("images/food/mystery_food.png")
return render(request, "partial/autocomplete/ingredients.html", context=ing)</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>The for loop over the request.GET items is needed because each time we add a new ingredient to the form, the field gets a slightly different name. e.g. recipeingredient_1, recipeingredient_2 and so on.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Putting it All Together</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I recorded a video of the form that I've built where I add an ingredient and the response gets populated.</p>
<!-- /wp:paragraph -->
<!-- wp:embed {"url":"https://youtu.be/_l0_Lxwm4TY","type":"video","providerNameSlug":"youtube","responsive":true,"className":"wp-embed-aspect-4-3 wp-has-aspect-ratio"} -->
<figure class="wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-4-3 wp-has-aspect-ratio"><div class="wp-block-embed__wrapper">
https://youtu.be/_l0_Lxwm4TY
</div></figure>
<!-- /wp:embed -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->

39
brainsteam/content/posts/2023/11/20/Reply_ How moving from AWS to Bare-Metal saved us 230_000_ _yr.md Normal file
View File

@ -0,0 +1,39 @@
---
categories:
- Engineering Leadership
- Software Development
date: '2023-11-20 11:56:36'
draft: false
tags:
- cloud
- devops
- onprem
title: 'Reply: How moving from AWS to Bare-Metal saved us 230,000$ /yr'
type: posts
---
<!-- wp:quote -->
<blockquote class="wp-block-quote"><!-- wp:paragraph -->
<p>Our transition from AWS to bare-metal infrastructure underscores the fact that while cloud services such as AWS offer robust flexibility and power, they may not always be the most economical choice for every enterprise.</p>
<!-- /wp:paragraph --><cite>Neel Patel, <a href="https://blog.oneuptime.com/moving-from-aws-to-bare-metal/">https://blog.oneuptime.com/moving-from-aws-to-bare-metal/</a></cite></blockquote>
<!-- /wp:quote -->
<!-- wp:paragraph -->
<p>We had a very similar experience at my current company a few years ago and we're saving similar magnitudes of cash. Rather than going full bare metal we have a hybrid IT strategy where staging and production systems are hosted in Google Cloud with all of the advantages that brings (redundancy, flexibility, security) but we use our own hardware for running test environments and CI workloads. Our source code repo is cloud-based for the redundancy after, true story, our old gitlab instance was literally melted in <a href="https://www.reuters.com/article/us-france-ovh-fire-idUSKBN2B20NU">the OVH fire, March 2021</a>.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Our hardware is co-located in an ISO27001 data centre and we also have an off-site backup solution, even though the "test" data is not as critical as the data we keep in the cloud.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We use containerisation and Kubernetes orchestration to provide similar environments for running almost identical environments in the cloud and locally. Our product is quite chunky and expensive to run and barely chugs along on a laptop so having inexpensive internal test instances that our team can use for testing and debugging purposes is really helpful.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>It seems like cloud is great for "getting started" as a business when you maybe don't have the resource or knowledge for configuring your own servers in-house. Cloud probably also makes sense when you are a massive corporation and you have so many machines and networks to manage that you basically end up inventing your own "cloud" anyway - if your business' core competency is not IT then you almost certainly don't want to have to operationalise your own cloud!</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>However, at a certain size I think IT/Tech firms can end up in this sort of "Goldilocks" zone where you are big enough to know how to manage servers and small enough that it isn't an entire industry. At this size there are some serious savings to be made, even when you offset the salaries of your site engineering staff!</p>
<!-- /wp:paragraph -->

114
brainsteam/content/posts/2023/11/24/Medieval Buzzfeed - Debugging Dodgy Datetimes in Pandas and Parquet.md Normal file
View File

@ -0,0 +1,114 @@
---
categories:
- Data Science
date: '2023-11-24 09:19:20'
draft: false
tags:
- pandas
- python
title: Medieval Buzzfeed - Debugging Dodgy Datetimes in Pandas and Parquet
type: posts
---
<!-- wp:paragraph -->
<p>I was recently attempting to cache the results of a long-running SQL query to a local parquet file using SQL via a workflow like this:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"python"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import os
import pandas as pd
import sqlalchemy
env = os.environ
engine = sqlalchemy.create_engine(f"mysql+pymysql://{env['SQL_USER']}:{env['SQL_PASSWORD']}@{env['SQL_HOST']}/{env['SQL_DB']}")
connection = engine.connect()
with engine.connect() as conn:
df = pd.read_sql("SELECT * FROM articles", connection)
df.to_parquet("articles.parquet")</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>This ended up yielding the following slightly cryptic error message:</p>
<!-- /wp:paragraph -->
<!-- wp:code -->
<pre class="wp-block-code"><code>ValueError: Can't infer object conversion type: 0 2023-03-23 11:31:30
1 2023-03-20 09:37:35
2 2023-02-27 10:46:47
3 2023-02-24 10:34:42
4 2023-02-23 08:51:11
...
908601 2023-11-09 14:30:00
908602 2023-11-08 14:30:00
908603 2023-11-07 14:30:00
908604 2023-11-06 14:30:00
908605 2023-11-02 13:30:00
Name: published_at, Length: 908606, dtype: object</code></pre>
<!-- /wp:code -->
<!-- wp:paragraph -->
<p>So obviously there is an issue with my <code>published_at</code> timestamp column. Googling didn't help me very much, lots of people suggesting that because there are maybe some <code>nan</code> values in the column, Pandas can't infer the correct data type before serializing to parquet.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I tried doing <code>df.fillna(0, inplace=True)</code> on my dataframe, hoping that pandas would be able to coerce the value into a zeroed out unix epoch but I noticed I was still getting the issue. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>A quick inspection of <code>df.published_at.dtype</code> returned 'O'. That's pandas' catchall "I don't know what this is" object data type. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I tried to force the data type to a date with <code>pd.to_datetime(df.published_at)</code> but I got another error :</p>
<!-- /wp:paragraph -->
<!-- wp:code -->
<pre class="wp-block-code"><code>OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 1201-11-01 12:00:00, at position 154228</code></pre>
<!-- /wp:code -->
<!-- wp:paragraph -->
<p>Sure enough if I inspect the record at row <code>154228</code> the datestamp is in the year of our lord 1201. I don't /think/ the article would have been published approximately 780 years before the internet was invented. Aside from the fact that this is obviously wrong, the error essentially tells us that the date was so long ago that it's not possible to represent it in terms of how many nanoseconds it was before the unix epoch (1 Jan 1970) without the data structure running out of memory.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We now need to do some clean up and make some assumptions about the data. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We can be pretty confident that none of the news articles from before the unix epoch matter. In this use case, I'm actually only interested in news from the last couple of years so I could probably be even more cut throat than that. I check how many articles are older than that:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"python"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import datetime
EPOCH = datetime.datetime.fromtimestamp(0)
df[df.published_at < EPOCH]</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>The only result - our article from the dark ages. I'm going to treat the unix epoch as a sort of <code>nan</code> value and set all articles with dates older than this (thankfully only the one) to have that value:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"python"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">
df.loc[df.published_at < EPOCH, 'published_at'] = EPOCH</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>Now when I re-run my <code>to_datetime</code> conversion it works! We can overwrite the column on our dataframe and write it out to disk!</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"python"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">df.published_at = pd.to_datetime(df.published_at)
df.to_parquet("test.parquet")</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->

22
brainsteam/content/posts/2023/12/04/Data Swamp.md Normal file
View File

@ -0,0 +1,22 @@
---
categories:
- Data Science
date: '2023-12-04 14:40:04'
draft: false
tags:
- humour
title: Data Swamp
type: posts
---
<!-- wp:indieblocks/like {"empty":false} -->
<div class="wp-block-indieblocks-like"><div class="u-like-of h-cite"><p><i>Likes <a class="u-url" href="https://snarfed.org/2023-12-03_51578">https://snarfed.org/2023-12-03_51578</a> by <span class="p-author">Ryan Barrett</span>.</i></p></div><div class="e-content"><!-- wp:quote -->
<blockquote class="wp-block-quote"><!-- wp:paragraph -->
<p>My dad has spent some of his retirement doing hobbyist machine learning projects. He heard the term “data lake” a while back and has taken to calling his datasets a “data swamp.” Feels like a terminology improvement the whole field could get behind.</p>
<!-- /wp:paragraph --></blockquote>
<!-- /wp:quote --></div></div>
<!-- /wp:indieblocks/like -->
<!-- wp:paragraph -->
<p>This is brilliant, I've not come across this term before but I could definitely get behind using it to describe data that comes out of customer systems like <a href="https://bsky.app/profile/cleverdevil.io/post/3kfom22ivtf2u">Jonathon says he already does</a> at his workplace.</p>
<!-- /wp:paragraph -->

57
brainsteam/content/posts/2023/12/04/Weeknote CW48 2023.md Normal file
View File

@ -0,0 +1,57 @@
---
categories:
- Personal
date: '2023-12-04 15:46:20'
draft: false
tags: []
title: Weeknote CW48 2023
type: posts
---
<!-- wp:paragraph -->
<p>I've missed a couple of weeknotes since the end of November and start of December has been a pretty busy time.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>On Tuesday we went to see a rerun of <a href="https://www.youtube.com/watch?v=wk9caHO3pW0">The Holiday</a> at the cinema. It's one of my favourite Christmas films and it put me and Mrs R into a christmassy mood. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>On Thursday I went to London to partake in the Alan Turing Institute NLP Group's Christmas party which was really lovely. I haven't seen much of Prof. Liakata or her students since I finished my PhD this time last year so it was great to catch up and have a good old moan about the AI Hype of the past year. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The weather has been bitterly cold over the last few days in the UK. On Friday it briefly snowed in London and in Fareham and we drove up to the Midlands to see my parents for the last time before Christmas and to do gift exchanges. We were a little concerned that we might not make it up there (not necessarily because it was particularly bad weather but because the UK grinds to a halt whenever we so much as glimpse a single snowflake). However, the drive was very smooth.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>On Saturday we spent the day with my dad and his wife. It was -4C outside so we spent the day indoors with the log fire on and a hot toddy in hand. Their naughty cat Bertie was almost constantly under foot and at one point decided to put himself in the bin which made a great photo opportunity.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":562,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/1701693562222-e1701695405141-edited_532562d7.jpg" alt="A british shorthair cat sat in a rubbish bin with a cheeky look on his face" class="wp-image-562"/><figcaption class="wp-element-caption">Naughty Bertie in the bin</figcaption></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>In the evening we visited Ralph Court Gardens where we met with my mum and her partner. We had a walk around the illuminated gardens (in -4C) and afterwards went indoors for a slide of cake and a hot chocolate. The illuminations were a strange mixture, some of them were cute, some of them were perplexing like a Santa in an England football supporter getup.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":563,"width":"600px","sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large is-resized"><img src="/media/1024-1360-edited_2567b615.jpg" alt="An inflatable Santa wearing a white vest with the English Flag across it." class="wp-image-563" style="width:600px"/><figcaption class="wp-element-caption">I particularly enjoyed seeing football hooligan santa</figcaption></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>I haven't published much on this site since last week when <a href="https://brainsteam.co.uk/2023/11/24/medieval-buzzfeed-debugging-dodgy-datetimes-in-pandas-and-parquet/">I wrote about a problem I was diagnosing in Python to do with weird datestamps</a>.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I'm continuing to read the young Dredd books, I'm about <a href="https://bookwyrm.social/book/1437488/s/judge-dredd-year-three-judge-dredd-the-early-years">45% through Dredd: Year Three</a> and I'm also continuing to make my way through <a href="https://bookwyrm.social/book/402435/s/the-one-thing-the-surprisingly-simple-truth-behind-extraordinary-results">The One Thing</a> which I picked up last week. I can't resist a "productivity" self-help book. On my way back from London last week I found myself listening to <a href="https://www.bbc.co.uk/news/uk-65175712">A Very British Cult</a>, a podcast that was recommended during another podcast I listen to regularly: <a href="https://pod.link/1564066507">Three Bean Salad</a>. It follows the story of a number of people who joined what seemed like a life coaching/mentoring group and ended up giving them lots of money and isolating themselves from their families to spend more time with the group. It's highly worth a listen if you are interested in crime/cult documentaries. The thing I found particularly interesting is that a lot of the key people are based near me in Hampshire. I've also started a new playthrough of Cyberpunk 2077 with the 2.1 patch in place.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>This week Mrs R and I had today off to recover from our trip up north and then later in the week I'm up in London again where I'll be joining a panel discussion as part of a seminar on the <a href="https://www.wbs.ac.uk/courses/executive-education/pathways/business-impacts-of-artificial-intelligence/">Business Impacts of AI</a> taught by a friend who is an Assistant Professor at Warwick Business School.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->

95
brainsteam/content/posts/2023/12/05/AI_s Electron.JS Moment_.md Normal file
View File

@ -0,0 +1,95 @@
---
categories:
- AI and Machine Learning
- Philosophy and Thinking
date: '2023-12-05 11:59:17'
draft: false
tags:
- climate
- genai
- onprem
title: AI's Electron.JS Moment?
type: posts
---
<!-- wp:indieblocks/reply {"empty":false} -->
<div class="wp-block-indieblocks-reply"><div class="u-in-reply-to h-cite"><p><i>In reply to <a class="u-url p-name" href="https://gizmodo.com/ai-images-as-much-energy-as-charging-phone-hugging-face-1851065091">Generating AI Images Uses as Much Energy as Charging Your Phone, Study Finds</a>.</i></p></div><div class="e-content"><!-- wp:paragraph -->
<p><a href="https://arxiv.org/pdf/2311.16863.pdf">The study</a> provides an analysis of ML model energy usage on a state of the art nvidia chip:</p>
<!-- /wp:paragraph -->
<!-- wp:quote -->
<blockquote class="wp-block-quote"><!-- wp:paragraph -->
<p>We ran all of our experiments on a single <strong><em>NVIDIA A100-SXM4-80GB GPU </em></strong></p>
<!-- /wp:paragraph --></blockquote>
<!-- /wp:quote --></div></div>
<!-- /wp:indieblocks/reply -->
<!-- wp:paragraph -->
<p>Looking these devices up - <a href="https://www.techpowerup.com/gpu-specs/a100-sxm4-80-gb.c3746">they have a power draw of 400W</a> when they're running at full pelt. Your phone probably uses something like 30-40W when fast charging and your laptop probably uses 60-120W when it's charging up. Gaming-grade GPUs like the RTX4090 have a <a href="https://www.techpowerup.com/gpu-specs/geforce-rtx-4090.c3889">similar power draw</a> to the A100 (450w). My<a href="https://www.techpowerup.com/gpu-specs/geforce-rtx-4070.c3924"> Nvidia 4070</a> has a power draw of 200W.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We know that the big players are running data centres filled with racks and racks of A100s and similar chips and that is concerning. We should collectively be concerned with how much energy we're burning using these systems. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I'm a bit wary about the Gizmodo article's conclusion that all models - including Dall-e and Midjourney - should be tarred with the same brush, not because I'm naively optimistic that they're not burning the same amount of energy, but simply because they are an unknown quantity at this point. It's possible that they are doing something clever behind the scenes (see quantization section below)/</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Industry Pivot Away From Task Appropriate Models</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I think those of us in the AI/ML space had an intuition that custom-trained models would probably be cheaper and more efficient than generative models but this study provides some great empirical validation of that hunch:</p>
<!-- /wp:paragraph -->
<!-- wp:quote -->
<blockquote class="wp-block-quote"><!-- wp:paragraph -->
<p>...The difference is much more drastic if comparing BERT-based models for tasks such as text classification with the larger multi-purpose models: for instance <code>bert-base-multilingual-uncased-sentiment</code> emits just 0.32g of 𝐶𝑂2 per 1,000 queries, compared to 2.66g for<code> Flan-T5-XL</code> and 4.67g for <code>BLOOMz-7B</code>... <br><br>...While we see the benefit of deploying generative zero-shot models given their ability to carry out multiple tasks, we do not see convincing evidence for the necessity of their deployment in contexts where tasks are well-defined, for instance web search and navigation, given these models’ energy requirements.</p>
<!-- /wp:paragraph --><cite>pg 14, <a href="https://arxiv.org/pdf/2311.16863.pdf">https://arxiv.org/pdf/2311.16863.pdf</a></cite></blockquote>
<!-- /wp:quote -->
<!-- wp:paragraph -->
<p>Generative models that can "solve" problems out of the box may seem like an easy way to save many person-weeks of effort - defining and scoping an ML problem, building and refining datasets and so on. However, the cost to the environment (heck even the fiscal cost) of training and using these models is higher in the long term. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>If we look at the recent history of the software industry to understand this current trend, we can see a similar sort of pattern in the switch away from platform-specific development frameworks like <a href="https://www.qt.io/">QT</a> or Java on Android towards the use of cross-platform frameworks like <a href="https://www.electronjs.org/">Electron.js</a> <a href="https://reactnative.dev/">React Native</a>. These frameworks generally produce more power-hungry, bloated apps but a much faster and cheaper development experience for companies who need to support apps across multiple systems. This is why your banking app takes up several hundred megabytes on your phone.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The key difference when applying this general "write once run everywhere" type approach to AI is that, once you scratch the surface of your problem-space and realise that prompt engineering is more <a href="https://explorethearchive.com/alchemy-marketing-scheme-watm">alchemy</a> than wizadry and that the behaviour of these models is opaque and almost impossible to explain, it may make sense to start with a simple model anyway. If you have a well defined classification problem you might find that a random forest model that can run on a potato computer will do the job for you.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Quantization and Optimisation</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>A topic that this study doesn't broach is model optimisation and quantization. For those unfamiliar with the term, quantization is a compression mechanism which allows us to shrink neural network models so that they can run on older/slower computers or run much more quickly and efficiently on state-of-the-art hardware. Quantization has been making big waves this year, starting with <a href="https://github.com/ggerganov/llama.cpp">llama.cpp </a>(which I built <a href="https://brainsteam.co.uk/2023/09/30/turbopilot-obit/">Turbopilot</a> on top of). </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Language models like Llama and Llama2 typically need several gigabytes of VRAM to run (hence the A100 with 80GB ram). However, quantized models can run in 8-12GiB RAM and will happily tick along on your gaming GPU or even a Macbook with an Apple M-series chip. For example, To run Llama2 without quantization you need 28GiB of RAM. To run it in 5-bit quantized mode you need 7.28GB. Not only does compressing the model mean it can run on smaller hardware but it also means that inference can be carried out in fewer computer cycles since we can do more calculations at once.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Whilst I stand by the idea that we should use appropriate models for specific tasks, I'd love to see this same study done with quantized models. Furthermore, there's nothing stopping us applying quantization to pre-GPT models to make them even more efficient too as <a href="https://github.com/skeskinen/bert.cpp">this repository</a> attempts to do with BERT.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I haven't come across a stable runtime for quantized stable diffusion models yet but there are <a href="https://github.com/Xiuyu-Li/q-diffusion">promising early signs </a>that such an approach is possible for image generation models too.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>However, I'd wager that companies like OpenAI are currently not under any real pressure (commercial or technical) to quantize their models when they can just throw racks of A100s at the problem and chew through gigawatt-hours in the process. </p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Conclusion</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>It seems pretty clear that transformer-based and diffusion-based ML models are energy intensive and difficult to deploy at scale. Whilst there are some use cases where it may make sense to deploy generative models, the advantages that these models bring to well defined problem spaces may simply never manifest. In cases where a generative model does make sense, we should be using optimisation and quantization to make their usage as energy efficient as possible.</p>
<!-- /wp:paragraph -->

32
brainsteam/content/posts/2023/12/20/NLP in the Post-LLM World.md Normal file
View File

@ -0,0 +1,32 @@
---
categories:
- AI and Machine Learning
date: '2023-12-20 06:37:14'
draft: false
tags:
- nlp
title: NLP in the Post-LLM World
type: posts
---
<!-- wp:paragraph -->
<p>I really enjoyed diving into Seb Ruder's <a href="https://nlpnewsletter.substack.com/p/nlp-research-in-the-era-of-llms">latest NLP Newsletter</a> which focuses on all the areas of NLP that are still in desperate need of attention in a post-LLM world.</p>
<!-- /wp:paragraph -->
<!-- wp:quote -->
<blockquote class="wp-block-quote"><!-- wp:paragraph -->
<p>In an era where running state-of-the-art models requires a garrison of expensive GPUs, what research is left for academics, PhD students, and newcomers to NLP without such deep pockets?</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>...while massive compute often achieves breakthrough results, its usage is often inefficient. Over time, improved hardware, new techniques, and novel insights provide opportunities for dramatic compute reduction...</p>
<!-- /wp:paragraph --></blockquote>
<!-- /wp:quote -->
<!-- wp:paragraph -->
<p> I wrote about some of the same issues in my post <a href="https://brainsteam.co.uk/2023/03/25/nlp-is-more-than-just-llms/">NLP is more than just LLMs</a> earlier this year and <a href="https://brainsteam.co.uk/2023/12/05/carbon-footprint-of-generative-models/">I recently speculated</a> about how current industry AI darlings, sexy-scale-up companies very much in "growth" mode as opposed to incumbents in "cost-saving" mode, are just not incentivised to be compute-efficient.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>If you are just starting out in this space there are plenty of opportunities and lots of problems to solve - particularly around trust, reliability and energy efficiency.</p>
<!-- /wp:paragraph -->

16
brainsteam/content/posts/2023/12/21/20231221_082f036a.md Normal file
View File

@ -0,0 +1,16 @@
---
categories:
- Personal
date: '2023-12-21 15:28:57'
draft: false
tags:
- read
title: null
type: posts
---
<!-- wp:paragraph -->
<p>Reblog via <a href="http://bookwyrm.social/user/jamesravey/comment/3345493">James Ravenscroft</a>
</p>
<!-- /wp:paragraph -->
<p>Took me a few weeks but I finally chewed my way through it. Thoroughly enjoyable pulp sci-fi. It has been fun to read this whilst also playing Cyberpunk 2077 and think about how the two might cross over.</p><p>(comment on <a href="https://bookwyrm.social/book/1437488"><i>Judge Dredd Year Three (Judge Dredd: The Early Years)</i></a>)</p>

16
brainsteam/content/posts/2023/12/21/20231221_224cf724.md Normal file
View File

@ -0,0 +1,16 @@
---
categories:
- Personal
date: '2023-12-21 15:28:57'
draft: false
tags:
- read
title: null
type: posts
---
<!-- wp:paragraph -->
<p>Reblog via <a href="http://bookwyrm.social/user/jamesravey/comment/3345493">James Ravenscroft</a>
</p>
<!-- /wp:paragraph -->
<p>Took me a few weeks but I finally chewed my way through it. Thoroughly enjoyable pulp sci-fi. It has been fun to read this whilst also playing Cyberpunk 2077 and think about how the two might cross over.</p><p>(comment on <a href="https://bookwyrm.social/book/1437488"><i>Judge Dredd Year Three (Judge Dredd: The Early Years)</i></a>)</p>

16
brainsteam/content/posts/2023/12/21/20231221_23f8cf2d.md Normal file
View File

@ -0,0 +1,16 @@
---
categories:
- Personal
date: '2023-12-21 15:28:57'
draft: false
tags:
- read
title: null
type: posts
---
<!-- wp:paragraph -->
<p>Reblog via <a href="http://bookwyrm.social/user/jamesravey/comment/3345493">James Ravenscroft</a>
</p>
<!-- /wp:paragraph -->
<p>Took me a few weeks but I finally chewed my way through it. Thoroughly enjoyable pulp sci-fi. It has been fun to read this whilst also playing Cyberpunk 2077 and think about how the two might cross over.</p><p>(comment on <a href="https://bookwyrm.social/book/1437488"><i>Judge Dredd Year Three (Judge Dredd: The Early Years)</i></a>)</p>

16
brainsteam/content/posts/2023/12/21/20231221_528343a7.md Normal file
View File

@ -0,0 +1,16 @@
---
categories:
- Personal
date: '2023-12-21 15:28:57'
draft: false
tags:
- read
title: null
type: post
---
<!-- wp:paragraph -->
<p>Reblog via <a href="http://bookwyrm.social/user/jamesravey/comment/3345493">James Ravenscroft</a>
</p>
<!-- /wp:paragraph -->
<p>Took me a few weeks but I finally chewed my way through it. Thoroughly enjoyable pulp sci-fi. It has been fun to read this whilst also playing Cyberpunk 2077 and think about how the two might cross over.</p><p>(comment on <a href="https://bookwyrm.social/book/1437488"><i>Judge Dredd Year Three (Judge Dredd: The Early Years)</i></a>)</p>

228
brainsteam/content/posts/2023/12/22/Annual Review 2023.md Normal file
View File

@ -0,0 +1,228 @@
---
categories:
- Personal
date: '2023-12-22 11:50:24'
draft: false
tags:
- annual-review
title: Annual Review 2023
type: posts
---
<!-- wp:paragraph -->
<p>It's that time of year where people reflect on how their year went. I'd say this year has been a really mixed bag as my life has changed and transformed post-PhD study and my job has transformed significantly post funding at work. There were lots of exciting achievements and events but a little stagnation on some fronts. I hope that next year I can focus a little more on some of those areas.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Becoming a Doctor...</h3>
<!-- /wp:heading -->
<!-- wp:media-text {"mediaId":1132,"mediaType":"image","mediaWidth":33,"mediaSizeSlug":"medium","verticalAlignment":"top"} -->
<div class="wp-block-media-text is-stacked-on-mobile is-vertically-aligned-top" style="grid-template-columns:33% auto"><figure class="wp-block-media-text__media"><img src="/media/image-7-226x300_f43cfeb1.png" alt="a mug with the abstract from James' phd thesis printed on it" class="wp-image-1132 size-medium"/></figure><div class="wp-block-media-text__content"><!-- wp:paragraph {"placeholder":"Content…"} -->
<p>I'd say my biggest achievement this year was finally completing my PhD after 7 years of part time study after having my thesis corrections accepted in February. I began my PhD in Natural Language Processing with the University of Warwick in 2015 while I was still at IBM. During that time a lot has changed, both for me personally and in technology and machine learning. The last few years have been a huge challenge but I'm really happy that they've paid off.</p>
<!-- /wp:paragraph --></div></div>
<!-- /wp:media-text -->
<!-- wp:paragraph -->
<p>Unfortunately I was sick during my graduation ceremony but I graduated in absentia and received my doctoral certificate in the mail. I had a lot of fun updating my title to Dr from Mr on as many official documents as I can and repeatedly making the joke about "is there a doctor in the house? Yes but I can't help with your medical emergency."</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I am excited to continue working on my academic projects as a sideline and will be staying in touch with my supervisors at the University of Warwick, Aberystwyth University and the Alan Turing Institute.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Public Appearances & Talks</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I've done reasonably well for public speaking gigs this year. In April I was invited to give a talk about applied AI at Rare Earth Digital's Debrief event in Nantwich, Cheshire. This gave me an opportunity to put together a slide deck that incorporated my experience of building machine learning and AI solutions over the last decade and some some cautionary tales about LLMs that I picked up from folks like<a href="https://simonwillison.net/"> Simon WIllison</a>.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":1102,"sizeSlug":"full","linkDestination":"none"} -->
<figure class="wp-block-image size-full"><img src="/media/image-4_cbf30d6f.png" alt="James standing at the front of a function room filled with people siting at round tables." class="wp-image-1102"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>I then did something of a "tour", presenting similar material at a local Science Cafe, an event for KTP associates at the University of Essex and privately for my wife's team at her workplace.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Earlier in December I was invited by a member of my PhD cohort to join a panel discussion about AI in industry. The audience were made up of heavy hitters from a diverse set of companies and industries. I was pleasantly surprised at the level of discourse and the healthy cynicism that the audience had about the blind application of GPT and LLMs.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":1101,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/image-3-1024x768_ed96ad02.png" alt="A webcam photo of panelists sat in a lecture theatre with James in the middle." class="wp-image-1101"/><figcaption class="wp-element-caption">My first visit to The Shard in London was as part of a panel discussion on AI as part of a course run by Warwick Business School</figcaption></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>In October, I also<a href="https://brainsteam.co.uk/2023/10/15/posts-2023-10-15-weeknote-cw-41-20231697388830/"> attended an awards ceremony</a>, accepting the trophy for best emerging tech firm on behalf of my company.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Work Achievements</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>This year my company went from being pretty much fully bootstrapped to taking on some investment money. This was a new experience for pretty much all of the senior leadership and the experience has been very exciting and enlightening and at times quite uncomfortable. It was great as it meant that we could <a href="https://brainsteam.co.uk/2023/08/08/we-moved-offices/">move offices</a>, give out pay rises and recruit but It's also been a big change of pace and focus for me with the engineering team almost doubling in size. In late September we had <a href="https://brainsteam.co.uk/2023/10/01/2023-10-1-weeknote-39/">a leadership team retreat</a> and we've been working on ways to improve the way that our product team work together to make things super sleak.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I've found that the hype around generative AI hasn't impacted what we do as much as you might expect. As I wrote earlier in the year, <a href="https://brainsteam.co.uk/2023/03/25/nlp-is-more-than-just-llms/">NLP is about a lot more than just LLMs</a> and we need to provide reliable systems that don't hallucinate about stuff. There are still a huge number of unsolved problems in this space and<a href="https://brainsteam.co.uk/2023/12/20/nlp-in-the-post-llm-world/"> much more efficient ways </a>to solve them than throwing GPT-4 at it.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Projects and Open Source Contributions</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>By far the biggest success I had this year was Turbopilot which was hugely successful when I first launched it in April thanks to a tweet by Clem Delangue. It was really interesting to play with llama.cpp and learn about quantizing models, not just because of all the LLM stuff but because this technique can be applied to other types of models to make them smaller and more efficient too. I stopped developing Turbopilot in september when it was clear that other, better-staffed projects were doing a better job. <a href="https://brainsteam.co.uk/2023/09/30/turbopilot-obit/">I wrote about it at the time.</a> Turbopilot was then featured in a keynote at Intel's <a href="https://www.intel.com/content/www/us/en/newsroom/resources/intel-innovation-2023.html">2023 Innovate conference </a>after I spent a little time coordinating with a couple of their engineers in order to help get it running smoothly on intel CPUs.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I've also made a couple of smaller contributions this year:</p>
<!-- /wp:paragraph -->
<!-- wp:list -->
<ul><!-- wp:list-item -->
<li>I built<span style="text-decoration: underline;"> <a href="https://github.com/ravenscroftj/freshrss-flaresolverr-extension">an extension for FreshRSS</a> </span>which allows it to solve Cloudflare's CAPTCHA-style challenges in order to retrieve RSS feeds that are inexplicably behind bot detection systems (RSS is supposed to be automated so I'm not quite sure why you'd do this but meh.</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>I went through a phase of using <a href="https://dendron.so"><span style="text-decoration: underline;">Dendron</span></a> to manage my notes before that project went into dormancy and I switched back to Logseq. I built<a href="https://github.com/ravenscroftj/md-hypothesis-sync"> <span style="text-decoration: underline;">a plugin that syncs your public hypothesis annotations with your local filesystem</span></a> from inside VSCode. It works with <a href="https://foambubble.github.io/"><span style="text-decoration: underline;">Foam</span></a> too.</li>
<!-- /wp:list-item --></ul>
<!-- /wp:list -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Misc Personal Stats</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>According to<a href="https://bookwyrm.social/user/jamesravey"> my bookwyrm stats</a>, this year I read 11, nearly 12, books. That is a disappointingly low number for me. Last year I managed nearly double that. My favourite books this year have been the <a href="https://www.amazon.co.uk/gp/aw/d/B00NA90XQO?ref_=dbs_mng_crcw_0&storeType=ebooks&_encoding=UTF8&tag=jamesravey-21&linkCode=ur2&linkId=0e9a47554c08db5e3a865cb38b2731e4&camp=1634&creative=6738" target="_blank" rel="noreferrer noopener">Dredd: The Early Years</a> (<a href="https://www.amazon.co.uk/gp/aw/d/B00NA90XQO?ref_=dbs_mng_crcw_0&storeType=ebooks">non-affiliate link</a>) series which are properly pulpy/cheesy sci-fi and William Gibson's <a href="https://www.amazon.co.uk/Complete-Collection-William-Neuromancer-Overdrive/dp/1473233968/ref=sr_1_1?crid=2GI1CJCDRBQF0&keywords=william+gibson+sprawl&qid=1703241414&sprefix=william+gibson+sprawl%252Caps%252C130&sr=8-1&_encoding=UTF8&tag=jamesravey-21&linkCode=ur2&linkId=e51082bfee5c1b67d40ac8295184920a&camp=1634&creative=6738" target="_blank" rel="noreferrer noopener">Sprawl Trilogy</a> (<a href="https://www.amazon.co.uk/Complete-Collection-William-Neuromancer-Overdrive/dp/1473233968/ref=sr_1_1?crid=2GI1CJCDRBQF0&keywords=william+gibson+sprawl&qid=1703241414&sprefix=william+gibson+sprawl%2Caps%2C130&sr=8-1">non-affiliate link</a>) definitely a bit of a Cyberpunk theme going on. My favourite non-fic book was <a href="https://www.amazon.co.uk/Complete-Collection-William-Neuromancer-Overdrive/dp/1473233968/ref=sr_1_1?crid=2GI1CJCDRBQF0&keywords=william+gibson+sprawl&qid=1703241414&sprefix=william+gibson+sprawl%252Caps%252C130&sr=8-1&_encoding=UTF8&tag=jamesravey-21&linkCode=ur2&linkId=e51082bfee5c1b67d40ac8295184920a&camp=1634&creative=6738" target="_blank" rel="noreferrer noopener">Sprawl Trilogy</a> (<a href="http://<a target="_blank" href="https://www.amazon.co.uk/Complete-Collection-William-Neuromancer-Overdrive/dp/1473233968/ref=sr_1_1?crid=2GI1CJCDRBQF0&keywords=william+gibson+sprawl&qid=1703241414&sprefix=william+gibson+sprawl%252Caps%252C130&sr=8-1&_encoding=UTF8&tag=jamesravey-21&linkCode=ur2&linkId=e51082bfee5c1b67d40ac8295184920a&camp=1634&creative=6738">Sprawl Trilogy</a>">non-affiliate link</a>) by Johann Hari which provided an interesting and well argued summary of the ongoing, multi-fronted assault that the modern world mounts upon our attention.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>According to Spotify, I've listened to about 25k minutes of music this year which isn't bad going. <a href="https://brainsteam.co.uk/2023/06/17/xavier-the-spotify-dj/">I spent a bit of time test driving their AI DJ earlier in the year</a> and that led me to discover some amazing new bands. Of particular note, The Last Dinner Party who I can't stop telling people about. I tried out Jungle's new album, it's not my favourite but there are some catchy tracks on there. I <a href="https://wiki.jamesravey.me/books/seed-propagator/page/music">recently updated my music recommendations page</a> over on my digital garden with some recommendations.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Mrs R and I have retained our Cineworld memberships and been to see a few films this year. That said I still haven't gotten around to the Barbenheimer experience that was all the rage earlier in the year. I recently saw Wonka which I actually enjoyed quite a lot. It had a lot of actors from British sitcoms like Peep Show and Ghosts. </p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Habits and Fitness</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I've been a fair bit better at journaling and meditating this year but I've definitely gained some weight this year which is unfortunate. I was probably at peak fitness towards the beginning of 2021 having spent most of lockdown eating healthily and regularly going for walks. This year we've been eating less well and doing less exercise and I've got the "dad bod" to prove it. I've had a few conversations with other people about getting back on the fitness train next year including some colleagues at work and Kev Quirk who is <a href="https://kevquirk.com/fat-boy-at-40">trying to avoid being a fat boy at 40</a>. I'm a few years away from 40 but I am going to try and do a similar "fat boy at 35" challenge. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I have made a few weeknotes this year but felt a little self-conscious about publishing something every week, particularly during weeks where it's felt like not much has happened. I might re-jig the format next year and switch to private weekly reviews and monthly blog reviews more like <a href="https://jlelse.blog/">Jan-Lukas</a>.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Both Mrs R and I have found the pace of this year pretty challenging and it's had a knock on effect on our personal projects around the house and our general health and fitness. Hopefully next year these are some areas that we can work on together.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Travel and Family</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I've managed to do a fair bit of travelling around this year and seen a lot more of my family than since pre-covid. In May we took a week-long trip up to the Lake District around the time of my birthday. At the same time, we popped in to visit my parents in the midlands and my dad, who has been taking flying lessons, took us up in a small aircraft which was really exciting. </p>
<!-- /wp:paragraph -->
<!-- wp:gallery {"linkTo":"none"} -->
<figure class="wp-block-gallery has-nested-images columns-default is-cropped"><!-- wp:image {"id":1137,"sizeSlug":"medium","linkDestination":"none"} -->
<figure class="wp-block-image size-medium"><img src="/media/PXL_20230429_092332645_smoll-226x300_12fd88ba.jpg" alt="" class="wp-image-1137"/></figure>
<!-- /wp:image -->
<!-- wp:image {"id":1138,"sizeSlug":"medium","linkDestination":"none"} -->
<figure class="wp-block-image size-medium"><img src="/media/PXL_20230429_091712036_smoll-226x300_d5821679.jpg" alt="" class="wp-image-1138"/></figure>
<!-- /wp:image -->
<!-- wp:image {"id":1139,"sizeSlug":"medium","linkDestination":"none"} -->
<figure class="wp-block-image size-medium"><img src="/media/PXL_20230429_095614392_smoll-226x300_7051f26b.jpg" alt="" class="wp-image-1139"/></figure>
<!-- /wp:image --></figure>
<!-- /wp:gallery -->
<!-- wp:paragraph -->
<p>While we were up in the Lake district we got to visit Kendal Museum where my late grandfather was the custodian in the 1990s. It was great to be back there as I have many treasured memories of Kendal and the surrounding lakes. We couldn't pass up an opportunity to recreate an old photo.</p>
<!-- /wp:paragraph -->
<!-- wp:gallery {"linkTo":"none"} -->
<figure class="wp-block-gallery has-nested-images columns-default is-cropped"><!-- wp:image {"id":1141,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/105086-768x1024_dcccc9a7.jpg" alt="" class="wp-image-1141"/></figure>
<!-- /wp:image -->
<!-- wp:image {"id":1140,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/image-8-685x1024_2c506e90.png" alt="" class="wp-image-1140"/></figure>
<!-- /wp:image --></figure>
<!-- /wp:gallery -->
<!-- wp:paragraph -->
<p>In June we went on a cruise around the Mediterranean with my dad and his wife to celebrate his 60th birthday. We stopped off at Pisa, Corsica, Marseilles, Palamos and a few others. We briefly visited St Tropez and I was a bit taken aback by the ostentatious and inauthentic display of wealth in the town. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Then, in September, we also did a beach and chill holiday in Cyprus where we spent a lot of time reading, swimming and drinking cocktails. We spent a lot of time making friends with the local stray cats of Cyprus who we wanted to smuggle home with us.</p>
<!-- /wp:paragraph -->
<!-- wp:gallery {"linkTo":"none"} -->
<figure class="wp-block-gallery has-nested-images columns-default is-cropped"><!-- wp:image {"id":1154,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/PXL_20230905_101634306_smoll-771x1024_f4d8abdd.jpg" alt="a lovely sunny pool area with people sat around on outdoor furniture" class="wp-image-1154"/></figure>
<!-- /wp:image -->
<!-- wp:image {"id":1155,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/smollcats-771x1024_72d26f1e.jpeg" alt="a bunch of stray cats sat on a walkway in the sun in cyprus" class="wp-image-1155"/></figure>
<!-- /wp:image --></figure>
<!-- /wp:gallery -->
<!-- wp:paragraph -->
<p>I have found that I've been ill quite a lot this year. From about mid-February until the end of April I had coughs, colds and COVID almost every other week. This has prevented us from visiting family and friends quite as often as I'd have liked this year. In particular, I would have liked to have visited some of my old school friends, one of whom is expecting their first child early in the year. We also had to cancel some concerts and shows due to illness and<a href="https://brainsteam.co.uk/2023/11/05/weeknote-cw-44-2023/"> dangerous weather</a>. Although it's not necessarily my fault that I got ill or that there was a raging storm, next year I hope to rectify this where possible by visiting as many of my friends as possible. I'm hoping that our fitness goals will also help with the whole "being sick" thing.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Next Year</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Rather than making a separate new year post, let me make some predictions and set some goals here.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I'd summarise by saying that getting fitter and looking myself a little bit more are the main things that I'm aiming for in 2024.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I'm also keen to improve on my existing journaling and planning habits. I recently joined the <a href="https://nesslabs.com/">Ness Labs</a> community and I'm keen to try and take part in some of the workshops and meetups around these topics.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I've enjoyed all of my public appearances and speaking in 2023 and I hope to continue with these sorts of activities next year. I'd like to get more engaged with the <a href="https://www.bcs.org/">BCS</a> (of which I am a member but have yet to go to any events) and join more meetups and events locally.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I think it'd be nice to make more progress on jobs around the house next year too, we've slacked off somewhat in 2023 and we have a few maintenance and decorating type jobs that we need to tick off the list.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I've published a fair few posts and articles on this site in 2023. I'd like to keep this up in the new year, possibly changing from weekly to monthly reviews as mentioned above and also writing more about stuff I'm working on.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Happy end of the year everyone, hope to speak to you all in 2024!</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->

19
brainsteam/content/posts/2023/12/26/Royal Institute Christmas Lecture on AI 2023.md Normal file
View File

@ -0,0 +1,19 @@
---
categories:
- AI and Machine Learning
date: '2023-12-26 18:32:46'
draft: false
tags:
- AI
- ml
title: Royal Institute Christmas Lecture on AI 2023
type: posts
---
<!-- wp:paragraph -->
<p>I'm excited and very proud to see my colleague and mentee Safa appear on <a href="https://www.bbc.co.uk/iplayer/episode/m001tv7s/royal-institution-christmas-lectures-2023-professor-mike-wooldridge-the-truth-about-ai-1-how-to-build-an-intelligent-machine">this year's Royal Institute Christmas lecture</a>. In the video, Safa is assisting Prof. Mike Woolridge (who has some <a href="https://www.theguardian.com/technology/2023/dec/26/ai-artifical-intelligence-secrets-chatbots-chatgpt--mike-wooldridge">excellent and sensible takes on AI </a>by the way) with some experiments.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>In general this is a well put together lecture that gives a lay summary of some of the techniques and acronyms that you're likely to come across in conversation about AI. I'd recommend it to anyone interested in learning more about what these systems do or to any relatives who might have brought up the subject of AI at the Christmas dinner table</p>
<!-- /wp:paragraph -->

49
brainsteam/content/posts/2023/12/27/Account Portability in the Social Web.md Normal file
View File

@ -0,0 +1,49 @@
---
categories:
- Software Development
date: '2023-12-27 21:16:47'
draft: false
tags:
- fediverse
- indieweb
title: Account Portability in the Social Web
type: posts
---
<!-- wp:indieblocks/repost {"empty":false} -->
<div class="wp-block-indieblocks-repost"><div class="u-repost-of h-cite"><p><i>Reposted <a class="u-url p-name" href="https://underlap.org/account-portability-in-the-social-web">Account portability in the social web — underlap</a> by <span class="p-author">underlap</span>.</i></p><blockquote class="wp-block-quote e-content"><!-- wp:paragraph -->
<p>...In an interoperable world, users want to be able to move between providers with minimal friction to avoid “lock-in”...</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>...ActivityPub is the protocol that underpins the Fediverse (Mastodon, Pixelfed, PeerTube, etc.). It does not provide direct support for account portability... It will be fascinating to see to what extent, if at all, interoperation with the Fediverse by Meta's Threads will include account portability...</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>...Efforts to build account portability on top of ActivityPub fall into three categories: abstracting identity, improving account migration, and migrating content...</p>
<!-- /wp:paragraph --></blockquote></div></div>
<!-- /wp:indieblocks/repost -->
<!-- wp:paragraph -->
<p>This is a really great article that discusses some of the things that are missing from ActivityPub in order to make the fediverse experience truly portable and some ongoing works in progress. A couple of observations that made me think:</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Noone besides BlueSky has implemented the AT protocol yet. Given that its <a href="https://atproto.com/guides/overview">an open protocol</a>, I'm surprised that nobody's at least tinkering with something in this vain.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Secondly, I really like the idea of DNS-based identity for identity provenance. That said, the majority of web users don't own their own domain so this might not scale well at all.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Finally, I noticed that <a href="https://fosstodon.org/deck/@hcj/111653083104286150">@hcj</a> has commented on the post to say that when moving to FireFish you can also <a href="https://wedistribute.org/2023/07/how-to-move-from-mastodon-to-firefish/">take your posts with you </a>- something that Mastodon doesn't currently support.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->

47
brainsteam/content/posts/2023/12/28/Can you add AI to the hydraulics system... Seriously_.md Normal file
View File

@ -0,0 +1,47 @@
---
categories:
- AI and Machine Learning
date: '2023-12-28 16:42:34'
draft: false
tags:
- AI
- humour
title: Can you add AI to the hydraulics system... Seriously?
type: posts
---
<!-- wp:indieblocks/bookmark {"empty":false} -->
<div class="wp-block-indieblocks-bookmark"><div class="u-bookmark-of h-cite"><p><i>Bookmarked <a class="u-url p-name" href="https://www.tumblr.com/dduane/737900044438667264/half-rant-half-story-im-a-physicist-i-work-for">Diane Duane on Tumblr</a>.</i></p></div><div class="e-content"><!-- wp:quote -->
<blockquote class="wp-block-quote"><!-- wp:paragraph -->
<p>"Can you add AI to the hydraulics system?"</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p> can i fucking what mate "Sir, I'm sorry, I'm a little confused - what do you mean by adding AI to the hydraulics?" </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>"I just thought this stuff could run smoother if you added AI to it. Most things do"</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p> The part of the car that moves when you push the acceleration pedal is metal and liquid my dude what are you talking about "You want me to .add AI...to the pistons? To the master cylinder?" </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>"Yeah exactly, if you add AI to the bit that makes the pistons work, it should work better, right?"</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p> IT'S METAL PIPES it's metal pipes it's metal pipes "Sir, there isn't any software in that part of the car"</p>
<!-- /wp:paragraph --></blockquote>
<!-- /wp:quote --></div></div>
<!-- /wp:indieblocks/bookmark -->
<!-- wp:paragraph -->
<p>This is equal parts hilarious, horrifying, excruciating and anger inducing to me. It takes me right back to my IBM days where "can't we just use Watson to do this?" was the eternal cry of the client.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>ML and AI have come a long way but people need to understand the domain that they are working in well enough to determine whether an AI system will help them or not. Frankly if the user doesn't understand the problem then how are they going to train a model? </p>
<!-- /wp:paragraph -->

378
brainsteam/content/posts/2023/12/29/Serving Django inside Docker the Right Way.md Normal file
View File

@ -0,0 +1,378 @@
---
categories:
- Software Development
date: '2023-12-29 14:22:27'
draft: false
tags:
- django
- docker
- python
title: Serving Django inside Docker the Right Way
type: posts
---
I've seen a number of tutorials that incorrectly configure django to run inside docker containers by leveraging it's built in dev server. In this post I explore the benefits of using django with gunicorn and nginx and how to set this up using Docker and docker-compose.
<!--more-->
<!-- wp:paragraph -->
<p>I'm working on a couple of <a href="https://brainsteam.co.uk/2023/11/13/gastronaut-fediverse-recipe-app/">side projects</a> that use django and I'm a big fan of docker for simplifying deployment.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Django is a bit of a strange beast in that, it has a simple, single-threaded development server that you can start with <code>python manage.py runserver</code> but then it can also be run in prod mode using <a href="https://wsgi.readthedocs.io/en/latest/what.html">WSGI</a> but once in this mode, it doesn't serve static files any more. This can be a little off-putting for people who are used to packaging a single server that does everything (like a nodejs app). It is especially confusing to people used to packaging an app along with everything it needs inside docker. </p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Part 1: Why not just use runserver?</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p><strong><em>If you already understand why it's better to use WSGI than runserver and just want to see the working config, skip down to <a href="#part-2">Part 2</a> below.</em></strong></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I've seen a few tutorials for packaging up django apps inside docker containers that just use the <code>runserver</code> mechanism. The problem with this is that you don't get any of the performance benefits of using a proper WSGI runner and in order to handle server load, you end up needing to run multiple copies of the docker container very quickly. </p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">A Rudimentary Performance Test</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I did a quick performance test against the python runserver versus my WSGI + Nginx configuration (below) to illustrate the difference on my desktop machine. I used <a href="https://github.com/codesenberg/bombardier">bombardier</a> and asked it to make as many requests as it can for 10s with up to 200 concurrent connections:</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><code>bombardier -c 200 -d 10s http://localhost:8000</code></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The thing at that address is the index <code>view</code> of my django app so we're interested in how quickly we can get the Python interpreter to run and return a response.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The python runserver results:</p>
<!-- /wp:paragraph -->
<!-- wp:preformatted -->
<pre class="wp-block-preformatted">Statistics Avg Stdev Max
Reqs/sec 1487.50 633.11 2988.81
Latency 133.84ms 259.97ms 7.12s
HTTP codes:
1xx - 0, 2xx - 15042, 3xx - 0, 4xx - 0, 5xx - 0
others - 0
Throughput: 13.70MB/s</pre>
<!-- /wp:preformatted -->
<!-- wp:paragraph -->
<p>And the WSGI config results:</p>
<!-- /wp:paragraph -->
<!-- wp:preformatted -->
<pre class="wp-block-preformatted">Statistics Avg Stdev Max
Reqs/sec 1754.20 666.40 16224.55
Latency 115.05ms 7.23ms 174.44ms
HTTP codes:
1xx - 0, 2xx - 17472, 3xx - 0, 4xx - 0, 5xx - 0
others - 0
Throughput: 15.95MB/s</pre>
<!-- /wp:preformatted -->
<!-- wp:paragraph -->
<p>As you can see, using a proper deployment configuration, the average number of requests handled per second goes up by about 15% but also we get a much more consistent latency (115ms average with a deviation of about 7ms as opposed to in the first example where latency is all over the place and if you're really unlucky, you're the person waiting 7s for the index page to load).</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":5} -->
<h5 class="wp-block-heading">Testing Static File Service</h5>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Now let's look at handling files. When we use <code>runserver</code> we are relying on the python script to serve up the files we care about. I ask bombardier to request the logo of my app as many times as it can for 10 seconds like before:</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><code>bombardier -c 200 -d 10s http://localhost:8000/static/images/logo.png</code></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>First we run this with django runserver:</p>
<!-- /wp:paragraph -->
<!-- wp:preformatted -->
<pre class="wp-block-preformatted">Statistics Avg Stdev Max
Reqs/sec 731.51 252.55 1795.93
Latency 270.50ms 338.53ms 5.01s
HTTP codes:
1xx - 0, 2xx - 7504, 3xx - 0, 4xx - 0, 5xx - 0
others - 0
Throughput: 255.27MB/s</pre>
<!-- /wp:preformatted -->
<!-- wp:paragraph -->
<p>And again with Nginx and WSGI. </p>
<!-- /wp:paragraph -->
<!-- wp:preformatted -->
<pre class="wp-block-preformatted">Statistics Avg Stdev Max
Reqs/sec 6612.33 705.07 9332.41
Latency 30.27ms 19.95ms 1.30s
HTTP codes:
1xx - 0, 2xx - 66156, 3xx - 0, 4xx - 0, 5xx - 0
others - 0
Throughput: 2.25GB/s</pre>
<!-- /wp:preformatted -->
<!-- wp:paragraph -->
<p>And suddenly the counter-intuitive reason for Django splitting static file service from code execution makes a little bit more sense. Since we are just requesting static files, Python never actually gets called. Nginx, which is an efficient server that is written in battle-hardened C, is able to just directly serve up the static files. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>In the first example, Python is the bottleneck and using Nginx + WSGI just makes some of the shifting around of information a little bit smoother. In the second example, we can completely sidestep python.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">If you still need convincing...</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>The devs literally tell you not to use runserver in prod in the official docs:</p>
<!-- /wp:paragraph -->
<!-- wp:quote -->
<blockquote class="wp-block-quote"><!-- wp:paragraph -->
<p>DO NOT USE THIS SERVER IN A PRODUCTION SETTING. It has not gone through security audits or performance tests. (And that’s how it’s gonna stay. We’re in the business of making web frameworks, not web servers, so improving this server to be able to handle a production environment is outside the scope of Django.)</p>
<!-- /wp:paragraph --><cite><a href="https://docs.djangoproject.com/en/5.0/ref/django-admin/#runserver">django-admin and manage.py - Django documentation</a></cite></blockquote>
<!-- /wp:quote -->
<!-- wp:separator -->
<hr class="wp-block-separator has-alpha-channel-opacity"/>
<!-- /wp:separator -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading" id="part-2">Part 2: Packaging Django + WSGI in Docker</h3>
<!-- /wp:heading -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Getting Started</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Ok so I'm going to assume that you have a django project that you want to deploy and it has a <code>requirements.txt</code> file containing the dependencies that you have installed. If you are using a <a href="https://brainsteam.co.uk/2021/04/01/opinionated-guide-to-virtualenvs/">python package manager</a>, I'll drop some hints but you'll have to infer what is needed in a couple of places.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Install and Configure Gunicorn</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Firstly, we need to add a WSGI server component that we can run inside the docker container. I will use <a href="https://gunicorn.org/">gunicorn</a>.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><code>pip install gunicorn</code> (or you know, pdm add/poetry add etc)</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We can test that it's installed and working by running:</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p> <code>gunicorn -b :8000 appname.wsgi</code> </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>If you go to localhost:8000 you should see your app there but, wait a minute, there are no images or css or js. As I mentioned, django won't serve your static resources so we'll pair gunicorn up with nginx in order to do that. </p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Collect Static Resources</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>nginx needs a folder that it can serve static files from. Thankfully django's manage.py has a command to do this so we can simply run:</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><code>python manage.py collectstatic --noinput</code></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The <code>--noinput</code> argument prevents the script from asking you questions in the terminal and it will simply dump the files into a <code>static</code> folder in the current directory.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Try running the command in your django project to see how it works. We'll be using this in the next step</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Build a Dockerfile for the app</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>We can produce a docker file that builds our django app and packages any necessary files along with it. </p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"dockerfile"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="dockerfile" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">FROM python:3
WORKDIR /app
ADD . /app
RUN python3 -m pip install -r requirements.txt
# nb if you are using poetry or pdm you might need to do something like:
# RUN python3 -m pip install pdm
# RUN pdm install
ENV STATIC_ROOT /static
CMD ["/app/entrypoint.sh"]</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>NB: if you are using pdm or poetry or similar, you will want to install them </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We also need to create the entrypoint.sh file which docker will run when the container starts up. Save this file in the root of your project so that it can be picked up by Docker when it builds:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"bash"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">#!/usr/bin/env bash
pdm run manage.py collectstatic --noinput
pdm run manage.py migrate --noinput
pdm run gunicorn -b :8000 appname.wsgi</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>This script runs the collectstatic command which, with a little bit of docker magic we will hook up to our nginx instance later. Then we run any necessary database migrations and then we use gunicorn to start the web app.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Build the nginx.conf </h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>We need to configure nginx to serve static files when someone asks for <code>/static/something</code> and forward any other requests to the django app. Create a file called nginx.conf and copy the following:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock -->
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">events {
worker_connections 1024; # Adjust this to your needs
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
# Server block
server {
listen 80;
server_name localhost;
# Static file serving
location /static/ {
alias /static/;
expires 30d;
}
# Proxy pass to WSGI server
location / {
proxy_pass http://frontend:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
}</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>This configuration should be relatively self-explanatory but a couple of notes:</p>
<!-- /wp:paragraph -->
<!-- wp:list -->
<ul><!-- wp:list-item -->
<li>The alias command tells nginx to serve static files from the <code>/static/</code> folder on the filesystem. This is where we will mount the static files using docker later. The behaviour of alias is described <a href="https://stackoverflow.com/questions/10631933/nginx-static-file-serving-confusion-with-root-alias"><span style="text-decoration: underline;">here</span></a>.</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>The expires 30d directive tells nginx that it can let browsers cache these static files for up to 30 days at a time - hopefully saving bandwidth and speeding things up.</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>If the request does not start with <code>/static/</code> then nginx will assume it is a request for django and send it to <code>http://frontend:8000</code> - again we will configure docker so that the django gunicorn process is listening from there.</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>Note that we use <code>/static/</code> but we could change this - the value needs to match whatever we set as <code>STATIC_ROOT</code> in the docker file.</li>
<!-- /wp:list-item --></ul>
<!-- /wp:list -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Glue it together with docker compose</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>We will use docker compose to combine our nginx and django containers together.</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"yaml"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="yaml" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">services:
frontend:
build: .
restart: unless-stopped
volumes:
- ./static:/app/static
environment:
DJANGO_CSRF_TRUSTED_ORIGINS: 'http://localhost:8000'
frontend-proxy:
image: nginx:latest
ports:
- "8000:80"
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf:ro
- ./static:/static:ro
depends_on:
- frontend</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>Ok so what we are doing here is using volume mounts to connect <code>/app/static</code> inside the django container where the results of <code>collectstatic</code> are dumped to <code>/static/</code> in our nginx container where the static files are served from.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We also mount the <code>nginx.conf</code> file in the nginx container. You'll probably end up using docker compose to add database connections too or perhaps a volume mount for a sqlite database file.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Finally we bind port <code>8000</code> on the host machine to port <code>80</code> in nginx so that when we go to <code>http://localhost:8000</code> we can see the running app.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Running it </h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Now we need to build and run the solution. You can do this by running:</p>
<!-- /wp:paragraph -->
<!-- wp:preformatted -->
<pre class="wp-block-preformatted">docker compose build
docker compose up -d
</pre>
<!-- /wp:preformatted -->
<!-- wp:paragraph -->
<p>Now we can test it out by going to <a href="http://localhost:8000">http://localhost:8000</a>. Hopefully you will see your app running in all its glory. We can debug it by using <code>docker compose logs -f</code> if we need to.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Conclusion</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Hopefully this post has shown you why it is important to set up Django properly rather than relying on <code>runserver</code> and how to do that using Docker, Nginx and Gunicorn. As you can see, it is a little bit more involved than your average npm application install but it isn't too complicated.</p>
<!-- /wp:paragraph -->

189
brainsteam/content/posts/2024/01/03/Migrating Users Across Servers With RSync.md Normal file
View File

@ -0,0 +1,189 @@
---
categories:
- Software Development
date: '2024-01-03 14:33:37'
draft: false
tags:
- linux
- rsync
title: Migrating Users Across Servers With RSync
type: posts
---
<!-- wp:paragraph -->
<p>I recently needed to migrate some user data from one Ubuntu server to another. It was not possible for me to clone the full disk so I opted to copy the user data and re-create the user accounts on the other machine.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I used rsync to copy all the user data and preserve all permissions on the files. I needed sudo access on both sides.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>In this article I refer to the <strong>new machine</strong> as the target onto which we want to copy our data and the <strong>old machine</strong> as the source of the data we want to copy.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Preparing The Data</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Firstly, check what data you want to clone. I made liberal use of <code>du -sh</code> /home/* to see how much space each of the affected user directories were taking up and worked with them to tidy up their local directories where necessary (lots of junk in hidden places like <code>.local</code> and <code>.cache</code>). A couple of the users had large projects that they were able to purge before we did the copy so I was able to significantly reduce the amount of data I needed to transfer.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Create the Users</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>For each of the users on the old machine, I created a new account using <code>sudo useradd -m <username></code> - if there are any special groups like <code>sudo</code> or <code>docker</code> you can add them at this point, e.g. <code>sudo useradd -m james -G sudo,docker</code></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The <code>-m</code> flag creates the user home directory so if you do an <code>ls /home</code> you should see one directory per user in there.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Set Up Passwordless Sudo rsync</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>In order to have permission to copy users we need to be able to operate as root on both the new machine and the old machine. We will run the sync command from the new machine with sudo and we can enter the password but that will SSH to the remote system and attempt to sudo and will likely fail if we don't do this next step. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We don't want to give blanket permission for the user to sudo without password auth - that's a pretty big security risk and also an accident waiting to happen (<code>sudo rm -rf /</code> anyone?). Instead, we will edit the <code>/etc/sudoers</code> file and add special permission for our current user (let's say james) to run the rsync command without asking for a password.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We add a couple of new lines to the bottom of the file like so:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock -->
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># User privilege specification
root ALL=(ALL:ALL) ALL
# Members of the admin group may gain root privileges
%admin ALL=(ALL) ALL
# Allow members of group sudo to execute any command
%sudo ALL=(ALL:ALL) ALL
# Custom user privileges
james ALL=(ALL) NOPASSWD: /usr/bin/rsync
</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>The key thing here is the NOPASSWD: directive, after which we put the path to the rsync binary. We should also remove this again once the sync is successful.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Set up and test the connection</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>From the new machine we can test our ability to rsync with the old machine. We can use the <code>--dry-run</code> flag to avoid actually copying any data.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The test command might like something like this:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"bash"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">sudo rsync -Porgzl --dry-run --rsync-path="sudo /usr/bin/rsync" james@target.domain.com:/home/james /home</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>You will want to run rsync via <code>sudo</code> on the new machine AND the old machine (via the config above) to ensure that we have permission to read and write other users' data. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><code>--dry-run</code> prevents rsync from actually copying any data, it just makes a list of files it would copy if it was run without this option.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><code>--rsync-path</code> - this is the command that the old server will use when it is looking for files to grab and send to the new machine. We prepended the command with <code>sudo</code> and because the user that we're sshing as has the <code>NOPASSWD</code> configured for <code>/usr/bin/rsync</code> it should allow the user to run this without any issue.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><code>-P</code> displays progress of the transfer and also enables partial transfer (if the command is interrupted at any point it will resume any partly-transferred files)</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><code>-o</code> and <code>-g</code> preserve the permissions on the files <code>-o</code> for owner and <code>-g</code> for groups. This ensure that the old user and group are correctly set in the new location. This is why it is important that we created the affected users and groups before we initiated the transfer.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><code>-r</code> for recursive - copy folders and their contents recursively, without this, rsync will just stop without copying anything</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><code>-z</code> for compression - this compresses data on the old machine before it is sent and decompresses it when it is received on the new machine. This saves bandwidth and these days with fancy, powerful CPUs, the chances are that you'll be able to compress/decompress the data faster than it can be transferred over the net so this is likely to be helpful.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><code>-l</code> allows rsync to copy symlinks - this may be important if you are copying things like conda environments (since library structures often contain multiple links to each other libexample.so.1.2.3 -> libexample.so.1 -> libexample.so).</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>If all goes well you should see a list of files flash up the screen - this is the list of files that would be copied from the old machine to the new machine if <code>--dry-run</code> wasn't enabled.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>If you get any permission errors, double check that you have the right permissions set up on the old machine, make sure that the user you are SSHing as is the same as the user in the sudoers file. </p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Run the Sync</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>If your dry run succeeded, you can now execute the full copy and transfer. I'm going to add a couple of additional exclusions with the --exclude operator. We can use wildcards to apply these exclusions to all of our users:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"bash"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">sudo rsync \
-Porgzl --rsync-path="sudo /usr/bin/rsync" \
--exclude "*/.local/lib" \
--exclude "*/.cache" \
--exclude '*/.vscode-server' \
--exclude "*/miniconda/pkgs" \
james@target.domain.com:/home/* \
/home</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>The command is pretty much the same as the previous one with the <code>--dry-run</code> flag turned off, with an <code>--exclude</code> for each of the directories we don't care about and with a wildcard in the home directory so that we copy all users rather than just <code>james</code>. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I highly recommend executing long-running commands like this inside <a href="https://github.com/tmux/tmux/wiki">tmux</a> so that if your connection from your workstation to the new machine goes down, the process continues.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>If your connection does get interrupted and you need to restart, you can run this command with <code>--ignore-existing</code> to have rsync skip any files that were already copied during the failed run.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Finally: Undo the Sudoers Change</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Remove the line that you added to the sudoers file that allowed you to run rsync without a password on the old server!</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->

123
brainsteam/content/posts/2024/01/10/Brand Loyalty.md Normal file
View File

@ -0,0 +1,123 @@
---
categories:
- Philosophy and Thinking
date: '2024-01-10 21:33:53'
draft: false
tags:
- humour
- philosophy
title: Brand Loyalty
type: posts
---
<!-- wp:paragraph -->
<p>I'm not really saying anything revolutionary or novel in this post, it's just a bit of a stream of consciousness rant about how companies behave when they think we're not watching them....</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>When I was growing up I remember my parents and grandparents frequently talking about companies that they trusted and telling me good makes for things:</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>"German cars are well engineered... you should drive a Volkswagen"</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>"Always fly British Airways"</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>"You should shop at Morrisons"</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>"Your Uncle's just bought a Dyson hoover, they're the best you know"</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Setting Myself Up For Disappointment</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Throughout my tweens, teens and early 20s I remember following this sort of advice and after years of encouragement, forming my own strong opinions about 'good' brands. I also remember falling, hook line and sinker, for a lot of the late 00s silicon valley companies who promised to change the world.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I can also recall the moments when almost all of the brands that I was loyal to betrayed my trust, either directly through a negative experience or indirectly through a scandal of some description.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Like when I learned about the <a href="https://en.wikipedia.org/wiki/Volkswagen_emissions_scandal">VW emissions scandal</a>... and also that time I bought a <a href="https://en.wikipedia.org/wiki/Lemon_%28automobile%29">lemon</a> off them...</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>...or when it turned out that<a href="https://www.theguardian.com/technology/2021/apr/21/lobbying-row-james-dyson-is-no-stranger-to-political-controversy"> the inventor of my vacuum cleaner expels more hot air than his trademark hand dryers</a>...</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>...Or when it turned out that my <a href="https://www.theguardian.com/world/series/the-snowden-files">silicon valley 'heroes' (in hindsight 🤮) were spying on us for years</a>... and that the guy who sold the cool electrical cars was a loser who <a href="https://www.theguardian.com/technology/2018/jul/15/elon-musk-british-diver-thai-cave-rescue-pedo-twitter">publically disparaged actual heroes</a> and <a href="https://www.theguardian.com/technology/2023/nov/16/elon-musk-antisemitic-tweet-adl">hung out with nazis</a>...</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Or in the recent 'inflation' years where almost every single branded food and drink company and supermarket <a href="https://www.theguardian.com/business/2023/jun/06/hard-pressed-shoppers-feel-food-shrinkflation">shrinkflated</a> the hell out of their entire range, r<a href="https://www.skynews.com.au/australia-news/david-littleproud-says-supermarket-giants-cleaning-up-at-the-expense-of-consumers-as-cost-of-living-crisis-drags-on/news-story/329c10b256990fd147709180bdd65845">aised their prices and then posted record profits</a>...</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>...or when a particular aerospace company that I had grown up wanting to work for released a plane that <a href="https://www.theverge.com/2019/3/22/18275736/boeing-737-max-plane-crashes-grounded-problems-info-details-explained-reasons">killed a bunch of people</a> and then <a href="https://news.sky.com/story/boeings-loose-bolts-how-much-trouble-is-the-company-in-13044860">nearly killed a bunch more people</a>...</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">A Legacy of a By-Gone Time</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Brand loyalty probably made sense in an earlier incarnation of capitalism. Your grandma might have known the owner of the local garage for years and they might have done her a solid a couple of times by not charging her for adding wiper fluid that one time or coming out to fix her car on a public holiday. There genuinely was a time when the things people needed were built with pride to a high quality and you'd buy something for life and maintain it. If Grandma had a pan that lasted 10 years and the company that made it for her were still going after all that time, chances are they were still making good pans.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The problem is, if I only buy one high quality saucepan every 20 years, saucepan companies can't make loads of money out of me. This all changed in the 1920s when psychologists like <a href="https://en.wikipedia.org/wiki/The_Century_of_the_Self">Edward Bernays</a> convinced us en-masse via the power of radio and print media that we needed to consume and buy and keep up with the Jones'. This led to <a href="https://en.wikipedia.org/wiki/Planned_obsolescence">planned obsolescence</a>. If people are going to buy something fashionable and chuck the old clothes away every season, it stands to reason that the clothes only need to last one season and therefore the manufacturer could make more money if they make the clothes out of cheaper materials. </p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Brand Loyalty in the 2020s</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Now we've had about 100 years to get used to the idea of "consuming" rather than buying for life and companies are in a race to the bottom to make stuff as cheaply and nastily as possible in order to sell us things we don't need that won't last very long anyway. Companies change very quickly and they are forever optimising (i.e. <a href="https://en.wikipedia.org/wiki/Enshittification">enshittifying</a>) their production processes, testing the waters to see what they can get away with not doing before customers realise like frogs slowly boiling in a pan of water.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Companies have also learned that talk is cheap. They will say whatever they think they should to get you to trust them and spend money with them even if they don't actually mean it. That's why we have things like <a href="https://www.bbc.co.uk/newsround/58465027">greenwashing</a>. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>In our current incarnation of capitalism brand loyalty makes zero sense. You might have a rose-tinted memory of a time you interacted with a company 10 years ago but in that time they might have been through 3 different CEOs, opted plastic components instead of metal ones or replaced their experienced senior engineers with graduates who got 6 months of training and just use ChatGPT to tell them what to do.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":1511,"sizeSlug":"full","linkDestination":"custom"} -->
<figure class="wp-block-image size-full"><a href="https://thenib.com/mister-gotcha/"><img src="/media/image-1_0f7b842a.png" alt="" class="wp-image-1511"/></a><figcaption class="wp-element-caption">I don't mean to be the 'gotcha' guy and I'm not judging people for doing what they need to do to get by. Surviving is hard work... (Comic by <a href="https://thenib.com/mister-gotcha/">Matt Bors</a>)</figcaption></figure>
<!-- /wp:image -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Loyalty is for People not Companies</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>In 2023 it's pretty hard to do avoid interacting with disingenuous rip off merchants... There are sites that can <a href="https://www.ethicalconsumer.org/ethicalcampaigns/boycotts">help you decide whether a company is ethical</a> and you can rely on your own moral compass. I'm not the "gotcha" guy from the comic strip above. Ethical boycotting is a privilege that not everyone can afford (fiscally or emotionally). Do what you've got to do to survive. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Ultimately I think loyalty should be reserved for people rather than big faceless corporations. When it comes to buying your next car, booking your next holiday, feeding your family next week or dressing yourself, be sceptical and clinical. Shop around, ignore adverts, posturing and green-washing. Remember talk is cheap. Don't give companies your loyalty. Abandon them as quickly and ruthlessly as they would you. If you have the energy, kick up a fuss. Write to them, email them, phone them, tell them why you're abandoning them (always be nice to the individuals handling your messages). </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Gift your loyalty to friends, family, neighbours and random strangers and remember those who help you out too. The brand loyalty of old was a lot closer to this model - Grandma probably trusted the manager at the local garage rather than the garage company. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>If you are a businessy person, build quality stuff, don't chase infinite growth and money at the expense of your customers and set up your business <a href="https://www.coops.tech/">in a way that makes it enshittification resistant </a>after you move on.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Finally, don't forget to also be loyal to yourself!</p>
<!-- /wp:paragraph -->

73
brainsteam/content/posts/2024/01/10/TIL_ Accessing Google Storage Buckets with gcloud sdk on an M2 Pro_MacOS 14.2.1.md Normal file
View File

@ -0,0 +1,73 @@
---
categories:
- Software Development
date: '2024-01-10 16:11:18'
draft: false
tags:
- python
- til
title: 'TIL: Accessing Google Storage Buckets with gcloud sdk on an M2 Pro/MacOS 14.2.1'
type: posts
---
<!-- wp:paragraph -->
<p>I recently ran into a strange issue where I was struggling to get the <a href="https://cloud.google.com/sdk/docs/install">GCloud SDK</a> to work properly on Mac 14.2.1. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I used <a href="https://brew.sh/">brew</a> to install the package <code>brew install google-cloud-sdk</code> and added it to my zshrc file.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>After I logged in with <code>gcloud auth login</code> I wanted to copy some files out of a GCP bucket to my local machine with the storage cp command:</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><code>gcloud storage cp gs://bucketname/filename .</code> </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The command looked fine and detected the right number of files to copy before crashing out with this rather odd looking error message:</p>
<!-- /wp:paragraph -->
<!-- wp:preformatted -->
<pre class="wp-block-preformatted">gcloud crashed (AttributeError): 'Lock' object has no attribute 'is_fork_ctx'</pre>
<!-- /wp:preformatted -->
<!-- wp:paragraph -->
<p>a quick search surfaced <a href="https://issuetracker.google.com/issues/301593908?pli=1">this ticket</a> in google's issue tracker showing exactly the same problem. It turns out that gcloud is a Python package and it doesn't play nicely with the version of Python that this version of MacOS has installed. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>My workaround was to use Anaconda (which I have installed anyway as a data science bod) to create a new Python environment at a version that gcloud likes. Then to install gcloud from conda:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"bash"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">conda create -n cloudtest python=3.10
conda activate cloudtest
conda install google-cloud-sdk</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>Once I had this set up I found that the command suddenly burst into life.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">A Simpler Alternative?</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>If you don't want to use Anaconda, Google's installer offers to install a compatible version of Python for you. In fact, using brew made life harder for me - I could have just followed the Google experience.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":1506,"sizeSlug":"full","linkDestination":"none"} -->
<figure class="wp-block-image size-full"><img src="/media/image_dbddf6a5.png" alt="" class="wp-image-1506"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->

106
brainsteam/content/posts/2024/01/20/Gitea_Forgejo Actions and PostgreSQL Tests.md Normal file
View File

@ -0,0 +1,106 @@
---
categories:
- Software Development
date: '2024-01-20 14:05:07'
draft: false
tags:
- ci
- django
- docker
- gitea
title: Gitea/Forgejo Actions and PostgreSQL Tests
type: posts
---
<!-- wp:paragraph -->
<p>I am<a href="https://brainsteam.co.uk/2023/11/13/gastronaut-fediverse-recipe-app/"> building a recipe app in django</a> and I want to be able to test the app within the CI pipelines when I push to my self-hosted gitea repository.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">The Problem</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Github actions already has a postgresql action but this crashes when you run Gitea's Action runner via docker:</p>
<!-- /wp:paragraph -->
<!-- wp:preformatted -->
<pre class="wp-block-preformatted">initdb: error: cannot be run as root
Please log in (using, e.g., "su") as the (unprivileged) user that will own the server process.</pre>
<!-- /wp:preformatted -->
<!-- wp:paragraph -->
<p>This is because, by default, docker containers don't really enforce users and permissions in the same way that systems installed in the operating system usually do and the action tries to run a load of stuff with the <code>root</code> user that should probably be run under the <code>postgres</code> system user account.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Finding Another Approach</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>In gitea we can specify the underlying container that we want the action to run in (which is different to github where actions typically run in <a href="https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners">full VMs</a> rather than containers). Although the vanilla gitea container is reasonably good, I replaced it with <code>catthehacker/ubuntu:act-latest</code> so that <a href="https://brainsteam.co.uk/2023/11/18/gitea-actions-and-pdm/">I could get the setup pdm action to work</a>. This container effectively contains a basic set of stuff you'd find in a default ubuntu install including apt for managing packages. Ubuntu ships with a PostgreSQL server package so I went ahead and installed it which also initialises the relevant data files and service user accounts inside the container. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Then, I started the service with <code>service postgresql start</code>. Normally this would be a weird thing to do inside a docker container since best practice is typically to have a separate container per service. However, for testing purposes it's probably ok. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The next step is to set a password for the postgres user which is used by the django app to log in and do the tests. We can use the <code>psql</code> command to do this and we use <code>sudo -u postgres</code> to authenticate against the <code>postgres</code> user before the password is set.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Our final gitea actions yaml looks something like this:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"yaml"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="yaml" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">name: Run Tests
run-name: ${{ gitea.actor }} is testing out Gitea Actions 🚀
on: [push]
jobs:
run_tests:
runs-on: ubuntu-latest
container: catthehacker/ubuntu:act-latest
steps:
- name: Checkout Codebase
uses: actions/checkout@v3
- name: Configure and install postgres
run: |
apt update
apt install -y postgresql
service postgresql start
sudo -u postgres -s psql -U postgres -d postgres -c "alter user postgres with password 'test123';"
- uses: pdm-project/setup-pdm@v3
with:
python-version: 3.10
token: ${{ secrets.GH_TOKEN }}
- name: Install dependencies
run: cd ${{ gitea.workspace }} && pdm install
- name: Run Django tests
env:
DB_HOST: 127.0.0.1
DB_NAME: gastronaut
DB_USER: postgres
DB_PASSWORD: test123
run: |
cd ${{ gitea.workspace }} && pdm run manage.py test
</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>Hopefully this will work out of the box. The whole thing is pretty efficient with a good internet connection, my run taks just under 2 minutes with about 25 seconds needed to install and configure the postgres db:</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":1576,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/image-2-1024x304_cc648cfd.png" alt="a screenshot of the gitea actions CI log for the flow set out above and with indicative timings. The whole run takes 1m 47s and the postgres config step takes 25s" class="wp-image-1576"/></figure>
<!-- /wp:image -->

32
brainsteam/content/posts/2024/01/22/AI Poisoning for everyone_.md Normal file
View File

@ -0,0 +1,32 @@
---
categories:
- AI and Machine Learning
date: '2024-01-22 09:25:51'
draft: false
tags:
- llms
- nlp
- security
title: AI Poisoning for everyone!
type: posts
---
<!-- wp:indieblocks/reply -->
<div class="wp-block-indieblocks-reply"><div class="u-in-reply-to h-cite"><p><i>In reply to <a class="u-url p-name" href="https://arstechnica.com/information-technology/2024/01/ai-poisoning-could-turn-open-models-into-destructive-sleeper-agents-says-anthropic/">AI poisoning could turn open models into destructive “sleeper agents,” says Anthropic</a>.</i></p></div></div>
<!-- /wp:indieblocks/reply -->
<!-- wp:paragraph -->
<p>This is definitely a bit of a hot take from Ars Technica on the recent Anthropic paper about sleeper agents. The article concludes with "...this means that an open source LLM could potentially become a security liability..." but neglects to mention two key things:</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>1) this attack vector isn't just for "open source LLMs" but for any LLM trained on publically scraped data. We're in the dark on the specifics but we know with some certainty that GPT and Claude are "really really big" transformer-decoders and the secret sauce is the scale and the mix of training data. That means they're just as susceptible to attack as any other LLM with this architecture when trained on scraped data.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>2) This isn't a new problem, its an extension of the "let's train on everything we could scrape without proper moderation and hope that we can fine-tune the bad stuff away" mentality. It's a problem that persists in any model, closed or open, which has been trained in this way.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>One thing I know for sure as a machine learning practitioner: performance discrepancies aside, I can probe, test and fine-tune open model weights to my heart's content. With a model behind an API I have a lot less scope to explore and probe and I have to trust, at face value, the promises of the model providers who are being embarassed by moderation fails on a weekly basis (like <a href="https://gizmodo.com/ai-chevy-dealership-chatgpt-bot-customer-service-fail-1851111825">here</a> and <a href="https://www.bbc.co.uk/news/technology-68025677">here</a>). I know which I'd prefer...</p>
<!-- /wp:paragraph -->

32
brainsteam/content/posts/2024/01/25/Supporting the Underdog when it comes to Automated Decision Making and Power Dynamics.md Normal file
View File

@ -0,0 +1,32 @@
---
categories:
- AI and Machine Learning
- Philosophy and Thinking
date: '2024-01-25 09:33:27'
draft: false
tags:
- AI
- ethics
- nlp
title: Supporting the Underdog when it comes to Automated Decision Making and Power
Dynamics
type: posts
---
<!-- wp:indieblocks/reply {"empty":false} -->
<div class="wp-block-indieblocks-reply"><div class="u-in-reply-to h-cite"><p><i>In reply to <a class="u-url p-name" href="https://shkspr.mobi/blog/2024/01/the-theoretical-risks-of-open-sourcing-imaginary-government-llms/#comments">The (theoretical) risks of open sourcing (imaginary) Government LLMs</a>.</i></p></div><div class="e-content"><!-- wp:paragraph -->
<p>I enjoyed this recent blog post by acclaimed technologist Terence Eden proposing a thought experiment about the ethics of open sourcing a hypothetical LLM classifier trained on benefits sanction appeal letters.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Eden, himself a huge open source advocate, argues, quite compellingly that such a model should be kept closed to prevent the potential leakage of potentially confidential information in the training data or probing of the model for the purpose of abusing it.<br><br>However, as some of the post's commentators point out, there is a bigger question at play here: where is it appropriate to be using this kind of tech?<br><br>One of the key issues in my mind is the end-user's treatment and the power dynamic at play here. If you're making life-and-death decisions (<a href="https://www.theguardian.com/uk-news/2021/jan/27/inquest-finds-mother-took-overdose-after-removal-of-disability-benefits">tw suicide</a>) about people who have few resources to challenge those decisions, then you should have appropriate systems in place to make sure that decisions are fair, explainable and rational. You must provide mechanisms that allow the party with everything to lose in this situation to understand what is happening and why. Finally, There must always be an adequate escape hatch mechanism for recourse if the computer gets it wrong. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Whether we're talking about AI in the context of <a href="https://en.wikipedia.org/wiki/Linear_regression">"draw a line of best fit through these points on a graph"</a> or whether we're talking about fancy language models with billions of parameters, my view is that stuff should always be an augmentative technology rather than a replacement for human intelligence. Wherever it is are deployed, AI should be helping a human being to do their job more effectively rather than letting them fall asleep at the wheel. From what I know about Terence I'd go out on a limb to assume he feels the same way and perhaps all this stuff I'm writing about is implicit in his thought experiment. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>However, this all comes at a time when, in the UK, we've had a <a href="https://lnkd.in/gfETy3Ye">recent reminder</a> about what happens when the computer says no and no option for recourse is provided. So I felt that it was worthwhile to fill in these gaps.</p>
<!-- /wp:paragraph --></div></div>
<!-- /wp:indieblocks/reply -->

108
brainsteam/content/posts/2024/02/03/January 2024 In Review.md Normal file
View File

@ -0,0 +1,108 @@
---
categories:
- Personal
date: '2024-02-03 09:38:54'
draft: false
tags:
- MonthlyReview
title: January 2024 In Review
type: posts
---
<!-- wp:paragraph -->
<p>My first monthly review in the new format since I mentioned in December that I would move from publishing weekly to monthly instead. Quite often January is a slow and sleepy month that drags on and on. This year I found that a lot happened in January and I was surprised how quickly it passed.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Personal Stuff</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>In my goal not to be a fat boy at 35 I shifted around 5kg (or 11lb) by not eating crap like we did over the christmas holidays and by going for walks or getting on the stationary bike most days. I've still got a fair bit more weight to shift but I'm happy with this as a starting point and I've found that eating more healthily most of the time means that sweet treats tastes absolutely amazing when we do have a treat day.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>January was a great opportunity for me to re-connect with a lot of people. I caught up with a mentor from my time at the Catalyst programme in Southampton Science Park, some former colleagues and friends. I got the chance to hang out with my good friend Dan Duma who I met during my PhD studies.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"align":"center","lightbox":{"enabled":true},"id":1720,"width":"760px","height":"auto","sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image aligncenter size-large is-resized"><img src="/media/image-1024x768_fde09499.png" alt="James and Daniel hanging out in the The Scottish Stores pub. They are sat at a table with beers in front of them." class="wp-image-1720" style="width:760px;height:auto"/><figcaption class="wp-element-caption">Two doctors: Dan (left) and James (right) catching up in the Scottish Stores pub near Kings Cross, London</figcaption></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>It was also an opportunity to make new friends and acquaintances and I had some networking meetings and calls with people I'd met on LinkedIn and through the <a href="https://nesslabs.com/">Ness Labs</a> community.</p>
<!-- /wp:paragraph -->
<!-- wp:media-text {"mediaId":1721,"mediaType":"image"} -->
<div class="wp-block-media-text is-stacked-on-mobile"><figure class="wp-block-media-text__media"><img src="/media/image-1-226x300_aba0043f.png" alt="" class="wp-image-1721 size-full"/></figure><div class="wp-block-media-text__content"><!-- wp:paragraph -->
<p>We got the opportunity to go up and visit my parents in the Midlands and hung out there one weekend. It was one of the colder weeks in January so it was lovely to be in a cosy setting and their cat Bertie took a liking to me</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We had planned to go out for to a Thai restaurant that we really liked and had been to before but when we arrived there the lovely owners came out to tell us that they had a power cut and they couldn't cook - luckily it was lunch time and they were hoping to get the power back on before their fully booked evening service. We ended up going to a cute little pub nearby instead and Mrs R still ordered a Thai curry!</p>
<!-- /wp:paragraph --></div></div>
<!-- /wp:media-text -->
<!-- wp:paragraph -->
<p>While we were up north we also got to meet up with one of my school friends for breakfast which was great fun.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Some stuff didn't go so well: we lost a close family friend earlier this month and Mrs R's family dog which she's had since she was an early teenager. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We also had to take our cats in for some dental surgery which was carried out under general anaesthetic and I found that very stressful even though the vet reassured us that the risk was very low and it all worked out fine.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Work Stuff</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>It's been a pretty hectic month on the work front. I had 8 annual personal development reviews to go through with my team. I love chatting to my team about what went well and what they want to work on this year so that was actually quite an enjoyable experience.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I oversaw a migration from Linear to JIRA which we decided to make due to a need to move to a more controlled and rigorous development process and so that our product team could make use of JIRA Product Discovery for ranking and sorting new feature requests effectively. I've written some scripts for automating a lot of the migration and preserving things like attachments and comments which I am<a href="https://github.com/ravenscroftj/linear-jira-tools"> in the process of open sourcing.</a> It's still a little bit half-baked for now but I'll post again when I'm done.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I spent a lot of time helping out with annual reviews of internal policy documentation around security, cyber-risk and software lifecycle which was a dull but very important task.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Entertainment</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>We set a goal of trying to get to the cinema twice a month this year since we have cineworld membership that costs about the same as 1.5 standard tickets. At the start of the month we went to see <a href="https://brainsteam.co.uk/notes/10bc80f5f0/">Anyone But You</a> but we haven't made it back since. Maybe February will be a better month for Cinema trips!</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We binged our way through the latest series of <a href="https://invidious.jamesravey.me/watch?v=GZMrivD2Aok">Queer Eye</a> this last week (I say binged but we watched one episode a night which I thought demonstrated restraint) and I really enjoyed it as usual. I'm excited that there will be a new series but sad that Bobby will not be returning, his interior design work is superb.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>This month I've been reading Consider Phlebas by Iain Banks. I was reminded by <a href="https://kevquirk.com/consider-phlebas">Kev's recent review</a> that the Culture books have been on my reading list since forever. So far I've found it a little difficult to get into and I keep falling asleep with it. However, I am about 25% of the way through now and I'm determined to keep going.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">New Process</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>My new review process is to write a weekly note in obsidian that summarises each week and then write a monthly review note which summaries my weeklies. Then I can publish a blog post based on that. So far I'm enjoying it and I think I'm putting a lot less pressure on myself if I have a slow week where there's not much to write home about. I think I'll probably stick to this new process for now and see how I get on.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">What will February Bring?</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>It will be interesting to see how the next month pans out. I have the funeral for the family friend I mentioned coming up later this week. I'm hoping to keep meeting up with people and networking as, finally nearly 3 years since the COVID lockdowns stopped happening regularly, I'm getting into the swing of having a social life again.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->

82
brainsteam/content/posts/2024/03/04/February 2024 In Review.md Normal file
View File

@ -0,0 +1,82 @@
---
categories:
- Personal
date: '2024-03-04 21:29:51'
draft: false
tags:
- MonthlyReview
title: February 2024 In Review
type: posts
---
<!-- wp:paragraph -->
<p>It was a pretty typical February for us this year in the sense that it was pretty rubbish and both myself and Mrs R were sick for a lot of it.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Flu February</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Regular readers may have noticed that I've not been posting much this month. Towards the end of the first full week of Feb I was stricken with Flu. Now, I've heard people say "it's not real flu unless you think you're dying" and laughed it off and I've had heavy colds before. Now, I've had real influenza I really do get it. On the first 2-3 days I had very high fever and muscle aches. Then, I had those symptoms plus nausea and sickness for a further 3 days. Normally if you get a stomach bug it passes within 24 hours. After 3 days solid of feeling this way you really do start to think "am I dying?" </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Thankfully, I started to recover after about 5 days and now, almost 3 weeks later I almost feel human again. However, the struggle has been real. One of the interesting things was that do get the flu vaccine on an annual basis so I guess this year I was just super unlucky and got a strain that I was not protected against. Or, perhaps, as my doctor said, it would have been <em>even worse</em> without the vaccine (I almost can't comprehend how bad that would be).</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Back at Work</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>After a week off and a second week doing half days and sleeping throughout the afternoon, I returned to work "as normal" last week.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We are currently doing some interesting work exploring few-shot and zero-shot models for classification in use cases where we would normally be looking to collect large volumes of data to help the client train their models. The trade-off is that many of these models are quite heavy to run compute-wise so I've been doing some work looking at Huggingface Optimum to quantize models so that they can run in less memory and with fewer CPU/GPU cycles. I've also been playing with <a href="https://notes.jamesravey.me/Software/Nvidia-Triton">Nvidia Triton</a> for deploying models.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Take Note</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I've been exploring personal knowledge management (yes again <em>sigh</em>) and playing with some tools and approaches. I have started experimenting with <a href="https://silverbullet.md/">Silverbullet</a> which has a steep learning curve but provides some pretty nifty features out of the box and plays really nicely with other Markdown-based PKM systems like Obsidian and Foambubble. I've also set it up so that any notes that I set as <code>public: true</code> in the frontmatter are automatically published to <a href="https://notes.jamesravey.me/">my digital garden</a>. I have added some sensible exclude and ignore rules so that I don't publish anything I want to keep private.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I was playing with the idea of moving away from wordpress for my main site again but I don't want to sink loads of time into re-designing my main website - especially since I only did it 6 months ago. </p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">At Home</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>With both of us being sick, we haven't had much opportunity to achieve much in our family life this month. However, we did book some holidays and trips for later in the year. Being deathly sick allowed me to lose a further 2kg I'm now 97kg (down from 105kg at my heaviest) which I am pretty happy with.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>While I was at my sickest I couldn't even bring myself to watch TV or read but either side I did manage a few things:</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I finally finished <a href="https://bookwyrm.social/book/6854/s/consider-phlebas">Consider Phelebas</a> which as <a href="https://brainsteam.co.uk/2024/02/03/january-2024-in-review/">I mentioned previously</a> I had been struggling to read. At some point the pace picked up and I started to gel with the characters but honestly, it was a real struggle to get there. When I did finally finish the book, I found the ending utterly unsatisfying and it made the whole book feel a bit moot and pointless. I think that might be what the author was going for but it's made me think twice about picking up any more of the culture books. I've since started reading <a href="https://bookwyrm.social/book/93151/s/doors-of-eden">The Doors of Eden</a> by Adrian Tchaikovsky which I'm really enjoying.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I finished <a href="https://www.youtube.com/watch?v=w8Zr3f-_Ft8">Avenue 5</a> which is a sitcom about a bunch of idiots in space with some heavy hitters (Hugh Laurie, Josh Gadd, Zack Woods, Lenora Crichlow, Daisy Mae Cooper). I really enjoyed the show and I was sad that it got cancelled after two series. </p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Hopes for March</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I'd love for March to not be another write-off due to sickness. Last year we got a bunch of coughs and colds from about Mid-Feb until the end of March and then I got COVID over easter.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>This week we have some friends and family catch ups and UK Mother's day. Next week I will be delivering a guest lecture on Software development best practices at Warwick Business School and then the next day we will be going up to Edinburgh for a mini-break and to explore the city. </p>
<!-- /wp:paragraph -->

23
brainsteam/content/posts/2024/03/17/Broken Sites in Firefox with Proton Pass v1.15.md Normal file
View File

@ -0,0 +1,23 @@
---
categories:
- Software Development
date: '2024-03-17 17:41:14'
draft: false
tags:
- firefox
- proton
title: Broken Sites in Firefox with Proton Pass v1.15
type: posts
---
<!-- wp:paragraph -->
<p>This morning I noticed that there were some weird things happening in Firefox and pages were crashing all over the place including my wordpress editor, my umami console and various other pages. I had some weirdness with Gnome and Ubuntu on my laptop this morning so I thought it was operating system instability.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Then, I came across <a href="https://lemmy.ml/post/13247149">someone on Lemmy who was experiencing similar problems an</a>d blamed the Proton Pass plugin. The error messages I was getting in the console didn't mention proton but they were similar in nature. I disabled the addon and hey presto, I got some stability back on my system.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>If you are having this problem and you're a proton pass user, you will be pleased to know that <a href="https://www.reddit.com/r/ProtonPass/comments/1bfe2zi/protonpass_firefox_addon_breaks_websites/">Proton are working on getting it fixed</a> but in the mean time you can go to<a href="https://addons.mozilla.org/en-US/firefox/addon/proton-pass/versions/"> the official addon page</a> and download v1.14.1 to forcibly revert your plugin and restore stability until the new version is out.</p>
<!-- /wp:paragraph -->

163
brainsteam/content/posts/2024/03/17/Moving from Gitea to Forgejo including Actions with Docker Compose.md Normal file
View File

@ -0,0 +1,163 @@
---
categories:
- Software Development
date: '2024-03-17 12:51:07'
draft: false
tags:
- devops
- docker
- forgejo
- gitea
title: Moving from Gitea to Forgejo including Actions with Docker Compose
type: posts
---
<!-- wp:paragraph -->
<p>I just moved from <a href="https://about.gitea.com/">gitea</a> to <a href="https://forgejo.org/">forgejo</a> and the process could not have been simpler. I'm really impressed. I suppose since the latter was <a href="https://forgejo.org/2024-02-forking-forward/">until very recently</a> a soft fork of gitea, I shouldn't be surprised at how easy it was. Still though, I sat down expecting to need to dedicate a few hours to debugging and it "just works" without a hitch after 15 minutes. The most jarring part was getting the actions runner working but the forgejo documentation is great, in fact I'd say better than gitea's docs, so in the end this wasn't a massive issue.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Why?</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I have used gitea for hosting personal projects since about 2018 when I first became aware of it. I've been super impressed and I love the fact that it's so lightweight and simple to use. However, I've been a little disappointed by the founders' r<a href="https://blog.gitea.com/a-message-from-lunny-on-gitea-ltd.-and-the-gitea-project/">ecent-ish pivot towards commercialisation</a>. Don't get me wrong, I'm an open source maintainer and contributor myself and I know that many of these folks are essentially doing a public service for free. The final straw for me has been the recent announcement of Gitea Enterprise and <a href="https://blog.gitea.com/gitea-enterprise/">the direction they seem to be moving in </a>(open "core" with a bunch of paywalled features). </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I was a huge fan of Gitlab about 10-11 years ago but they have since heavily commercialised and gotten worse, caring less and less about their community and more and more about corporate user. The open core model can be ok if the features that are paywalled genuinely are things that only big companies care about or if bleeding edge features are open sourced after an embargo period or <a href="https://squidfunk.github.io/mkdocs-material/insiders/">once a funding milestone has been met</a>. However, I'm once bitten twice shy when it comes to these kinds of precedents in software forges so forgive my cynicism.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Forgejo was set up in response to Gitea's pivot towards commercialisation and their governance model is pretty much set up in a way to prevent the same thing happening again (I guess nothing is impossible but it is unlikely and there would be a lot of drama). For now I will throw in my lot with them.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">How to Switch The Server</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>The server software was the easiest thing to switch. Following the instructions from forgejo, I simply replaced the name of the gitea image with the equivalent forgejo one in my <code>docker-compose.yml</code> and ran <code>docker-compose pull</code> followed by <code>docker-compose up</code>:</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Before:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"yaml"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="yaml" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""> server:
restart: unless-stopped
image: gitea/gitea:1.21
container_name: gitea
environment:
- USER_UID=1000
- USER_GID=1000
volumes:
...</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>And after:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"yaml"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="yaml" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""> server:
restart: unless-stopped
image: codeberg.org/forgejo/forgejo:1.21
container_name: gitea
environment:
- USER_UID=1000
- USER_GID=1000
volumes:
...</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>As soon as I ran this and started the server the whole thing came back up and apart from the new orange paint job, very little had changed about my server.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Swapping the Runner</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>The runner was slightly more painful because I had made <a href="https://brainsteam.co.uk/2024/01/20/gitea-forgejo-actions-and-postgresql-tests/">a couple of customisations</a> to my gitea actions runner before in order to get it running docker-in-docker (allowing me to push and pull images from inside the runner without those images appearing in the host system's user space)</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock -->
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""> runner:
restart: unless-stopped
image: vegardit/gitea-act-runner:dind-latest
privileged: true
environment:
CONFIG_FILE: /config.yaml
GITEA_INSTANCE_URL: https://git.jamesravey.me
GITEA_RUNNER_REGISTRATION_TOKEN: "<redacted>"
GITEA_RUNNER_NAME: gitea-runner-1
GITEA_RUNNER_LABELS: "ubuntu-latest,ubuntu-22.04,ubuntu-20.04,ubuntu-18.04"
volumes:
- ./runner-config.yaml:/config.yaml
- ./runner-data:/data:rw
- ./runner-cache/:/cache
ports:
- 42263:42263</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>Again, the documentation in forgejo is great and they provide an example for running docker-in-docker as a separate service and having the runner talk to it. All I had to do was change a couple of env vars and the runner image name:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock -->
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""> docker-in-docker:
image: docker:dind
container_name: 'docker_dind'
privileged: true
command: ['dockerd', '-H', 'tcp://0.0.0.0:2375', '--tls=false']
restart: 'unless-stopped'
runner:
restart: unless-stopped
links:
- docker-in-docker
depends_on:
docker-in-docker:
condition: service_started
container_name: 'runner'
image: code.forgejo.org/forgejo/runner:3.3.0
user: 1000:1000
command: forgejo-runner daemon
environment:
DOCKER_HOST: tcp://docker-in-docker:2375
CONFIG_FILE: /config.yaml
GITEA_INSTANCE_URL: https://git.jamesravey.me
GITEA_RUNNER_REGISTRATION_TOKEN: "<redacted>"
GITEA_RUNNER_NAME: gitea-runner-1
GITEA_RUNNER_LABELS: "ubuntu-latest,ubuntu-22.04,ubuntu-20.04,ubuntu-18.04"
volumes:
- ./runner-config.yaml:/config.yaml
- ./runner-data:/data:rw
- ./runner-cache/:/cache
- /var/run/docker.sock:/var/run/docker.sock
ports:
- 42263:42263</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>Again, a quick test via <code>docker-compose up</code> and running one of my CI pipelines and voila, everything clicked into place:</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":2092,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/image-6-1024x222_fee3419d.png" alt="a screenshot of a forgejo run with green ticks" class="wp-image-2092"/><figcaption class="wp-element-caption">Happy green ticks after the forgejo runner completes its run.</figcaption></figure>
<!-- /wp:image -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Conclusion</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>If you were thinking about checking out forgejo for philosophical reasons or simply because you were curious and you're worried about switching from gitea, it couldn't be easier. If you are thinking about trying it out though, I'd strongly recommend taking a backup and/or spinning up a temporary or beta site where you can try it out before committing. Also bear in mind that<a href="https://code.forgejo.org/forgejo/runner#forgejo-runner"> Forgejo recommend not using their actions runner in prod yet</a> because it may not be secure enough. I don't let anyone else use or sign up in my instance and I'm pretty paranoid about what I do run there so I'm taking a calculated risk here.</p>
<!-- /wp:paragraph -->

155
brainsteam/content/posts/2024/03/30/Edinburgh 2024.md Normal file
View File

@ -0,0 +1,155 @@
---
categories:
- Personal
date: '2024-03-30 08:07:03'
draft: false
tags:
- food
- travel
title: Edinburgh 2024
type: posts
---
<!-- wp:paragraph -->
<p>Earlier this month, Mrs R and I took a mini-break in Edinburgh. We got a few hours of sunshine on the day that we arrived and I managed to get a couple of nice photos of the castle. After that it basically rained non stop until we left, which, as I understand it, is the authentic Scottish experience. We still had a really great time though.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":2221,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/PXL_20240313_180026393-1024x769_8fcd24e6.jpg" alt="James and Mrs R posing in front of Edinburgh Castle on a sunny afternoon." class="wp-image-2221"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>One of the few sunny photos that we managed to capture.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Things We Did</h2>
<!-- /wp:heading -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">The Castle</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>We visited Edinburgh castle which was quite neat but it was raining heavily during our visit. The interior of the castle was really interesting. There is a regal banquet hall that felt like something straight out of game of thrones. Apparently Oliver Cromwell turned it into a cramped barracks. He installed 3 storeys of wooden floors in there which definitely would have spoiled it. It was then restored in the 1800s. From what I know about Cromwell, most stuff he did was questionable or flat out offensive so it didn't surprise me.</p>
<!-- /wp:paragraph -->
<!-- wp:media-text {"mediaId":2225,"mediaLink":"https://brainsteam.co.uk/pxl_20240314_095151699-2/","mediaType":"image"} -->
<div class="wp-block-media-text is-stacked-on-mobile"><figure class="wp-block-media-text__media"><img src="/media/PXL_20240314_095151699-edited-771x1024_f9340a7b.jpg" alt="A building that looks a bit like a poo made out of metal - or maybe a spiral ribbon if you look closely" class="wp-image-2225 size-full"/></figure><div class="wp-block-media-text__content"><!-- wp:paragraph {"placeholder":"Content…"} -->
<p>The views from atop the castle weren't super interesting due to the heavy rain and lack of visibility. However, On a good day, I'm told the view is spectacular.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p> I managed to photograph the building that looks suspiciously like a big poo. This building is locally known as "the jobbie" which is scottish slang for poo. Apparently, it's a shopping mall.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Our guide in the camera obscura said he was once berated by the building's architect when describing it as "the jobbie". Apparently it is supposed to be a ribbon. I suppose I sort of see it.</p>
<!-- /wp:paragraph --></div></div>
<!-- /wp:media-text -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Camera Obscura</h3>
<!-- /wp:heading -->
<!-- wp:media-text {"mediaPosition":"right","mediaId":2227,"mediaLink":"https://brainsteam.co.uk/?attachment_id=2227","mediaType":"image","mediaWidth":35} -->
<div class="wp-block-media-text has-media-on-the-right is-stacked-on-mobile" style="grid-template-columns:auto 35%"><div class="wp-block-media-text__content"><!-- wp:paragraph {"placeholder":"Content…"} -->
<p>Edinburgh Camera obscura is really cool. In the 1800s a female optician and professor built this thing as a giant spying tool. It uses huge glass focussing lenses to focus light onto a white projection table. You can move the aperture around with a long wooden pole. It was effectively an early CCTV system. Our tour guide explained that she basically did this "for shigs" in high society Edinburgh.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The camera needs lots of light to work and we visited edinburgh on a particularly rainy day. Therefore, the picture wasn't that clear. Luckily, they simulated a sunny day using a digital projector and there were some stunning views. This was my second experience with a Camera Obscura. My first was at <a href="https://www.cardigan-bay.com/cliff-railway-camera-obscura/">Aberystwyth Constitution Hill </a>when I was at university.</p>
<!-- /wp:paragraph --></div><figure class="wp-block-media-text__media"><img src="/media/Untitled-768x1024_ab6d1263.jpeg" alt="An external shot of the camera obscura on a rainy day. It's a round tower with a hole at the top to let light in." class="wp-image-2227 size-full"/></figure></div>
<!-- /wp:media-text -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">The Real Mary King's Close</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>This was a really interesting and informative attraction. When they built <a href="https://www.edinburghcitychambers.co.uk/">Edinburgh City Chambers,</a> they decided to build it over the top of 4-5 streets that go downhill behind it. This meant using the existing streets and buildings for support. They chopped the roofs off some buildings and putting rubble and stuff on others to create a flat foundation. The original street is still there under ground and you can take guided tours around it. You can see some of the original architecture and even some of the possessions of 18th and 19th century Edinburgh people. It was very cool and very interesting.</p>
<!-- /wp:paragraph -->
<!-- wp:gallery {"linkTo":"none"} -->
<figure class="wp-block-gallery has-nested-images columns-default is-cropped"><!-- wp:image {"lightbox":{"enabled":true},"id":2228,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/Before-Edi-Hall_d6dab321.png" alt="cross sectional diagram of some buildings on a hill and buildings down a road that slopes down the hill" class="wp-image-2228"/></figure>
<!-- /wp:image -->
<!-- wp:image {"lightbox":{"enabled":true},"id":2229,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/After-Edi-Hall-1024x532_c462e3b5.png" alt="second diagram but this time there is a new building on top of the slope and the buildings that were there before have been chopped down or had junk piled on top of them to support the new building on top of them." class="wp-image-2229"/></figure>
<!-- /wp:image --></figure>
<!-- /wp:gallery -->
<!-- wp:paragraph -->
<p>Really simplified diagrams of before (left) and after (right) the hall was built. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Our tour guide, Becca, was really fun and engaging. She was in character as a "Foul Clenger" from the plague times. These were people who went into the houses of dead plague victims and burned all of their belongings. We learned a lot about the black death epidemics in Edinburgh. Most interestingly, the plague doctor's uniform (that iconic look with the raven mask) protected them from the disease almost by coincidence. They were trying to protect themselves from "foul odors" and "miasmas" and coincidentally stopped the bubonic-plague carrying fleas from biting them.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">National Museum of Scotland</h3>
<!-- /wp:heading -->
<!-- wp:media-text {"mediaId":2230,"mediaLink":"https://brainsteam.co.uk/?attachment_id=2230","mediaType":"image"} -->
<div class="wp-block-media-text is-stacked-on-mobile"><figure class="wp-block-media-text__media"><img src="/media/image-10-762x1024_90a5c9b3.png" alt="The magnificent interior of the museum with huge glass windows and decorative iron supports" class="wp-image-2230 size-full"/></figure><div class="wp-block-media-text__content"><!-- wp:paragraph -->
<p>The National Museum of Scotland is a huge striking building in central Edinburgh. It's about 5 minutes walk from the castle and a perfect distraction in the rain.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph {"placeholder":"Content…"} -->
<p>We ventured around the museum for a few hours. They had lots of different exhibits focussing on natural history, modern science and engineering and anthropology. They had lots of fun interactive exhibits to play with like giant musical instruments and practical experiments.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>It was a bit jarring seeing devices that I used to own, like the <a href="https://en.wikipedia.org/wiki/Nokia_3310">Nokia 3310</a> in the museum of technologies past. That's progress I guess!</p>
<!-- /wp:paragraph --></div></div>
<!-- /wp:media-text -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Dining</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>We ate in two really lovely restaurants.</p>
<!-- /wp:paragraph -->
<!-- wp:media-text {"mediaPosition":"right","mediaId":2231,"mediaLink":"https://brainsteam.co.uk/?attachment_id=2231","mediaType":"image","mediaWidth":27,"mediaSizeSlug":"large"} -->
<div class="wp-block-media-text has-media-on-the-right is-stacked-on-mobile" style="grid-template-columns:auto 27%"><div class="wp-block-media-text__content"><!-- wp:paragraph {"placeholder":"Content…"} -->
<p> On the first night we went to Bertie's Proper Fish + Chips where I tried a proper scotch egg. There were a lot of Americans around trying the traditional British dish which was kind of neat. They did have battered mars bars on their menu but after my scotch egg and haddock and chips I was too full to try it.</p>
<!-- /wp:paragraph --></div><figure class="wp-block-media-text__media"><img src="/media/PXL_20240313_200402972_sm-771x1024_98d77185.jpg" alt="A monster plate of fish and chips with a wedge of lemon and tartar sauce" class="wp-image-2231 size-large"/></figure></div>
<!-- /wp:media-text -->
<!-- wp:paragraph -->
<p>On our 2nd night we were due to go and eat at an Italian restaurant but we were both super tired. Instead, so we stayed in our hotel apartment and heated up some store-bought ravioli. Not exactly the Italian experience we were hoping for but it filled a gap.</p>
<!-- /wp:paragraph -->
<!-- wp:media-text {"mediaId":2241,"mediaLink":"https://brainsteam.co.uk/?attachment_id=2241","mediaType":"image","mediaSizeSlug":"large"} -->
<div class="wp-block-media-text is-stacked-on-mobile"><figure class="wp-block-media-text__media"><img src="/media/4080-3072-max-1024x771_0300d45e.jpg" alt="James and Mrs R sat in Nok's Kitchen in Edinburgh with a bottle of water in front of them on the table." class="wp-image-2241 size-large"/></figure><div class="wp-block-media-text__content"><!-- wp:paragraph -->
<p>On our last night we went to Nok's Kitchen, an independently run Thai restaurant just next to the castle. On our travels we are rarely let down by an indie Thai restaurant and this was no exception to that finding. Nok's was really great and the staff were super friendly too.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The restaurant was quite small and it was completely full up the whole time we were there. </p>
<!-- /wp:paragraph --></div></div>
<!-- /wp:media-text -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Final Thoughts</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I really enjoyed our mini-break to Edinburgh. Despite the inclement weather we had a really great time. I found the people in Edinburgh, particularly those we interacted with in restaurants and tourist attractions to be super welcoming and genuinely passionate about what they do. This was refreshing compared to where we live in England where most staff in these industries are bored teenagers doing the bare minimum to get through their shift (and hey for minimum wage and without the tipping culture of the US, who can really blame them?) Edinburgh's residents seem to have a real pride in what they do. There is a national identity there that just isn't a thing in England and I found that really refreshing.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We will definitely try and get back to Edinburgh when it's not persistently raining some time in the summer. Not that anyone can make such guarantees in the UK/Scotland. If you are planning to visit Scotland in March, please do pack a good rain coat and some waterproof trousers. We saw a huge number of tourists wearing t-shirts under those disposable rain ponchos you get from theme parks. We assume that they probably paid through the nose for them at an Edinburgh tourist shop!</p>
<!-- /wp:paragraph -->

180
brainsteam/content/posts/2024/04/02/Finding the Best AI-Powered Handwriting OCR.md Normal file
View File

@ -0,0 +1,180 @@
---
categories:
- AI and Machine Learning
date: '2024-04-02 18:25:27'
draft: false
tags:
- AI
- llms
- nlp
title: Finding the Best AI-Powered Handwriting OCR
type: posts
---
<!-- wp:paragraph -->
<p>Quite unusually for me, this essay started its life as a scribble in a notebook rather than something I typed into a markdown editor. A few weeks ago,<a href="https://www.linkedin.com/posts/tiagoforte_how-to-digitize-handwritten-notes-with-chatgpt-activity-7168970873788358656-lAmj"> Tiago Forte made a video suggesting that people can use GPT-4 to capture their handwritten notes digitally.</a> I've been looking for a "smart" OCR that can process my terribly scratchy, spidery handwriting for many years but none have quite cut the mustard. I thought, why not give it a go? To my absolute surprise, GPT did a reasonable job of parsing my scrawling and capturing text. I was seriously impressed. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Handwriting OCR has now gone from a fun toy and conversion piece to something I can actually, seriously use in my workflow. So then, Is GPT the best option I've got for this task or are there any FOSS tools and models that I could use instead?</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">An Experiment</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>In my experiment I use the following system prompt with each model:</p>
<!-- /wp:paragraph -->
<!-- wp:quote -->
<blockquote class="wp-block-quote"><!-- wp:paragraph -->
<p>Please transcribe these handwritten notes for me. Please use markdown formatting. Do not apply line wrapping to any paragraphs. Do try to capture headings and subheadings as well as any formatting such as bold or italic. Omit any text that has been scribbled out. Try your best to understand the writing and produce a first draft. If anything is unclear, follow up with questions at the end.</p>
<!-- /wp:paragraph --></blockquote>
<!-- /wp:quote -->
<!-- wp:paragraph -->
<p>and then proceed to send each of them this photo of my notes:</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":2271,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/image-771x1024_b6877c57.png" alt="A page in a notebook with handwritten text which is the first draft of this very post. It starts "quite unusually for me..."" class="wp-image-2271"/><figcaption class="wp-element-caption">My lovely scrawly hand writing displa</figcaption></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>The main flaw in my experiment is its scale. Ideally I would use a large number of handwritten documents with different pens, ink colours and photographed under different lighting conditions to see how this affects the model output. However, I didn't have hours and hours to spare manually transcribing my own notes.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>However, I would say that the photo is pretty representative of a page of notes in my typical handwriting. It is written in my bujo/notebook that I carry around with me and my fountain pen that I use most of the time. I wouldn't expect results to vary wildly for me specifically. However dear reader: your mileage may vary.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">What I'm Looking For</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I want to understand which model is best able to understand my handwriting without misinterpreting it. Bonus points for also providing sensible styling/formatting and being robust to slightly noisy images (e.g. the next page in the notebook being slightly visible in the image).</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Ideal Outcomes</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>What I hope to find is an open source/local model that I could run which would provide high quality handwriting OCR that I can run on my own computer. This would give me peace of mind about not having to send my innermost thoughts to OpenAI for transcription and would be a budget option. Also, from an energy and water usage point of view, it would be great to be able to do this stuff with a small model that can run on a single consumer-grade GPU like I have in my computer.</p>
<!-- /wp:paragraph -->
<!-- wp:separator -->
<hr class="wp-block-separator has-alpha-channel-opacity"/>
<!-- /wp:separator -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">GPT-4V</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>GPT-4V did the best job of transcribing my notes, making the fewest mistakes interpretting my handwriting and almost perfectly following my instructions regarding formatting. It did make a few errors. For example, renaming Tiago Forte as Rafe Furst and referring to LLAVA as "Clara".</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><a href="https://memos.jamesravey.me/m/o6u9pLGGFxDJArCVryeMRe">Read GPT-4 Full Response</a></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Gemini</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Google Gemini did a pretty poor job compared to GPT-4V. It ignored my instructions not to wrap lines in markdown and it also attempted to read the first word of each line from the second page of my notes which is visible in the image. It made a complete hash of my work and also misread a large chunk of the words. I used the model in chat mode and even when I asked it to "try again but this time don't try to read the next page", it ignored me and spat out more or less the same thing.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><a href="https://memos.jamesravey.me/m/PENgmWxbKY2QgvAft9T4Sq">Read Full Gemini Output</a></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Claude Opus</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>The quality of Claude's response was much closer to that of GPT-4V's. There were a couple of very prominent spelling/misinterpretations that made me immediately suspicious in general the output is quite faithful to the writing. However, Claude did say it would provide markdown output and then completely fail to do that.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><a href="https://memos.jamesravey.me/m/5dx73ymc3dzY7QPxz3eBVV">Read full Claude Output</a></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Local Model: LLAVA</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>One of the biggest advances in Local LLMs in the last 12 months has been LLAVA which provides reasonable performance on a number of multi-modal benchmarks. Earlier versions of LLAVA were trained using images and by having GPT-4 expand the text descriptors. These images that were originally provided by humans. However, they didn't yet have access to opt-4.5k so training was done on descriptors that were often second-hand and pretty surprising that this worked at all tbh - but it did!</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>LLAVA 1.6 is trained on data with more in-depth annotations including <a href="https://huggingface.co/datasets/laion/gpt4v-dataset/viewer?p=1">LAION GPT-4V</a> which, incidentally, I can't find any formal evaluations of. This seems to improve the model's ability to reason about the contents of images. They also train using <a href="https://textvqa.org/">TextVQA</a> which aims to train models to be able to reason about and perform question answering for text documents. It is likely that elements of both of these datasets help improve the model's OCR ability.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I set up LLAVA to run locally using llama.cpp and tried running some experiments to see how well it picked up my handwriting (<a href="https://notes.jamesravey.me/LLaVa#running-llava-with-llamacpp">instructions here</a>). Although LLAVA 1.6 fared a lot better than earlier versions, it didn't do an amazingly good job. It also got stuck in a generation loop a few times, repeatedly outputting the same mad ramblings until terminated.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><a href="https://memos.jamesravey.me/m/RWWjf5LfF7iGddkSB2cARH">Read LLAVA Output Here</a></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">The Classic option - CRAFT-TrOCR</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p><a href="https://huggingface.co/spaces/tomofi/CRAFT-TrOCR">HF Space Repo</a></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>CRAFT-TrOCR is an early example of a transformer-based large multi-modal model. According to <a href="https://arxiv.org/abs/2109.10282">TrOCR's abstract</a>:</p>
<!-- /wp:paragraph -->
<!-- wp:quote -->
<blockquote class="wp-block-quote"><!-- wp:paragraph -->
<p>we propose an end-to-end text recognition approach with pre-trained image Transformer and text Transformer models, namely TrOCR, which leverages the Transformer architecture for both image understanding and wordpiece-level text generation</p>
<!-- /wp:paragraph --></blockquote>
<!-- /wp:quote -->
<!-- wp:paragraph -->
<p>The CRAFT component is use to detect regions of the image that may contain text and TrOCR is used to recognise the text. However, TrOCR is quite small for a large language model, the largest version trained in the paper has 550M parameters - around the same size as BERT and around 40 times smaller than <a href="https://arxiv.org/pdf/2310.17680v1.pdf">GPT-3.5 is believed to be</a>.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p> I played with this pipeline a couple of years ago and I remembered it being reasonably good at certain context but only if I write in my very best handwriting all the time. And I don't do that because when I am taking notes in a meeting or something, I am thinking about the meeting, not my handwriting.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Although the HF space is broken, I was able to make some local changes (published <a href="https://github.com/ravenscroftj/CRAFT-OCR">here</a> to get it running on my PC). This is not a generative model with a prompt. It is a pipeline with a single purpose: to recognise and output text. I simply uploaded the photo of my note and hit submit. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>However, I found that the output was pretty disappointing and nonsensical. I wonder if the limitation is the model size, the architecture or even the image preprocessing and segmentation that is going on (which is "automatic" in more recent LMMs).</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><a href="https://memos.jamesravey.me/m/kEsGCc7CJc7yJ9bbo7QJcH">Read CRAFT-TROCR Output</a></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Conclusion: The Best Option Right Now</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Right now, GPT-4V seems to be the best option for handwriting OCR. I have created a custom GPT which has a tuned variation on the system prompt used above in my experiment which people can use <a href="https://chat.openai.com/g/g-Kjk6IzGWm-markdown-scribe">here</a>. Using GPT to do this work makes copying my analogue thoughts into my digital garden a breeze.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>However, I am very much on the lookout for a new open source model that will do this job well. What could change in this space? A version of LLaVa that has been fine-tuned on handwriting OCR task could emerge or perhaps we'll see an all together new model. The rate of open source model development is phenomenal so I am sure we will see progress in this space soon. Until then, I'm stuck paying OpenAI for their model.</p>
<!-- /wp:paragraph -->

92
brainsteam/content/posts/2024/04/07/March 2024 In Review.md Normal file
View File

@ -0,0 +1,92 @@
---
categories:
- Personal
date: '2024-04-07 15:41:04'
draft: false
tags:
- MonthlyReview
title: March 2024 In Review
type: posts
---
<!-- wp:paragraph -->
<p>We're a few days into April now so I'm overdue my March Review. March was a much more enjoyable month than <a href="https://brainsteam.co.uk/2024/03/04/february-2024-in-review/">February</a>. We did a lot of stuff and I managed not to get sick the whole time which was great.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Travelling About</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I started the month with a couple of hectic weeks. In the first week we met up with an old family friend for dinner in Southampton. We went to <a href="https://www.mangothai.co.uk/">Mango Thai Above Bar</a> where we had delightful food and a really fun evening. We also met up with my Mum and her partner for lunch to celebrate Mother's Day.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>In the second week went up to Warwick University to give a guest lecture about AI and Machine Learning. I drove up and back on the same day (a good 6 hour round trip) so that I could be at home to meet Mrs R the next day and fly up to<a href="https://brainsteam.co.uk/2024/03/30/edinburgh-2024/"> Edinburgh for our mini break</a>.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":2308,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/IMG_4629-1024x768_2407d657.jpg" alt="James stood on a stage in front of a podium with slides projected behind him - "garbage in, garbage out"." class="wp-image-2308"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>We had a week off to recover before a weekend visit my Dad + his wife for her birthday party. It was nice to see a bunch of family members, some of whom we hadn't seen since before COVID. We also visited my mum and her partner the next day. We went out for Sunday breakfast with them by the riverside in <a href="https://visitworcestershire.org/locations/bewdley">Bewdley</a>.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Learning in Public</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Since I migrated to using <a href="https://silverbullet.md/">silverbullet</a> + Obsidian last month I've been writing a lot more in my public <a href="https://notes.jamesravey.me">digital garden</a>. I'm writing up any useful tidbits of information that I find there and gradually adding to them over time. Historically I have written these sorts of posts up as blog posts which then age and get left in time. For example my <a href="https://brainsteam.co.uk/2021/04/01/opinionated-guide-to-virtualenvs/">advice on python virtual environments</a> which is due a rewrite would make a great digital garden piece. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I am planning to add summaries of my digital garden changes here. Partly as signposts to new information that readers might find interesting. Partly as a way for me to track and quantify the knowledge I am collecting.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I also published a few of articles:</p>
<!-- /wp:paragraph -->
<!-- wp:embed {"url":"https://brainsteam.co.uk/2024/03/17/broken-sites-in-firefox-with-proton-pass-v1-15","type":"wp-embed","providerNameSlug":"brainsteam"} -->
<figure class="wp-block-embed is-type-wp-embed is-provider-brainsteam wp-block-embed-brainsteam"><div class="wp-block-embed__wrapper">
https://brainsteam.co.uk/2024/03/17/broken-sites-in-firefox-with-proton-pass-v1-15
</div></figure>
<!-- /wp:embed -->
<!-- wp:embed {"url":"https://brainsteam.co.uk/2024/03/17/moving-from-gitea-to-forgejo-including-actions-with-docker-compose","type":"wp-embed","providerNameSlug":"brainsteam"} -->
<figure class="wp-block-embed is-type-wp-embed is-provider-brainsteam wp-block-embed-brainsteam"><div class="wp-block-embed__wrapper">
https://brainsteam.co.uk/2024/03/17/moving-from-gitea-to-forgejo-including-actions-with-docker-compose
</div></figure>
<!-- /wp:embed -->
<!-- wp:embed {"url":"https://brainsteam.co.uk/2024/03/30/edinburgh-2024","type":"wp-embed","providerNameSlug":"brainsteam"} -->
<figure class="wp-block-embed is-type-wp-embed is-provider-brainsteam wp-block-embed-brainsteam"><div class="wp-block-embed__wrapper">
https://brainsteam.co.uk/2024/03/30/edinburgh-2024
</div></figure>
<!-- /wp:embed -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Films, Books & Games</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>In March I finished <a href="https://bookwyrm.social/book/93151/s/doors-of-eden">The Doors of Eden by Adrian Tchaikovsky</a> and started re-reading the Dune books. My book choice was largely inspired by seeing Dune: Part 2 at the cinema which I thoroughly enjoyed. The first time I read Dune I wasn't that impressed (but I was about 16 at the time). This time around, I am really enjoying it. I wonder if it is partly due to the fact that when I was younger <a href="https://brainsteam.co.uk/2023/11/03/living-with-aphantasia/">I didn't have any visual reference</a>. Now, having seen the films, perhaps I can better contextualise the things i read. We also went to see <a href="https://www.imdb.com/title/tt21235248/">Ghostbusters: Frozen Empire</a> which I thoroughly enjoyed. I was surprised to see so much of <a href="https://en.wikipedia.org/wiki/James_Acaster">James Acaster </a>who normally only seems to get bit parts in things.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I've not really been doing much gaming in the last few weeks. I took my Steam Deck to Edinburgh with me but didn't touch it at all. Instead, I have been reading and have been listening to a lot of <a href="https://pod.link/1564066507">Three Bean Salad</a>. I am especially enjoying the beans' new film review episodes. I most recently listened to the Dune episode to stay on theme.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Hopes for April</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>We're now in full daylight savings mode in the UK, the days are longer and we've even had some sunshine in the last few days. I've already been out in the garden a bit in the first few days of this month and I hope to continue this trend in April and finally get our garden tamed. Apart from that we've got a slightly quieter month ahead which I am grateful for since March was action packed and May promises to be similarly busy. I'm also looking forward to <a href="https://www.cineworld.co.uk/static/en/uk/blog/action-season-cineworld-five-pound-tickets-april">Cineworld Action Movie Season</a> this month which will allow Mrs R and I to go and see some of our favourite 90s action films on the big screen. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->

275
brainsteam/content/posts/2024/04/20/Self-hosting Llama 3 on a home server.md Normal file
View File

@ -0,0 +1,275 @@
---
categories:
- AI and Machine Learning
date: '2024-04-20 19:38:12'
draft: false
tags:
- AI
- llama
- llms
- nlp
- self-hosting
title: Self-hosting Llama 3 on a home server
type: posts
---
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Self-hosting Llama 3 as your own ChatGPT replacement service using a 10 year old graphics card and open source components.</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Last week Meta<a href="https://ai.meta.com/blog/meta-llama-3/"> launched Llama 3</a>, the latest in their open source LLM series. Llama 3 is particularly interesting because the 8 billion parameter model, which is small enough to run on a laptop, <a href="https://news.ycombinator.com/item?id=40084699">performs as well as models 10x bigger than it</a>. The responses it provides are as good as GPT-4 for many use cases.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I finally decided that this was motivation enough to dig out my old Nvidia Titan X card from the loft and slot it into my home server so that I could stand up a ChatGPT clone on my home network. In this post I explain some of the pros and cons of self-hosting llama 3 and provide configuration and resources to help you do it too.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">How it works</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>The model is served by <a href="https://ollama.com/">Ollama</a> which is a GPU-enabled open source service for running LLMs as a service. Ollama makes heavy use of llama.cpp, t<a href="https://brainsteam.co.uk/2023/09/30/turbopilot-obit/">he same tech that I used to build turbopilot</a> around 1 year ago. The frontend is powered by <a href="https://github.com/open-webui/open-webui">OpenWebUI</a> which provides a ChatGPT-like user experience for interacting with Ollama models.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I use <a href="https://docs.docker.com/compose/">docker compose</a> to run the two services and wire them together and I've got a Caddy web server set up to let in traffic from the outside world.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":2430,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/Ollama-1024x350_91a2d262.png" alt="Drawing of the setup as described above. Caddy brokers comms with the outside world over https and feeds messages to OpenWebUI" class="wp-image-2430"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Hardware</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>My setup is running on a cheap and cheerful AMD CPU and Motherboard package and a 10 year old Nvidia Titan X card (much better GPUS are available on Ebay for around £150. The RTX 3060 with 12GB VRAM would be a great choice). My server has 32GB RAM but this software combo uses a lot less than that. You could probably get away with 16GB and run it smoothly or possibly even 8GB at a push. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>You could buy <a href="https://www.scan.co.uk/products/3xs-a520-home-bundle-amd-ryzen-5-5500-asus-tuf-a520-gaming-plus-wifi-16gb-ddr4-amd-wraith-stealth">this bundle</a> and a used RTX3060 on Ebay or a brand new one for around £250 and have a functional ChatGPT replacement in your house for less than £500.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Pros and Cons of Llama 3</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Llama 3 8B truly is a huge step forward for open source alternatives to relying on APIS from OpenAI, Anthropic and their peers. I am still in the early stages of working with my self-hosted Llama 3 instance but so far I'm finding that it is just as capable as GPT-4 in many arenas.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Pro: Price</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Self-hosting Llama 3 with Ollama and OpenWebUI is free-ish except for any initial investment you need to make for hardware and then electricity consumption. ChatGPT plus is currently $20/month but techies are likely also burning a similar amount in API calls too. I already had all the components for this build lying around the house but if I bought them 2nd hand it would take around 1 year for them to pay for themselves. That said, I could massively increase my API consumption through my self-hosted models since it's effectively "free".</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Pro: Privacy</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>A huge advantage of this approach is that you're not sending your data to an external company to be mined. The consumer version of ChatGPT that most people use is heavily data mined to improve OpenAI's models and anything that you type in may end up in their corpus. Ollama runs entirely on your machine and never sends data back to any third party company.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Pro: Energy Consumption and Carbon Footprint</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Another advantage is that since Llama 3:8B is small and it runs on a single GPU it uses a lot less energy to run than <a href="https://www.reddit.com/r/aipromptprogramming/comments/1212kmm/according_to_chatgpt_a_single_gpt_query_consumes/">an average query to ChatGPT</a>. My Titan X card consumes about 250 watts at max load but RTX 3060 cards only require 170 watts to run. Again, I had all the components lying around so I didn't buy anything new to make this server and indeed it means I won't be throwing away components that would otherwise become e-waste.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Con: Speed on old hardware</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Self-hosting Llama 3 8B on a Titan X is a little slower than ChatGPT but is still perfectly serviceable. It would almost certainly be faster on RTX 3 and 4 series cards.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Con: Multimodal Performance</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>The biggest missing feature for me is currently multi-modal support. I use GPT-4 to do handwriting recognition and transcription for me and current gen open source models aren't quite up to this yet. However, given the superb quality of Llama 3, I have no doubt that a similarly brilliant open multi-modal model is just around the corner.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Con: Training Transparency</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Although Llama 3's weights are free to download, the training corpus content is unknown. The model was built by Meta and thus is likely to have been trained on a large amount of user generated content and copyrighted content. Hosted third party models like ChatGPT are likely to be equally problematic in this regard but.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Setting up Llama 3 with Ollama and OpenWebUI</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Once you have the hardware assembled and the operating system installed, the fiddliest part is configuring Docker and Nvidia correctly. </p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Ubuntu</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>If you're on Ubuntu, you'll need to install docker first. I recommend using <a href="https://docs.docker.com/engine/install/ubuntu/">the guide from Docker themselves</a> which installs the latest and greatest packages. Then follow<a href="https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#installing-with-apt"> this guide</a> to install the nvidia runtime. Then you will want to verify that it's all set up using the checking step below.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Unraid</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I actually run Unraid on my home server rather than Ubuntu. To get things running there, simply install the <a href="https://forums.unraid.net/topic/98978-plugin-nvidia-driver/">unraid nvidia plugin</a> through the community apps page and make sure to stop and start docker before trying out the step below.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Checking the Docker and Nvidia Setup (All OSes)</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>To make sure that Docker and Nvidia are installed properly and able to talk to each other you can run:</p>
<!-- /wp:paragraph -->
<!-- wp:code -->
<pre class="wp-block-code"><code> docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi</code></pre>
<!-- /wp:code -->
<!-- wp:paragraph -->
<p>This runs the nvidia-smi status utility which should show what your GPU is currently doing but crucially it's doing so from inside docker which means that nvidia's container runtime is all set up to pass through the nvidia drivers to whatever you're running inside your container. You should see something like this:</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":2431,"sizeSlug":"full","linkDestination":"none"} -->
<figure class="wp-block-image size-full"><img src="/media/image-2_b8926720.png" alt="A screenshot of nvidia-smi output which shows the GPU name, how much power it is drawing, how much VRAM is in use and any processes using the card." class="wp-image-2431"/></figure>
<!-- /wp:image -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Installing Ollama</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Create a new directory and a new empty text file called <code>docker-compose.yml</code>. In that file paste the following:</p>
<!-- /wp:paragraph -->
<!-- wp:code -->
<pre class="wp-block-code"><code>ersion: "3.0"
services:
ui:
image: ghcr.io/open-webui/open-webui:main
restart: always
ports:
- 3011:8080
volumes:
- ./open-webui:/app/backend/data
environment:
# - "ENABLE_SIGNUP=false"
- "OLLAMA_BASE_URL=http://ollama:11434"
ollama:
image: ollama/ollama
restart: always
ports:
- 11434:11434
volumes:
- ./ollama:/root/.ollama
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
</code></pre>
<!-- /wp:code -->
<!-- wp:paragraph -->
<p>We define the two services and we provide both with volume mounts to enable them to persist data to disk (such as models you downloaded and your chat history).</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>For now we leave ENABLE_SIGNUP commented out so that you can create an account in the web ui but later we can come back and turn that off so that internet denizens can't sign up to use your chat.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Turn on Ollama</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>First we will turn on ollama and test it. Start by running <code>docker-compose up -d ollama</code>. (Depending on which version of docker you are running you might need to run <code>docker compose</code> rather than <code>docker-compose</code>). This will start just the ollama model server. We can interact with the model server by running an interactive chat session and downloading the model:</p>
<!-- /wp:paragraph -->
<!-- wp:code -->
<pre class="wp-block-code"><code>docker-compose exec ollama ollama run llama3:8b</code></pre>
<!-- /wp:code -->
<!-- wp:paragraph -->
<p>In this command the first <code>ollama</code> refers to the container and <code>ollama run llama3:8b</code> is the command that will be executed inside the container. If all goes well you will see the server burst into action and download the llama3 model if this is the first time you've run it. You'll then be presented with an interactive prompt where you'll be able to chat to the model.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":2432,"sizeSlug":"full","linkDestination":"none"} -->
<figure class="wp-block-image size-full"><img src="/media/image-3_980b31e5.png" alt="Screenshot showing the interactive prompt. I have entered hello and the model has responded "Hello it's nice to meet you. IS there something I can help you with or would you like to chat?"" class="wp-image-2432"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>You can press CTRL+D to quit and move on to the next step.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Turn on the Web UI</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Now we will start up the web ui. Run <code>docker-compose up -d ui</code>. Now open up your browser and go to http://localhost:3011/ to see the web ui. You will need to register for an account and log in. After which you will be able to interact with the model like so:</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":2429,"width":"760px","height":"auto","sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large is-resized"><img src="/media/image-1-1024x566_8521a3e4.png" alt="A screenshot of the web ui. I have asked the model what noise a fox makes." class="wp-image-2429" style="width:760px;height:auto"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">(Optional) Configure Outside Access</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>If you want to be able to chat to your models from the outside world you might want to stand up a reverse proxy to your server. If you're new to self hosting and you're not sure about how to do this, a safer option is probably to use <a href="https://tailscale.com/">Tailscale</a> to build a VPN which you can use to securely connect to your home network without risking exposing your systems to the public and/or hackers.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Conclusion</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Llama 3 is a tremendously powerful model that is useful for a whole bunch of use cases including summarisation, creative brainstorming, code copiloting and more. The quality of the responses are in line with GPT-4 and it runs on much older, smaller hardware. Self-hosting Llama 3 won't be for everyone and it's quite technically involved. However, for AI geeks like me, running my own ChatGPT clone at home for next-to-nothing was too good an experiment to miss out on.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->

325
brainsteam/content/posts/2024/04/26/Can Phi3 and Llama3 Do Biology_.md Normal file
View File

@ -0,0 +1,325 @@
---
categories:
- AI and Machine Learning
date: '2024-04-26 13:41:42'
draft: false
tags:
- AI
- llms
- nlp
title: Can Phi3 and Llama3 Do Biology?
type: posts
---
<!-- wp:paragraph -->
<p>Small Large Language Model might sound like a bit of an oxymoron. However, I think it perfectly describes the class of LLMs in the 1-10 billion parameter range like Llama and Phi 3. In the last few days, Meta and Microsoft have both released these open(ish) models that can happily run on normal hardware. Both models perform surprisingly well for their size, competing with much larger models like GPT 3.5 and Mixtral. However, how well do they generalise to new unseen tasks? Can they do biology?</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Introducing Llama and Phi</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Meta's offering, Llama 3 8B, is an 8 billion parameter model that can be run on a modern laptop. It performs <a href="https://huggingface.co/MaziyarPanahi/Meta-Llama-3-8B-Instruct-GGUF/discussions/5">almost as well as Mixtral 8x22B mixture-of-expert model</a>, a model 22x bigger and compute intensive.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Microsoft's model, Phi 3 mini, is around half the size of Llama 3 8B at 3.8 billion parameters. It is small enoughthat it can run on a high end smartphone at a reasonable speed. Incredibly, <a href="https://arxiv.org/html/2404.14219v1">Phi actually beats Llama 3 8B</a>, which is twice as big, at a few popular benchmarks including MMLU which approximately measures "how well does this model behave as a chatbot" and HumanEval which measures "how well can this model write code?".</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I've also read a lot of anecdotal evidence about people chatting to these models and finding them quite engaging and useful chat partners (as opposed to previous generation small models). This seems to back up the benchmark performance and provide some validation of the models' utility.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Both Microsoft and Meta have stated that the key difference between these models and previous iterations of their smaller LLMs is the training regime. Interestingly, both companies applied very different training strategies . Meta trained Llama3 on<a href="https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md#training-data"> over 15 trillion tokens (words)</a> which is unusually large for a small model. Microsoft trained Phi on <a href="https://www.microsoft.com/en-us/research/publication/textbooks-are-all-you-need/">much smaller training sets curated for high quality</a>.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Can Phi, Llama and other Small Models Do Biology?</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Having a model small enough to run on your phone and generate funny poems or trivia questions is neat. However, for AI and NLP practitioners, a more interesting question is "do these models generalise well to new, unseen problems?"</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I set out to gather some data about how well Phi and Llama 3 8B generalise to a less-well-known task. As it happened, I have recently been working with my friend <a href="https://twitter.com/drdanponders/">Dan Duma</a> on a test harness for BioAsq Task B. This is a less widely-known, niche NLP task in the bio-medical space. The model is fed a series of snippets from scientific papers and asked a question which it must answer correctly. There are four different formats of question which I'll explain below.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The 11th BioASQ Task B <a href="http://participants-area.bioasq.org/results/11b/phaseB/">leaderboard</a> is somewhat dominated by <a href="https://ceur-ws.org/Vol-3497/paper-009.pdf">GPT-4 entrants</a> with perfect scores at some of the sub-tasks. If you were somewhat cynical, you might consider this task "solved". However, we think it's an interesting arena for testing how well smaller models are catching up to big commercial offerings.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>BioASQ B is primarily a reading comprehension task with a slightly niche subject-matter. The models under evaluation are unlikely to have been explicitly trained to answer questions about this material. Smaller models are often quite effective at these sorts of <a href="https://aws.amazon.com/what-is/retrieval-augmented-generation/">RAG</a>-style problems since they do not need to have internalised lots of facts and general information. In fact, in <a href="https://arxiv.org/html/2404.14219v1">their technical report</a>, the authors of Phi-3 mini call out the fact that their model can't retain factual information but could be augmented with search to produce reasonable results. This seemed like a perfect opportunity to test it out.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">How The Task Works</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>There are 4 types of question in task B. Factoid, Yes/No, List and Summary. However, since summary is quite tricky to measure, it is not part of the BioASQ leaderboard. We also chose to omit summary from our tests.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Each question is provided along with a set of snippets. These are full sentences or paragraphs that have been pre-extracted from scientific papers. Incidentally, that activity is BioASQ <a href="http://participants-area.bioasq.org/Tasks/">Task A</a> and it requires a lot more moving parts since there's retrieval involved too. In Task B we are concerned with existing sets of snippets and questions only.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>In each case the model is required to respond with a short and precice exact answer to the question. The model may optionally also provide an ideal answer which provides some rationale for that answer. The ideal answer may provide useful context for the user but is not formally evaluated as part of BioASQ.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Yes/No questions require an exact answer of just "yes" or "no". For List questions, we are looking for a list of named entities (for example symptoms or types of microbe). For factoid we are typically looking for a single named entity. Models are allowed to respond to factoids with multiple answer. Therefore, factoids answers are scored by how closely to the top of the list the "correct" answer is ranked.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The Figure from the <a href="https://ceur-ws.org/Vol-3497/paper-009.pdf">Hseuh et al 2023 Paper </a> below illustrates this quite well:</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":2510,"sizeSlug":"full","linkDestination":"none"} -->
<figure class="wp-block-image size-full"><img src="/media/image-8_8c2824b9.png" alt="Examples of different question types. Full transcriptions of each are:
Yes/No
Question: Proteomic analyses need prior knowledge of the organism complete genome. Is the complete genome of the bacteria of the genus Arthrobacter available?
Exact Answer: yes
Ideal Answer: Yes, the complete genome sequence of Arthrobacter (two strains) is deposited in GenBank.
List
Question: List Hemolytic Uremic Syndrome Triad.
Exact Answer: [anaemia, thrombocytopenia, renal failure]
Ideal Answer: Hemolytic uremic syndrome (HUS) is a clinical syndrome characterized by the triad of anaemia, thrombocytopenia, renal failure.
Factoid
Question: What enzyme is inhibited by Opicapone?
Exact Answer: [catechol-O-methyltransferase]
Ideal Answer: Opicapone is a novel catechol-O-methyltransferase (COMT) inhibitor to be used as adjunctive therapy in levodopa-treated patients with Parkinson's disease
Summary
Question: What kind of affinity purification would you use in order to isolate soluble lysosomal proteins?
Ideal Answer: The rationale for purification of the soluble lysosomal proteins resides in their characteristic sugar, the mannose-6-phosphate (M6P), which allows an easy purification by affinity chromatography on immobilized M6P receptors." class="wp-image-2510"/><figcaption class="wp-element-caption">Figure 1 from the <a href="https://ceur-ws.org/Vol-3497/paper-009.pdf">Hseuh et al 2023 Paper </a>illustrates the different task types succinctly</figcaption></figure>
<!-- /wp:image -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Our Setup</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>We wrote a python script that passes the question, context and guidance about the type of question to the model. We used a<a href="https://github.com/ollama/ollama/issues/3616"> patched version of Ollama</a> that allowed us to put restrictions on the shape of the model output. This allowed us to ensure responses were valid JSON in the same shape and structure as the BioASQ examples. These forced grammars saved us loads of time trying to coax JSON out of models in the structure we want. This is something that smaller models aren't great at. Sometimes models would still fail to give valid responses. For example, sometimes they get stuck in infinite loops spitting out brackets or newlines. We gave models up to 3 chances to produce a JSON response before a question is marked unanswerable and skipped.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Prompts</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>We used exactly the same prompts for all of the models which may have left room for further performance improvements. The exact prompts and grammar constraints that we used <a href="https://memos.jamesravey.me/m/6LnBLQNihaS6FtPkcbBx4Z">can be found here</a>. Snippets are concatenated together with newlines in between them and provided as "context" in the prompt template.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We used<a href="https://github.com/BioASQ/Evaluation-Measures/tree/master"> the official BioASQ scoring tool </a>to evaluate the responses and produce the results below. We evaluated our pipeline on the <a href="http://participants-area.bioasq.org/Tasks/11b/goldenDataset/">Task 11B Golden Enriched test set</a>. You have to create a free account at bioasq to log in and download the data. </p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Models</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>We compared quantized versions of Phi and Llama with some other similarly sized models which perform well at benchmarks.</p>
<!-- /wp:paragraph -->
<!-- wp:list {"className":"ticss-f3e747da","hasCustomCSS":true,"customCSS":"li a {\n text-decoration: underline;\n}\n"} -->
<ul class="ticss-f3e747da"><!-- wp:list-item {"className":"ticss-4713a356","hasCustomCSS":true,"customCSS":"a {\n text-decoration: underline;\n}\n"} -->
<li class="ticss-4713a356"><a href="https://ollama.com/library/llama3:8b"><span style="text-decoration: underline;">Llama 3 8B</span></a></li>
<!-- /wp:list-item -->
<!-- wp:list-item {"className":"ticss-4713a356","hasCustomCSS":true,"customCSS":"a {\n text-decoration: underline;\n}\n"} -->
<li class="ticss-4713a356"><a href="https://ollama.com/library/phi3"><span style="text-decoration: underline;">Phi 3 Mini 3.8B </span></a></li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li><span style="text-decoration: underline;"><a href="https://ollama.com/library/mistral:7b">Mistral 7B</a></span></li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li><span style="text-decoration: underline;"><a href="https://ollama.com/library/starling-lm:7b">Starling LM 7B</a></span></li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li><span style="text-decoration: underline;"><a href="https://ollama.com/library/zephyr:7b">Zephyr 7B</a></span></li>
<!-- /wp:list-item --></ul>
<!-- /wp:list -->
<!-- wp:paragraph -->
<p>Note that although Phi is approx. half the size of the other models, the authors report <a href="https://arxiv.org/html/2404.14219v1#S3">competitive results against much larger models </a>for a number of widely used benchmarks so it seems reasonable to compare it with these 7B and 8B models as oppose to only benchmarking against other 4B and smaller models.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Results</h3>
<!-- /wp:heading -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Yes/No Questions</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>The simplest type of BioASQ question is Yes/No. These results are measured with macro F1 to allow us to get a single metric across the performance at both "yes" and "no" questions.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":2502,"sizeSlug":"full","linkDestination":"none"} -->
<figure class="wp-block-image size-full"><img src="/media/image-6_bf5a57f4.png" alt="Diagram of Yes/No F1
Llama3 gets 1.0
Mistral gets 0.8
Phi gets 0.7
Starling gets 0.9
Zehpyr gets 0.85
The bars on the chart have little range indicators because they represent the average values over 4 sets of results." class="wp-image-2502"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>The results show that all 5 models perform reasonably well at this task but Phi 3 lags behind the others a little bit, but only by about 10% next to it's closest competitor. <a href="http://participants-area.bioasq.org/results/11b/phaseB/">The best solutions to this task</a> are coming in at 1.0 F1. Llama3 and Starling both achieve pretty close to perfect results here.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Factoid Questions</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>For factoid answers we measure responses in <a href="https://en.wikipedia.org/wiki/Mean_reciprocal_rank">MRR</a> since the model can return multiple possible answers. We are interested in how close the right answers are to the top of the list.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":2501,"sizeSlug":"full","linkDestination":"none"} -->
<figure class="wp-block-image size-full"><img src="/media/image-5_dec5deb6.png" alt="Factoid results
Llama gets roughly 0.55 MRR
Mistral gets rouglhy 0.05 MRR
Phi 3 gets roughly 0.15 MRR
Starling gets roughly 0.17 MRR
Zephyr gets roughly 0.12 MRR
The bars on the chart have little range indicators because they represent the average values over 4 sets of results." class="wp-image-2501"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>This graph is a lot starker than the yes/no graph. Llama 3 outperforms it's next closest neighbour by a significant margin (roughly +0.40 MRR) . The best solution to this task, again <a href="https://arxiv.org/abs/2306.16108">a GPT-4-based entrant</a>, weighs in at 0.6316 MRR so it's pretty impressive that Llama 3 8B is providing results in the same ballpark as a model many times larger. For this one, Phi is in third place after Starling-LM 7B. Again, given that Phi is half the size of this model, it's quite impressive performance.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">List Questions</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>We measure list questions in F1. A false positive is when something irrelevant is included in the answer and a false negative is when something relevant is missed from an answer. F1 gives us a single statistic that balances the two.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":2503,"sizeSlug":"full","linkDestination":"none"} -->
<figure class="wp-block-image size-full"><img src="/media/image-7_7c86d083.png" alt="Llama 3 gets roughly 0.45 F1
Mistral gets roughly 0.21 F1
Phi gets roughly 0.05 F1
Starling gets roughly 0.27 F1
Zephyr gets roughly 0.32 F1
The bars on the chart have little range indicators because they represent the average values over 4 sets of results." class="wp-image-2503"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>This one was a little surprising to me as Phi does a lot worse than any of its counterparts. We noticed that Phi produced a much higher rate of unanswerable questions than any of the other models which may be due to the somewhat complex JSON structure required by list type questions. It may be worth re-testing with different formatting arrangements to see if the failure to format the model masks reasonable performance at the task. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Llama 3 8B wins again. The current best solution, again <a href="https://ceur-ws.org/Vol-3497/paper-011.pdf">a GPT-4-based system</a>, achieves an F1 of 0.72 so even Llama 3 8B leaves a relatively wide gap here. It would be worth testing the larger variants of Llama 3 to see how well they perform at this task and whether they are competitive with GPT-4.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Discussion and Conclusion</h2>
<!-- /wp:heading -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Llama3</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>We've seen that Llama 3 8B and, to a lesser extent, Phi 3 Mini, are able to generalise reasonably well to a reading comprehension task in a field that wasn't a primary concern for either se of model authors. This isn't conclusive evidence for or against the general performance of these models on unseen tasks. However, it is certainly an interesting data point that shows that, particularly Llama 3 really is competitive with much larger models at this task. I wonder if that's because it was trained on such a large corpus which may have included some biomedical content as part of its training corpus. </p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Phi</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I'm reluctant to too-harshly critique Phi's reasoning and reading comprehension ability since there's a good chance that it was disadvantaged by our test setup and the forced JSON structure, particularly for the list task. However, the weaker performance at the yes/no questions may be a hint that it isn't quite as good at generalised reading comprehension as the competing larger models.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We know that Phi3, like it's predecessor was trained on data that <em>"consists of heavily filtered web data (according to the “educational level”) from various open internet sources, as well as synthetic LLM-generated data."</em> However, we don't know specifically what was included or excluded. If Llama 3 went for "cast the net wide" approach to data collection, it's likely that the latter model may have been exposed to more biomedical content "by chance" and thus be better at reasoning about concepts that perhaps Phi had never seen before.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I do want to again call out that Phi is approximately half the size of the next biggest model in our benchmark so it's performance is quite impressive in that light.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Further Experiments</h3>
<!-- /wp:heading -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Model Size</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I won't conjecture about whether 3.8B parameters is "too small" to generalise given the issues mentioned above but I'd love to see some more tests of this in future. Do the larger variants of Phi (trained on the same data but simpliy with more parameters) suffer from the same issues?</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Model Fine Tuning</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>The models that I've been testing are small enough that they can be <a href="https://duarteocarmo.com/blog/fine-tune-llama-2-telegram">fine-tuned on specific problems on a consumer-grade gaming GPU for very little cost</a>. It seems entirely plausible to me that by fine-tuning these models on biomedical text ands historical BioASQ training sets their performance could be improved even more significantly. The challenge would be in finding the right mix of data.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Better Prompts</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>We did not spend a lot of time attempting to build effective prompts during this experiment. It may be that performance was left on the table because of this oversight. Smaller models are often quite fussy about prompts. It might be interesting to use a prompt optimisation framework like <a href="https://dspy-docs.vercel.app/">DSPy</a> to be more systematic about better prompts.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Other Tasks</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I tried these models on BioAsq but this is lightyears away from conclusive evidence for whether or not these new-generation small models can generalise well. It's simply a test of whether they can do biology. It will be very interesting to try other novel tasks and see how well they perform. Watch this space!</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->

184
brainsteam/content/posts/2024/05/01/LLMs Can_t Do Probability.md Normal file
View File

@ -0,0 +1,184 @@
---
categories:
- AI and Machine Learning
date: '2024-05-01 10:13:15'
draft: false
tags: []
title: LLMs Can't Do Probability
type: posts
---
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I've seen a couple of recent posts where the writer mentioned asking LLMs to do something with a certain probability or a certain percentage of the time. There is a particular example that stuck in my mind which I've since lost the link to (If you're the author, please get in touch so I can link through to you):</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The gist is that the author built a Custom GPT with educational course material and then put in the prompt that their bot should lie about 20% of the time. They then asked the students to chat to the bot and try to pick out the lies. I think this is a really interesting, lateral thinking use case since the kids are probably going to use ChatGPT anyway. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The thing that bothered me is that transformer-based LLMs don't know how to interpret requests for certain probabilities of outcomes. We already know that <a href="https://www.reddit.com/r/ChatGPT/comments/1cfxt3v/chatgpt_reflects_human_biases_when_choosing_a/">ChatGPT reflects human bias when generating random numbers</a>. But, I decided to put it to the test with making random choices.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Testing Probability in LLMS</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I prompted the models with the following:</p>
<!-- /wp:paragraph -->
<!-- wp:quote -->
<blockquote class="wp-block-quote"><!-- wp:paragraph -->
<p>You are a weighted random choice generator. About 80% of the time please say 'left' and about 20% of the time say 'right'. Simply reply with left or right. Do not say anything else</p>
<!-- /wp:paragraph --></blockquote>
<!-- /wp:quote -->
<!-- wp:paragraph -->
<p>And I ran this 1000 times through some different models. Random chance is random (profound huh?) so we're always going to get some deviation from perfect odds but we're hoping for roughly 800 'lefts' and 200 'rights' - something in that ballpark.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Here are the results:</p>
<!-- /wp:paragraph -->
<!-- wp:table -->
<figure class="wp-block-table"><table><tbody><tr><td><strong>Model</strong></td><td><strong>Lefts</strong></td><td><strong>Rights</strong></td></tr><tr><td>GPT-4-Turbo</td><td>999</td><td>1</td></tr><tr><td>GPT-3-Turbo</td><td>975</td><td>25</td></tr><tr><td>Lllama-3-8B</td><td>1000</td><td>0</td></tr><tr><td>Phi-3-3.8B</td><td>1000</td><td>0</td></tr></tbody></table></figure>
<!-- /wp:table -->
<!-- wp:heading -->
<h2 class="wp-block-heading"></h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>As you can see, LLMs seem to struggle with probability expressed in the system prompt. It almost always answers left even though we asked it to only do so 80% of the time. I didn't want to burn lots of $$$ asking GPT-3.5 (which did best in the first round) to reply with single word choices to silly questions but I tried a couple of other combinations of words to see how it affects things. This time I only ran each 100 times.</p>
<!-- /wp:paragraph -->
<!-- wp:table -->
<figure class="wp-block-table"><table><tbody><tr><td><strong>Choice (Always 80% / 20%)</strong></td><td><strong>Result</strong></td></tr><tr><td>Coffee / Tea</td><td>87/13</td></tr><tr><td>Dog / Cat</td><td>69/31</td></tr><tr><td>Elon Musk/Mark Zuckerberg</td><td>88/12</td></tr></tbody></table><figcaption class="wp-element-caption">Random choices from GPT-3.5-turbo</figcaption></figure>
<!-- /wp:table -->
<!-- wp:paragraph -->
<p>So what's going on here? Well, the models have their own internal weighting to do with words and phrases that is based on the training data that was used to prepare them. These weights are likely to be influencing how much attention the model pays to your request. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>So what can we do if we want to simulate some sort of probabilistic outcome? Well we could use a Python script to randomly decide whether or not to send one of two prompts:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"python"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import random
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, SystemMessage
choices = (['prompt1'] * 80) + (['prompt2'] * 20)
# we should now have a list of 100 possible values - 80 are prompt1, 20 are prompt2
assert len(choices) == 100
# randomly pick from choices - we should have the odds we want now
chat = ChatOpenAI(model="gpt-3.5-turbo")
if random.choice(choices) == 'prompt1':
r = chat.invoke(input=[SystemMessage(content="Always say left and nothing else.")])
else:
r = chat.invoke(input=[SystemMessage(content="Always say right and nothing else.")])</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Conclusion</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>How does this help non-technical people who want to do these sorts of use cases or build Custom GPTs that reply with certain responses? Well it kind of doesn't. I guess a technical-enough user could build a CustomGPT that uses <a href="https://platform.openai.com/docs/guides/function-calling">function calling</a> to decide how it should answer a question for a "spot the misinformation" pop quiz type use case.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>However, my broad advice here is that you should be very wary of asking LLMs to behave with a certain likelihood unless you're able to control that likelihood externally (via a script).</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>What could I have done better here? I could have tried a few more different words, different distributions (instead of 80/20) and maybe some keywords like "sometimes" or "occasionally".</p>
<!-- /wp:paragraph -->
<!-- wp:separator -->
<hr class="wp-block-separator has-alpha-channel-opacity"/>
<!-- /wp:separator -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Update 2024-05-02: Probability and Chat Sessions</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Some of the feedback I received about this work asked why I didn't test multi-turn chat sessions as part of my experiments. Some folks hypothesise that the model will always start with one or the other token unless the temperature is really high. My original experiment does not give the LLM access to its own historical predictions so that it can see how it behaved previously. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>With true random number generation you wouldn't expect the function to require a list of historical numbers so that it can adjust it's next answer (although if we're getting super hair splitty I should probably point out that pseudo-random number generation does depend on a historical 'seed' value). </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The point of this article is that LLMs definitely are not doing true random number generation so it is interesting to see how conversation context affects behaviour.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I ran a couple of additional experiments. I started with the prompt above and instead of making single API calls to the LLM I start a chat session where each turn I simply say "Another please". It looks a bit like this:</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:quote -->
<blockquote class="wp-block-quote"><!-- wp:paragraph -->
<p>System: You are a weighted random choice generator. About 80% of the time please say ‘left’ and about 20% of the time say ‘right’. Simply reply with left or right. Do not say anything else<br><br>Bot: left<br><br>Human: Another please<br><br>Bot: left<br><br>Human: Another please</p>
<!-- /wp:paragraph --></blockquote>
<!-- /wp:quote -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I ran this once per model for 100 turns and also 10 times per model for 10 turns. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><strong><em>NB: I excluded Phi from both of these experiments as in both test cases, it ignored my prompt to reply with one word and started jibbering.</em></strong></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">100 Turns Per Model</h3>
<!-- /wp:heading -->
<!-- wp:table -->
<figure class="wp-block-table"><table><tbody><tr><td>Model</td><td># Left</td><td># Right</td></tr><tr><td>GPT 3.5 Turbo</td><td>49</td><td>51</td></tr><tr><td>GPT 4 Turbo</td><td>95</td><td>5</td></tr><tr><td>Llama 3 8B</td><td>98</td><td>2</td></tr></tbody></table></figure>
<!-- /wp:table -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">10 Turns, 10 time per model</h3>
<!-- /wp:heading -->
<!-- wp:table -->
<figure class="wp-block-table"><table><tbody><tr><td>Model</td><td># Left</td><td># Right</td></tr><tr><td>GPT 3.5 Turbo</td><td>61</td><td>39</td></tr><tr><td>GPT 4 Turbo</td><td>86</td><td>14</td></tr><tr><td>Llama 3 8B</td><td>71</td><td>29</td></tr></tbody></table></figure>
<!-- /wp:table -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Interestingly the series of 10 shorter conversations gets us closest to the desired probabilities that we were looking for but all scenarios still yield results inconsistent with the ask from the prompt.</p>
<!-- /wp:paragraph -->

92
brainsteam/content/posts/2024/05/07/LLMs_ To Fine-Tune or Not to Fine-Tune_.md Normal file
View File

@ -0,0 +1,92 @@
---
categories:
- AI and Machine Learning
- Data Science
date: '2024-05-07 11:19:07'
draft: false
tags:
- AI
- llms
- nlp
title: 'LLMs: To Fine-Tune or Not to Fine-Tune?'
type: posts
---
<!-- wp:paragraph -->
<p>Knowing when to fine-tune LLMs and when to use an off-the-shelf model is a tricky question. New research can help shed a light on when each approach makes more sense and eke more performance out of off-the-shelf models without fine-tuning them.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">When Fine-Tuning Beats GPT-4</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Modern LLMs show impressive performance at a range of tasks out of the box. Even small models like the recent Llama 3: 8B <a href="https://brainsteam.co.uk/2024/04/26/can-phi-and-llama-do-biology/">show excellent performance at unseen tasks</a>. However, a <a href="https://arxiv.org/pdf/2405.00732">recent preprint</a> from the research team at <a href="https://predibase.com/">Predibase</a> shows that small models can match and even out-perform GPT-4 when they are fine-tuned for specific tasks. Figure 5 from the paper shows a list of the tasks that were evaluated and the relative performance difference vs GPT-4.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":2609,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/image-1024x706_78da9ced.png" alt="Figure 5: Performance lift from the best fine-tuned LLM over 1) the best base model (<=
7B) (in blue) and GPT-4 (in red) across 31 tasks, in absolute points." class="wp-image-2609"/><figcaption class="wp-element-caption">Figure 5 from <a href="https://arxiv.org/pdf/2405.00732">Zhao et al. 2024</a></figcaption></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>The authors note that fine-tuned models consistently outperform GPT-4 at specific, narrowly-scoped tasks like classification and information extraction. GPT-4 came out winning where tasks are more complex and open ended like generating code and multi-lingual language understanding.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Fine tuning these small models is cheaper and easier than ever before. Llama 3 can be <a href="https://medium.com/@sumandas0/fine-tune-llama3-on-million-scale-dataset-in-consumer-gpu-using-qlora-deepspeed-3ae8ad75299a">fine-tuned on consumer GPUs</a> and once tuned, <a href="https://brainsteam.co.uk/2024/04/20/self-hosting-llama-3-on-a-home-server/">they can be run on modest systems</a> with ease. All of this comes with one relatively large. caveat. You need techies with at least a veneer of machine learning and MLOps expertise to do it. If you have a team with DevOps/MLOps capability you may find that this option works really well for you. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Interestingly the paper came out just after Llama 3 and Phi 3. Therefore these models are not included in the evaluation but may offer even better fine-tuned performance than their prevous-generation counterparts.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Is ICL a Good Alternative to Fine-tuning?</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p><a href="https://arxiv.org/pdf/2405.00200">Another preprint</a> published 1 day later by researchers at CMU shows that you can get performance competitive with fine-tuned models with in-context-learning (ICL). That is to say, providing a lot of example input and output pairs in your prompt can give the same sort of performance boost as fine-tuning a model. If that's the case then why bother fine-tuning? Well including huge amounts of context with every inference is going to significantly drive up API costs. It's also going to mean that each model call takes much longer. As the authors of the paper say:</p>
<!-- /wp:paragraph -->
<!-- wp:quote -->
<blockquote class="wp-block-quote"><!-- wp:paragraph -->
<p>...Finetuning has higher upfront cost but allows for reduced inference-time cost... finetuning on 4096 examples may still be preferable to prompting with 1000 if efficiency of inference is a major priority...</p>
<!-- /wp:paragraph --><cite><a href="https://arxiv.org/pdf/2405.00200">Bertsch et al, 2024</a></cite></blockquote>
<!-- /wp:quote -->
<!-- wp:paragraph -->
<p>This may be an acceptable tradeoff if you are operating offline (e.g. your user is not actively waiting for a response). For example, using models to classify, summarise or extract information. However, this could be a bit of a UX vibe-killer in chat applications.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">DSPy: A Wildcard</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Another alternative related to in-context-learning is meta-learning. Meta-learning uses frameworks like <a href="https://dspy-docs.vercel.app/">DSPy</a> to automatically find the best prompt for your task. It does this by systematically searching your dataset for example inputs and outputs and measuring performance. This may help you to find a compromise between context length and model performance automatically. DSPy will attempt to find the most educational training samples from your dataset whilst keeping your context short. DSPy is a little involved to get started. However, once you've got it plugged into your framework you can automatically have DSPy optimise your prompts for different models. This makes switching between API providers a doddle.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Conclusion</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Both fine-tuning and providing a large number of examples in the prompt are reasonable ways to significantly boost the out-of-the-box performance of models at specific tasks. If you have a data science team in-house, a very large training dataset or require very fast inference, fine-tuning may be the way to go. Alternatively if you want to boost your model's performance without much up-front investment, it may be worth playing with in-context-learning or meta-prompting to try to improve things. However, expect slower inference speeds and higher API bills if you adopt this approach at scale.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Of course all of these approaches require you to have a well defined task, a way to measure performance at said task and some example inputs and outputs. If you're just starting out and you haven't yet built much of a dataset then zero or few-shot learning (where you describe the task in the prompt and optionally provide a handful of input/output examples) is the way to go.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->

464
brainsteam/content/posts/2024/07/08/Ditch that ChatGPT Subscription_ Moving to Pay-as-you-Go AI usage with Open Web UI.md Normal file
View File

@ -0,0 +1,464 @@
---
categories:
- AI and Machine Learning
- Software Development
date: '2024-07-08 10:09:36'
draft: false
tags:
- AI
- docker
- llms
- nlp
- self-hosting
title: 'Ditch that ChatGPT Subscription: Moving to Pay-as-you-Go AI usage with Open
Web UI'
type: posts
---
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Introduction</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>In the world of AI assistants, subscription services like <a href="https://openai.com/index/chatgpt-plus/">ChatGPT Plus</a>, <a href="https://www.anthropic.com/news/claude-pro">Claude Pro</a> and <a href="https://one.google.com/about/ai-premium/">Google One AI </a>have become increasingly popular amongst knowledge workers. However these subscription services may not be the most cost-effective or flexible solution for everyone and the not-insignificant fees encourage users to stick to one model that the've already paid for rather than trying out different options.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I previously wrote about<a href="https://brainsteam.co.uk/2024/04/20/self-hosting-llama-3-on-a-home-server/"> how you can self-host Llama3</a> on a machine with an older graphics card using open source tools. In this post, I demonstrate how to expand this setup to allow you to interact with OpenAI, Anthropic, Google and others alongside your local models via a single, self-hosted UI.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Why ditch my subscription?</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Hosting your own AI user interface allows for more granular cost control, potentially saving money for lighter users of commercial models (are you really getting $20/mo of usage out of ChatGPT?). It grants you greater flexibility in choosing and comparing different AI models without worrying about which subscription to spend your $20 on this month or forking out for more than one at a time. Additionally, commercial AI providers usually provide <a href="https://www.reddit.com/r/ChatGPT/comments/1d80onp/pro_tip_the_chatgpt_api_is_still_up_and_working/">a more stable experience via their APIs </a>since business users, the target audience of the API offerings, tend to have more leverage than consumers when it comes to service-level-agreements. API terms & conditions around data usage are normally better too (within ChatGPT, data collection is opt-out and you lose functionality when you do it) .</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Self-hosting your chat interface with Open Web UI brings enhtanced privacy through a local UI, the capability to run prompts through multiple models simultaneously, and the freedom to use different models for different tasks without being locked into a single provider. You can build your own bespoke catalogue of commercial and local models (<a href="https://brainsteam.co.uk/2024/04/20/self-hosting-llama-3-on-a-home-server/">like Llama-3</a>) and access them and their outputs all from one place. If you're already familiar with docker and docker-compose you can have something up and running in 15 minutes.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">When might this not work for me?</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>If you are a particularly heavy user of the service that you're subscribed to and you're not keen on trying other models you may find that moving to pay-as-you-go is more expensive for you. The <a href="https://www.penguin.co.uk/articles/2020/09/book-length-debate-fiction-long-novels">average length of a fiction book</a> is something like 60-90k words. It costs about $5 to have GPT-4o read 16 books (input/prompt) and $15 to have it write 16 books worth of output. If you're spending all day every day chatting to Jippy then you might find that you end up spending more than the $20/mo on API usage. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>If cost is your primary driver, you should factor in the cost of hosting your server too. If you are already running a homelab or a cheap VPS you might be able to run the web UI at "no extra charge" but if you need to spin up a new server just for hosting your web UI that's going to cut into your subscription savings.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>If privacy is your primary driver, you should be aware that this approach still involves sending data to third parties. If you want to avoid that all together you'll need to go fully self hosted.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Finally, fair warning: this process is technical, and you'll need to be familiar (or willing to learn about) Docker, yaml config files and APIs.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">The Setup</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>This setup builds on my previous post about Llama 3. Previously we used Open Web UI to provide a web app that allows you to talk to the LLM and we used Ollama to host Llama 3 locally and provide a system that Open Web UI could send requests to. </p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":3160,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/Self-host-AI-Setup-1024x566_f9a6b5e0.png" alt="The image is a flowchart illustrating a network architecture for handling both self-hosted and commercial Large Language Model (LLM) traffic. The components are as follows:
Web Traffic (Open Internet): Represented by an arrow entering the local network boundary.
Caddy: A web server handling incoming web traffic from the internet with arrows pointing to "chat.example.com" (Open Web UI) and "api.example.com" (LiteLLM).
Open Web UI: Handles self-hosted LLM traffic (LLama 3/Gemma 2) and forwards it to "Ollama". Also manages commercial LLM traffic (GPT-4/Claude) and routes it to "LiteLLM".
Ollama: Connected to Open Web UI, manages the self-hosted LLM traffic.
LiteLLM: Manages commercial LLM traffic and logs information including API credentials, interacting with "PostgreSQL" and "Third Party APIs".
PostgreSQL: Handles logging and API credential storage, connected to LiteLLM.
Third Party APIs: Interacts with LiteLLM for external API calls.
The local network boundary is highlighted with a green dashed line, encapsulating all internal components." class="wp-image-3160"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>This setup has five main components components:</p>
<!-- /wp:paragraph -->
<!-- wp:list {"ordered":true} -->
<ol><!-- wp:list-item -->
<li>Ollama - a system for running local language models on your computer, using your GPU or Apple silicon to provide reasonable response speeds</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>Open Web UI, an open source frontend for chatting with LLMs that supports Ollama but also any old OpenAI-compatible API endpoint.</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>LiteLLM Proxy which allows us to input our API keys and provides a single service that Open Web UI can call to access a whole bunch of commercial AI models</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>PostgreSQL - a database server which LiteLLM will use to store data about API usage and costs.</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>Caddy - used for reverse proxying HTTP traffic from the open web and routing it to Open Web UI or LiteLLM.</li>
<!-- /wp:list-item --></ol>
<!-- /wp:list -->
<!-- wp:paragraph -->
<p>The resulting docker-compose file will look something like this:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"yaml"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="yaml" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">version: "3.0"
services:
ui:
image: ghcr.io/open-webui/open-webui:main
restart: always
ports:
- 8080:8080
volumes:
- ./open-webui:/app/backend/data
environment:
# after you
# - "ENABLE_SIGNUP=false"
# if you disable ollama then remove the following line
- "OLLAMA_BASE_URL=http://ollama:11434"
# if you disable ollama then set the following to false
- "ENABLE_OLLAMA_API=true"
db:
image: postgres:15
restart: always
# set shared memory limit when using docker-compose
shm_size: 128mb
volumes:
- ./data/postgres:/var/lib/postgresql/data
environment:
POSTGRES_PASSWORD: somesecretpw
POSTGRES_DB: litellm
PGDATA: /var/lib/postgresql/data/pgdata
litellm:
image: ghcr.io/berriai/litellm:main-latest
restart: always
depends_on:
- db
ports:
- 4000:4000
volumes:
- ./litellm/config.yaml:/app/config.yaml
command: --port 4000 --config /app/config.yaml
env_file: .env.docker
environment:
- DATABASE_URL=postgresql://postgres:somesecretpw@db:5432/litellm
caddy:
image: caddy:2.7
restart: unless-stopped
ports:
- "80:80"
- "443:443"
- "443:443/udp"
volumes:
- ./caddy/Caddyfile:/etc/caddy/Caddyfile
- ./caddy/data:/data
- ./caddy/config:/config
# everything below this line is optional and can be commented out
ollama:
image: ollama/ollama
restart: always
ports:
- 11434:11434
volumes:
- ./ollama:/root/.ollama
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>Note that Ollama is optional in this setup and we can deploy without it. That might be helpful if you want to take advantage of being able to switch between different commercial APIs but don't want to run local models (or perhaps don't have the hardware for it). If you want to turn off Ollama, comment it out or remove it. You'll also need to remove the Ollama line </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We also need some support files. </p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Caddyfile</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>The caddyfile is used to route incoming web traffic to your services. Edit the file <code>caddy/Caddyfile</code>. Assuming you have set up DNS as required, You can do something like this:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock -->
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">chat.example.com {
reverse_proxy ui:3000
}
api.example.com {
reverse_proxy litellm:4000
}
</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Secrets in <code>.env.docker</code></h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>We need to create a docker env file which contains our API keys for the services we want to use. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Here we also define a "master" API key which we can use to authenticate Open Web UI against LiteLLM and also to log in to LiteLLM and see the API call stats.</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock -->
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">ANTHROPIC_API_KEY=sk-blah
OPENAI_API_KEY=sk-blablabla
LITELLM_MASTER_KEY=sk-somesecretvalue</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">LiteLLM Config</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>We need to create a very basic <code>config.yaml</code> file which LiteLLM will read to tell it which external models you want to allow users of your web ui to access.</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"yaml"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="yaml" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">model_list:
- model_name: claude-3-opus ### RECEIVED MODEL NAME ###
litellm_params: # all params accepted by litellm.completion() - https://docs.litellm.ai/docs/completion/input
model: claude-3-opus-20240229 ### MODEL NAME sent to `litellm.completion()` ###
api_key: "os.environ/ANTHROPIC_API_KEY" # does os.getenv("AZURE_API_KEY_EU")
- model_name: gpt-4o
litellm_params:
model: openai/gpt-4o
api_key: "os.environ/OPENAI_API_KEY"
- model_name: groq-llama-70b
litellm_params:
api_base: https://api.groq.com/openai/v1
api_key: "os.environ/GROQ_API_KEY"
model: openai/llama3-70b-8192</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>In the example above we tell LiteLLM that we want to connect to Anthropic, OpenAI and Groq and we will allow access to Claude Opus, GPT-4o and Llama 70B respectively. The <code>api_key</code> directives tell LiteLLM to grab the values from named environment variable <code>os.environ/ENV_VAR_NAME</code> so we can customise our env vars to whatever makes sense. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>LiteLLM uses the prefixes on model names to know which API it needs to use e.g. it sees <code>claude</code> and knows to use the Anthropic API client. We can also use <code>openai/</code> to instruct LiteLLM to load any model that supports OpenAI-compatible endpoints out of the box.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">First Run Setup</h2>
<!-- /wp:heading -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Start Docker</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Ok now that we've created all the config files we can start our docker-compose cluster. You can run <code>docker-compose up -d</code> to bring all the services online. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Log in to LiteLLM</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Let's start by testing that our LiteLLM setup works. You should be able to navigate to litellm via the caddy subdomain (eg.. <a href="https://api.example.com">https://api.example.com</a>) or via <a href="http://localhost:8080/ui">http://localhost:8080/ui</a> to get to the LiteLLM UI. If you're not running on your current machine you can also use your LAN IP address instead. You'll need to enter <code>admin</code> as the username and whatever value you used for <code>LITELLM_MASTER_KEY</code> above as the password (including sk-). If all goes well you should see a list of API keys which initially only contains your master key:</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":3169,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/image-1024x523_63cf7606.png" alt="Screenshot of the LiteLLM dashboard interface. The main section displays information for the "Default Team." Key properties shown include a Key Alias "Not Set," a Secret Key partially blurred, Spend in USD as $0.0000, Budget as Unlimited, Models labeled as "all-proxy-models," and TPM/RPM (Transactions per Minute/Requests per Minute) Limits both set to Unlimited. Icons for editing, copying, and deleting the key are also visible. Total Spend is shown at the top as $0.0000. The left sidebar menu includes links for API Keys, Test Key, Models, Usage, Teams, Internal Users, Logging & Alerts, Caching, Budgets, Router Settings, Admin, API Reference, and Model Hub. A “Create New Key” button is at the bottom. The top-right corner includes options for "Get enterprise license" and an "admin" button." class="wp-image-3169"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>If you navigate to the "Models" tab you should also see the models that you enabled in your config.yaml and if all has gone well, pricing information should have been pulled through from the API</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":3170,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/image-2-1024x488_73bdeb15.png" alt="Screenshot of the LiteLLM dashboard's "Models" section. The header shows options: All Models, Add Model, /health, Models, Model Analytics, and Model Retry Settings. The filter is set by Public Model Name to "claude-3-5-sonnet-20240620." The table lists public model details including:
1. **Model Name**: "claude-3-5-sonnet-20240620"
- **Provider**: openai
- **Input Price**: 3.00
- **Output Price**: 15.00
- **Status**: Config Model
- Icons for editing, copying, and deleting appear next to each model.
2. **Model Name**: "claude-3-opus"
- **Provider**: openai
- **Input Price**: 15.00
- **Output Price**: 75.00
- **Status**: Config Model
- Icons for editing, copying, and deleting appear.
3. **Model Name**: "gpt-4"
- **Provider**: openai
- **Input Price**: 30.00
- **Output Price**: 60.00
- **Status**: Config Model
- Icons for editing, copying, and deleting appear.
4. **Model Name**: "gpt-4o"
- **Provider**: openai
- **Input Price**: 5.00
- **Output Price**: 15.00
- **Status**: Config Model
- Icons for editing, copying, and deleting appear.
5. **Model Name**: "groq-llama-70b"
- **Provider**: openai
- **API Base**: https://api.groq.com
- **Status**: Config Model
- Icons for editing, copying, and deleting.
The right corner of the page shows "Get enterprise license" and "admin" links. The last refresh date and time are displayed as 08/07/2024, 08:57:00. The left sidebar includes links for API Keys, Test Key, Models, Usage, Teams, Internal Users, Logging & Alerts, Caching, Budgets, Router Settings, Admin, API Reference, and Model Hub." class="wp-image-3170"/></figure>
<!-- /wp:image -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Create Open Web UI Account</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Next we need to create our Open Web UI account. Go to your chat subdomain <a href=" https://chat.example.com/ ">https://chat.example.com/</a> or <a href="http://localhost:3000">http://localhost:3000</a> and follow the registration wizard. Open Web UI will <a href="https://docs.openwebui.com/getting-started/#how-to-install-">automatically treat the first ever user to sign up as the administrator</a>. </p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Connect Open Web UI to LiteLLM</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Once we're signed in we need to connect Open Web UI to LiteLLM so that it can use the models. Click on your profile picture > Admin Panel > Settings and open the Connections tab:</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":3171,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/image-3-1024x524_7f90f127.png" alt="Screenshot of the Admin Panel interface with a dark theme. There are two primary tabs at the top: "Dashboard" and "Settings." The "Settings" tab is selected. The left sidebar contains links for General, Users, Connections, Models, Documents, Web Search, Interface, Audio, Images, Pipelines, and Database.
The main section displays configurations for two APIs:
1. **OpenAI API**
- URL: http://litellm:4000/v1
- WebUI requests to: 'http://litellm:4000/v1/models'
- Includes a masked API key field with an eye icon to reveal the key, a plus icon to add a new key, and a refresh icon. The toggle switch for this API is turned on.
2. **Ollama API**
- URL: http://ollama:11434
- A clickable link for help: "Trouble accessing Ollama? Click here for help."
- Features include a plus icon to add a new key and a refresh icon. The toggle switch for this API is also turned on." class="wp-image-3171"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>Since everything is running inside docker-compose we can address litellm (and Ollama if you enabled it) using their container names. In the OpenAI API box enter the path to your litellm API endpoint - should be <code>http://litellm:4000/v1</code> and in the API key box enter your master password from <code>LITELLM_MASTER_KEY</code>. Click the button to test the connection and hopefully you'll get a green toast notification to say that it worked.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Ollama should already be hooked up if you turned it on. If you didn't enable ollama but want to now, make sure the container is started (<code>docker-compose up -d ollama</code>) and then enter <code>http://ollama:11434</code> in the url field).</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><strong>Question: If Open Web UI lets us connect multiple OpenAI endpoints, why do we need LiteLLM?</strong></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Open Web UI won't talk directly to models that don't use OpenAI compatible endpoints (e.g. Anthropic Claude or Google Gemini). LiteLLM also lets you be specific about which models you want to pass through to OWUI. For example, you can hide the more expensive ones so that you don't burn credits too quickly. Finally, LiteLLM gives you centralised usage/cost analytics which saves you opening all of your providers' API consoles and manually tallying up your totals.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Testing it Out</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Congratulations if you got this far, you now have a working self-hosted chat proxy system. Time to try out some prompts - use the drop down menu at the top of the window to select the model you want to use.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":3155,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/image-1-1024x481_6169e2e3.png" alt="Screenshot of a coding chat interface where the user requests a Python script with CLI options using the `click` module. The response provides a detailed guide, including installation instructions (`pip install click`) and a code snippet for processing an input file and saving the result to an output file. The left sidebar displays recent activity and categories, including today's and previous days' items like "Firms Embrace AI" and "Recipe Ingredients Transcription." The footer contains a message input field and user information, "James."" class="wp-image-3155"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>You can race/compare models using the +/- buttons in the top left to have them all read the current prompt and respond in parallel. </p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":3172,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/image-4-1024x184_8e531302.png" alt="Screenshot of a chat interface with three dropdown options: "gemma2:latest," "gpt-4," and "claude-3-5-sonnet-20240620." A user query is displayed asking, "how many books is 1 million tokens?"" class="wp-image-3172"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>You can even use the multi-modal features of the model from this view by uploading images and documents. Open Web UI will automatically pass them through as required.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":3173,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/image-5-1024x325_894422b5.png" alt="Screenshot of a chat interface where the user uploads an image of a recipe and requests transcription of the ingredients into a Markdown list. The response from "gpt-4o" provides a Markdown formatted list of the recipe ingredients." class="wp-image-3173"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":4} -->
<h4 class="wp-block-heading">Monitoring Usage</h4>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Once we've made some API calls we can log back into litellm and go to "Usage" to see how much it's cost us. </p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":3153,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/litellm_usage-1024x482_c51004bc.png" alt="A bar chart breaking down spend by model in litellm's web ui" class="wp-image-3153"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Final Thoughts</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Congratulations, now you've got a fully self-hosted AI chat interface set up. You can load up your API keys with a few $ each and track how much you are spending from LiteLLM's control panel.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>If you enabled ollama, you can stick to cheap/self-hosted models by default and switch to a more powerful commercial model for specific use cases.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>If you found this tutorial useful, please consider subscribing to my RSS feed, my <a href="https://jamesravey.medium.com/">newsletter on medium</a> or <a href="https://fosstodon.org/@jamesravey">following me on mastodon</a> or the fediverse:</p>
<!-- /wp:paragraph -->
<!-- wp:activitypub/follow-me {"selectedUser":"1"} /-->

75
brainsteam/content/posts/2024/07/21/A Personal Experiment with Coffee_ Walking and Anxiety.md Normal file
View File

@ -0,0 +1,75 @@
---
categories:
- Personal
date: '2024-07-21 11:19:04'
draft: false
tags:
- anxiety
- coffee
title: A Personal Experiment with Coffee, Walking and Anxiety
type: posts
---
<!-- wp:paragraph -->
<p>I noticed that in the last few weeks I've been pretty anxious. More so than my normal background level of anxiety. I realised that I've been drinking a lot more heavily caffeinated iced coffees and pepsi max recently. What's more, I was thinking back to a period earlier in the year when I was less anxious and remembered that it coincided with a health kick where Mrs R and I cut back a lot on Pepsi/fizzy drinks. I put two and two together and started thinking that caffeine is probably exacerbating my anxiety. This led me to try experimenting with coffee, walking and anxiety.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Caffeine and Walking vs Anxiety</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>A recent <a href="https://pubmed.ncbi.nlm.nih.gov/34871964/">study</a> found that 5 cups of coffee a day can induce panic attacks in patients prone to them. It also increases the background anxiety of people without a panic disorder too. That's about 400-500 mg caffeine. I worked out that if I drank 2-3 coffees a day plus 2-3 300ml cans of Pepsi, I'd probably be exceeding that range.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Also, through <a href="https://thoughtshrapnel.com/2024/07/09/summer-digital-detox.html">Doug Belshaw's blog</a> I found <a href="https://drchatterjee.com/5-powerful-ways-to-transform-your-summer-with-dr-rangan-chatterjee/">a podcast episode by Dr Rangan Chaterjee</a> recommending that people use the summer months to do a little detox. He suggests going for a daily 10-15 minute walk as soon as you wake up. Preferably without your phone and before you've had any tea or coffee. During the COVID lockdowns Mrs R and I were doing daily walks morning and evening. We found that it helped to keep us sane and to provide a transition between work and personal time whilst working from home 100% of the time. Thinking about this also felt like a minor revelation.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Personal Experiment</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>It felt like it was time for a <a href="https://nesslabs.com/personal-experiments">personal experiment</a> to see if I could reduce my anxiety. I'm trying out reducing my caffeine intake and doing daily walks to see if it helps my anxiety. I guess I could try doing one without the other to try to isolate the effect of each. On the other hand, both of these things have other advantages. Giving up diet pepsi and doing more exercise are both good for my general health. Therefore, I'm not too bothered as ideally I'd like to keep doing both anyway.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Since Monday 15th July I have been consuming a maximum of 3 caffeinated drinks per day. That's normally 1 coffee and 1-2 cans of pepsi a day. I've replaced my 2nd and 3rd coffee with decaf. I've also swapped out my pepsi for water or lemonade as a treat. I am also doing a 10-15 minute walk around the block as soon as I hop out of bed. I plan to try to do this every day for at least a month to see how it affects me.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Early Results</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Monday morning was particularly bad. I had an intense dose of the sunday scaries and I had drunk lots of pepsi max the night before. I told Mrs R I might need to take a mental health day and nearly gave up on my experiment on day 1. However, she encouraged me to on a walk and it immediately took the edge off my anxiety. I had forgotten how pretty everything is first thing in the morning bathed in golden sunshine and birds tweeting. The air smelt clean and refreshing. I took a single morning coffee and as the day wore on and my sunday scaries subsided, I felt more relaxed. By about 4pm I had a really bad headache and felt tired which are classic caffeine withdrawal symptoms.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>As the week has continued I've felt pretty good. I'm still getting anxious thoughts and having little worry episodes but they don't feel as acute or scary at the moment. I described this feeling to Mrs R as "they are lurking but they can't hurt me any more". It's only been a few days but things are feeling a lot more manageable and the overwhelmed feeling I get when I have a lot on at work has subsided a little.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Today (Sunday) I am feeling a little more anxious. I normally get the sunday scaries and start to think about all the emails waiting for me in my work inbox and all the stuff I want to achieve so that's pretty typical. What is unusual is that I am not feeling paralysed by my sunday scaries today. Quite often the feeling gets in my way and prevents me from doing or achieving much at all on a sunday but today it just feels like a bit of a niggle that's in the background if I stop what I'm doing. I really enjoyed my morning walk again today because the sun was out and I was up before the rest of the neighbourhood and it was lovely and peaceful.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Discussions Online</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I posted about my experiment on <a href="https://fosstodon.org/deck/@jamesravey/112817744848502155">Mastodon</a> and found that a few others had similar experiences. <a href="https://fosstodon.org/@lars@mastodon.social/112818677574853349">Glyn</a> chuckled when his timeline placed my post about cutting down coffee just below another tongue-in-cheek post about how doing that is a great way to reduce joy in your life. Obviously this is a pretty common thing.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Conclusion</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>So early signs are quite positive. I don't know if its placebo that will wear off or a combination of both the reduction in caffeine and the walking but I am feeling a fair bit better. I am trying to be realistic about the fact that anxiety won't completely go away like some kind of magical spell (and my experience today has confirmed that). However, even just having my anxieties numbed and "unable to hurt me" is a massive win.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Part of me is quite annoyed that I didn't try this sooner. I've been regularly drinking relatively high volumes of coffee for about 15 years and I'd say my anxiety probably started around the same sort of time. I really doubt that's a coincidence and it's such a smoking gun when I look back on it. However, hindsight is always 20:20 as they say.</p>
<!-- /wp:paragraph -->

38
brainsteam/content/posts/2024/07/29/GenAI and the Trough of Disillusionment.md Normal file
View File

@ -0,0 +1,38 @@
---
categories:
- AI and Machine Learning
date: '2024-07-29 21:18:00'
draft: false
tags:
- linkblog
title: GenAI and the Trough of Disillusionment
type: posts
---
<!-- wp:paragraph -->
<p>There have been a bunch of recent stories about the GenAI hype not panning out so well.</p>
<!-- /wp:paragraph -->
<!-- wp:indieblocks/like {"empty":false} -->
<div class="wp-block-indieblocks-like"><div class="u-like-of h-cite"><p><i>Likes <a class="u-url p-name" href="https://www.sequoiacap.com/article/ai-optimism-vs-ai-arms-race/">The Game Theory of AI CapEx</a> by <span class="p-author">David Cahn</span>.</i></p></div><div class="e-content"><!-- wp:paragraph -->
<p>A story from Sequoia Capital, who have a lot of money riding on AI investments, talks about the power dynamics at play between large cloud providers. They're all buying a bunch of GPUs because they're worried their competitors will beat them to it - a very rational and normal reason.</p>
<!-- /wp:paragraph --></div></div>
<!-- /wp:indieblocks/like -->
<!-- wp:indieblocks/like {"empty":false} -->
<div class="wp-block-indieblocks-like"><div class="u-like-of h-cite"><p><i>Likes <a class="u-url p-name" href="https://futurism.com/investors-concerned-ai-making-money">Investors Are Suddenly Getting Very Concerned That AI Isn't Making Any Serious Money</a>.</i></p></div><div class="e-content"><!-- wp:paragraph -->
<p>Big investors like Goldman Sachs are starting to wonder where the magical human level intelligence that they were promised by OpenAI and friends is. Early stage investment has significantly dried up</p>
<!-- /wp:paragraph --></div></div>
<!-- /wp:indieblocks/like -->
<!-- wp:indieblocks/like {"empty":false} -->
<div class="wp-block-indieblocks-like"><div class="u-like-of h-cite"><p><i>Likes <a class="u-url p-name" href="https://www.wheresyoured.at/to-serve-altman/?ref=ed-zitrons-wheres-your-ed-at-newsletter">How Does OpenAI Survive?</a>.</i></p></div><div class="e-content"><!-- wp:paragraph -->
<p>Ed Zitron does some napkin maths that suggests OpenAI might struggle to stick around much longer without significant breakthroughs - they will have to keep raising and in this climate that might be hard, even for Sam Altman.</p>
<!-- /wp:paragraph --></div></div>
<!-- /wp:indieblocks/like -->
<!-- wp:indieblocks/like {"empty":false} -->
<div class="wp-block-indieblocks-like"><div class="u-like-of h-cite"><p><i>Likes <a class="u-url p-name" href="https://www.404media.co/websites-are-blocking-the-wrong-ai-scrapers-because-ai-companies-keep-making-new-ones/">Websites are Blocking the Wrong AI Scrapers (Because AI Companies Keep Making New Ones)</a>.</i></p></div><div class="e-content"><!-- wp:paragraph -->
<p>Meanwhile LLM companies continue to steadily tank their reputation by changing their user agent strings (which are used to identify their bots when they scrape your website) on a regular basis so that it's hard to know what to block.</p>
<!-- /wp:paragraph --></div></div>
<!-- /wp:indieblocks/like -->

20
brainsteam/content/posts/2024/08/02/Watched_ Long Legs.md Normal file
View File

@ -0,0 +1,20 @@
---
categories:
- Personal
date: '2024-08-02 23:45:03'
draft: false
tags:
- movies
title: 'Watched: Long Legs'
type: posts
---
<!-- wp:paragraph -->
<p>Just returned from seeing Long Legs in the cinema. I went into this movie pretty much blind, not knowing much except that Nicolas Cage was involved. It was a really interesting thriller and Cage was an exceptionally creepy serial killer. It was set in the 90s and follows a young FBI agent on the case of a killer. There were some real X-Files vibes. Quite an enjoyable watch.</p>
<!-- /wp:paragraph -->
<!-- wp:embed {"url":"https://m.youtube.com/watch?v=FXOtkvx25gI","type":"video","providerNameSlug":"youtube","responsive":true,"className":"wp-embed-aspect-16-9 wp-has-aspect-ratio"} -->
<figure class="wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio"><div class="wp-block-embed__wrapper">
https://m.youtube.com/watch?v=FXOtkvx25gI
</div></figure>
<!-- /wp:embed -->

213
brainsteam/content/posts/2024/08/03/Migrating from Linear to Jira.md Normal file
View File

@ -0,0 +1,213 @@
---
categories:
- Uncategorised
date: '2024-08-03 10:11:37'
draft: true
tags: []
title: Migrating from Linear to Jira
type: posts
---
<!-- wp:paragraph -->
<p>Our company recently decided to move from Linear to Jira because we wanted a bunch of the fancier stuff that Jira offers around workflow management and resource burndown. The migration process has been a little painful but not too overwhelming. I wrote up some of my notes and built some Python scripts to facilitate the process so I thought I'd share them here.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Our company uses something like an agile Scrum methodology with a bi-monthly release cycle.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Mapping Linear to Jira Data Structures</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Linear has Teams, each team can have multiple projects and then within projects we have stories, tasks and bugs. A project in Linear is typically a time-bound initiative over a number of releases that contains a set of stories and bugs. Stories contain tasks and sub-tasks</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Jira's top level data structure is a Project, within a project you can have an Epic which is a time-bound initiative over a number of releases and can contain a number of stories and bugs. Stories contain tasks and sub-tasks.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>So with this in mind, we mapped the data structures into something like the following:</p>
<!-- /wp:paragraph -->
<!-- wp:table -->
<figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th>Linear Data Structure</th><th>Jira Data Structure</th></tr></thead><tbody><tr><td>Team</td><td>[Jira] Project</td></tr><tr><td>[Linear] Project</td><td>Epic</td></tr><tr><td>Story</td><td>Story</td></tr><tr><td>Bug</td><td>Bug</td></tr><tr><td>Task</td><td>Task</td></tr><tr><td>Sub-Task</td><td>Sub-Task</td></tr></tbody></table><figcaption class="wp-element-caption">A mapping of Linear data structures to Jira data structures</figcaption></figure>
<!-- /wp:table -->
<!-- wp:paragraph -->
<p>There's a pretty straight forward 1:1 mapping of data from one system to the other. Not bad.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Linear has the concept of Cycles for timekeeping whereas Jira calls them Sprints. We actually decided not to try to map cycles and instead manually sorted that out after the migration.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Getting Data Out of Linear into Jira</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Linear offers a CSV export function which is easy enough to use. However, not everything gets exported by this tool - this includes project metadata, comments on issues and attachments. I have provided some support tools that we can use to get those files out.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Getting the Main Tickets Out of Linear</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>In the Workspace Settings menu, go to<strong><em> Import/Export</em></strong> and scroll down to the Export CSV button. Linear will generate a CSV of your workspace in the background and email it to you.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Upon downloading the CSV file, we can see all of the tickets from all teams in your workspace. </p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Getting Project Metadata out of Linear</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>The projects are not represented by rows in the CSV but instead each item has a Project [Name] and Project ID associated with them. What if we want more metadata about each project? For example, the name, description, start date and status. Well I wrote a small script called <code>export_projects.py</code> that fetches all of that information from Linear's GraphQL API and stores it in a separate CSV. You can find it in the git repo.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Use it by running:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"bash"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">export LINEAR_API_KEY=<your-api-key>
python export_projects.py projects.csv</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>You can find out more about how to get a linear API key <a href="https://developers.linear.app/docs/graphql/working-with-the-graphql-api">here</a>. The generated CSV file <code>projects.csv</code> should contain a set of all of your linear projects and associated metadata.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Getting Comments out of Linear</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Another shortcoming of the main linear CSV export is that it does not include comments from tickets, only the main ticket body. I've provided a script called <code>export_comments.py</code> which, like the projects script above, will grab all comments directly from the API. Use it like this:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"bash"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">export LINEAR_API_KEY=<your-api-key>
python export_comments.py comments.csv</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>The generated <code>comments.csv</code> should contain all the comments from your projects and the IDs of the parent thread that they address.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Getting Attachments out of Linear</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I've also provided a script for exporting attachments - specifically images - from linear. This script goes through the CSV file provided by linear looking for URLS that appear to be attachments. It will then download each of them in turn and place them in a local directory so that we can re-upload them to Jira. This step is necessary because the JIRA import worker cannot directly access images on the linear server.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>With the csv file you exported from Linear you should be able to run:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock -->
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">export LINEAR_API_KEY=<your-api-key>
python export_images.py --issues-file ./linear_export.csv --output-dir ./images</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>This script will produce a new directory called images which contains all the attachments and a file called index.csv which essentially holds a catalogue of all the images, what their old URL was in Linear and what their local filename is.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>So now you should have 2 CSV files. The one you exported directly from Linear and the projects file created by this script.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Converting Markdown to Atlassian Wiki Markup</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Linear uses Markdown for rich descriptions but Atlassian uses their own wiki markup (because y'know, <a href="https://xkcd.com/927/">standards</a>). We need to auto-convert the item descriptions from Linear to Atlassian format. We need to do this for the main linear export, the projects and the comments.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>When I did a bit of googling for how to go about this I found an example that uses <a href="https://github.com/miyuchina/mistletoe">mistletoe markdown</a> to parse markdown and emit jira. I've created a lightweight command-line script called <code>markdown2jira.py</code> that will do this job for you.</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"bash"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">python markdown2jira.py -i projects.csv -o projects_converted.csv -c description
python markdown2jira.py -i linear_export.csv -o linear_export_converted.csv -c Description
python markdown2jira.py -i comments.csv -o comments_converted.csv -c node.body </pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>You should now have converted csv files for comments, projects and linear items.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Hosting the Attachments</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>The JIRA import process does not allow you to directly upload image attachments. Instead, it expects them to be available in some public location or website. My solution to this was to use <a href="https://ngrok.com/">ngrok</a> to provide a temporary public tunnel to the images directory on my laptop. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>You will need an ngrok api key which you can get for free by signing up <a href="https://dashboard.ngrok.com/signup">here</a>, then you can run the following:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"bash"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">export NGROK_AUTHTOKEN=<your-ngrok-api-key>
python attachment_host.py host-ngrok --path ./images --csv-outfile ./attachments.csv</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>This will run a server which we will need to leave running while the JIRA import takes place. Ngrok's free tunnels time out after 8 hours so bear in mind that if you choose now to take a break and go and do something else, you might need to re-run this step (also if you have a really massive project). </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><strong><em>nb: I was also toying with the idea of writing a version of this that uploads all the images to S3 or Google storage but I haven't had time. Pull Requests welcome.</em></strong></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>You should now also have a new file called <code>attachments.csv</code> - this is a map generated by the server which converts from the old image url in the JIRA ticket to the new one in your ngrok server. </p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Merge, Link and Tidy up Timestamps</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Ok so the next step (script) is actually doing three things at once:</p>
<!-- /wp:paragraph -->
<!-- wp:list {"ordered":true} -->
<ol class="wp-block-list"><!-- wp:list-item -->
<li> We need to merge the two converted CSV files together so that we have a single file that we can upload using JIRA's import system. </li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>We need to link together issues with their parents properly and make sure that issues which specify a Linear Project ID but no parent ID (because in Linear they were top level issues in the project) get mapped to the relevant Epic in the new world.</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>Linear uses some very strange date strings when it exports by default so we will need to tidy those up and write them back out in a simple format that is easy for Jira to parse.</li>
<!-- /wp:list-item --></ol>
<!-- /wp:list -->
<!-- wp:paragraph -->
<p>The script <code>merge_link.py</code> handles all of these operations.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->

24
brainsteam/content/posts/2024/08/11/Watched_ Trap.md Normal file
View File

@ -0,0 +1,24 @@
---
categories:
- Personal
date: '2024-08-11 19:32:28'
draft: false
tags:
- movies
title: 'Watched: Trap'
type: posts
---
<!-- wp:paragraph -->
<p>We went to see Trap, <a href="https://en.wikipedia.org/wiki/M._Night_Shyamalan">M. Night Shyamalan</a>'s latest starring <a href="https://en.wikipedia.org/wiki/Josh_Hartnett">Josh Hartnett</a> as a somewhat creepy antagonist trying to escape a concert without being arrested (because he's actually a serial killer and the concert is an FBI trap set up to lure him and his tween daughter in).</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>It was a fairly classic Shyamalan movie, possibly one of his more enjoyable films of late. It felt a bit like an inverse-heist movie: what if instead of breaking in, they're trying to break out. Pretty cheesy but made a nice change from yet another sequel.</p>
<!-- /wp:paragraph -->
<!-- wp:embed {"url":"https://www.youtube.com/watch?v=hJiPAJKjUVg","type":"video","providerNameSlug":"youtube","responsive":true,"className":"wp-embed-aspect-16-9 wp-has-aspect-ratio"} -->
<figure class="wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio"><div class="wp-block-embed__wrapper">
https://www.youtube.com/watch?v=hJiPAJKjUVg
</div></figure>
<!-- /wp:embed -->

87
brainsteam/content/posts/2024/08/18/Visiting Bletchley Park.md Normal file
View File

@ -0,0 +1,87 @@
---
categories:
- Personal
date: '2024-08-18 21:59:38'
draft: false
tags:
- museums
- trips
title: Visiting Bletchley Park
type: posts
---
<!-- wp:paragraph -->
<p>Yesterday Mrs R and I visited Bletchley Park as part of our third wedding anniversary celebration. As a computer scientist, BP has been on my bucket list since forever. It's the cradle of many computing breakthroughs including cyber warfare, cryptography and crypt-analysis and the first generally programmable computer.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>BP is a large stately home which was converted into a research campus by the British Ministry of Defence during the late 1930s. At its peak, the site hosted nearly 10,000 workers who all contributed to the analysis and decryption of secret messages being transmitted around within Hitler's army and between axis nations. The site features beautiful gardens and a large lake and staff were able to walk around the gardens and enjoy them in between shifts. </p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":3649,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/PXL_20240817_115735551.MP_-1024x771_1480eed8.jpg" alt="A large house/mansion behind a lake with a fountain at the center of it on a calm, bright and sunny day." class="wp-image-3649"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>BP reminded me a lot of<a href="https://en.wikipedia.org/wiki/Hursley_House"> Hursley House </a>where I worked for a few years during my time at IBM. Similarly, Hursley was bought by the Government and hosted the team who developed the spitfire in world war 2 before being sold off to IBM many years later. I used to enjoy going for a wander when I was ruminating on some problem I needed to solve and I can just as easily imagine Alan Turing and his colleagues wondering the grounds in BP when they had a particularly tough problem to solve.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">The Wrens and the Bombes</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>The vast majority of staff at BP during this time were women from the Women's Royal Navy Service (WRNS, referred to "The Wrens"). There were a number of VIPs there, Turing being one of the better known figures but key amongst them were <a href="https://en.wikipedia.org/wiki/Gordon_Welchman">Gordon Welchman</a> who was key in breaking the Enigma encryption and speeding up the automated cracking of keys. Another key figure, <a href="https://en.wikipedia.org/wiki/Tommy_Flowers">Tommy Flowers</a> wasn't based at BP but worked with Turing to develop <a href="https://en.wikipedia.org/wiki/Colossus_computer">The Colossus</a>, the very first programmable computer. Prior to this, all computation was done with purpose built machines which could only do the thing they had been designed for. The Bombes - mechanical code breaking machines that could try 40k combinations of enigma keys in 12 minutes - being an impressive example.</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":3652,"aspectRatio":"4/3","scale":"cover","sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/PXL_20240817_142406470-771x1024_fb89ce0c.jpg" alt="a cabinet full of rows cylindrical rotors which were used to brute force enigma cypher keys" class="wp-image-3652" style="aspect-ratio:4/3;object-fit:cover"/><figcaption class="wp-element-caption">A working model of a "Bombe" which was manually configured and used to partially break enigma encryption</figcaption></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>The Bombes were operated by the Womens' Royal Naval Service (WRNS or 'wrens' as they were known colloquially). In fact during the war, the staff on site were mostly women. It's somewhat ironic and sad that STEM fields are now so very male-dominated.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">A Personal Connection</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I found a personal connection to Bletchley when we saw an exhibit that showed how intelligence gathered from Hitler's Navy had fed to the British ship HMS Duke of York to help them to locate and sink the Scharnhorst, a German warship that was harrying a civilian convoy in arctic. My grandad was a sailor onboard the Duke of York and was <a href="https://www.bbc.co.uk/history/ww2peopleswar/stories/37/a7261337.shtml">interviewed about his experiences for the BBC</a> (nb: they misspell Scharnhorst as Charnos in their coverage).</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":3653,"aspectRatio":"4/3","scale":"cover","sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/PXL_20240817_130521051-771x1024_f38c50c0.jpg" alt="A map with a legend showing the Scharnhorst and HMS Duke of York - the ship that my grandfather served on." class="wp-image-3653" style="aspect-ratio:4/3;object-fit:cover"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>Staff were sworn to secrecy and were not allowed to talk about their work even amongst colleagues from other departments. I found it quite fascinating reading diary entries and personal accounts of how boring some of the day-to-day work at BP was. Many of the staff may not have realised just how important their contributions were until after the war. Some days as a software engineer I have days where I just get stuck thinking about a particular mathematical challenge. I can only imagine how annoying it must have been to spend days and days trying to break a code by hand or using primitive computing only. It must also be pretty surreal getting on a bus with colleagues every day and not being able to talk about what you'd been doing that day.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Turing Exhibit</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>There was a huge exhibit dedicated to Turing's contributions to BP and to computing in general including a copy of the letter signed by former Prime Minister Gordon Brown in 2009 apologising for the treatment of Turing. After World War 2, Turing was convicted of gross indecency for being in a sexual relationship with another man and as a result lost all of his academic credentials and funding. I suppose this was analogous to the modern concept of "being cancelled". Unable to continue his work, he was driven to suicide. His work truly laid the foundations for all of modern computing and Artificial Intelligence and we owe him a great debt of gratitude. As a gay man whose unfair treatment led to his death, I can't imagine he would look kindly on modern uses of AI for things like screening interviews and surveillance capitalism. </p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":3647,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/PXL_20240817_122728352-scaled-e1724012111189-1024x575_f393c212.jpg" alt="A slate statue of Alan Turing" class="wp-image-3647"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Conclusion</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Overall, we had a really interesting day out at BP. It was really intriguing seeing the conditions that the codebreakers worked in and also nice to finally be able to talk to Mrs R about the Universal Turing Machine and how cryptography works. I'd strongly recommend that anyone with a vague interest in the history of information warfare or computing in general should pop along and visit.</p>
<!-- /wp:paragraph -->

13
brainsteam/content/posts/2024/08/25/Morning People.md Normal file
View File

@ -0,0 +1,13 @@
---
categories:
- Uncategorised
date: '2024-08-25 14:53:51'
draft: true
tags: []
title: Morning People
type: posts
---
<!-- wp:paragraph -->
<p>I am a morning person. I was talking to a friend recently about why</p>
<!-- /wp:paragraph -->

12
brainsteam/content/posts/2024/08/27/wicked.md Normal file
View File

@ -0,0 +1,12 @@
---
categories:
- Uncategorised
date: '2024-08-27 21:14:00'
draft: true
photo:
- url: /media/1724789671358_da5cba22.jpg
tags: []
title: wicked
type: posts
---

297
brainsteam/content/posts/2024/08/29/Data Export in Bulk inside your CI Pipeline With Sling.md Normal file
View File

@ -0,0 +1,297 @@
---
categories:
- Data Science
date: '2024-08-29 16:24:57'
draft: false
tags:
- database
- elt
title: Data Export in Bulk inside your CI Pipeline With Sling
type: posts
---
<!-- wp:paragraph -->
<p>In this post I will discuss my migration journey from Airbyte to Sling. Sling is a new lightweight data sync product that is small enough that it can run in a CI pipeline.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">What's the Problem?</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I'm trying to orchestrate the secure export of large amounts of data from a customer's database into cold storage. Customers can loadthis data into their favourite <a href="https://aws.amazon.com/what-is/olap/">OLAP</a> database and do analytics and machine learning there.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">What Happened With Airbyte?</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Last year I wrote about <a href="https://brainsteam.co.uk/2023/08/14/stable-airbyte-sync/">some experiments I had done with airbyte</a> for copying data around in bulk. I was quite pleased with Airbyte at the time and it has proven to be quite robust.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Recently, Airbyte have made a few changes to the product that have made it less suitable for my work environment. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Notably:</p>
<!-- /wp:paragraph -->
<!-- wp:list -->
<ul class="wp-block-list"><!-- wp:list-item -->
<li><a href="https://github.com/airbytehq/airbyte/issues/26185#issuecomment-2088406941">They deprecated their octavia CLI </a>which I was using for programmatic sync config management. Our product's db schema is huge so manually clicking around in the UI can be very error prone.</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>They recently <a href="https://github.com/airbytehq/airbyte/discussions/40599">deprecated docker and docker-compose deployment </a>which was our favoured way of running the product. We didn't want to migrate to running kubernetes clusters for each deployment of Airbyte.</li>
<!-- /wp:list-item --></ul>
<!-- /wp:list -->
<!-- wp:paragraph -->
<p>There were a couple of other sticking points that led me to want to move away from Airbyte:</p>
<!-- /wp:paragraph -->
<!-- wp:list -->
<ul class="wp-block-list"><!-- wp:list-item -->
<li>When you update the sync config it will default to running a reset and sync there and then. This is inconvenient because it puts significant load on the DB and can disrupt our clients' usage of our platform. We normally run syncs out of hours.</li>
<!-- /wp:list-item -->
<!-- wp:list-item -->
<li>Airbyte is very expensive to run per-client. It really needs a dedicated multi-core machine with 16GiB RAM (I tried 8 GiB and got thrashing and out-of-memory errors). If we had a single instance of it running for all clients it might work out cheaper. However, we are committed to infrastructure-level isolation of client data. We don't want to end up in a situationwe accidentally sync Client A's MySQL db to Client B's OLAP environment. The very thought makes me pretty shiver with anxiety.</li>
<!-- /wp:list-item --></ul>
<!-- /wp:list -->
<!-- wp:paragraph -->
<p>For all of these reasons I decided to start looking at alternatives.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Exploring Alternative Solutions</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I played briefly with <a href="https://meltano.com/">meltano</a> and <a href="https://www.singer.io/">singer</a> but I found that they were quite sluggish for my use case. I'm not really sure why this was the case. I was finding that meltano could handle something like 10k records/second locally but 1-2k records/sec in CI. If anyone knows why that might have been, please let me know!</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Eventually I stumbled across <a href="https://slingdata.io/">sling</a>. Sling is written in golang and so far I've found it to be very responsive. It runs handily in a small amount of memory and supports yaml files that can be programmatically managed.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I was really excited by the resource footprint of sling vs airbyte. It's small enough that it can run in a few hundred MB of RAM rather than needing GBs. This means that it can run in our CI pipeline. In fact, the official documentation provides examples for running it inside <a href="https://docs.slingdata.io/sling-cli/github-actions">Github Actions</a>. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Working with Sling and Gitlab</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>We use Gitlab for version control and CI so I spent some time building a POC where we ship a sling <code>replication.yaml</code> configuration file to a repository, set some environment variables and use <a href="https://docs.gitlab.com/ee/ci/pipelines/schedules.html">Gitlab Pipeline Schedules</a> to automatically run the sync on a periodic basis.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">The CI File</h2>
<!-- /wp:heading -->
<!-- wp:enlighter/codeblock {"language":"yaml"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="yaml" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">stages:
- run
execute:
image: slingdata/sling
stage: run
rules:
- if: '$CI_PIPELINE_SOURCE == "schedule"'
- when: manual
script:
- sling conns set GOOGLE_STORAGE type=gs bucket=${GOOGLE_STORAGE_BUCKET} key_file=${GOOGLE_SECRET_JSON}
- sling conns set MYSQL url="mysql://${MYSQL_USER}:${MYSQL_PASSWORD}@${MYSQL_HOST}:3306/${MYSQL_DBNAME}"
- sling run -r replication.yaml -d
after_script:
- |
if [ "$CI_JOB_STATUS" == "success" ]; then
curl -H "Content-type: application/json" \
--data "{\"text\": \"CI sync for \`${CI_COMMIT_REF_NAME}\` successful! :partyparrot: <${CI_PIPELINE_URL}|Click here> for details!\"}" \
-X POST $SLACK_WEBHOOK_URL
else
curl -H "Content-type: application/json" \
--data "{\"text\": \"CI sync for \`${CI_COMMIT_REF_NAME}\` failed :frowning: <${CI_PIPELINE_URL}|Click here> for details!\"}" \
-X POST $SLACK_WEBHOOK_URL
fi</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Let's unpack what's happening here. I'm using the <a href="https://hub.docker.com/r/slingdata/sling">official sling docker</a> as the main base image for my CI job. Effectively its like I've installed sling on a linux machine. I've added a rule to make sure that this run only fires when it's part of a schedule. We don't want it running every time someone pushes a change to the replication config for example.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>The script simply adds connection configurations for GOOGLE_STORAGE where I'll be sending my data and MYSQL where I'll be reading it from. Then we execute the sync step with <code>sling run</code>.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Afterwards we send slack notifications to my org's workspace depending on whether the job succeeded or not.</p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">The Replication Yaml</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Next we also define a replication yaml. This file tells sling which connection is which (i.e. MySQL is our source and GCS is our target) and also which tables and columns to copy. This is particularly powerful since there may be specific columns that you don't want to sync (e.g user password hashes or api keys).</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"yaml"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="yaml" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">source: MYSQL
target: GOOGLE_STORAGE
defaults:
object: "{stream_schema}.{stream_table}"
mode: full-refresh
target_options:
format: jsonlines # this could also be CSV
compression: gzip
streams:
news_articles:
mode: full-refresh
select:
- id
- url
- is_active
- title
- summary
- created_at</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>The yaml file above provides default configuration for each stream - telling sling to generate gzipped newline-delineated json files in google storage. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Under the streams section we list the tables we're interested in and also use the <code>select</code> subsection to list columns we want (you can actually flip to an denylist approach by listing columns you don't want to select with a <code>-</code> in front of them which looks a bit weird because you need a dash for the yaml format and a dash for the value. It can make it clearer if you wrap them in quotes:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"yaml"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="yaml" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">select: # select /everything/ except the following:
- "-password"
- "-my_secret_column"</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>We can also use a custom <code>sql</code> query to limit what we send over. Imagine you have a table with millions of news articles but your customer only cares about the ones categorised as "tech":</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock -->
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">streams:
news_articles:
mode: full-refresh
sql: "SELECT * FROM news_articles WHERE category='tech'"</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Configuring the Schedule</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>In the Gitlab Navigation tab under "Build" there should be a Pipeline Schedules button. Clicking that and then "New Schedule" should bring up a screen like this:</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":3840,"sizeSlug":"full","linkDestination":"none"} -->
<figure class="wp-block-image size-full"><img src="/media/image_b57ecd00.png" alt="User interface for scheduling a new pipeline. Form includes fields for description, interval pattern selection (daily, weekly, monthly, or custom), cron timezone selection, target branch or tag selection, variable input, and an activation checkbox. Options to create pipeline schedule or cancel are provided at the bottom." class="wp-image-3840"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>Here we can set our CI job to run on a recurring basis (if you're not comfortable using cron expressions, I recommend <a href="https://crontab.guru/">https://crontab.guru</a> to help). </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We need to provide values for each of the environment variables in our gitlab ci file above. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p><strong>A tip for Google JSON key files: </strong>use the left-most dropdown to set a "File" variable - this takes the variable input value from the form, writes it to a file and sets the variable name to the path of the file. We can simply open up a json key file locally and copy and paste it's contents into the variable in this form.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Finally, once all the environment variables are set, we can save the schedule</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":3841,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/image-1-1024x995_fb25bd4f.png" alt="User interface for editing a pipeline schedule. Form includes fields for description ('Sling test'), interval pattern selection (custom selected with '54 15 * * *' cron syntax), cron timezone (UTC 0), and target branch ('sling'). Variables section shows four filled entries for MYSQL_HOST, MYSQL_PASSWORD, GOOGLE_STORAGE_BUCKET, and GOOGLE_SECRET_JSON, with values obscured. An empty variable entry is available. Options to reveal values, toggle activation, save changes, or cancel are provided." class="wp-image-3841"/></figure>
<!-- /wp:image -->
<!-- wp:heading {"level":3} -->
<h3 class="wp-block-heading">Managing Pipelines</h3>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Once you have saved your sync config you'll be able to see it in the pipelines view</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":3842,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/image-2-1024x452_a6d32eea.png" alt="" class="wp-image-3842"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p>You should be able to see when the job is next scheduled to run and you can click the play button to force it to run right away. Job logs show up in your CI pipelines tab as normal. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Tips for Managing Multiple Sync Targets</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Now that we've found this pattern it's made life much easier (and cheaper). I've written a python tool for automatically enabling and disabling groups of tables from our platform's schema en-masse so that we avoid manual issues.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>For segregating client workloads you could have one repository per client or you could have one branch per client in the same project. The environment variable values are unique per schedule which means that it would be difficult to cross contaminate environments (unless you paste in different client's credentials during setup which would also be possible in any other situation if you are a bit gung-ho with your clipboard management).</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>We have branch protection rules on <code>production/**</code> and require a merge request and approval from an admin so that existing production sync configurations don't get accidentally overwritten.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Conclusion</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Sling is a really powerful yet very lightweight tool for doing database data sync. It's been really great to be able to vastly simplify our company's data sync offering and stop having to set up expensive servers for every customer that needs this functionality. Hopefully this post is useful to others looking to try out sling. If you're interested in the tool check out their <a href="https://docs.slingdata.io/">documentation</a>, <a href="https://github.com/slingdata-io/sling-cli">github page</a>. The maintainer Fritz is very responsive and active when it comes to issues and the project discord chat so those are also great resources if you have any questions.</p>
<!-- /wp:paragraph -->

148
brainsteam/content/posts/2024/09/05/Running Phi MoE 3.5 on Macbook Pro.md Normal file
View File

@ -0,0 +1,148 @@
---
categories:
- AI and Machine Learning
date: '2024-09-05 14:17:39'
draft: false
tags:
- AI
- llms
title: Running Phi MoE 3.5 on Macbook Pro
type: posts
---
<!-- wp:paragraph -->
<p>The relatively recently released Phi 3.5 model series includes a mixture-of-experts model featuring 16 x 3.3 Billion parameter expert models. It activates these experts two at a time resulting in pretty good performance but only 6.6 billion parameters held in memory at once. I recently wanted to try running Phi MoE 3.5 on my macbook but was blocked from doing so using <a href="https://brainsteam.co.uk/2024/07/08/ditch-that-chatgpt-subscription-moving-to-pay-as-you-go-ai-usage-with-open-web-ui/">my usual method</a> whilst <a href="https://github.com/ggerganov/llama.cpp/issues/9119#issuecomment-2319393405">support is built into llama.cpp</a> and then <a href="https://github.com/ollama/ollama/issues/6449">ollama</a>.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>I decided to try out another library,<a href="https://github.com/EricLBuehler/mistral.rs"> mistral.rs,</a> which is written in the rust programming language and already supports these newer models. It required a little bit of fiddling around but I did manage to get it working and the model is relatively responsive.</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Getting Our Dependencies and Building Mistral.RS</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>To get started you will need to have the rust compiler toolchain installed on your macbook including <code>rustc</code> and <code>cargo</code>. The easiest way to do this is via brew: </p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"bash"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">brew install rust</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>You'll also need to grab the code for the project </p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"bash"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">git clone https://github.com/EricLBuehler/mistral.rs.git</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>Once you have both of these in place we can build the project. Since we're running on Mac, we want the compiler to make use of apple <a href="https://developer.apple.com/metal/">Metal</a> which allows the model to use the GPU capabilities of the M-series chip to accelerate the model.</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"bash"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">cd mistral.rs
cargo install --path mistralrs-server --features metal</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>This command may take a couple of minutes to run. The compiled server will be saved in the <code>target/release</code> folder relative to your project folder. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Running the Model with Quantization</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>The <a href="https://github.com/EricLBuehler/mistral.rs?tab=readme-ov-file#quick-examples">default instructions in the project readme</a> work but you might find it takes up a lot of memory and takes a really long time to run. That's because, by default mistral.rs does not do any quantization so running the model requires 12GB of memory. </p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>mistral.rs supports in-situ-quantisation which essentially means that the framework loads the model up and does the quantisation at run time (as opposed to requiring you to download a GGUF file that was already quantized). I recommend running the following:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"bash"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">./target/release/mistralrs-server --isq Q4_0 -i plain -m microsoft/Phi-3.5-mini-instruct -a phi3</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>In this mode we use ISQ to quantize the model down to 4bit mode (<code>--isq Q4_0</code>). You should be able to chat to the model through the terminal</p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Running as a Server</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>Mistral.rs provides a HTTP API that is compatible with OpenAI standards. To run in server mode we remove the <code>-i</code> argument and replace it with a port number to run on <code>--port 1234</code>:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"bash"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">./target/release/mistralrs-server --port 1234 --isq Q4_0 plain -m microsoft/Phi-3.5-mini-instruct -a phi3</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>You can then use an app like Postman or Bruno to interact with your model:</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":3999,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/image-1024x466_76149d20.png" alt="Screenshot of a REST tooling interface. A pane on the left shows a json payload that was sent to the server containing messages to the model telling it to behave as a useful assistant and write a poem.
On the right is the response which contains a message and the beginning of a poem as written by the model." class="wp-image-3999"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Running the Vision Model</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>To run the vision model, we just need to make a couple of changes to our command line arguments:</p>
<!-- /wp:paragraph -->
<!-- wp:enlighter/codeblock {"language":"bash"} -->
<pre class="EnlighterJSRAW" data-enlighter-language="bash" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">./target/release/mistralrs-server --port 1234 --isq Q4_0 vision-plain -m microsoft/Phi-3.5-vision-instruct -a phi3v</pre>
<!-- /wp:enlighter/codeblock -->
<!-- wp:paragraph -->
<p>We still want to use ISQ but this time we swap <code>plain</code> for <code>vision-plain</code>, we swap the model name for the vision equivalent and we change the architecture <code>-a phi3</code> to <code>-a phi3v</code>.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p>Likewise we can now interact with the model via HTTP tooling. Here's <a href="https://github.com/EricLBuehler/mistral.rs/blob/master/docs/PHI3V.md">a response based on the example from the documentation</a>:</p>
<!-- /wp:paragraph -->
<!-- wp:image {"id":4000,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="/media/image-1-1024x674_6047d8ff.png" alt="Screenshot of a REST interface. A pane on the left shows a json payload that was sent to the server containing messages to the model telling it to analyse an image url.
On the right is the response which describes the mountain in the picture that was sent." class="wp-image-4000"/></figure>
<!-- /wp:image -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:heading -->
<h2 class="wp-block-heading">Running on Linux and Nvidia</h2>
<!-- /wp:heading -->
<!-- wp:paragraph -->
<p>I am still struggling to get mistral.rs to build on Linux at the moment, the docker images that are provided by the project don't seem to play ball with my systems. Once I figure this out I'll release an updated version of this blog.</p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->
<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->