ravenscroftj
/
brainsteam.co.uk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add memos links

This commit is contained in:
James Ravenscroft 2024-11-03 15:25:03 +00:00
parent 8cc1e5601b
commit b7c0c744a4
1 changed files with 4 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
brainsteam/content/posts/2024/11/03-annomemo-telegram-bot.md
View File

@ -20,15 +20,15 @@ url: /2024/11/3/03-annomemo-telegram-bot
Earlier this year [I wrote about using VLM models to do OCR on my terribly scribbly hand writing](https://brainsteam.co.uk/2024/04/02/finding-the-best-ai-powered-handwriting-ocr/). Models like GPT-4o are actually quite good at interpreting my rubbish writing and converting it to markdown. However, my workflow for using these models was a bit fiddly.
I have just finished an early version of AnnoMemo, a telegram bot that can receive images of handwritten notes and respond with a transcription of them. AnnoMemo also integrates with the popular memos app. It will automatically upload the photo of your hand written notes alongside the transcription as a new note and include a link to that note in its telegram response.
I have just finished an early version of AnnoMemo, a telegram bot that can receive images of handwritten notes and respond with a transcription of them. AnnoMemo also integrates with the popular [memos](https://www.usememos.com/) app. It will automatically upload the photo of your hand written notes alongside the transcription as a new note and include a link to that note in its telegram response.
AnnoMemo is a portmanteau of Annotation and Memorandum.
## Motivation
I went through a phase of manually uploading photos to ChatGPT or [my self hosted LLM portal](https://brainsteam.co.uk/2024/07/08/ditch-that-chatgpt-subscription-moving-to-pay-as-you-go-ai-usage-with-open-web-ui/) and copying and pasting the resulting text into my notes app. There are a few friction points in this process including the need to take the photo with my phone's camera app before opening [Open Web UI](https://openwebui.com/) since it currently doesn't provide a way of launching the camera in-app. I also need to highlight and copy the response and paste it into my memos app of choice. Another fairly major annoyance is that if the OCR model does get some words wrong I have to go and find the page to make sure I remember what I actually wrote.
I went through a phase of manually uploading photos to ChatGPT or [my self hosted LLM portal](https://brainsteam.co.uk/2024/07/08/ditch-that-chatgpt-subscription-moving-to-pay-as-you-go-ai-usage-with-open-web-ui/) and copying and pasting the resulting text into my notes app. There are a few friction points in this process including the need to take the photo with my phone's camera app before opening [Open Web UI](https://openwebui.com/) since it currently doesn't provide a way of launching the camera in-app. I also need to highlight and copy the response and paste it into my [memos](https://www.usememos.com/) app of choice. Another fairly major annoyance is that if the OCR model does get some words wrong I have to go and find the page to make sure I remember what I actually wrote.
AnnoMemo simplifies this process by allowing me to simply open Telegram, take a photo in-app and send it. The bot takes care of the rest of the process including sending the image and the prompt to the model, sending a copy of the response and the initial input image to my memos instance and finally sending the transcription and link to the newly created memo back to me inside telegram.
AnnoMemo simplifies this process by allowing me to simply open Telegram, take a photo in-app and send it. The bot takes care of the rest of the process including sending the image and the prompt to the model, sending a copy of the response and the initial input image to my [memos](https://www.usememos.com/) instance and finally sending the transcription and link to the newly created memo back to me inside telegram.
## How I built it
@ -48,7 +48,7 @@ I've been doing some testing and I've actually found that [Qwen-2 VL 2B Instruct
### Other PKMS Integrations
I love memos but I also make extensive use of other Personal Knowledge Management Systems (PKMS) like Obsidian and Joplin. Therefore, I may end up providing integrations with those tools at some point too.
I love [memos](https://www.usememos.com/) but I also make extensive use of other Personal Knowledge Management Systems (PKMS) like Obsidian and Joplin. Therefore, I may end up providing integrations with those tools at some point too.
## Conclusion