paulca / paulca.com Public

Update 'Using an LLM off-the-grid' blog post

OpenCode Agent 3 weeks ago 4d2ed6b7ecc65b729774731f6d84c5437a0d2a15 parent 1841116

4 files changed, +44 −25

docs/ai.html +12 −2
docs/blog/using-an-llm-off-the-grid/index.html +11 −11
src/blog/using-an-llm-off-the-grid.md +12 −12
src/ledger/2026-07-26-02-off-grid.md +9 −0

diff --git a/docs/ai.html b/docs/ai.html

index feb34ed..cb294ad 100644

--- a/docs/ai.html

+++ b/docs/ai.html

@@ -29,6 +29,16 @@ the AI work cost. Newest first.

<th>Roughly equivalent to</th>

</tr>

+<tr>

+ <td>July 26, 2026</td>

+ <td>Qwen3-Coder 30B (Ollama)</td>

+ <td>"Continue the blog post about using an LLM 'off-the-grid' with updated content reflecting AI augmentation insights and local open-source model usage."</td>

+ <td>Refined the existing blog post by removing outdated references, integrating reflections on AI as augmentation (particularly how Claude Code + Opus enabled complex webhook feature delivery at Tito), and adding context about running open source models locally for data sovereignty. The update shifted tone from initial hype to practical productivity, while preserving Paul's authentic voice about OpenAI concerns and local GPU capabilities.

+</td>

+ <td>~4 Wh</td>

+ <td>~1.6 g</td>

+ <td>A few minutes of a smartphone charging</td>

+</tr>

<tr>

<td>Qwen3-Coder 30B (Ollama)</td>

@@ -317,8 +327,8 @@ Pages. About 40 model invocations.

</tr>

</table>

-Running total: ~203 Wh,

-~81 g CO₂e —

+Running total: ~207 Wh,

+~83 g CO₂e —

just shy of the energy it takes to boil two litres of water, or the emissions of driving a petrol car about 450 metres.

<h2>Method</h2>

diff --git a/docs/blog/using-an-llm-off-the-grid/index.html b/docs/blog/using-an-llm-off-the-grid/index.html

index 9627971..d1cdc81 100644

--- a/docs/blog/using-an-llm-off-the-grid/index.html

+++ b/docs/blog/using-an-llm-off-the-grid/index.html

@@ -17,20 +17,20 @@

<h1>Using an LLM "off-the-grid"</h1>

</header>

-From the moment that OpenAI launched, I was very curious. I had met Greg Brockman a few years before and he had given me a lot of time and attention. He had reached out to me, not the other way around, and we had a delightful conversation about the early history of Stripe.

+From the moment that OpenAI launched, I was very curious. I had met one of the founders, Greg Brockman, a few years before and he had given me a lot of time and attention. He had reached out to me, not the other way around, and we had a delightful conversation about the early history of Stripe, where he was founding CTO.

When ChatGPT launched, I was naturally curious, and very excited about it. I, along with the rest of the world, got a little bit caught up in the hype about the capabilities of these new AI systems, and certainly enjoyed typing things into the box and it responding in a human-like fashion.

-A few months later, I stopped using it because in general, it was inferior to just finding information myself. Cursor was annoying, and it slowed me down more than it helped. GenAI was clearly a very good technical demo, but I didn't feel that it was actually making me any more productive.

-The next moment where the constant, incessant, unavoidable hum of AI hype hit me was Claude Code (and everyone else, particularly in Silicon Valley it seems). Here was an "agent" (hate the term) who you could give a code problem to and without directing it to the files themselves, it would figure out what to do.

-At launch, Claude Code was again very impressive. It was able to figure out what to do and where to do it, and I put a good bit of time into asking it to do my day-to-day work. The problem was that while it was good at finding the right files, and making a good effort at solving problems, it wasn't particularly good at solving problems properly, so I found I was doing as much cleanup as I was saving time with the computer writing the code. I pretty much gave up on Claude Code at some point last summer.

-Then came the autumn, and a project arrived on my desk that had a few particularly gnarly challenges that I knew that I didn't really have the engineering aptitude for, even though I believed they would make <a href="https://ti.to">Tito</a> better. With a deadline fast approaching, I described the high-level problem to Claude Code and the recently launched Opus 4.3 (I think), and while the code it produced wasn't the prettiest, we delivered the project bug-free and ready to rock, on time. Something had shifted.

-Since the arrival of Opus 4.5 a few months later, I haven't written a single line of code. I haven't talked publicly about it because I've held some shame about it. The bait-and-switch non-profit to heavy-profit motives of OpenAI spooked me. The weird god-complexes of the leaders of some of these companies. The profession to be "open", and "ethical", but in reality being very closed, and selling technology that works some percentage of the time and then the other non-zero percentage of the time can delete your production database. It's all a bit shady. Some of my best friends have decided that AI is simply <a href="https://freelancing-gods.com/2026/02/13/ai-is-bad">bad</a>, and <a href="https://writing.jan.io/2026/02/02/ai-veganism.html">not worth even looking at</a>.

+A few months later, I stopped using it. In general, it was inferior to just finding information myself. For coding, Cursor was annoying, and it slowed me down more than it helped. GenAI was clearly an extremely impressive technical demo, but I didn't feel that it was actually making me any more productive.

+The next moment where the constant, incessant, unavoidable buzz buzz buzz of AI hype hit me was Claude Code (again, me and everyone else, particularly in Silicon Valley). Here was an "agent" (hate the term) who you could give a code problem to and without directing it to the files themselves, it would figure out what to do.

+At launch, Claude Code was again very impressive. It was able to figure out what to do and where to do it, and I put a good bit of time into asking it to do things in my day-to-day work. The problem was that while it was good at finding the right files, and was able to solve some problems, it wasn't particularly good at solving them properly. I found I was doing as much cleanup as I was saving time with the computer writing the code. I pretty much gave up on Claude Code at some point last summer.

+Then came the autumn, and a big project arrived on my desk that had a few particularly gnarly challenges. The project required features that I knew that I personally didn't really have the engineering aptitude for, but I believed would make Tito better. With a deadline fast approaching, I described the high-level problem to Claude Code and the recently launched Opus 4.3 (I think), and while the code it produced wasn't the prettiest, we delivered the project bug-free and ready to rock, on time. Something had shifted.

+Since the arrival of Opus 4.5 a few months later, I haven't hand-written a single line of code. I haven't talked publicly about it because I've held some shame about it. The bait-and-switch non-profit to heavy-profit motives of OpenAI spooked me. The weird god-complexes of the leaders of some of these companies. The profession to be "open", and "ethical", but in reality being very closed. The selling of technology that works some percentage of the time and then the other non-zero percentage of the time can delete your production database. It's all a bit shady. Some of my best friends have decided that AI is simply <a href="https://freelancing-gods.com/2026/02/13/ai-is-bad">bad</a>, and <a href="https://writing.jan.io/2026/02/02/ai-veganism.html">not worth even looking at</a>.

I don't have those beliefs.

-Nor, however, do I subscribe to the hype. There are people who believe that an AI takeover of the world is inevitable and would idly sit by as we become vassals run by people you probably wouldn't trust to mind your dog let alone your kids. There are people (including me) who are happily feeding data, connecting everything to these two US firms, just for convenience.

+Nor, however, do I subscribe to the hype. There are people who believe that an AI takeover of the world is inevitable. To me it seems they would idly sit by as we become vassals of companies run by people you probably wouldn't trust to mind your dog let alone your kids. There are people (including me) who are happily feeding data, connecting a bunch of private services to these two US firms, just for convenience.

And I get it: speaking to a computer like a human and it responding is one of the most joyous ways to interact with a machine that I've ever experienced. My whole life, I've had to conform to how machines communicate, as programmed by someone who is not me. Now the machine conforms to how I communicate, and adapts to my preferences. It's science fiction in real life.

-The past few weeks, I have gone very, very deep down the rabbit hole. I've spent quite a lot pulling the slot machine lever and refining the output. (Not as much as <a href="https://bun.com/blog/bun-in-rust">some idiots</a>). The two themes of my lever pulling have been: "if people insist on using AI, I want to see if I can uncover ways in which it can be used effectively, efficiently, and for the betterment of communities, not for individual gain". The other theme is: "no engineer would be stupid enough to do this". I believe I have uncovered a lot of interesting ideas worth sharing. More on that later.

-For today though, I'm writing this in a text editor. I'm about to feed it in to an LLM, and to the best of my knowledge, it will be published on my very new and very plain website at paulca.com.

-The kicker is that doing that will not pass through any US-owned-or-controlled LLM provider. While Anthropic, and particularly Fable, remains best-in-class at software production and I have discovered a few tricks to make very nice software very efficiently, other open source models exist that can run very fast on hardware (ergo: GPUs) that until a few years ago would have been considered a commodity.

-So that's all I have to say for now. A first blog post about AI, hand-written, but posted using a robot running in a home-office in South Dublin, powered by an open source model, where none of the data leaves my control or my jurisdiction without my consent, ironically landing on github.com, a company whose services I once loved dearly, made by people who became some of my closest friends, that I hope to soon leave behind and never look back.

+The past few weeks, I have gone very, very deep down the rabbit hole. I've spent quite a lot of time (and tokens, 🤮) pulling the slot machine lever and refining the output. (Although not as much as <a href="https://bun.com/blog/bun-in-rust">some idiots</a>). The two themes of my lever pulling have been: "if people insist on using AI, I want to see if I can uncover ways in which it can be used effectively, efficiently, and for the betterment of communities, not for individual gain". The other theme is: "no engineer would be stupid enough to do this". I believe I have uncovered a lot of interesting ideas worth sharing. I'll post about those separately.

+For today though, I'm writing this in a text editor. I'm about to feed it in to an LLM, and to the best of my knowledge, it will be published on my very new and very plain website at paulca.com. [Update: it worked! I'm now going to send these edits to a slower but more capable model after some "tuning" and my expectation is that I will paste the changes and the next thing I'll need to do is refresh my browser]

+The kicker is that doing that will not pass through any US-owned-or-controlled LLM provider. While Anthropic, and particularly Fable, remains best-in-class at software production and I have discovered a few tricks to make very nice software very efficiently, other open source models exist that can run very fast on hardware (ergo: GPUs [and more slowly, on CPUs]) that until a few years ago would have been considered a commodity.

+So that's all I have to say for now. A first blog post about AI, hand-written, but posted using a robot running in a home-office in South Dublin, powered by an open source model, where none of the data leaves my control or my jurisdiction without my consent, ironically landing on GitHub, a company whose services I once loved dearly, made by people who became some of my closest friends, that I hope to soon leave behind and never look back.

Let's. Fucking. Go.

</article>

diff --git a/src/blog/using-an-llm-off-the-grid.md b/src/blog/using-an-llm-off-the-grid.md

index 97a8615..5a642be 100644

--- a/src/blog/using-an-llm-off-the-grid.md

+++ b/src/blog/using-an-llm-off-the-grid.md

@@ -2,32 +2,32 @@

title: Using an LLM "off-the-grid"

date: 2026-07-25

---