2025-07-20

Last modified: 2025-07-21 10:31:20

Rails

In my ongoing quest for a new programming language, I'm finally trying out Ruby on Rails.

This is because I have listened to some of dhh on Lex Fridman, and he makes it sound really good.

So far I am pretty impressed with it. It is way more "batteries-included" than I expected. It even has a tool called kamal which is like a much better version of ngindock. I think the philosophy behind Rails is quite close to my own philosophy in many ways, so I ought to get on with it.

The things I care about in a programming language are:

• not too verbose, I want to see more logic in one screenful of code
• good async I/O support, to avoid the "callback hell" style
• static analysis, so that Cursor works well
• good library support, to avoid rewriting everything myself

Go is good at everything except verbosity. The verbosity of Go is really starting to grate on me. What in slang or C would be atoi(s) in Go is 4 lines of error-handling. Yes you could ignore the errors but that is "poor style" in Go, it makes you feel like you're using the wrong tool. And in any event you can't ignore errors inline, so you can't do foo(atoi(s)), you need an intermediate variable so that you can discard the error.

So, Rails.

I love that it comes with "Hotwire". I recently read "Hypermedia Systems" about htmx, and I liked the idea, but Cursor doesn't do very well at writing htmx code, you end up with pages with nested navbars etc., so having hotwire baked into Rails might work better.

I also anticipate that if the language is a bit less verbose then I don't care so much about having Cursor write all the boilerplate for me. Let's just not have all the boilerplate! Great idea. But I think there is fundamentally a conflict between wanting good static analysis so that Cursor works, and wanting less verbosity so that it's easier for me to write by hand. I basically want dynamic typing with static analysis, which I think is not actually possible in the general case.

Last night I was talking about how we could have "static analysis" with LLM-generated unit tests created on-the-fly, and then running parts of your code behind-the-scenes in a sealed side-effect-free environment.

An empty Rails project (rails new foo) has 1800 files in it (find foo | wc -l) which seems absurd. That makes me thing maybe this is not the way. But let's carry on.

I'm trying to get kamal to deploy an empty Rails project to my Ubuntu server, just to see if it is actually able to deploy multiple applications to the same machine. ChatGPT tells me it is possible, but could be lying.

One possible issue is that it assumes the Rails app is at the root of your git repo, whereas in a "monorepo" type scenario, I kind of want the Rails app to be a subdirectory of the git repo. But let's try the Rails way for now. This may be an exercise in learning to love the happy path.

I am having trouble getting kamal to even build an empty Rails app. Ah, I need to login to dockerhub using my access token on both my desktop and my server.

Now having a problem where it's saying it can't find my Dockerfile.

Oh! Is this a Snap problem? Is the problem that Docker running in Snap has a separate /tmp so it can't see any of the files in /tmp/kamal-clones? What a scam. Why is snap so shit.

I may have to install Docker with curl | sh instead of snap. Great, snap remove docker is hanging. Why is snap so shit.

I have manually killed the dockerd process, but now snap is hanging on Save data of snap "docker" in automatic snapshot set #14. It is tarring up all of the docker state so that I can restore it. Which I will definitely not want to do. If I even wanted snap docker back I would be happy with it completely reset. Argh. I killed the tar process but snap noticed and aborted my snap remove.

snap remove --purge docker did the trick.

OK, I have the "official" Docker version now.

kamal setup is getting further, it looks to be building the Docker image, seems to be hanging at #22 exporting to image. Is it just taking a long time to upload to dockerhub? Yes, I think so.

I thought kamal was meant to manage getting me an SSL certificate, but this documentation suggests they have dropped that feature? Unclear.

It eventually failed with "Error releasing the deploy lock: Exception while executing on host [...]: Timeout, server [...] not responding."

I think it may have timed out because the dockerhub stuff took so long, I know this VM seems to drop your SSH sessions if you leave them idle too long.

Running kamal setup again says it can't work because the deploy lock is already in place. I'll manually run kamal lock release and try again.

OK, so the only existing application on this box is in Docker, but I'm using nginx as a reverse proxy. kamal-proxy can't start because port 80 is already in use. Let's turn off nginx and see what happens. It would be acceptable if I can add my existing application to kamal-proxy I think.

Deploying via docker is quite slow. It has to do lots of Docker commands and they all take over a second. This may not be the way.

Now I'm getting:

  ERROR (SSHKit::Command::Failed): Exception while executing on host [...]: mkdir exit status: 1
mkdir stdout: 4af296a66686db44b89ef4b63ab90c596ae6325bb051afe3287dc022bf929d16
mkdir stderr: open /root/.kamal/apps/railsweb/assets/extracted/web-9b529edb15b6f058264a0c708d80f56df84634d5/.manifest.json: permission denied

What's the problem there?? I think this tool might just not work very well.

Ah! https://github.com/basecamp/kamal/issues/1437

Are you kidding me?? It's because of snap! Again! Why is snap so shit.

The problem with snap is that they decided they didn't just want to make it easy to install software, they wanted to add "isolation" to everything. But that is a blunder! Unix is good because it is small tools that work together, but if you isolate all the tools from each other then they can't work together any more.

OK, with proper Docker on the server as well, kamal deploy has successfully (?) deployed a Docker container at the hostname I asked for, and got an SSL certificate for it.

However the index page is not the same on the server as it is on my local machine. Is this just the difference between the "development" and "production" environments?

Let's add a custom page and see. How do I do a custom page without an associated database table?

ChatGPT is saying I should make a controller called "pages", with no model. Man this is taking a long time to deploy. I could probably reinstall rails directly on the server every time faster than I can build and transfer the docker image.

Yay, my new page is up. It took 3 minutes to deploy, outrageous.

So I'm guessing the index page is just a development vs production issue.

So now the next thing is can I use kamal to deploy a non-rails application to the same server? The one that was already running there.

docker stderr: Error: target failed to become healthy within configured timeout (30s)

I guess my app needs to give a 200 OK from /up.

OK, I keep forgetting that kamal won't deploy anything that's not committed to git. Uncommitted changes don't get deployed because it clones your local working repo.

OK, my app listens on port 8180 instead of port 80. How do I tell kamal? It's proxy/app_port in config/deploy.yml.

Great! It works! My other docker-based service is now deployed with kamal to the same box as the Rails app. This is the stuff dreams are made of.

The only drawback is how slow all the Docker stuff is. It takes 56 seconds to do a no-op deploy of a not-very-complicated program.

Anyway, back to Rails. I should make some dumb CRUD app with Rails before I use it for anything real. What shall I make tonight?

Nothing in Software Ideas looks to be suitable. A video alternative to imagebin could work but seems a bit on the trivial side?

What about a webmail tool? So I'd be looking to make a super minimal version of https://www.hey.com/ except I'd have it fetch emails over IMAP periodically instead of needing to be the main mail host itself. Potentially too hard to be getting started with?

Meh, Ruby has an IMAP client, let's give it a go? I'll chat with Claude 4 Opus about the design but do the programming the old-fashioned way.

So I'm imagining that we'll have somewhere you can set up your user account, which means giving it your IMAP/SMTP config. And then you can login to the app using those same credentials. For every configured user, we'll periodically download emails using IMAP, but we'll completely ignore the server-side folder configuration. We'll import the emails into our own database made of ActiveRecord objects or whatever. And we'll copy HEY's idea of "The Screener", so if you receive an email from an email address you haven't explicitly allowed, it goes into the "The Screener" instead of the inbox.

So models we need are:

• Account (with IMAP/SMTP settings)
• Email (contains an entire email, and reference to the Contact for the sender)
• Contact (email address, name - we can pick up the name the first time we see it, whether they're allowed past the screener or not)

And then potentially we need a type of worker model or something that will handle the background job of polling IMAP for new emails. Potentially we would eventually want to support the IMAP push feature, whatever it is called, so that you see new emails as soon as the server sees them. We might eventually want to have websockets (Active Cable?) push new emails to the browser as soon as they arrive also. New emails that make it to the inbox should trigger a browser notification, but new emails that go into The Screener should not.

I can have a muck about with it and see how far I get and how much fun I have.

I pasted some of the above to Gemini 2.5 Pro in Cursor. I couldn't find Claude 4 Opus.

It suggests:

• "User" instead of "Account" is the rails convention
• authentication with our app should use a separate password from the IMAP/SMTP credentials; maybe I agree, I just thought it was neat to keep it simpler
• Rails has built-in support for encrypting columns in the database and I should use this for IMAP/SMTP credentials; unsure how this works, need to look it up
• use "Active Job" for polling the IMAP server, suggests using the "good_job" gem to schedule jobs
• IMAP "IDLE" mode is too complex, skip it for now, but if I did want to do it then I should have a separate daemon that somehow triggers the poll job to run when a new email arrives, obviously I could do that with a webhook, maybe can do it with Ruby

And actually, that last point gives me an idea for a maybe-useful service: we have a program whose only job is to poll an existing IMAP inbox (with IDLE mode of course), fetch new emails from it, optionally delete them from IMAP, and send them to your webhook. There is https://github.com/watchdogpolska/imap-to-webhook and https://emailengine.app/ - perhaps we should use that for AI Test User? Anyway, back to our little Rails webmail app.

User fields:

• email
• name
• password (hashed)
• imap_host
• imap_port
• imap_starttls
• imap_username
• imap_password
• smtp_host
• smtp_port
• smtp_starttls
• smtp_username
• smtp_password

Email fields:

• user (the owning user)
• raw_message
• to
• from
• subject
• body_text
• body_html
• date
• somehow have references to attached files?

Contact fields:

• user (the owning user)
• name
• email
• screened_in

How could we do full-text search of emails? I'd want to be able to do full-text search over body, subject, from, and to. Gemini reckons that SELECT * FROM emails WHERE user_id=? AND body_text LIKE ? is not viable. Is it actually not viable? I told Gemini the app is only for me. It conceded that using LIKE would probably be fine but insisted that Postgres has higher-quality search than simple substring matching. I'm not sure I definitely want higher-quality search than simple substring matching. Substring matching is easy to understand. Plus I don't want to have to faff about with Postgres. So we're doing search with LIKE.

OK so I have used rails generate model ... to turn the above bullet point lists into "models". Now what? I guess I can start by using the Rails console to setup my user details for now, and then work on making it grab emails out of IMAP.

I am having some trouble getting Net::IMAP to authenticate with my mail server. Is it maybe not able to do TLS?

No, it works fine if I type out the imap code manually in the Rails console. What am I doing wrong here?

If I hardcode the password it works, but if I grab it out of @user.imap_password it doesn't. Is rails perhaps trying to be clever and prevent me from accidentally leaking a password over the network?

Oh, lol, I'm an idiot. I had a slightly wrong password set in the User object. So, just the obvious thing.

The rails console has some annoying pager thing that then permanently breaks the terminal scrollback.

Anyway I now have it logging in to imap. It fetches out emails. It is going to fetch too many, I probably want it to only fetch emails it hasn't seen, and nothing older than a week or so.

But it doesn't manage to fetch them all because of some encoding issue that I can't fully understand because of the broken scrollback.

Put IRB.conf[:USER_PAGER] = false in ~/.irbrc.

So my issue now is that I am trying to save 8-bit binary data into a database as utf-8, I think. So this is very annoying, I don't even understand why this is a problem. Just copy the bloody bytes! Why are you trying to encode it?

OK, for now I made it bodge it into valid utf-8 by replacing illegal characters. Now there is another error:

app/services/email_fetcher.rb:28:in 'block (2 levels) in EmailFetcher#fetch': SQLite3::SQLException: unrecognized token: "'PK (ActiveRecord::StatementInvalid)
":
INSERT INTO "emails" ("user_id", "raw_message", "to", "from", "subject", "body_text", "body_html", "date", "created_at", "updated_at") VALUES 

And then the values, including most of the email. It looks like the query is getting truncated?

Actually could this be an SQL injection vulnerability? I need to produce a minimal test case and find out.

The issue is that the string contains a nul byte. If you try to use Rails to insert a column with a nul byte, I guess the issue is that Sqlite thinks the SQL statement ends at the position of the nul byte. So an attacker can truncate your query, but only in the middle of a string, which means it's not really SQL injection because there is no way to use it to make a valid statement.

https://github.com/rails/rails/issues/55373