Scale Hardening
Today I decided to tackle scale hardening. I’d rather build a bit of capability early, while the data and the codebase are still small, than bolt it on later.
The technical thing I wanted to get ahead of is a cap in SQLite: a single query can only use 999 variables, and long-running TV shows hit that easily because they carry so many cast and crew.
Before, the code grabbed everyone who worked on a show and asked the database to look them all up at once, to find other shows those same people were in. That group can get big, big enough to hit the cap. So the fix is to stop asking about everyone at once. For the cast and crew, I let the database do the matching on its own. That’s called a subquery. For the weighted list, I ask about the people in small groups instead of all at once. That’s called chunking.
That was interesting to learn about.
For a more in-depth explanation, read this ADR.