pragmatist

The Time We Ran a Super Bowl Ad

I’ve got a story to tell.

It’s been 14 years, so I think we’re past the statute of limitations on this one. I’m also pretty sure nothing here qualifies as proprietary anymore.

One disclaimer: I’m working from 14-year-old memories, so I might get some of the details wrong, but I’m pretty sure I’ve got most of the story right.

Playing With LLMs As A Binary Classifier

A friend of mine leads an internal data team for hospital group.

They recently received the updated 2025 ICD-10 codes which is a coding system of ~74,000 codes that are used by healthcare providers and insurance companies for billing, tracking public health trends, and medical recordkeeping.

Basically, they got handed a plain text file with 74,260 lines and were asked to annotate the data with various additional bits of information. For example, they want to record whether each ICD code represents an infection:

• A200 Bubonic plague is an infection
• Y9366 Activity, soccer is not an infection

Annotating more than 74,000 lines is tedious and time consuming¹, so he hit up the group chat asking about how to script an LLM to do this.

This seemed like a fun toy problem to try out some stuff I personally hadn’t done yet, so I decided to see how well I could use LLMs as a binary classifier for “infection/not infection”

What I Read in 2024

Looking back, I noticed a couple of themes in my reading this year. One was biographies of people who had achieved success and then figured out how to change so that the drive that propelled their success didn’t eat them from the inside (Becoming Steve Jobs, Pep Guardiola)

Another theme was books that prompted some reflection that I would read (or listen to) in the morning as I got the kids ready for school (What We Owe The Future, Not The End Of The World, A Decent Life)

And then there is my normal mix of trying to better understand the world and pick up additional tools for functioning in it (The Laws of Trading, Thinking in Systems, Sludge, Noise, Stories that Stick). I’m happy with those books if I take away one mental model or skill that I can use.

What I Read in 2023

I read 18 books in 2023. Including 5 in Spanish. Here they are.

Monitor For The Absence Of Success, Not Just the Presence of Failures

Let’s start with a brief story: last year one of my teams built a new system that was in a limited beta. The system was dependent on receiving messages from an upstream system owned by a different team in an entirely different part of the company. It processed those messages, using them to display something to customers that had a meaningful positive impact on purchase conversion for the small number of merchants that were using the new feature.

After an intense and successful Black Friday / Cyber Monday period, most of the team was looking forward to taking a well-earned rest around the end of the year. We were confident that our on-call rotations and monitoring would let people mostly unplug and recharge, while knowing that if anything went wrong, we’d know about it.

Unfortunately, when we came back from a week off, we realized that the buyer-facing feature had essentially been disabled for a week and we didn’t know.