Things I wrote

Getting people to try these tools

I know people who are very good at their jobs, keep detailed notes, and manage complicated spreadsheets, but still haven’t tried these tools on their own files. That surprises me, and the post was a bit of a frustration post. People replied about organizational restrictions, incentives, and how hard it is to change how you work. I think they’re right. I also think some people use those things as a reason to never dig in. I appreciated the replies, though. There’s more going on than I acknowledged in the post.

Missing the old flow state

I used to really enjoy a big refactoring task. Tracing through the code, figuring out how the data moved, finding things I could delete. That could be my whole day, and I didn’t want anyone to interrupt me. I like how much I can get done with agents, but I miss that way of working. I get why some very good engineers don’t enjoy the change.

My initial thoughts on Jev

A lot of companies have problems that classical ML could solve, but labeling data, training a model, and hosting it takes work. Calling an API is much easier to get started with, which is part of why Jev interests me. I’m still wondering how much know-how it takes to use it well, and at what point you’d be better off having an LLM help you build a classical model. I wrote a follow-up about making these tools easier to use.

Prompt debt

If you upgrade the model, you should revisit the prompts and context you’ve built around it too. You can ask the new model to help with that. I’d like to see a more systematic, automated way of doing it.

Forecasting inside these workflows

TimeCopilot looks interesting because it helps you try and compare forecasting models without wiring up each one yourself. Seeing what people are doing with Jev makes me curious what they’d do with forecasting in those same workflows.

Things I’m trying

An interactive explanation of how language models learn. I built this for an upcoming talk. I wanted enough detail that people wouldn’t think the models were magic, without making it too technical to follow. More on building it here.

A remote-control workaround. I’ve been keeping one Claude Code remote session open and asking it to launch sessions in other directories through cmux. Probably a naive way to do it, but it’s working for me.

Small Jev experiments. I’ve mostly been making toy API calls to see how it works and what comes back. The computer-use demos got me thinking the most: it’s a complicated task to build out of smaller pieces like this. Here’s one browser-use example I saved, combining LangChain and Jev.

Things I listened to

Tobi Lütke on The Knowledge Project

I recognized the problem of generating work that someone else has to review. I do this to myself when I don’t think enough about what I’m trying to do upfront. I get into a slot-machine mentality, end up with something that sort of works, and then have a hard time fixing it.

• Shopify’s agent works in open Slack channels. People can see what their colleagues ask it to do and how they use it.
• Tobi talks about sending someone a large generated pull request or email. The person who generated it should understand it before asking someone else to read it.
• He asks agents for different perspectives on a decision, but still considers himself responsible for the call he makes.

Matt Pocock on The Pragmatic Engineer

Matt talks through how he uses agents to build software, including how he breaks up the work and checks the tests.

• He recommends building a small feature all the way through the system. If you build the database, application, and UI separately, you can get pretty far before finding out they don’t work together.
• Check that a test would fail without the change. A passing test doesn’t tell you much if the agent just copied the implementation into the test.
• He uses agents to review code too. Those reviewers need checking, just like the agent writing the code.

What’s the Future of Data Engineering in an AI World? - Hedgineer

The hosts discuss when to invest in data infrastructure and describe a ticket-writing agent they decided was doing too much.

• They suggest trying an agent on the actual work, with the business context it needs, before starting a big warehouse project. Doing that first can change what you decide to build.
• They suggest writing down domain knowledge in Markdown first, then moving the logic into code and tables once you understand what needs to happen.
• They stopped having the agent do so much of the ticket writing because writing the tickets was how people worked out what they were agreeing to build. It still summarizes discussions, and people review and correct those summaries.

More episodes and takeaways are in my Listening Index.

Other things that caught my attention

Posts

• Thariq on what to do with more model capability: His suggestion is to use better models to spend more time understanding users and trying prototypes before deciding what to build.
• Zach Davis on code quality: Humans never had code quality under control either. I wrote more about this: someone can now generate a lot of plausible code that your good engineers have to spend their time sorting through.
• @FundamentEdge on investment research: On what investment researchers can contribute as summarizing public information gets easier, including relationships and knowledge from people working in an industry.
• @copyconstruct on agents amplifying weaknesses: On teams producing more code with agents while making more work for the people reviewing it.
• Tobi Lütke on MCP versus CLI: He argues that agents need somewhere to run code and keep state between calls. That’s the part he wants people to focus on in the MCP/CLI debate.
• Mitchell Hashimoto on agency: Starting my business made me feel more strongly about this. I don’t have everything figured out, but every day I have to go figure it out. Seeing that work has made me more willing to try new things.
• Sydney Runkle on labeling repository issues: An example of using a decision model to label repository issues, including how she checks the threshold for assigning a label.
• @_lopopolo on progressive disclosure: How to organize files so an agent can find the relevant instructions without loading everything at once.

Articles

• Isaac Flath’s six Jev experiments: Examples include checking citations, ranking a feed, and grouping review notes. Useful for seeing what you could try with the API.
• Simon Willison on Jev and decision models: A walkthrough of how to use it, including what you can and can’t learn from the scores it returns.
• Jason Cohen on ruthless prioritization: On deciding which problems to leave unfixed, and explaining those decisions to people who keep running into them.
• Dan Luu on resetting an agent’s context: He tested starting an agent with fresh context against continuing with the existing context. Continuing worked better on that task.
• Hamel Husain and Shreya Shankar on finding AI failures (subscriber article): They look at a leasing assistant that gives a polite answer but fails to suggest a cheaper apartment. It’s a concrete example of finding a problem by reading the conversations that a generic eval could miss.
• Boris Cherny on being wrong: On changing your mind about both the problem you’re solving and how you’re trying to solve it.