Ask whether the work has a measurable signal
One point from Karpathy's post stood out to me: if a metric is reasonably efficient to evaluate, an agent can try changes and measure whether they help.
I wanted people to consider whether their own problem fits that description, even if they don't work in AI.