The decision tier can now see the screen

TypeSafe's decision tier now has eyes.

Jev 1.14 shipped last week and you can now pass screenshots to the state, use box questions to find specific items in an image, and take advantage of 262k tokens of context.

I can imagine some interesting use cases for these features. For example, as of now the only thing between a browser agent and an irreversible click is a hand-coded allowlist, which can't see the screen, or a second LLM being asked to review it, which gives you an opinion.

Jev gives you a number. Is that button a send, submit or pay action, with a probability. Is this screen even part of the authorized task. Where exactly is the thing the agent claims it's clicking, so you can check whether the agent is making up its own UI. And across the whole session history, has it quietly wandered.

It’s still experimental though. Box returns a box even when nothing’s there and requests needs the TypeSafe-Danger-Zone-Images header.

But the shape is right. The planner proposes, the decision model adjudicates, and now the adjudicator can see the screen.