4 Comments
User's avatar
Philip's avatar

Is it just the accessibility tree being returned? Need to try it out later

Tal Raviv's avatar

Oh, that's super interesting! I didn't make that connection!

You inspired me to look at the accessibility tree using Chrome Dev inspection tools for that same web page, and it's definitely similar but definitely not one-to-one either.

So I wouldn't be surprised if there was a lot of inspiration here, maybe adaptation of the same tools.

Neural Foundry's avatar

This is brilliant hands-down. The YAML conversion detail is what I've been looking for becuase I kept wondering why agent responses felt so diffreent from regular browsing. I've noticed my own Cursor setup does this selective approach too and it makes total sense now why some pages load faster than others. The tradeoff between DOM extraction and screenshots kinda explains those weird latency patterns I've been seeing.

Tal Raviv's avatar

It's super interesting. How would you describe how they feel so different from regular browsing? I'm curious how you sensed that, will help me piece the puzzle together!