I'm working on https://artifacts.iofold.com, a way to use artifacts (self contained html + optional assets like image/video/json) easily across agents, with feedback loop and docsend style wttribiti gates.
Have made it agent friendly enough that my teammates' agents can read and drop commennts on specs/storyboards etc, and my agent can close the loop by iterating with a new artifact version.
I'm very curious to see a benchmark for this - have toyed with the idea myself but haven't put in the hard work to test these hypothesis on extracting learning signal from deep-agent traces.
The profile as a memory primitive is pretty interesting, how do you scale it for a scoped access/hierarchical setting? (for example, overall memory of processes => customer specific => project specific memory)
I've struggled with adding evals to my AI agents for last few months, and felt that vibe evals should have a path to building a robust system down the line.
Working on a plugin for langfuse to create evals functions and dataset from ingested traces automatically, based on ad-hoc user feedback.
Have made it agent friendly enough that my teammates' agents can read and drop commennts on specs/storyboards etc, and my agent can close the loop by iterating with a new artifact version.