Google+ would like a word

Google Wave is waving

I was @ Google I/O at live keynote when they announced/demoed it. Ironically, at the exact same time, me and the guy next to me said (in different words), "Why do we need this?" That was a collaborative Google Wave moment IRL.

Google gears is grinding

Gears has been superseded, not canceled

Google Buzz is buzzing

It’s clear that maintaining everything in perpetuity is how you become #2

Yahoo would like a word with you

Actually, I think it's drowning.

The capabilities of local LLM text-to-image is honestly pretty damn impressive. IMO, I think local image generation is currently ahead of local code generation. I can get an image in seconds locally with the quality being way higher than what I'd expect from a local model. However with coding it's much slower and much less impressive. I'm sure there's a reason for this and I'm not an AI expert so I'll let the smarter folks tell me why, but that's just been my observation thus far.

I've played with diffusion models on and off since the first release of Stable Diffusion - just for amusement, without a particular goal.

Recently, I've been helping a friend's wife with some basic vector images for her sewing hobby (she has what is essentially a CNC sewing machine) and have been super-impressed with FLUX.1-Kontext, which I've been running on my Macbook Pro with mflux. Its ability to (for example) take a photo of a human or an animal and return a line drawing which is recognisably them (rather than just a generic similarish image as I've experienced with other models) is excellent.

It's an older model now, but (AIUI) has the text-handling features baked in, and in my various testing is very reliable at giving me the outputs that I want, without the randomness I've experienced previously. It's big and relatively slow (~3 mins per 512x512 image edit on my M1 Max Mac) but excellent to work with. It's also very straightforward to set up, without the harness complexity of e.g. comfyui.

is the cnc sewing machine an off the shelf model or something DIY? I'd love to hear more

Off the shelf - it’s a Brother. It prints via a proprietary file format (.PES) but there’s an extension for Inkscape that supports creation and export.

Sounds ripe for vibe... sewing

I think they meant an embroidery machine

How are you converting the bitmaps into vector images?

This is currently the weakest part of the workflow. I'm tracing the output images with VTracer [0] and then finishing them manually.

Through a combination of careful initial prompting (e.g. requesting line drawings, requesting limited colours ["posterization" works] if you need colour at all, requesting background removal) and then using the settings in VTracer to limit the colours used for the resulting SVG to the minimum needed, you can end up with an SVG that doesn't take too long to clean up in Inkscape. It usually takes a few minutes of deletions and path unions to produce something usable.

Diffusion models outputting SVGs would obviously be very interesting for this, but this is probably a small niche compared to everything that diffusion models can do with raster images, and I've not found anything usable online so far. LLMs (understandably, given their core technology) are still pretty bad at creating SVG art.

[0] https://github.com/visioncortex/vtracer

Claude and I have spent most of the summer on exactly this problem for Skechworks (skechworks.com), a free little Mac design app I made to help my own workflow that is very similar to yours. I use it for my custom coin shop The Achieve Mint (theachievemint.com).

It runs VTracer under the hood too. Most of the cleanup you're doing in Inkscape could possibly be pushed to before and after the trace?

Skechworks picks the palette before the trace and forces it again after. I quantize to 8 colors with pngquant, trace, then snap every fill back to the nearest palette color. VTracer fills each region with its own average color, otherwise you get hundreds of nearly identical grays. One design went from 279 fill colors to 3.

I upscale 2x before I quantize. The anti-aliasing is what tells the tracer where an edge really sits. Quantize at 1x and you throw that away, and small text comes out lumpy. Related gotcha: filter_speckle is an area, so don't double it when you double the image.

I also flatten transparency onto white first. VTracer counts every alpha value as a different color. I had a 5 color picture that it saw as 688 colors, and the trace took 6 GB of memory. Flattened, the same picture took 300 MB.

For the background I sample the whole edge of the picture. If 85% of it is one color, I delete every path in that color that touches the edge. White inside the drawing never touches the edge, so it stays.

For black and white I threshold with Otsu, then trace in color mode with hierarchical stacked instead of bw mode. bw mode hands you one giant black path full of holes. Stacked gives you every region as its own shape, which is much easier to edit. You have to pad the image with the background color first, or VTracer decides your subject is the canvas.

Your FLUX step is the same trick I use for photos and messy art. I have an image model redraw the picture as flat art, then trace the redraw. That came out about a quarter the file size of the best direct trace I could get. The catch is that the model moves and resizes the subject a little every run. I measure the box around the ink in both pictures and fit the trace back onto the original.

What I haven't solved is curve fitting. vectorizer.ai gives you real arcs and straight lines. VTracer gives you splines, so circles are never quite circles.

Remember that quality output is a necessary but insufficient property of a generative model.

Prompt-adherence is really hit-or-miss—especially if one lacks the visual vocabulary. Likewise with coding, I find junior devs don't think to prompt re: respecting this-or-that interface, or refactoring to point-free style, etc.

So, as others have said, the artist knows better.

I think there was a lot more brainpower invested in the media generation side of things. The noise-based diffusion technique is further developed. It had a discovery of applying a physics-based understanding of Brownian motion to guide it. Image generation has comparatively simple training process - this is an image with dog, and without dog (contrastive learning).

Might be worth to watch the diffusion based LLMs.

I mean I’m sure it’s the reverse for an artist. They would be less impressed with the image and more impressed with the code quality

To generalize, LLMs are great at what you are not skilled at.

That's how they're sold, isn't it?

Well, not exactly. Many, many companies and managers think that they can replace skilled people or have unskilled monitoring them.

The point I think is interesting is that this is just 7B. The current SOTA 7B LLMs are barely usable for quite simple coding.

Text is in a sense way harder to do than images because of radical nonlocality. A word at the start of one paragraph can directly influence the meaning of a word five paragraphs away. Whereas images typically represent the real world, or at least a spatial domain, which gives you a lot of structure 'for free'. If you are drawing a human, you can make a reasonable guess where their hands go in relation to their face. If someone hands you the first half of an essay, finishing it is not trivial.

That's a fair statement, I agree. I'm quite an abysmal artist so I could be a victim of my own bias here

I think there's a lot of setup and context required for an AI agent to consistently write good code. Once the agent has these guard rails in place I usually get great quality- far better than what I would write in most cases.

I think where things get dicey is being able to write in any language. I write and review code in many languages and frameworks I'm not fluent in, so it's hard for me to distinguish between working code and great code. I can spot when the fundamental logic is wrong, but when it comes to "best fit" choices I'm clueless.

The issue is that in order to have the agent write good code, you need to implement standard SWE best practices. But that also means a lot of manual intervention in terms of writing specs, checking acceptance criteria, and reviewing code. So you end up spending a lot of time on managing your agent, which means you won't get a 1000% productivity gain, you get maybe 50 or 100, possible less in some areas and with some issues.

> implement standard SWE best practices

The thing is, if you follow SWE best practices indiscriminately, then you ll have a shit code base in no time.

There is no silver bullet, and no replacement for experience and mindfulness.

A 1000% productivity gain is quite possible on solo greenfield projects.

At work, with a team and code reviews, the 50%-100% figure seems much more likely.

This can probably move towards the more spectacular productivity gains as the AI's output becomes more reliable, people realize this, and less time is spend on code review and cleaning up the output.

50 or 100 seems unlikely. Even with all these improvements, custom setups and guardrails it just isn't that much faster for me.

You get 0% productivity gains if you are careful and actually reviewing the code the LLM produces. The only way to actually get the massive productivity gains that AI bros claim is to throw quality out the window.

I'm curious as to what guardrails you've tried.

This is something I have been trying to get right as well. I've attempted to use lots of linting and things like strong typing, duplicate checks, cyclomatic complexity, and robust tests. However, I still happen to find issues, which requires me to look at the code (at least at a high level)

For example, I can say "Don't repeat yourself, and don't re-write helper functions" and I will even have a duplicate linter check, but inevitably the LLM will always want to re-write a similar yet slightly different helper function. Like it will always want to re-write something small like a trim() or a toString() function in every file.

I find that if I leave an instruction in AGENTS.md to "do not do X", there's a good chance the agent will forget it.

But if I add a separate post-implementation pass to "find and fix X" by the agent, it'll usually find and fix the issues.

So I've started doing it for everything from naming conventions to duplicate code to other problems. It does cost more tokens, but now I get less frustrated at having to fix basic issues in the PRs.

> Don't repeat yourself, and don't re-write helper functions

It's worth reflecting on why these things are important to you and whether they remain important in an agent-developed codebase.

Yes, they are both still very important for consistency throughout your codebase and any user interface for it.

Consistency of behaviour and UI can be tested.

Have you tried something like "Always consult the utils/ package before writing helper functions. When adding a new generic helper function justify it in your design or PR description."

I have better luck telling it positive things rather than lots of "never do X" style things.

That's a good idea to give more positive instructions as opposed to negative instructions. I think you've stated it well, I suppose the problem with negative instructions is that the LLM doesn't know what to do instead.

"Never re-write a helper function" vs "Always search for helper functions before writing one" the "never... " one doesn't tell the LLM what to do, so it would have to make the logical leap from not re-writing to knowing that it should search. While it's a minor leap to make in isolation, I suppose stacking many negative rules in an AGENTS.md would assume that every time it will always make that logical conclusion on what to do.

I would say that it is like gardening. If you let them go havoc from the start, the weed will take over. If you keep focusing on removing the weed and enforce specific standards and practices over the code base and it keeps growing, over time LLMs start to suddenly follow that and they don't make so much slop anymore. At least that is my experience. But I force specific audit agent after every added feature which says them to force compliance with AGENTS.md and check the consistency with the code base.

Rust is a serious contender for the next mainstream, widely adopted low level language.

I would say it already is. But it takes time for all software to be rewritten in Rust.

It's worrying to say the least. Today it's "get this working", tomorrow it's "can it also do XYZ?", next week it's "we have a 40% spike in crashes, you MUST resolve this IMMEDIATELY!"

Meanwhile the devs are furiously asking AI how to fix it, every flavor of every model will give you a different diagnosis, GH Copilot will throw a million high/critical at you, and you still have no idea if it's fixed or not.

It's the exact reason why you still need to know the languages you're using despite what leadership/product teams demand.

Yeah. It’s also far too tempting and far too easy to keep adding features if you don’t make yourself disciplined about it (which is okay if you’re in a position to control that, not so good if it’s the higher-ups demanding it).

He's incapable of empathy, his only other option would be Lake Trump. He is quite literally the American Nero.

Nero was just a poor boy who wanted to play the fiddle. Are you sure Caligula isn't a better comparison? He's the one who constantly tried to own the ̶l̶i̶b̶s̶ senate. That's what the whole deal with the horse was about.

I think historians might have something to say about Nero just playing the fiddle. I mean, he literally ordered the murder of his own mother.

I'm reading Iain McGhilchrist's The Master and his Emissary. There's a bit about right brain damage correlating (fact?) to lacking empathy.

    Patients with right frontal deficits, but not left frontal deficits, suffer a change of personality whereby they become incapable of empathy.

Perhaps try reading more history about Nero.

Lake America, Gulf of America, New America, South America

It's consistent. This administration is a colossal failure.

Rust is such an awesome language. The execution times are blazing fast.

I just wish Rust itself compiled faster. Serde, proc macros, it's all so slow.

I'm going to start prototyping systems in Go, then porting them to Rust when they're more concrete.

I feel as though the Rust code LLMs emit is generally higher quality than most other languages, perhaps due to the error handling being a chief concern of the language, but the iteration on making changes and running tests is so slow.

Rust could grow into "the final language" if it wasn't so slow to compile. Assuming the WASM bridge also continues to improve.

LLMs being good with Rust makes sense, because the language itself gives you more safeguards than many others. What's good for the (human) gander is good for the (agent) goose.

LLMs are ungodly good at Rust, and I say this as a 10-year veteran of the language.

My problem is that now iteration with LLMs is so quick when prototyping new things that I now spend an inordinate amount of time waiting on agents to compile Rust and run tests. It's a huge slow down.

I still want to serialize production logic to Rust (and TypeScript for the frontend as the WASM for frontend ecosystem isn't there yet), but it's really slow to iterate. I'm starting to build prototypes in Golang now and then convert them to Rust when I'm done.

Hopefully the Rust team makes strides over the coming years to address this. I know they have a lot of irons in the fire for many speedups.

But once you start to close the loop (as in allowing the LLM to build and run its changes) what’s good turns suddenly bad again. Long build times also increase LLM turnaround times. Especially bad when you have a fast model.

Quite surprising for an ahead of time compiled language. /s

I still have serious questions about the validity of the ChatGpt hugging face debacle. How is it that OpenAI being the tech giant they are, didn't have a completely air gapped environment for this to run in?

I'd say it is because of the time factor. It is one thing to have a lot of money, it is another thing to have robust systems that have been developed and tested for years. Money can "buy development time" only up to a certain factor.

I guess the sandboxing problem, that is easily giving access to enough resources while restraining the critical parts is still open for most of the cases, given all the startups and bit tech companies (docker, etc...) working on their solutions.

If they wanted air gapped environment they would’ve it. I mean, if you want to sabotage your trial by hard constraints you can do it, or you do not do it to see interesting results. They even said it that some constraints were disabled for the test.

> How is it that OpenAI being the tech giant they are, didn't have a completely air gapped environment for this to run in?

They don't care.

Yeah - and if they wanted some cheap PR, that's one way to get it.

He also hit the nail on the head- reaction videos are just low effort click bait.