I made something silly to this effect last year:

https://gitlab.com/AsbjornOlling/nixllm

You provide the prompt to a nix function, and it deterministically generates the source code using llama.cpp.

LLMs as compilers