Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Show HN: Personal Replit Ghostwriter (repl.co)
4 points by MLH6ft1 on July 1, 2023 | hide | past | favorite | 2 comments
What it is: A website to run a Ghostwriter like code generation assistant for free!

Backstory: Hi All, I recently stumbled upon GGML 4 bit quantized LLMs and the fact that small version of these GGML models (i.e., upto 7b) can run smoothly on a CPU! The GGML 4 bit quantized version of replit’s codeInstruct-3b model, only requires 2GB of RAM! So I quickly tested it and hosted the model on a free HuggingFace Space and it works!

Let me know your thoughts on it!



I’m not familiar with code generating models in general, so I don’t know how much to expect, but I tried one moderately simple task and one very simple task, and both times it produced something that looked like code but didn’t make any sense and used the wrong keywords.

It’s neat, and probably beyond what I would have expected a few years ago, but it doesn’t seem useful in practice.

It’s very cool that it works on CPU quickly and doesn’t take too much RAM, though! Thanks for letting me try it!


I think that's because Replit's model was trained to generate small snippets of code or parts of your code, which you can glue together later on, rather generating the entire code in the first shot itself.

But I was more surprised by the fact we can run such a model on a CPU!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: