I would put myself in the 100x camp but my start in AI (and programming) was symbolic approaches in lisp/scheme, then I put that away for about a decade and got into NLP and then got into question answering and search/information retrieval. I have read probably at least 20 papers on LLMs a week every week since December 2022.
So I have some familiarity with the domain but I also would not exactly call myself an expert on LLMs. However, I would say I am confident I mostly understand what is required for good results.
Getting the best possible results requires:
- an LLM trained to have the "features" (in the ML/DL sense of the word) required to follow instructions to complete your task
- an application that manages the context window of the LLM
- strategically stuffing the context window with preferences/conventions, design information, documentation, examples, your git repo's repo map, and make sure you actually use rules and conventions files for projects.
Do not assume the LLM will be able to retrieve or conjure up all of that for you. Treat it like a junior dev and lead it down the path you want it to take. It is true, there is a bit of micromanagement required but Aider makes that very very simple to do. Aider even makes it possible to scrape a docs page to markdown for use by the LLM. Hooking up an LLM to search is a great way to stuff the context window BTW, makes things much simpler. You can use the Perplexity API with Aider and quickly write project plans and fetch necessary docs quickly this way; just turn that into markdown files you'll load up later after you switch models to a proper code gen model. Assume that you may end up editing some code yourself, Aider makes launching your editor easy though.
This mostly just works. For fun the first thing I did with Aider was to write a TUI chat interface for ollama and I had something I could post to github in about an hour or two.
I really think Aider is the missing ingredient for most people. I have used it to generate documentation for projects I wrote by hand, I have used it to generate code (in one of my choice languages) for projects written in a language I didn't like. It's my new favorite video game.
Join the Aider discord, read the docs, and start using it with Gemini and Sonnet. If you want local, there's more to that than what I'm willing to type in a comment here but long story short you also need to make a series of correct decisions to get good results from local but I do it on my RTX4090 just fine.
I am not a contributor or author of Aider, I'm just a fanatical user and devotee to its way of doing things.
First responder who put themselves into the 100x category. Do you mind sharing what setup you use on your local RTX4090? Like what models you run and what kinds of prompt you feed it? I'm contemplating building an LLM rig to do some experimentation.
It also sounds like you've read a lot of papers (20 per week since 2022 December would equate to 2000 by today). Do you have any favorites that really stood out to you?
Aider, AIchat (both are TUIs), and crawl4ai are my daily essentials. I don't even bother with other coding agents or IDEs like Cursor although I did try it and others throughout 2024 but this year I'm just doing those.
I would say unless you can find cheap 3090s (a pair would be good for most models, anything bigger and maybe you should buy GPU time or use options like Groq and Gemini; Sonnet 3.5 is still a very good model for Aider use) the cost may not be worth it for new hardware. I also have a 2080Super I can run other models on and sometimes if I need AIChat and Aider with a local model that's what I do, IBM Granite on the 2080S and codegen models on the 4090. FWIW most laptops will be great for RAG chats with AIChat and any of the Qwen or IBM models.
I constantly model hop but I always IBM Granite models for RAG in AIChat. I am currently testing out Bolt.new and am in the hackathon, but when I'm done with that I'm going to try Tesslate's latest UIGEN model extensively, it seems to be quite decent but I imagine some prompt engineering will be needed to get truly polished Tailwind CSS out of it, not a big deal as I'm sure that is a large part of Bolt's strategy with their use of Claude. I would like to find a local multi-modal vision model to help with getting a more informal vibe coding workflow going with frontend stuff but I haven't gone looking yet tbh.
For most of my code generation (most of which is python) usually Qwen2.5-Coder 14B is really good, I just have to load up context with whatever materials I'm using. Also I recently switched to LMStudio for inference instead of Ollama.
OpenAPI Generator is a game changer, I'm currently using that and Aider and trying different models for going over a project I have right now building something that integrates the API of some backend stuff my client uses, there are thousands of API calls in this and OpenAPI Generator takes the spec and spits out a library in my language of choice. Then it's a matter of using Aider and AIChat to build using the generated library.
I use Perplexity and Claude for making my development plans (which go into AIChat) and then sometimes depending on how big the whole plan is I might split it up. Also repomix is extremely good and useful, especially if you need to fix someone else's codebase.
If you are building a project with a frontend and you roughly know the anatomy of your project and what your core UX is, make the frontend first, make your UI components and get them lined up where you want them, doing this locally is totally possible and could save you money if you do the rest of your project with Gemini.
You might find that you are building the same thing over and over again, of course code reusability is as big of a deal in using LLMs for code gen too. Turn the things you rebuild often into something you can just user over and over. I have some UI components I'm doing this to but to.
This stuff is hard at first, I will admit, but a lot of what works for me super well is just tweaking a lot of random conventional dev wisdom I've picked up over the years and treating LLMs as a code completer, question answerer, task manager/planner, and editor. I think pair-programming with Aider and limited use of scripting (Aider's scripting potential is yuge) goes such a long way. I am finishing projects I didn't really even want to attempt or waste time on before because of LLMs.
My favorite papers seem to mostly revolve around steering attention in LLMs, but the one that I think everyone should read is Textbooks is All You Need. If you curate, if you provide high quality data, you get high quality results. Also, I recommend everyone to learn about test time compute (which I plan on exploiting with small models soon and possibly some homebaked PEFTs) and in context learning.
Anyway I'm starving, waiting for groceries to show up so I can start cooking dinner so I may have missed something here, but the most important piece of advice I can give anyone is to join every single AI coding related discord or IRC channel you can possibly find and check it every day and ask them stupid questions and EXTENSIVELY USE NOTE TAKING! I handle too much information to reliably hang onto it for more than a couple of months so everything goes into the obsidian vault. The stuff I haven't figured out on my own I learned off of other people in these places. Fully read the docs for Aider and AIChat too, they have many key features.
If there's something particular you want to do I might have better more specific advice.
funny enough I am AuDHD but I just smke w*d (low basal dopamine gang gang) and no-life this stuff because I made the classic blunder of turning my hobby into my jobby
Getting the best possible results requires: - an LLM trained to have the "features" (in the ML/DL sense of the word) required to follow instructions to complete your task - an application that manages the context window of the LLM - strategically stuffing the context window with preferences/conventions, design information, documentation, examples, your git repo's repo map, and make sure you actually use rules and conventions files for projects. Do not assume the LLM will be able to retrieve or conjure up all of that for you. Treat it like a junior dev and lead it down the path you want it to take. It is true, there is a bit of micromanagement required but Aider makes that very very simple to do. Aider even makes it possible to scrape a docs page to markdown for use by the LLM. Hooking up an LLM to search is a great way to stuff the context window BTW, makes things much simpler. You can use the Perplexity API with Aider and quickly write project plans and fetch necessary docs quickly this way; just turn that into markdown files you'll load up later after you switch models to a proper code gen model. Assume that you may end up editing some code yourself, Aider makes launching your editor easy though.
This mostly just works. For fun the first thing I did with Aider was to write a TUI chat interface for ollama and I had something I could post to github in about an hour or two.
I really think Aider is the missing ingredient for most people. I have used it to generate documentation for projects I wrote by hand, I have used it to generate code (in one of my choice languages) for projects written in a language I didn't like. It's my new favorite video game.
Join the Aider discord, read the docs, and start using it with Gemini and Sonnet. If you want local, there's more to that than what I'm willing to type in a comment here but long story short you also need to make a series of correct decisions to get good results from local but I do it on my RTX4090 just fine.
I am not a contributor or author of Aider, I'm just a fanatical user and devotee to its way of doing things.