Rivers caught on fire for a hundred years before the EPA was formed.

And even then, I've used a batch writer pattern to get 180k writes per second on a commodity vps.

ex: main.db + fts.db. reading and writing to main.db is always available; updating the fts index can be done without blocking the main database — it only needs to read, the reads can be chunked, and delayed. fts.db keeps the index + a cursor table — an id or last change ts

could also use a shard to handle tables for metrics, or simply move old data out of main.db

* some examples:

  conn = sqlite3.connect("data.db")
  conn.execute("PRAGMA journal_mode=WAL")        # concurrent reads (see above)
  conn.execute("PRAGMA synchronous=NORMAL")      # fsync at checkpoint, not every commit
  conn.execute("PRAGMA cache_size=-62500")       # ~61 MB page cache (negative = KB)
  conn.execute("PRAGMA temp_store=MEMORY")       # temp tables and indexes in RAM
  conn.execute("PRAGMA busy_timeout=5000")       # wait 5s on lock instead of failing

my replies are being ratelimited, so let me add this

the heavy duty server other databases have is doing that load bearing work that folks tend to complain about sqlite can't do

the real dmbs's are doing mostly the same work that sqlite does, you just don't have to think about it once they're set up. behind that chunky server process the database is still dealing with writing your data to a filesystem, handling transaction locks, etc.

by default sqlite gives you a stable database file, that when you see the transaction complete, it means the changes have been committed to storage, and cannot be lost if the machine were to crash exactly after that.

you can decide to wave some, or all of those guaranties in exchange for performance, and this doesn't even have to be an all or nothing situation.

https://the-php-bench.technex.us/

There's a huge performance difference between memory and file storage within sqlite itself. Not even getting into tuning specifics.

"Batch writer pattern" is a good idea to get rid of expensive commits.

Developers should write their own code and use LLMs to design and verify. Better, faster architecture and planning, pre-cleaned PRs and no skill atrophy or loss of understanding on the part of the developer.

I come in knowing what I need to build and at least one idea or more of how it should be done. I present the problem, constraints, potential solutions, and ask for criticisms and alternatives. I can keep it as broad as possible or I can get more granular like struct layouts, api endpoints, etc. I go back and forth until there's an approach I prefer and then I code that approach.

| it can code pretty well given a very tight and limited scope.

It's wildly better at tight and limited scope than large scale changes but even then I would rather code it myself.

One thing I would like to see is the use of LLMs for smarter semi-manual editing.

While programming I often need to make very similar changes in several places. If the instances are similar enough I can get away with recording a one-off keyboard macro to repeat, but if there are differences that are too difficult to handle this way I end up needing to do a lot of manual editing.

It would be nice to see LLMs tightly integrated into the editor so I can do a simple "place the cursor at things like this" based on an example or two. I'm sure more ideas for using LLMs more quickly perform semantic changes you intended are possible, instead of just prompting for a big diff. I feel there's a lot more innovation possible in this direction, where you're still "coding it yourself" but just faster.

What I did was make one commit by hand (involving multiple files), and then told Codex (last year's Codex!) to make the equivalent changes to other instances in the code base.

Never understood that argument. Because there’s two steps in design. Finding a good solution (discussing prior art, tradeoffs,…) and then nailing the technical side of that solution (data structures, formula,…). Is it the former, the latter or both?

Throughout human history, it was rare for only two people to raise a child, let alone one. Or for women to not bring money into the home.

Like many "trad" trends, it's based more on advertising and television than history.

"It takes a village to raise a child" was meant literally. However, the glory of capitalism required people to move to where the jobs were, turning that millennia-old principle upside down ever since industrialization. And car culture was the ultimate fatal blow, when children can't even walk their own neighborhood any more.

And when BLM made it part of their charter to encourage community support for children beyond the typical nuclear unit they were accused of a radical Marxist agenda to "destroy families."

For some reason the very concept of extended families and community engenders deep anger and hostility from some Americans, and that's odd for a nation of immigrants considering how common the "whole society of aunts and uncles and grandparents and cousins" is in the rest of the world.

I think because excessive individualism plays into the hands of large companies. There is an individualist culture that has naturally grown over time in the US, but it has also been pushed by big corporations because if you can't depend on your neighbors and extended family, you need to spend money to fill the gaps.

Also given how many people espousing "traditional family values" among the right turn out to be abusers, pedophiles, rapists, deadbeats, etc, what you might consider "traditional" values don't actually mapped to the left-right political axis at all.

And I assume you didn't bother reading my comment or this thread very hard and just wanted to dunk on the left, but the American nuclear family isn't "traditional family values" to begin with.

Given those precautions if it is just memory or some form of deanonymization that's also cause for concern.

The double edged brilliance/danger of capitalism is that it constantly opens up and moves into new markets. This is good, it means once the market determines a need, capital investment can accelerate production of the good that meets that need.

But the flip side is it is coming for everything. Everything will be marketized and monetized and accelerated and made efficient. And there are genuine problems with that.

Regulation has been the historical response, but we've seen concentrated wealth chip away at regulations for decades or even rip them apart overnight.

This is a contradiction that needs to be resolved. One can be pro-capitalism or anti-capitalism and come to the same conclusion.

There are more and more regulations every day. Oil refineries are being abandoned in California due to regulations so heavy there's no way for them to operate anymore. A friend of mine pulled his business out of California due to stifling regulations.

> Everything will be marketized and monetized and accelerated and made efficient.

I give my unwanted items to the thrift store rather than the landfill. Others sell it on eBay. This is monetizing/making things more efficient. And it's good.

them moving to another state is a regulatory failure (they shouldn't have another jurisdiction to move to, they should just operate without imposing negative externalities on others, spelling of the refineries).

what value is clean air? what is the value of a human life? how much is your attention worth?

these are questions that capitalism should not answer, but will nevertheless try to.

Films on film using in camera effects are still made on occasion but they’re art films for niche audiences.

But we’ll never get another Ben Hur. And that doesn’t sit well with me even if society can’t yet fully explain why.

It would take a well-respected org pushing a standard library that has clear benefits over "package shopping."