Maybe because they still have to pay for part replacements?

Maybe, I guess I assumed it was just already plug and play but probably there is some hardware changes there.

Edit :Yes, it seems your right, I should have checked,

> Support services replacement of defective hardware

It wasn't clear from the examples, and the gist doesn't have a `deinit` method, so what happens if an error needs to own data?

> Here, sqlite.ErrorPayload.init saves 500 bytes of error message from sqlite

Who owns those 500 bytes and where are they being freed?

It's just stored as a [256]u8 in the struct.

  // sqlite.zig
  pub const ErrorPayload = struct {
      message: [256]u8,
  
      pub fn init(db: *c.sqlite3) @This() {
          var self = std.mem.zeroes(@This());
          var fw = std.Io.Writer.fixed(self.message[0..]);
          _ = fw.writeAll(std.mem.span(c.sqlite3_errmsg(db))) catch |err| switch (err) {
              error.WriteFailed => return self, // full
          };
          return self;
      }
  };

And what pattern would you recommend if you needed to allocate?

It's worth every effort to avoid situations where a function creates extra clean up responsibilities to the caller only on error conditions.

If I really needed a large error payload, too big to fit on the stack, I'd probably want to do something like this:

  const errbuf = try alloc.alloc(u8, 1024*1024*1024);
  module.setErrorBuf(&errbuf)
  defer {
    module.setErrorBuf(null);
    alloc.free(errbuf);
  }

  var func_diag = diagnostics.OfFunction(module.func){};
  module.func(foo, bar, &func_diag) catch |err| switch (err) {
    error.BigPayload => {
      const payload = func_diag.get(error.BigPayload);

      // The payload can reference the 1MiB of data safely here,
      // and it's cleaned up automatically.
    }
  }

The diagnostic struct could contain a caller-provided allocator field which the callee can use, and a deinit() function on the diagnostic struct which frees everything.

Claude code has those "thoughts" you say it never will. In plan mode, it isn't uncommon that it'll ask you: do you want to do this the quick and simple way, or would you prefer to "extract this code into a reusable component". It also will back out and say "Actually, this is getting messy, 'boss' what do you think?"

I could just be lucky that I work in a field with a thorough specification and numerous reference implementations.

I agree that Claude does this stuff. I also think the Chinese menus of options it provides are weak in their imagination, which means that for thoroughly specified problem spaces with reference implementations you're in good shape, but if you want to come up with a novel system, experience is required, otherwise you will end up in design hell. I think the danger is in juniors thinking the Chinese menu of options provided are "good" options in the first place. Simply because they are coherent does not mean they are good, and the combinations of "a little of this, a little of that" game of tradeoffs during design is lost.

This has happened to me too. Claude has stopped and said on occasions "this is a big refactor, and will affect UI as well. Do you want me to do it?"

I recently asked Claude to make some kind of simple data structure and it responded with something like "You already have an abstraction very similar to this in SourceCodeAbc.cpp line 123. It would be trivial to refactor this class to be more generic. Should I?" I was pretty blown away. It was like a first glimpse of an LLM play-acting as someone more senior and thoughtful than the usual "cocaine-fueled intern."

Makes me think of an iolist in erlang/elixir.

Aren't you missing something? Gemini tells me that non-NA market makes up 33% of Ford's sales, and 51% of GMs.

So a better way to put it is "protects US automakers in the US." And that assumes NA manufacturers would be unaffected by declining sales abroad.

Isn't the problem exactly the opposite? A corporation has personhood which shields its executives?

Fair enough. That’s a more accurate way of putting it. Corporations are shields for the ruling elite to get away with pretty much anything. And in addition to that the corporation can be sued by shareholders if it doesn’t maximize shareholder value by externalizing costs. It’s the kind of policy that would be very popular among cancer cells.

Thanks - we've merged that thread hither.

Then in the top-most snippet, what size should `buffer` be?

It shouldn't matter.

Decompress's Reader shouldn't depend on the size of the buffer of the writer passed in to its "stream" implementation.

So that's a bug in the Decompress Reader implementation.

The article confuses a bug in a specific Reader implementation with a problem with the Writer interface generally.

(If a reader really wants to impose some chunking limitation for some reason, then it should return an error in the invalid case, not go into an infinite loop.)

So how would the Decompress Reader be implemented correctly? Should it use its own buffer that is guaranteed to be large enough? If so, how would it allocate that buffer?

At the very least, the API of the Readera and Writers lends itself to implementations that have this kind of bug, where they depend on the buffer being a certain size.

> So how would the Decompress Reader be implemented correctly? Should it use its own buffer that is guaranteed to be large enough?

yes

> If so, how would it allocate that buffer?

as it sees fit. or, it can offer mechanisms for the caller to provide pre-allocated buffer(s). in any case the point is that this detail can't be the responsibility of the producer to satisfy, unless that requirement is specified somehow explicitly

in general the new zig io interfaces conflate behavior (read/write) with implementation (buffer size(s))

I ... don't disagree with you. Thanks. It helps my understanding.

I know this is moving the goalpost, but it's still a shame that it [obviously] has to be a runtime error. Practically speaking, I still think it leaves lot of friction and edge cases. But what you say makes sense: it doesn't have to be unsafe.

Makes me curious why they asserted instead of erroring in the first place (and I don't think that's exclusive to the zstd implementation right now).

Author here. I see it both ways.

Blog posts are collaboration (1). I did get the sense that Andrew doesn't see it that way. (And for this post in particular, and writegate in general, I have been discussing it on the discord channel. I know that isn't an official channel).

My reasons for not engaging more directly doesn't have anything to do with my confidence / knowledge. They are personal. The linked issues, which I was aware of, are only tangentially related. And even if they specifically addressed my concerns, I don't see how writing about it is anything but useful.

But I also got the sense that more direct collaboration is welcome and could be appreciated.

(1) - I'm the author of The Little MongoDB Book, The Little Redis Book, The Little Go Book, etc... I've always felt that the appeal of my writing is that I'm an average programmer. I run into the same problems, and struggle to understand the same things that many programmers do. When I write, I'm able to write from that perspective.

No matter how inclusive a community you have, there'll always be some opinions and perspectives which get drowned out. It can be intimidating to say "I don't understand", or "it's too complicated" or, god forbid, "I think this is a bad design"; especially when the experts are saying the opposite. I'm old enough that I see looking the fool as both a learning and mentoring experience. If saying "io.Reader is too complicated" saves someone else the embarrassment of saying it, or the shame of feeling it, or gives them a reference to express their own thoughts, I'm a happy blogger.

I don't even like Zig but I read your blog for the low level technical aspects. I agree completely that blog posts are collaborative. I read all kinds of blogs that talk about how computers work. I can't say if it brings value to the Zig people, but it certainly brings value to me regardless!

Author here.

I finally got it working. I had to flush both the encrypted writer and then the stream writer. There was also some issues with reading. Streaming works, but it'll always return 0 on the first read because Writer.Fixed doesn't implement sendFile, and thus after the first call, it internally switches from streaming mode to reading mode (1) and then things magically work.

Currently trying to get compression re-enabled in my websocket library.

(1) https://github.com/ziglang/zig/blob/47a2f2ddae9cc47ff6df7a71...

Ha, "Don't Forget to Flush" https://www.youtube.com/watch?v=f30PceqQWko

That's why I like RAII.

It's a bug to flush (fallible operation) in a destructor (infallible operation).

I know you've thought carefully about these issues, but still it can't be that simple, can it? Closing a file or a socket is a fallible operation too.

Wrong.

Why is he wrong?

Here's an excerpt from the close(2) syscall description:

RETURN VALUE close() returns zero on success. On error, -1 is returned, and errno is set to indicate the error.

ERRORS EBADF fd isn't a valid open file descriptor.

       EINTR  The close() call was interrupted by a signal; see signal(7).

       EIO    An I/O error occurred.

       ENOSPC
       EDQUOT On NFS, these errors are not normally reported against the first write which exceeds the available storage space, but instead against a subsequent
              write(2), fsync(2), or close().

       See NOTES for a discussion of why close() should not be retried after an error.

It obviously can fail due to a multitude of reasons.

It's unfortunate that the original authors of this interface didn't understand how important infallibility is to resource deallocation, and it's unfortunate that NFS authors didn't think carefully about this at all, but if you follow the advice of the text you pasted and read the section about how you can't retry close() after an error, it is clear that close is, in fact, a fundamentally infallible operation.

If the flush (syscall) fails, it's not possible to recover in user space, therefore the only sensible option is to abort() immediately. It's not even safe to perror("Mayday, mayday, flush() failed"), you must simply abort().

And, the moment you start flushing correctly: if(flush(...)) { abort(); }, it becomes infallible from the program's point of view, and can be safely invoked in destructors.

File closure operations, on the other hand, do have legitimate reasons to fail. In one of my previous adventures, we were asking the operator to put the archival tape back, and then re-issuing the close() syscall, with the driver checking that the tape is inserted and passing the control to the mechanical arm for further positioning of the tape, all of that in the drivers running in the kernel space. The program actually had to retry close() syscalls, and kept asking the operator to handle the tape (there were multiple scenarios for the operator how to proceed).

If the tape drive failed close() in a way that did not deallocate the file descriptor, that was just straight up a bug.

Retrying close() is dangerous, if the file descriptor was successfully deallocated, it might have already been re-allocated by another thread. I'd guess the program you're describing was single threaded though (it can still bite there though)

> In one of my previous adventures, we were asking the operator to put the archival tape back, and then re-issuing the close() syscall, with the driver checking that the tape is inserted and passing the control to the mechanical arm for further positioning of the tape, all of that in the drivers running in the kernel space.

Why can't the OS itself do the prompting in this case, as part of processing the original close()? MS-DOG had its (A)bort/(R)etry/(I)gnore prompt for failing I/O operations, and AmigaOS could track media labels and ask the user to "insert $MEDIA_LABEL in drive".

Because DOS relied on BIOS interrupt 10h to handle I/O:

  mov si, GREETINGS_STRING
  print_loop:
    lodsb                  ; Load next byte into AL, advance SI
    cmp al, 0              ; Check for null terminator
    je done

    mov ah, 0Eh            ; BIOS teletype output
    mov bh, 0              ; Page number = 0
    mov bl, 07h            ; Light gray on black in text mode
    int 10h                ; Print character in AL

    jmp print_loop
  done:
    ...

  GREETINGS_STRING db "Hello, BIOS world!", 0

And linux doesn't rely on BIOS for output I/O, it provides TTY subsystem and then programs use devices like /dev/tty for I/O. Run $ lspci in your console: which of those devices should the kernel use for output? The kernel wouldn't know that and BIOS is no longer of any help.

> which of those devices should the kernel use for output?

Whatever facility it uses for showing kernel panics, perhaps. Though one could also use IPC facilities such as dbus to issue a prompt in the session of whatever user is currently managing that media device.

Yeah, close() can't fail, but it can return an error. It's kind of odd.

How could one fix that though? It seems pretty unavoidable to me because write() is more or less asynchronous to actual disk I/O.

You could add finalize() which is distinct from close(), but IMHO that's even more confusing.

Hidden control flow violates the zig manifesto.

Whatever happened to the principle of least surprise?

Unsurprisingly, it’s occasionally disregarded—seemingly when you least expect it.

Going from the previous interface to what ever this is, is certainly something. Yeesh.