Only very theoretically - in Go, you don't control whether memory goes on your stack or the heap, and heap escape analysis is notoriously unpredictable. There is no explicit free. You would have to write Go in a completely crazy way to be able to turn off the GC and have the program not grow unbounded.
You might think "I'll just use static buffers everywhere", but allocations can occur in unexpected places. The compiler does some very basic lifetime analysis to eliminate some obvious cases (loops...), but it's really hard to avoid in general.
Ok so garbage collection is optional, how about garbage generation? Is there any way to manually clean up resources if GOCG=off, or will memory usage continue to grow unbounded as new objects are created?
Grows unbounded. I wasn't recommending that one should set GOGC=off. Just making a remark that one could should they choose to do so.
EDIT: Sorry, I misunderstood part of your question. The memory grows unbounded unless you call runtime.GC() which triggers garbage collection. But this is a blocking call and essentially block your whole program.
Arguably, one can't truly say the GC is optional, unless the language and its libraries were designed to work without it. In languages like Vlang, that's the case, as the GC was added later. If turning the GC off cripples the functionality and usefulness of the language, then there is little point in using the option or claiming it as optional.
Probably a better argument for Go (and other languages like Java) is how "tweakable" the GC is or at least describe it as the GC can be turned off, but it's not designed or useful to do so.
Not sure if this really is required. Most cases in Go are served well by GoRoutines and for yield/resume semantics, 2 blocking channel are enough. This seems to add complexity for the sake of it and not sure it actually adds any new power to Go that already didn't exist.
"Enormous amount of overhead" is the operative phrase. In general, you want your concurrency operations to be significantly smaller than the payload of the operation. In the case of using channels as iterators with goroutines behind them, it works fine for something like a web page scraper, where the act of fetching a web page is enormously larger than a goroutine switch, but as a generalized iteration mechanism it's unusably expensive because a lot of iteration payloads are very small compared to a channel send.
I've encountered a lot of people who read that on /r/golang and then ask "well why are send operations so expensive", and it's not that. It's that a lot of iteration operations are on the order of single-digit cycles and often very easy to pipeline or predict. No concurrency primitive can keep up with that. A given send operation is generally fairly cheap but there are enough other things that are still an order or two of magnitude cheaper than even the cheapest send operation that if you block all those super cheap operations on a send you're looking at multiple factor of magnitude slowdowns. Such as you would experience with your code.
Yes, that's exactly what the article is about: how we can model "plain for-loop" levels of performance in the presence of complex (intricate state at multiple levels and high potential for nesting iterators) code that is supplying the loop(s)
Then you lose the separation of iteration from the looping construct.
You can use a function or a method, but you lose the nice continuation/generator ability to write a function that can do complicated yields without having to write the state machine yourself, plus you run a risk that the call won't inline in which case you're incurring non-trivial function call overhead.
The problem with iteration in Go isn't that you can't solve any given individual problem, the problem is that you can't solve all of them simultaneously the way you can in Rust or Python. (Though one of these days I want to get around to benchmarking Python's iteration versus Go channel-based iteration, I'm not actually sure which would win. What Go considers dangerously slow can still be baseline performance for other languages.) So you can get a defeat-in-detail sort of thing where a person cites a problem, and someone posts as solution to that, and then they cite another problem, and there's a solution for that, and then there's another problem, and a solution is posted for that, and all the solutions do indeed more-or-less solve the given problem... but you can't combine them into one.
Since all recursive programs can be converted into iterative programs then you can "simply" (not always simple) convert recursive solutions like McCarthy's Lisp solution to a loop: https://dl.acm.org/action/showFmPdf?doi=10.1145%2F1045283 (page 5)
Aside from the massive performance penalty, cache thrashing and context switching, this code will also leak a goroutine (and so, memory) if you don't finish receiving from `ch`. It's more brittle, longer to write, less local and in every other way worse than a for loop. Why would you ever do it?
It’s not insane at all. How did you come to that conclusion?
* Mutex lock+unlock: 10ns
* Chan send buffered: 21ns
* Try send (select with default): 3.5ns
Missing from here is context switches.
In either case, the overhead is proportional to how fast each iteration is. I have channels of byte slices of 64k and the channel ops don’t even make a dent compared to other ops, like IO.
You should absolutely use channels if it’s the right tool for the job.
Fwiw, I wouldn’t use channels for “generators” like in the article. I believe they are trying to proof-of-concept a language feature they want. I have no particular opinion about that.
> On my laptop, a C thread switch takes a few microseconds. A channel operation and goroutine switch is an order of magnitude cheaper: a couple hundred nanoseconds. An optimized coroutine system can reduce the cost to tens of nanoseconds or less.
Yeah I 100% understand wanting to optimize this for something like generators if we imagine them as first-class constructs. But they’re not at all a replacement for channels – they would be an addition, or specialization. I’ve never seen real world Go code that has needed it but maybe this will change with generics. It’s worth keeping an eye on, at least.
Channels otoh are very versatile: everything from spsc to mpmc with buffering and starvation protections, fast cancelation and notifications, etc etc. They’re not perfect, but it’s a helluva bang-for-the-buck for a single primitive. Literally all you have to do for performance is add buffering and coalesce “units of work”, and you’re good to go.
Where did you get this '..it uses same under the hood'? The article clearly says:
..Next I added a direct coroutine switch to the runtime, avoiding channels entirely. That cuts the coroutine switch to three atomic compare-and-swaps (one in the coroutine data structure, one for the scheduler status of the blocking coroutine, and one for the scheduler status of the resuming coroutine), which I believe is optimal given the safety invariants that must be maintained. That implementation takes 20ns per switch, or 40ns per pulled value. This is about 10X faster than the original channel implementation.
... That means the definition of coroutines should be possible to implement and understand in terms of ordinary Go code. Later, I will argue for an optimized implementation provided directly by the runtime,..
At the end he implements an experimental runtime mechanism that permits a goroutine to explicitly switch execution to another goroutine rather than using the generic channel scheduling plumbing.
You can export Jaeger traces via OTel as well. I am assuming the question here is why is there a different SDK rather than just re-using the standardized OTel APIs + libs for tracing and then providing a simple exporter for Traceo.
As a potential user, I am quite unsure as to why I would use this over something like TexStudio or even VSCode enabled with some LaTeX plugins. Could you tell me/us some pro/cons of this?
I am the co-author of a very similar application (CoCalc for LaTeX) and our landing page https://cocalc.com/doc/latex-editor.html lists many of the reasons people use it. A number of the reasons apply also to JupyterLab LaTeX, or will soon. A quick summary: realtime collaboration, having the paper you're writing and the data you're computing in the same place, having a very high-resolution history of edits, using latex in course management, using a Chromebook or other lightweight client, and zero configuration support for PythonTex and R (knitr). Note that some of these reasons for using JupyterLab or CoCalc are things that https://www.overleaf.com/ doesn't provide.
lolwut