For single-threaded, cooperative multitasking systems (such as JavaScript and what OP is discussing), async mutexes[1] IMO are a strong anti pattern and red flag in the code. For this kind of code, every time you execute code it's always "atomic" until you call an await and effectively yield to the event loop. Programming this properly simply requires making sure the state variables are consistent before yielding. You can also reconstruct the state at the beginning of your block, knowing that nothing else can interrupt your code. Both of these approaches are documented in the OP.
Throwing an async mutex to fix the lack of atomicity before yielding is basically telling me that "i don't know when I'm calling await in this code so might as well give up". In my experience this is strongly correlated with the original designer not knowing what they are doing, especially in languages like JavaScript. Even if they did understand the problem, this can introduce difficult-to-debug bugs and deadlocks that would otherwise not appear. You also introduce an event queue scheduling delay which can be substantial depending on how often you're locking and unlocking.
IMO this stuff is best avoided and you should just write your cooperative multi-tasking code properly, but this is does require a bit more advanced knowledge (not that advanced, but maybe for the JS community). I wish TypeScript would help people out here but it doesn't. Calling an async function (or even normal functions) does not invalidate type narrowing done on escaped variables probably for usability reasons, but is actually the wrong thing to do.
The traditional thing would be to have an init() function that is required to be called at the top of main() or before any other methods that need it. But I agree with your point.
Return the cached item if it exists else spawn a task to update it (doing nothing if the task has already been spawned), await the task and return the cached item.
Developers should properly learn the difference between push and pull reactivity, and leverage both appropriately.
Many, though not all, problems where an async mutex might be applied can instead be simplified with use of an async queue and the accompanying serialization of reactions (pull reactivity).
This is like fighting complexity with even more complexity. Nix and bazel are definitely not close to actually achieving hermetic build at scale. And when they break the complexity increases exponentially to fix.
What's not hermetic with Nix? Are you talking about running with the sandbox disabled, or and macOS quirks? It's pretty damn hard to accidentally depend on the underlying system in an unexpected way with Nix.
My experience with nix, at a smaller scale than what you're talking about, is that it only worked as long as every. single. thing. was reimplemented inside nix. Once one thing was outside of nix, everything exploded and writing a workaround was miserable because the nix configuration did not make it easy.
> every. single. thing. was reimplemented inside nix
That's kinda what hermetic means, though, isn't it? Whether that's painful or not, that's pretty much exactly what GGP was asking for!
> Once one thing was outside of nix, everything exploded and writing a workaround was miserable because the nix configuration did not make it easy.
Nix doesn't make it easy to have Nix builds depend on non-Nix things (this is required for hermeticity), but the other way around is usually less troublesome.
Still, I know what you mean. What languages were you working in?
It was the dev environment for a bunch of wannabe microservices running across node/java/python
And like, I'm getting to the point of being old enough that I've "seen this before"; I feel like I've seen other projects that went "this really hard problem will be solved once we just re-implement everything inside our new system" and it rarely works; you really need a degree of pragmatism to interact with the real world. Systemd and Kubernetes are examples of things that do a lot of re-implementation but are mostly better than the previous.
> Systemd and Kubernetes are examples of things that do a lot of re-implementation but are mostly better than the previous.
I feel the same way about systemd, and I'll take your word for it with respect to Kubernetes. :)
> "this really hard problem will be solved once we just re-implement everything inside our new system" [...] rarely works
Yes. 100%. And this is definitely characteristic of Nix's ambition in some ways as well as some of the most painful experiences users have with it.
> you really need a degree of pragmatism to interact with the real world
Nix is in fact founded on a huge pragmatic compromise: instead of beginning with a new operating system, or a new executable format with a new linker, or even a new basic build system (a la autotools or make)! Instead of doing any of those things, Nix's design manages to bring insights and features from programming language design (various functional programming principles and, crucially, memoization and garbage collection) to build systems and package management tools, on top of existing (even aging) operating systems and toolchains.
I would also contend that the Nixpkgs codebase is a treasure, encoding how to build, run, and manage an astonishing number of apps (over 120,000 packages now) and services (I'd guess at least 1,000; there are some 20,000 configuration options built into NixOS). I think this does to some extent demonstrate the viability of getting a wide variety of software to play nice with Nix's commitments.
Finally, and it seems you might not be aware of this, but there are ways within Nix to relax the normal constraints! And of course you can also use Nix in various ways without letting Nix run the show.[0] (I'm happy to chat about this. My team, for instance, uses Nix to power Python development environments for AWS Lambdas without putting Nix in charge of the entire build process.)
However:
- fully leveraging Nix's benefits requires fitting within certain constraints
- the Nix community, culturally, does not show much interest in relaxing those constraints even when possible[1], but there is more and more work going on in this area in recent years[2][3] and some high-profile examples/guides of successful gradual adoption[4]
- the Node ecosystem's habit of expecting arbitrary network access at build time goes against one of the main constraints that Nix commits to by default, and *this indeed often makes packaging Node projects "properly" with Nix very painful*
- Python packaging is a mess and Nix does help IME, but getting there can be painful
Maybe if you decide to play with Nix again, or you encounter it on a future personal or professional project, you can remember this and look for ways to embrace the "heretical" approach. It's more viable and more popular than ever :)
1: See Graham Christensen's 2022 NixCon talk about this here. One such constraint he discusses relaxing, build-time sandboxing, is especially useful for troublesome cases some Node projects: https://av.tib.eu/media/61011
2: See also Tom Bereknyei's NixCon talk from the same year; the last segment of it is representative of increasing interest among technical leaders in the Nix community on better enabling and guiding gradual adoption: https://youtu.be/2iugHjtWqIY?t=830
3: Towards enabling gradual adoption for the most all-or-nothing part of the Nix ecosystem, NixOS, a talk by Pierre Penninckx from 2024: https://youtu.be/CP0hR6w1csc
4: One good example of this is Mitchell Hashimoto's blog posts on using Nix with Dockerfiles, as opposed to the purist's approach of packaging your whole environment via Nix and then streaming the Nix packages to a Docker image using a Nix library like `dockerTools` from Nixpkgs: https://mitchellh.com/writing/nix-with-dockerfiles
So I don't understand where the meme of the blurry super-resolution based down sampling comes from. If that is the case, what is super-resolution antialiasing[1] then? Images when rendered at higher resolution than downsampled is usually sharper than an image rendered at the downsampled resolution. This is because it will preserve the high frequency component of the signal better. There are multiple other downsampling-based anti-aliasing technique which all will boost signal-to-noise ratio. Does this not work for UI as well? Most of it is vector graphics. Bitmap icons will need to be updated but the rest of UI (text) should be sharp.
I know people mention 1 pixel lines (perfectly horizontal or vertical). Then they go multiply by 1.25 or whatever and go like: oh look 0.25 pixel is a lie therefore fractional scaling is fake (sway documentation mentions this to this day). This doesn't seem like it holds in practice other than from this very niche mental exercise. At sufficiently high resolution, which is the case for the display we are talking about, do you even want 1 pixel lines? It will be barely visible. I have this problem now on Linux. Further, if the line is draggable, the click zones becomes too small as well. You probably want something that is of some physical dimension which will probably take multiple pixels anyways. At that point you probably want some antialiasing that you won't be able to see anyways. Further, single pixel lines don't have to be exactly the color the program prescribed anyway. Most of the perfectly horizontal and vertical lines on my screen are all grey-ish. Having some AA artifacts will change its color slightly but don't think it will have material impact. If this is the case, then super resolution should work pretty well.
Then really what you want is something as follows:
1. Super-resolution scaling for most "desktop" applications.
2. Give the native resolution to some full screen applications (games, video playback), and possibly give the native resolution of a rectangle on screen to applications like video playback. This avoids rendering at a higher resolution then downsampling which can introduce information loss for these applications.
3. Now do this on a per-application basis, instead of per-session basis. No Linux DE implements this. KDE implements per-session which is not flexible enough. You have to do it for each application on launch.
> So I don't understand where the meme of the blurry super-resolution based down sampling comes from. If that is the case, what is super-resolution antialiasing
It removes jaggies by using lots of little blurs (averaging)
Neat! I worked on a similar formal verification of Ghostferry, which is a zero downtime data migration tool that powers the shard balancing tool at Shopify, also using TLA+:
I also was able to find an concrrency bug before a single line of code was written with the TLC which saved a lot of time. It took about 4 weeks to design and verify the system in spec and about 2 weeks to write the initial code version, which mostly survived to this day and reasonably resembles the TLA+ spec. To my knowledge (I no longer work there) the correctness of the system was never violated and it never had any sort of data corruption. Would be a much harder feat without TLA+.
There is the kawai novus5 which is a digital piano with the action and soundboard of a real upright piano and enough speakers to sound almost exactly like a real piano. There are also some new roland models I haven't tried. Many dealers lump these into their acoustic piano offering and don't market them differently because they are that good.
Comparing gas vs electric seems incorrect. Should be comparing with induction instead. It is way more energy efficient (altho not necessarily more cost efficient). It produces no combustion byproduct like gas (which your air quality meter may or may not be able to detect), and it is way faster and gives you better control. It also is safer without flames or leaks, and less likely to burn you. It is the way to cook in 2024 imo.
The progressive loss in consumer robotics company in the West to their Chinese counter parts has been disappointing. Much like drones, I suspect this is short sighted as the underlying technology eventually have national security concerns.
Now maybe these companies are likely just mismanaged and the cost of North American engineering is too high? That said, it still seems like there is a structural problem here that very few hybrid software-hardware companies succeed.
The problem is, Chinese consumer tech is full of extremely competitive and cut-throat companies. Some countries don’t like how their government is giving a tons of subsidies for them to progress like crazy (see BYD in 2012 vs now), but they’re delivering results. Combined with their low cost of engineering, the prices in the products are also pretty low, so it’s a no-brained for an average person to buy something for double the price for half of the functionality, just because it was designed in US.
It's competition. Competition made US what it is and now the US economy has moved from competing on product quality to competing on who buys a better lawyer and politician to block competing companies from existing. Or just buying them.
I wonder how much money is burned in the economy just paying people to write EULAs, laws and service agreements to more effectively avoid liability and screw over customers vs. how many is actualy going into improving products and services?
You're exactly right. I'm not asking for people to choose an inferior, pricier product. My thoughts is that China has the environment to have extreme competition which is leading to better product. This is distinctly not the case here. This is the structural problem that will eventually lead to a loss of competitive edge.
Your call out to BYD is a good one, because it is conceivable that even western-made cars will be made non competitive in 10 years and it seems that we are sleeping through the news (or even encouraging it). I hope I'm wrong, but the road ahead is filled with challenge because the direction is fundamentally wrong, and it will take a lot of effort to reverse course, if that is even possible.
A lot of western companies do not compete anymore - established european countries are basically oligopolies and their lunch is slowly eaten by the more aggressive chinese companies and Tesla. American companies - aside Tesla - is in the same situation. Rich on government contracts and control over their home market.
Basically a lot of established manufacturers are IBMs of this era.
Uh, Tesla got tons of money direct via carbon credits and indirect via consumer EV credits. They're actually in trouble for possibly lying about their range and getting more credits than they should have.
Yeah it sucks. We’re trying to play catch up game for manufacturing industry, but it’s abysmally hard to get it going. I don’t think we can easily pour down money and the talent and processes would just reappear in a couple of years either. So, my assumption is high-scale tech protectionism wars are going to start.
I think people focus on protectionism because that is the traditional tool to fight things like foreign government's unfair subsidy practices. However, you cant just have protectionism without fostering competition and innovation in order to succeed in creating a more competitive product/market. Example would be USA protectionism against Canada's bombardier. It only protected Boeing but didn't actually make Boeing make better planes, as we can see from all the recent issues.
So I think protectioism is fine as long as we properly setup an environment that allows for and encourages competition and innovation. However, that doesn't seem to be a path we are used to taking .
Absolutely agreed. I do think it will go the Argentina way if/when we start mass banning imports of Chinese consumer tech. Well, unless, as you mentioned we start heavily investing and encouraging local competition. I guess, time will show, but I hope we don’t cut ourselves out of good products just because “they’re foreign”.
It's hard to call it mismanaged when they did the playbook that is expected by prevailing finance and economic views since the mid 70s: paring down to one thing and increasing what you give to shareholders over time. Or perhaps that is the structural problem.
The problem is Chinese companies are subsidized by their government to manufacture things of little or no intrinsic or critical value. Automated vacuum cleaners and consumer drones are niche electronic novelties. Electric cars using solid state batteries are also novelty that will be obsolete once electric engines that use liquid fuels become mainstream (fuel-cells).
The purpose of subsiding what are zombie companies is to maximize employment to ensure internal stability. The wins these companies show are propaganda wins only and don’t make the country more competitive. Foreign manufacturing is also migrating out of China at an alarming rate as shown by falling exports and GDP growth.
None of the development in the Chinese technology sector is sustainable. These companies would never survive on their own without subsidies and are dependent on them. It’s a cascading failure waiting to happen in the Chinese economy and will likely be a global shock. At least the Americans may appear to take longer to develop winning companies but once they do they tend to be sustainable and long lasting as organic enterprises.
Edit: The American free market is working as intended because it rightly values robotic vacuums as useless devices.
> Electric cars using solid state batteries are also novelty that will be obsolete once electric engines that use liquid fuels become mainstream (fuel-cells).
This seems like a big statement, can other experts comment?
Current battery tech has a slow and steady progression of improvement. There is bound to be a market disruptor at some point in the future, but it’s far from a guarantee that it’s going to be hydrogen.
I believe the biggest hurdle to any changes to current battery tech is that it costs so much to develop an entirely new process and build factories. Most innovation is in the form of small adjustments. For hydrogen to overcome this hurdle it would have to either be extremely cheap or have some unique property. For cars I don’t see hydrogen having that much of an advantage, but maybe electric planes would be feasibly powered by hydrogen due to the much improved weight to energy ratio.
you have so much incorrect views of Chinese companies, the technologies these companies have, what is actually happening on the ground in China. You also vastly underestimate the real complexity of making today's products, even as mundane as a hair dryer or a toy. Chinese manufacturing makes making them look easy, people think all you need is bunch of cheap labor and you are set. No it's not. Also, for white label products, examples like hair dryer, washing machines, air conditions, its the Chinese companies who design, build and test, the entire lifecycle of the product, importers buy them and slap on their own brand.
Think what goes into a hair dryer? Exterior design, looks good and functional. How you make the plastic cover, do the plastic injection molding? How you design all the internal parts, fan, motor housing, heating wire, power circuits, micro-controllers etc, and make sure everything fits. Some companies even do individual components themselves, like the brushless motor, or there is a Chinese supplier that makes them, which provides much faster time to response. Then do the testing for each component, electric, heat, water, moisture testing. Then design a mass manufacturing system with automation and human labor that achieves really high yield and low wasted materials. This is the hardest part, its easy to make a hair dryer by hand taking 100 human hours and make sure it works. It's much harder to make 1M hair dryers per month, that is going to be used in all sorts of environments and with all kind of abuse, make sure they work well for a number of years so customers don't return them, or you go bankrupt from recalls and warranty, and make sure you only have to throw out the absolute smallest number of manufacturing defects, and really control your cost structure so you still make a profit when importers are squeezing your price. Then the supply chain and logistics, shipping from suppliers and shipping to customers. Then create a number of products for different markets. China can manufacture for cheap, but people don't realize manufacture for cheap and at massive quantities is a technology itself. It's also management, business process, even company and worker culture. China doesn't have the cheapest labor cost. It's the combination of everything that produces a physical product with the level of quality, fit and finish at the price point.
It’s only disappointing if you care about the relative power of national economies.
As someone who doesn’t care at all about stack ranking or any nation’s “national security”, as a consumer, more competition, and more and cheaper products is a simple and uncomplicated win.
Almost all of my favorite companies are in Shenzhen presently. I would move there if I could do so easily.
> Almost all of my favorite companies are in Shenzhen presently.
All my favorite devices were designed/engineered either in Japan or in the USA. I'd take good engineering over cheap manufacturing every time. And we could do with lower number of devices. While they are probably made in the same factory, I'd love a focus on quality instead of price.
Following your own logic, you'd never have had any Japanese engineering if everyone thought the same thing about japan in the 60s and 70s. It used to be considered the place where cheap stuff and knock offs were made, but it evolved from there. Same seems to be happening to China right now
As someone who actually care about those but not a US citizen, I welcome all of these! It's just funny seeing the free trade principles that's been repeated over the ages getting reversed like this. Now this is the end of colonization.
unless you are a descendant of Chinese or at least Asian people, if you move, you may find what "national security" is about.
countries compete, albeit on different rules - having a monopoly on violence and a centrally controlled money printer tends to do that - so your dream of 'just pure free market competition' can only ever be that - a dream.
With China's policy being what it currently is, we're going to feel the economic consequences, in the US and in Shenzhen alike :( Good thing if it's going to be only limited to economic consequences.
