Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I know the guy that heads up the team that did this work -- he and I spent 2+ years fighting Broadcom's old, god-awful bluetooth code. Our whole team used to play what-if games about replacing the thing while massive code dumps came in from vendors, making the task ever larger.

Zach, if you're reading this, HUGE kudos to holding the line in replacing that, and double kudos for doing it in a verifiable, sane language!



> fighting Broadcom's old, god-awful bluetooth code

Correction: god-awful host side bluetooth code.

There is still the bluetooth firmware residing on the BCMxxx chip (or Qualcomm chip) - >1MB of god-awfulerer closed-source code, half of it is in ROM (with limited number of available patch slots), full of bugs. You can see it crash from time to time in the kernel debug logs (and auto-restart itself)


On Glass, we actually went to Broadcom and had them on the ropes for fixing parts of their firmware. Sadly, we couldn't bring those fixes out of the closed source world of hardware, so it's still up to the system integrator to fight those battles...


Serious question, why doesn't Google build its own bluetooth & BLE chips? Please put some competition on Broadcoam and the like and either push them entirely out of the market (good riddance), or force them to step up their game.


I can't speak to this directly because of numerous reasons, chiefly among them being that I don't get to make those decisions. Standard disclaimer follows: I rejoined in the last two years, what follows are my opinions, these opinions are my own, blah blah.

Android has never been about driving the hardware narrative -- it's always been about building a phone with mostly open contributions and driving the start of a wedge to open up the phone industry a bit. It's always been a software answer to a hardware problem, even today. Prior to Android, all we had were closed source low powered feature phones and Blackberries.

That being said, building silicon is non trivial work, and building a BLE stack and controller is even more so. Will a solid BLE stack sell phones? Hard to say how it could drive that narrative, realistically, and even harder to say if such a controller could be made cost effectively. Given Android's archetype (software solution to closed hardware), this puts such a project into a much more difficult position politically and financially.

I can't see this kind of thing having much in the way of legs in a large corp. That being said, I do think if a startup could challenge this landscape, it is a HUGE opportunity.


    Prior to Android, all we had were closed source low 
    powered feature phones and Blackberries.
I...what? Even if we want to ignore the iPhone for whatever reason, the Palm Treo, Nokia N900, and Windows Phone were firmly established by the time Android started getting demoed--and that was the variant that was very much reminiscent of Windows Phone, with a strong emphasis on the cursor keys over a touchscreen.

The rest of your comment makes sense, but the cognitive dissonance of that sentence was so extreme I had to respond.


I missed a comma in that: closed source phones, low power feature phones, and Blackberries. The N900 is a different, rare breed that did not see widespread adoption.

Fun fact: the first Android phone was the sooner, and it looked and behaved much like a blackberry. Still have mine, in fact.


I loved my Symbian E62


Yeah from the Android platform side it would be weird to build chips. For products like the Pixel phone though, that would be a great place to innovate. And realistically, Google needs to get into the custom chip game sooner rather than later... GCE needs to start competing with Amazon's Graviton ARM processors. The sooner you get the expertise and talent to churn out chips (and maybe even a fab or two?), the better. The global shortage for fab usage and chips could kind of force your hand soon.


Google doesn't have the volume on its first-party hardware to drive something like this yet & they would never start with BLE chips. Additionally, there's some really good vendors out of China already challenging BCOM & QCOM FWIW, which further complicates the "build it yourself" narrative (look at the Pixel buds which have an AirPods-like experience running a Chinese BLE chip for the buds which other Western vendors weren't able to match).

The Android org generally isn't the right org to build hardware, let alone do chip design. Maybe the camera team got closest when I worked there?

Source: worked on Pixel Buds @ Google & was one of several engineers responsible for selecting the chip vendor. We got source access to the entire stack/OS except for the microcode & some parts of the stack they hid. I found BES a way better partner to work with than the BCOM/QCOM mess.


Are you talking about the 1st or 2nd Gen Buds? Because the 2nd gens seem to have severe connection issues:

a) they frequently flip flop between at least two BT profiles, leading to a lackluster listening experience b) they lose connection to each other (leading to intermittend profile switches during reconnects, possibly due to lack of available bandwidth) c) they have severe signal quality issues, leading to limited range and audio interruptions

All these issues only happen with the 2nd Gen Pixel Buds, but not with a random sample of various other true wireless earbuds (I tested Sony WF-1000XM3, 1More True Wireless ANC, Airpods Pro) leading me to believe that this must be a hardware issue on Google's side, especially because the amount of people with the same issues is pretty high.

No other pair of true wireless earbuds hat any issues with music playback while I'm on a road bike, with my phone being on my back, only the Pixel Buds do - and I'm on my 2nd replacement already.


Aren’t they already with TPUs?


Huge difference between designing for their own data centers vs consumer chips.


Yes. And several generations in.


> Prior to Android, all we had were closed source low powered feature phones and Blackberries.

Symbian was open source, ran on millions of smartphones - most of which had app stores and web browsers. Some of which had touchscreens, GPS, augmented reality features etc.

Don't get me wrong - Android has been brilliant. But let's not completely rewrite history, eh?


Right. I didn't think I was trying to rewrite history -- I'm simply pointing out a gap in the market that Android was attempting to fill. Asking me to enumerate everything out there at the time is silly -- especially given the haze of memory.

Symbian was far more popular in Europe than it ever was Stateside, so bear that in mind. I had to import my Nokia E70 gullwing phone before I received my sooner, and what functionality it had was okay, but the browser was hardly more than a WAP browser in a feature phone. The app store was barely there as well, including only a handful of very simple apps at the time.


The browser in S60 phones was actually WebKit. In fact it was Nokia that started the work of fitting WebKit into memory-constrained devices.


Oh, neat! I had no idea it was WebKit on there. Still, what I had wasn't exactly what you'd be comfortable using for more than a few moments. I bought one of the original Nokia Internet Tablets and put together a full wearable system back then to make things a bit better for myself, but I never used the browser in S60 for anything serious because it was so cut down.


Google throws money at problems which don't generate revenue all the time. I feel like all it takes is someone inside Google with enough leverage to push it through without the thing having to make business sense


That could work but the other side of that coin is: if it stops making financial sense Google will do the responsible thing and end it. "Your BT hardware's software is no longer supported" will do more damage to the problem they're trying to solve here


I’ve already had Android phones that didn’t get more than a single major update where the BT/WiFi blobs were locked to some ancient kernel.


tbh, Android consistently having a good bluetooth experience would probably be good for the platform.


> Android has never been about driving the hardware narrative

Apple has been building its own hardware from the beginning, but still also uses Broadcom chips.


They have been integrating hardware from the beginning, not making it all themselves. What they do, though, is demand the ability to vet and fix vendor firmware.


>Apple has been building its own hardware from the beginning

do you mean designing? I'm not familiar with any point in time where Apple was building phones, but maybe i'm mistaken.

A quick google search indicates that even the first generation phones were built by Hon Hai.


They acquired Intel's modem division, so that may change in the future as well.


[flagged]


> What a hopelessly naive perspective.

Maybe, but I was there, helping it develop in the early days, which is the time span we're talking about. I can tell you the core of the Android team was fighting to keep it open -- so much so that about three years in one of the guys who dedicated his job to open sourcing drivers and kernel patches burnt out because of it.

What Android is about is decidedly not what Play Services and GCM core are about.

> Try seeing how willing they are to add support for chipsets on phones that don't include Google Services

It's always been the system integrator's job to work with the OEM vendors to integrate drivers and functionality into Android -- not the Android team's. Even on Glass we had to do this as though we were outside system integrators.


too bad more and more of android was moved into play services over time


It wasn't. Play services is just add on features that proxy app permissions and centralize push notifications -- no actual Android features moved into Play services that I know of. Ie: auto filling SMS OTP codes. It feels like Android is being sucked up that way, but that's because there are a lot of really nice features in there, like push notifications, geofencing, etc.

You can still run Android without GMS core and Play services -- I do that, myself, on a Pixel 3a running Graphene. The trick is that you lose some nice functionality (which Android actually makes up for in some cases, ie: SMS OTP copy buttons), and the mapping experience is god-awful (mostly because the OSS mapping scene is hopelessly stuck in the 1990s GPS model)


one example i can think of without having to look things up is that the music player used to be part of AOSP and then got replaced with a google play variant

e: here's an article from 2018 with some more examples:

https://arstechnica.com/gadgets/2018/07/googles-iron-grip-on...


So the old music player isn't still included by default. It should still work just fine on modern android if you install it, and there's plenty of foss media players on fdroid (ie not Google-dependent), most of them thin wrappers around the android media apis with a few extras like playlists. Kodi and VLC should work fine without Google services, and LineageOS Eleven doesn't require Google, though I'm not sure if it still works on modern Android.

Lack of media playback options shouldn't be able to stop you from using AOSP.


What would get open-source mapping out of this "1990s GPS model"?


Stop geocoding by separating out addresses into singular fields, for one. Stop showing mostly irrelevant contours on maps, for another. OsmAnd~ is unfortunately the only game in town, and the UX is freaking terrible.


how do i get SMS OTP copy buttons on my google-free android device?


> Serious question, why doesn't Google build its own bluetooth & BLE chips?

There is a lot of overlap between WiFi and Bluetooth & BLE such that you just have a Wireless chip that does both. In fact I think with Bluetooth 4 the file transfer profile just establishes an adhoc WiFi network. You don't have separate Bluetooth and WiFi chips anymore.

Furthering that, in the mobile space the Wireless capabilities are usually integrated into the mobile SoC. So you don't even have separate chips for CPU and Wireless.

About the only time you see a separate Wireless chip is when a new technology is emerging like 5G and it's usually only an external chip for a generation or two until it can be integrated into the SoC.

So, if you were to design your own Bluetooth chip today you would also be designing a WiFi chip and then you'd probably just roll all that into a SoC with a CPU. No small feat.


> In fact I think with Bluetooth 4 the file transfer profile just establishes an adhoc WiFi network.

This was a feature of Bluetooth 3.0, but almost nothing ever used it. I was once at a big BT testing company and asked about it, and they had like one device that could do it (a crazy feature-packed HTC WinMo device I think).

And then Bluetooth 4.0 added BLE, and it seems like there hasn't been much development of classic BT since then.


Building radio ASICs is a world rife with patents. Pretty much nobody new can enter the market.


Aren't they considered "essential patents" that must be made available at a reasonable price?


Generally yes. But you would need to spend 3 years in court to get that to work.


Follow-on question for the sake of debate: what if you got a bunch of smaller wannabe startups and interests backing a group effort to do exactly this for a pool of things? Would the legal fees scale linearly? :/


Care to expand? Is that why Software Define Radio is still so much a niche and expensive?


Nearly all modern radio chipsets are mostly software defined. That includes WiFi, LTE and GPS.

The radio frontend is typically a downmixer and then straight into digital.

Some of the typically "software" bits like FFT's, various encodings, checksums, clock recovery etc. are frequently done in digital hardware acceleration blocks for performance, and saving power. If you were writing the firmware of the device, you needn't use them though.

With enough human years of effort, you could take almost any radio hardware for sale today and repurpose it to speak nearly any other radio protocol in similar frequency bands. Performance will probably be terrible though!

It's rare people do this though - all the chips don't have their firmware documented (again mostly to avoid publishing documentation that proves they are violating someone elses patents), and many have various cryptographic elements that makes reverse engineering hard.

The one exception to this is WiFi chips used in the Nexus 5 by Broadcom, which has had a reasonable amount of reverse engineering because Broadcom accidentally published the source code because the firmware code was in part shared with published Linux kernel driver source code.


> all the chips don't have their firmware documented (again mostly to avoid publishing documentation that proves they are violating someone elses patents)

:O

Enlightenment moment :(


There are a gazillion patents on radio related stuff. Nearly all standards have associated patents.

Companies already in the industry typically have cross-licensing agreements - ie. I can use your patents if you can use mine. Either that or they just violate each others patents knowing that a patent war would be mutual destruction and in neither companies interests.

But a newcomer has nothing to offer - the minute they release any product, every incumbent company will go through their patent portfolio and sue them out of the water.


Yikes, that sounds pretty harsh. So how would a new player be able do enter?

I thought only the hardware could be patented, not the software, and so SDR would level the playing field, but that's perhaps too naive?


Google's involvement in any kind of hardware is already a distraction from their core product line. Making their own chips is a distraction on a distraction.

Distraction might not be the right word but I can't conjure up the right one.

Hardware is one of the end goals for Apple, for example. For Google, Android hardware is not. It's just there to serve their goal of selling ads.


IMHO those dynamics are changing. Custom chips are becoming table stakes for new products. Look at Apple M1, W1, or Amazaon's Graviton2 chips. All of those are core parts of products and services which would not be possible without the custom silicon. The pool of talent and resources to build these things is extremely small, and there are geopolitical issues putting ever more pressure and scarcity on them (i.e. China pivoting to reduce dependency on Western designed chips and hoovering up as much chip design talent as possible). The TL;DR is amazing new hardware and services need custom silicon, but custom silicon is only getting more difficult and more challenging to build in the near future.


Getting good yield/performance on the radio is non trivial. Aside from having domain experts you have a pretty significant investment in equipment to do the testing. Broadcom and other semis split that cost over many customers.


Tangential - in the BLE land nrf52832 and friends are pretty neat, and there is an ongoing work on the rust stack: https://github.com/jonas-schievink/rubble

I tried it and it can do a few basic things, though it’s in the early stages, apparently.


Nrf52 are great BLE chips, however the full bluetooth spec (classic+BLE) is orders of magnitude more complex...


Even if some courageous developer there fixes the bugs and updates the firmware; how many end-users would actually receive the update and actually apply the update? That's the problem Linux's LVFS[1] solves but it's unfortunate that not all manufacturers support it.

I got update for my half a decade old Logitech's 2.4 GHz receiver (nRF24L) for wireless keyboard as soon as I plugged it on Linux, I've used the same keyboard on Mac and the official Logitech software doesn't even detect the device properly let alone update the receiver's firmware(no issues using the device though).

[1] https://fwupd.org/


Do you have any more info about ROM patch slots? I have never heard of this before. I assume this is a small amount of r/w memory that is somehow overlaid over specific locations in the ROM?


Correct. It's a small table of:

  address1, 4 bytes overlay data
  address2, 4 bytes overlay data
  etc
The data is overlayed over the specified addresses, in runtime. On some chips its 8 bytes instead of 4. On a typical Broadcom/Cypress chip you have 128 or 256 entries.

By the time the chip is 2-3 years in the market and still getting firmware updates, ~98% of them are used by existing firmware, so there are only 5-10 free entries by the time the chip is considered "obsolete".

Case in point: the Broadcom/Cypress BCM43455 chip on the raspberry pi is almost out of patch entries. Broadcom have switched to their usual tactic of stalling for years on known, reproducible bug reports.


> Case in point: the Broadcom/Cypress BCM43455 chip on the raspberry pi is almost out of patch entries. Broadcom have switched to their usual tactic of stalling for years on known, reproducible bug reports.

And it's still really buggy. I had to write a service on the RPI and the only way to reliably connect was to restart bluetooth before every attempt.

That kind of fix makes a person feel dirty.


Sadly that's common in the hardware world.

Step 1. Have a reliable hardware watchdog that restarts everytime there's a software problem.

Step 2. There is no step 2.


Such is the sad world of Bluetooth. The dirty secret to this industry is that this, while seeming hacky, is the bare minimum de-facto standard in most cases.


So given these data points, isn’t it reasonable for Apple to refuse to play along this broken tune and just roll out their own dialect of a wireless protocol? Why, if not in the name of scarcely affirmed “standards”, drag suppliers through an endless contractual game, when you can direct your own capacity toward the quality standards that fit you?


I don’t follow the leap. The grandparent’s point was about the quality and terrible lack of long term support of Broadcom chips. How does that translate to issues of the standard itself?

Nobody would complaining about Apple creating their own radio chips (which they seem to plan for 5G/6G). Apple creating their own standard protocols is an issue though.


Well, if the implementations of the standard are such a garbage fire what's the point chasing them? Just to check the box "standards compliant" and likely providing an abysmal UX and poor interoperability?

I fixated on Apple because they're often picked on for taking the highway, but on the other hand what's the point doing otherwise? What's a common ground if it's just a pipe dream?


Just because a chip is shitty doesn't mean it's worthless. In practice, Bluetooth is quite interoperable, and reliable enough for many use cases (especially the common, better-tested ones).

Breaking compatibility with that ecosystem out of spite is not conducive to getting adoption for a better product.


Well, the original posts report some frankly tragic scenarios - so bad that they “reboot to initialize” just to keep sane - in what are some pretty ubiquitous devices. Or not?


"Reboot to initialize" is ugly as hell and very brittle, but it's good enough for most I/O devices like keyboards or headphones. If the kernel is able to properly reinitialize the chip with all of its old association information, it might even be indistinguishable from a few hundred ms of interference. (Rebooting on errors is in fact quite common for all kinds of hardware in high-radiation environments, and is a pretty standard kernel technique for working around buggy hardware.)

Now, of course, multiple nines of uptime would be very nice to have (and open up new use-cases), but 2-3 nines is still a lot better than 0.


Fine, but you're rationalizing living with poor execution. I'm rationalizing with deep (perhaps goldplating) correctness.


I'm not "rationalizing", and neither are you.

You're arguing that it's not enough to make a correct implementation, but that it's also important to break compatibility with incorrect implementations.

I'm arguing that a better implementation that is compatible with current protocols is strictly better than a better implementation that is not Bluetooth-compatible.

If you make a piece of hardware that is good (e.g. doesn't randomly crash and need to be rebooted by the kernel), why is it a bad thing for it to try to connect to some flaky BT headset?


The real brokenness here seems to be that the chips are not engineered with say ten times the patch capacity.

And the root of the brokenness is that there isn't the end-to-end awareness and acceptance that ten times the capacity is obviously needed.

Owww.


Easy to say, and I can’t know for sure exactly what factored impacted Broadcom’s decisions here, but I can tell you that chip manufacturers are under extreme pressure to keep costs down, which means that they may under-spec systems at times. Also, with the long design cycles involved in chip design the patch capabilities may have been decided years in advance, before realizing how much would be needed.

In general I agree with your comment, though it’s a lot easier to say this in hindsight.


For more illustrations of bugs/vulns in the firmware: https://www.youtube.com/watch?v=7tIQjPjjJQc&t=1986s


Broadcom's firmware seems to be just absolutely terrible across the stack, NIC included. They seem to have solid product design / engineering chops, but firmware just defies them.


My question would be what were the motivations to move into a stack written in Rust, if the much of the bugs are in the closed source FW running in the peripheral?

What would happen if consistency is lost outside the Rust domain but still in the BT stack?


I really hope it will fare better than NewBlue and BlueDroid... https://www.androidpolice.com/2020/09/14/the-rise-and-fall-o...


NewBlue died because I quit Google. I started that project and was its main motive force.


> NewBlue died because I quit Google. I started that project and was its main motive force.

Unclear why you're being downvoted here -- I seem to remember you, and I definitely remember hearing about NewBlue while we were working on Bluedroid. At the time it wasn't clear what happened. When did you leave?


I am also not sure about the downvotes, but c'est la vie. I was dmitrygr@, i left apr 2019


Bluedroid is the old stack. Zach and co started the floride project to clean it up, but it seems GD is their attempt to totally rewrite it.


Any insight into why Apple seems to have a much better bluetooth stack? I do a lot of BLE development that has to work cross platform, and we see constant GATT connection issues on android compared to almost none on ios.


Apple has a unique relationship to its hardware, and the money to bend vendor's firmware to their will. Half of the problem with Bluetooth is the host stack. The other half is the firmware running on the controller on the other side of HCI. If the controller ever gets screwed up, the host stack can only disconnect from HCI and totally restart the controller.


The third half is the bluetooth firmware on the other device. The fourth half is the other firmwares on the other device. The fifth half is the specification(s). The sixth half is the RF environment.


Haha -- yes, indeed. Bluetooth the spec and implementation are an absolute dumpster fire.

"Bluetooth is a layer cake of sadness" is the turn of phrase we used for a while on the Glass connectivity team. One of our project founders, Thad Starner actually apologized to me for the mess it became; apparently it was supposed to be simpler, but when Nokia took ownership back in the 90s, it started to go downhill.

Our lead on the connectivity team at the time had a crazy idea to rewrite the spec using only ACL and L2CAP, but never really went anywhere with it because of the the Glass org implosion.


But those are the same for iOS, yet Bluetooth on iOS is far better than on Android.


A huge factor is that iPhone has a large, long-standing market share, with relatively few different OS and hardware versions. This means that everyone else has been able to test their Bluetooth implementations for interoperability with iPhones.


A lot of people using Apple Bluetooth hosts use them with Apple Bluetooth devices (other hosts, mice, keyboard, headphones, etc)


Anecdotal, but my new 16 inch MBP drops BT connections all the time. I had to give up the keyboard and mouse I've used for years across at least 5 computers, mostly Mac's because they disconnected so much. Even the Apple keyboard and mouse I switched to drop occasionally.


FWIW, I also have a 16" MBP and I've also had a ton of problems with Bluetooth on this thing.

I've noticed that Bluetooth connectivity is significantly worse when the laptop is closed. You might see if keeping it open helps.

Resetting the Bluetooth module also helped resolve some persistent connectivity problems I was having (shift+option+click on the Bluetooth menubar item; choose "reset" from the menu).

Eventually this thing started having kernel panics every time I plugged something into a USB-C port and I had to send it back for replacement. Not a great experience.


Wow, never knew about that combination of keys and the fact that it brings extended options in the drop down menu. Thank you!


the 16 inch mbp is full of bugs, so it's probably a problem with the device (typing with one and still crying because of https://forums.macrumors.com/threads/16-is-hot-noisy-with-an...)


This HN thread from yesterday may help!

https://news.ycombinator.com/item?id=26625356

tl;dr Using USB3 ports can cause Bluetooth dropouts on Macs (and lots of other machines).


I think there might be some issue on that hardware where 2.4ghz wifi and bluetooth share an antenna, and can interfere with each other? If you're using 2.4ghz and can use 5, give that a try.


Honestly sounds like faulty hardware or interference.


There's something odd about the 2019 MBP Bluetooth hardware. The most interesting part is that just plugging in a Cambridge Silicon -based USB dongle can kill it permanently: https://discussions.apple.com/thread/250944058


Doesn't kill it permanently; there's a fix involving connecting a CSR 2.0 dongle. There's also a workaround that sets an nvram var. Overall it's an absolute clusterfuck, though, because Apple still haven't fixed the problem.


The workaround with nvram doesn't work if you already made the mistake of plugging in the CSR dongle. I don't know about the fix with CSR 2.0 dongle, it seems that nobody's selling them any more. Could work, could be an urban legend like all the reported fixes with nvram and pram resets and restoring various files.


I can confirm the 2.0 BT dongle fix works because I had that exact issue and that's how I fixed it. Of course, I still made Apple replace the laptop (was less than 2 weeks old), but doesn't really seem like they've noticed seeing how it's still not patched almost a year later.


My guess would be that they can apply much more pressure to the Bluetooth chip vendor. The buying power that Apple has for designing in a particular chip is much bigger than any individual Android manufacturer (even Samsung). That gets them the leverage to get the chip vendor to do what they want.


> The buying power that Apple has for designing in a particular chip is much bigger than any individual Android manufacturer (even Samsung)

Why does Samsung have smaller buying power than Apple? Doesn't Samsung sell more phones than them? Or is it because while Samsung sells more phones in total, Apple still has the most successful single models?


A typical manufacturer approach is to beat the hell out of your suppliers on price to make your margin which is likely most of what Samsung does with its buying power. Apple does that but is also willing to throw money at issues and go as far as financing facilities for vendors. Where most manufacturers don't care about the device they sold the second it's out of warranty[1], Apple takes a longer term view of the relationship. So Apple is far more likely to say 'we need this fixed and are willing to provide X million dollars and a team of our own engineers to solve the problem' while Samsung probably goes more like 'make cheaper and make it better... and make it cheaper!'

So the reason Samsung typically has less influence is that when all you do is crush your suppliers margins to make your own, said suppliers don't tend to make much of an investment in making things better since they are incentivized to just make them cheaper.

[1] in fairness, they can't afford to: while Apple has an ongoing revenue stream from its devices, most other manufacturers don't. It's Google/Facebook/etc who monetize the devices post-sale while for the original device manufacturer it's merely a liability at that point. This is a factor in why Android has a rather dismal track record re: updates on older devices.


It's because Samsung can get away with selling total garbage. I work on an audio-related app, and a huge majority of the hacks in the app to work around bugs in phones are for different Samsung models. Still more than half of our users keep buying them.


This, but slightly revised: it's because Samsung is in the market of selling cheap hardware. Apple sells luxury products and part of their value-add is leverage with their vendors.

Apple can come to a vendor and say "these are our constraints on what we are willing to buy. Here's testing benchmarks. You are required to meet them. Failure to do so voids the purchase contract."

The vendor will then, of course, say "We'll have to charge you more if we're spinning up a test infrastructure we don't have."

And then Apple will negotiate a price point and pass the anti-savings onto the consumer.

They do this at multiple levels at multiple points in their hardware story. I met someone once who worked on the USB integration specs back in the first-couple-generation iBooks. Apple built a rig for physically testing the connectors (including multiple cycles of intentional mis-plug, i.e. flipping a USB-A connector over wrong-side-up and pushing hard against the socket). They told the vendors selling them USB sockets that the sockets had to survive the rig, or the sale was void. Vendors priced accordingly. But the resulting product is more robust than others on the market.


My social network orbits around several people who experience apple’s testing rigor- and the stories I’ve heard aligns with the parent comment.

Apple sounds like it really is that rigorous with the quality of hardware and drivers/firmware they use.

Some of the stories of nitpicking that I’ve heard are truly awe inspiring. On the other hand, I’d hate to be the engineer on the vendor side trying to please apple. (Mostly because some crap PM or sales person made promises without consulting engineering on what’s possible with the available time and resources)


> Samsung is in the market of selling cheap hardware

I'm not sure about this. There are people who pay >1000€ for their flagship phones and believe them to be premium products, but they are just as buggy as the cheap ones. Huge amount of CPU power and impressive camera specs, though.


I've never had one, can you expound more on this topic?

The S21 Ultra seems like a very good purchase from reviews. QHD screen with dynamically adjusted 120Hz seems really like the best spec.


IIRC that model doesn't give apps audio route change notifications when Bluetooth is disconnected, which screws up audio timing. Or something similar, each model has different bugs. But yeah, it doesn't spontaneously catch fire, so on Samsung scale it's good. Reviewers generally don't know about this stuff, because app developers have already worked around it.


Are Samsung devices buggier than other android brands, or just so much more popular within your userbase, that more bugs surface?


My impression as an Android developer (not backed by any systemic research, just personal experience) is that Samsung feels less bound by the platform standards because - due to their large market share - they can simply get away with it.


It's something like 50% of user base and >75% of device-specific bugs.


Bruv. I have a samsung. I'm touched.

We've been together 4 years, its USB port is a little loose now. Too much charging everyday, but the screen is still pristine. The camera is fine so we can go out and take photos at the beach. I'm happy man. It does what it said it would do and did it. And still doing it.


You don't get to see the hacks in the apps you're using, starting with comments like "On some Samsung phones X happens even though API documentation says Y. We work around it by Z" or don't directly pay for the work that went into discovering the workaround.


> Apple still has the most successful single models?

Not just that, every apple device has wireless access, and Apple has thrice the operating income and more than twice the net income of Samsung Electronics.

But yes the "value" of individual devices is also part of the equation, in the sense that Samsung has a lot of cheap-ish devices with fairly short lifetimes, they're not going to fight for device quality. Apple has a very limited number of devices they support for a long time. And they're probably bringing in a lot of baggage from having been screwed over by the plans and fuckups of their suppliers in the past.

And even then, the more time passes the more they just go "fuck'em all" and move chip design in-house, not just the "main" chips (AX, MX, SX) but the ancillary as well: the WX and HX series integrate bluetooth on the SoC. There's no doubt they'll eventually go their own way on larger devices as well, the U1 is probably the first steps towards that.


I didn't realise that Samsung actually sold more phones than Apple (I've just looked at the numbers and you're right). Apple's smaller set of devices does help them a little but there isn't a lot to choose there.

I think my overall point is still valid because I have had Samsung phones for a while and have found their Bluetooth to be pretty good. This is not surprising as Samsung actually bought one of the biggest Bluetooth chip vendors (CSR) at one point, so they do have control over the full stack.


Samsung bought what was advertised as mobile handset business after CSR failed to spot that combo BT/Wifi chips were the future and threw away their lead. Qualcomm got the rest a few years later.

("what was advertised as" because there wasn't really a differentiation in the R&D bits, so there was a somewhat arbitrary split and hasty redacting of repos given to Samsung to avoid names of other customers in comments).

Samsung's phone division and electronic parts division aren't the same thing, so there was no guarantee that the phones would buy the Bluetooth/Wifi from their new acquisition, although I hear they did eventually.


I'd imagine lots of reasons. Apple has the hardware guys to design whatever chip they need. They also have a mountain of reserve capital (Something like $1 trillion I believe?). Further, Apple's closed ecosystem means that if you want them to sell them your hardware you have to conform to their standards.

With Samsung and android, it's a different story. There are many android vendors and the people producing the Bluetooth chips are selling to many of them (broadcom). Samsung has the ability to make their own chips, but to get everything working flawlessly they not only have to make their chips awesome but also improve the android driver stack to work with their new awesome chips. That stack has to also be compatible with the other bluetooth manufactures on the market making it a harder change to make.

In other words, with apple and a vendor, there are pretty much just the 2 parties involved which control everything. With Samsung and vendor it's not just them but also the likes of google and other bluetooth vendors that can get in the way of really fixing things.


I get what you're saying, but Samsung is also huge and Bluetooth is used in lots of electronics that Samsung sells -- including laptops, tablets, smart watches, TVs... or conceivably could sell, like future IoT products.

If someone senior at Samsung said to their vendor "good Bluetooth or you lose the Samsung account", that would provoke some, um, intense conversations at the vendor between sales and engineering.

Incidentally Apple sometimes has more than one vendor too, so it's not just two parties. I know cases where they've had two suppliers. Displays and modems come to mind, although I've not Googled to verify.


> If someone senior at Samsung said to their vendor "good Bluetooth or you lose the Samsung account", that would provoke some, um, intense conversations at the vendor between sales and engineering.

The problem: There aren't that many suppliers left in the field, and Broadcom knows that their customers are pretty much locked in. The notable exception is once again Apple, they have proven that they can and will go and implement the technology on their own if their suppliers fail to meet their expectations.


> go and implement the technology on their own

Samsung could obviously do this too, but as you say, they lack the will - and motivation.


Besides that they also lack the amount of control over the software side. Google is the entity that controls the stack, not Samsung - their responsibility ends at the kernel / HAL interface.

So why should Samsung invest more than the bare minimum when they can't get anything measurable in return?


This is, of course, totally untrue. Samsung changes a ton of things about their flavor of Android, at every level of the stack.

Edit: Samsung has apparently already ripped out the Bluetooth stack and replaced it with their own - https://news.ycombinator.com/item?id=26650447


I get what you are saying but they are simply in different positions. Broadcom can say "Hey, there's a bunch of work here to make this not suck" and Samsung, even if they wanted to do their own thing, realizes they too would have to go through that effort to make everything work. It wouldn't be a simple matter of just making non-sucky bluetooth, they'd have to also work with the android OS to improve the bluetooth stack and no guarantee that those changes can be merged.

Apple controls their whole stack. They've already written the Bluetooth stack for their OS. They only have to service their devices.

> Incidentally Apple sometimes has more than one vendor too, so it's not just two parties.

Not the point I was making. It's not an issue with multiple vendors, its and issue of who controls what. Those other vendors also have to conform to apples standards if they want to sell apple their products. What I'm talking about is the fact that a bluetooth device manufacture has to conform to google's android standards if they want to sell their chips to android manufactures, not to samsung standards. That's where the leverage goes away.

If samsung ever pulls the trigger and uses Tizen everywhere, then they'll be more in Apples position. Until that happens, they need to work with google to get stuff done.


Except that's not how it works? Chip manufacturers just care about selling chips, not about being in the iPhone or being in an Android phone. It doesn't matter that some chip works in OnePlus phones, if it doesn't work in Samsung phones Samsung's not going to buy it.

If broadcom says it's going to be expensive to fix, either you pay or you don't. But it has nothing whatsoever to do with "controlling the whole stack".

As an aside, and it's totally irrelevant to the above, but there's nothing other than the amount of work involved preventing Samsung writing their own bluetooth stack. They write plenty of custom drivers and they created a folding device prior to OS support. If they wanted to they could; they just apparently don't think it's worth the cost.

Edit: according to a comment down thread they have done exactly that.


> If someone senior at Samsung said to their vendor "good Bluetooth or you lose the Samsung account", that would provoke some, um, intense conversations at the vendor between sales and engineering.

The same thing could be said for Qualcomm's failure to support it's SOCs for more than a couple of years. Device makers could force their hand.


> support it's SOCs for more than a couple of years

That works, even works well, as long as device makers feel that supporting SOCs for a limited period helps them sell more phones.

Apple has changed the rules of the game by supporting its devices for longer -- and as phone upgrades become more infrequent due to plateauing technology, I think device makers will realize this.

Samsung has already committed to supporting recent Galaxy devices for at least 4 years -- at least with security updates[1]. I suspect this was also because they found that their extremely short-sighted prior policy re security updates encouraged corporate phone procurers to ditch Samsung and go with Apple.

[1] https://www.theverge.com/2021/2/22/22295639/samsung-galaxy-d...


Any movement on the Android side is helpful and hopefully Samsung can shame Google into following suit.

However, if we're going to count years where you only got a security update, the iPhone 5s is currently on it's 8th supported year.


I wouldn't be surprised if Apple write their own firmware for things like bluetooth chips.


I would be surprised if they didn't. They advertise their own Bluetooth chips H1 and W1 in AirPods and some Beats products. These chips definitely have their own firmware and it would be rather ridiculous not to also have their own firmware on the host side, maybe even re-using code.


Most of them run RTKit, I believe.


There's quite a bit of evidence that they do. Apple's strength has always been at the point at which hardware is built and interfaces to the software, including firmware.


I still find it buggy on my iPhone 12 Pro, as a user at least. Often says 'connected' to my headphones (Sony WH-100XM3's) when it's not, connecting to Alexa devices to stream audio often fails and requires a reboot to connect properly.

I can't believe Bluetooth is still such a pain in the bum in 2021.


Buggy Bluetooth on the other end... Alexa is running Linux, and the Sony is probably some custom bluetooth stack.

Once you move to all Apple bluetooth, things really smooth out. It seems that Apple does way more testing/validating of their Bluetooth stack.


I had the idea in my head (so take with a grain of salt) that parts of the BT stack are underspecified such that different implementations tend to have slightly different interpretations of the standard, so the problem isn't even which vendor you use so much as going all-in on any single vendor so the devices all agree on which reading to use.

Although of course Apple might well be better anyways; one would hope that billions of dollars in R&D plus caring about quality makes a difference.


I've actually worked in this area. You are basically correct.

Sometimes you have to make a choice on which brands/chipsets you support. Devices on different ends of the compatibility spectrum can basically be mutually exclusive. IIRC if you advertise A2DP some devices supporting only HSP won't work, so you can make some hacky workaround but then your nicer A2DP equipment is harder to use. If you only need to guarantee support for X subset of devices you control, it's easy to tweak the settings so they work well together.


Actually the spec is overspecified and repeats itself multiple times in different layers. L2CAP specifies a TTL, so does RFCOMM, so does SOC, so does...


I buy this. All Apple works great, but Apple headset with Windows PC doesn’t seem better than anything else with Windows PC.


They don't smooth out, for me, at least. Airpods Max, for example, can just decide to transmit nothing but deafening static to the other side in the conversation.


Eh, I’m still having weird connection issues between my iPhone and OG AirPods. Less than with my old headphones, but more than I’d like


As a long time Apple user, I really don't know where that's coming from. Endless issues with bluetooth over the years, including full system crashes. Curiously iOS bluetooth stack seems to be more stable than on macOS. Or maybe it's just hardware differences.

Apple's stack does work great with Apples hardware, though.


How many hardware devices are you talking about? That's very different than what I've seen personally or peripheral to enterprise support.


Well, at least Magic Trackpad, Keyboard etc. work really well with a Mac. Never had any issues.

Bose bluetooth headphones, third party "high end" bluetooth devices... not so great. Lately it's been better, though.


They have less hardware variability and also their bluetooth stack is still quite buggy but everyone works around their bugs on device side due to popularity.


Whilst they also use a lot of Broadcom silicon, the stack they use is written 100% in house.


Could it be because they only have a handful of hardware variants to support, vs hundreds on Android? Presumably, the stack itself is somewhat sane on both sides, but the hardware's bugs, poorly specified or undefined behavior is constantly throwing wrenches in the machinery - Apple managed to fix most of them for the hardware they support, but Android has no chance as they have to support hundreds of different chips.


This is not specific to bluetooth or drivers, but I don't fully buy the thesis/deflection that Apple just has a much smaller set of hardware to support. I run a hackintosh system myself and it's more stable than Windows. I'm starting to think Apple just has stronger general QC. Windows is just as glitchy and crashy on Surface devices as anywhere else.

The counter point does apply to drivers, but it's really not just that.


Possibly, but half the guys on the dev/qa team have google pixel devices and even they throw connection errors fairly regularly.


It's having to support every phone that introduces the instability even the refernece devices.


> Any insight into why Apple seems to have a much better bluetooth stack?

This is a joke right? My M1 BT goes out to lunch several times an hour. It is literally unusable.


A wild guess: less hardware variability.


Can you shed some light on what we’re actually looking at here? I see some Rust but most of the actual BT stack looks to be C++ code. Is the HN headline accurate?


Did you miss this on the main page?

> We are missing Rust support in our GN toolchain so we currently build the Rust libraries as a staticlib and link in C++.


Right, but the headline suggests the BlueTooth stack is being rewritten in Rust while the code seems to suggest that the stack is in C++ still. There isn’t much Rust code in the directory relative to all of the C++

4K lines of Rust is not a BlueTooth stack.


It's almost impossible to be worse than the existing Bluewooth stack. IT's the single biggest problem I have with Google phones and it's shameful it took this long to fix.


I wouldn’t assume it’s fixed yet. Even if they’ve managed to write bug free code, you’re still at the mercy of driver supplied by the chip manufacturer.


Broadcom seems to pump out a lot of garbage in general. The SoCs on the raspberry pis are particularly terrible.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: