It is similar to an article I read a while back saying we need "data engineers" not "data scientists". I think its generally true. Why my company, while it can do data science consulting, is choosing to focus on data engineering consulting first.
I've always been annoyed by the title "data scientist" because I don't consider their methods worthy of the name science. It seems like the winds are turning and more people are questioning data "science."
But talking about changing the branding to deliver the product that was advertised all along feels like putting the cart before the horse. Sure, maybe it will work. Or maybe in a few years we'll be talking about how we need more "data sleuths" to replace all those haphazard "data engineers" that just didn't have a job title that emphasized solving the problems rigorously enough.
No they do this because horizontal scalability is more general. Once you cross the threshold of what your meganode can handle you have to rewrite your code from the bottom up
The only way to do this is to build an integration that makes the switch DEAD SIMPLE. If it involves someone being highly motivated to switch over themselves without any help its not going to gain steam.
Was SparkSQL ever intended to replace hive? My impression was that it was supposed to supplement spark for times it was convenient. I kind of suspected at one point they got caught up in the SQL hadoop race, but I always felt like it was best to do SQL elsewhere, and save spark for things that couldn't be easily expressed in SQL.
The original SparkSQL was pretty much modelled after the Hive flavour of SQL, down to the available udfs. The compatibility was never complete and has somewhat diverged again with respective releases of the frameworks, but for the most part, Hive was the big data framework to beat at the time (2015-2016), and not everyone wanted to write Scala.
I think that now, maintaining that compatibility is less of a need for Spark and Hive has introduced a lot of goodies in the meantime, so there might not be a need for the SQL flavors to be in lockstep anymore.
SQL can be used as a dataframe, or a hive temp view that can be called from other SQL. That gives flexibility to mix and match SQL and programmatic logic within the same spark app.
A company I worked at embraced nix. Beforehand I was a big docker fan and had few issues with it (yes there were occasionally caching issues and damn it docker figure out the issue with hyperkit on mac os, but largely its a productive tool). From an outside perspective nix just felt like alot more work, and nearly no one (except the people that set it up) could ever get it to work. So basically the entire build process for the core of the system was something that literally no one in the company wanted to touch. I'm sure if I was an insider on the tool I would appreciate the added stability you get on the backend, but at face value it seemed like it just made builds unapproachable on the front end. I'm happily back to using docker and haven't looked back.
Estimates for anything sufficiently complex are pure fantasy. They just serve to place an engineer in a bargaining conundrum and makes them implicitly decrease the scope for the 1000's+ decisions they will need to make during the course of implementation that they are not going to deliver the desired (also implicit) expectations. Its better to work from deadlines, then at least the engineer can try to hedge down the scope and deliver something they are comfortable with.
I don't think that's how you define a bug. I think technically you could argue its a "logical error", however that assumes that you know the intent of the creator of bash was specifically to not allow this behavior which I don't think is the case.
Firefox is now objectively better than Chrome in every way. I would recommend switching moral reasons aside. Chrome's memory management is abysmal. I now get the same shudder when I see someone with chrome on their machine that I used to get when I saw someone running Internet Explorer (which may even be better than chrome now too).
I wish this were true but certainly on Linux I don't find this to be the case. I periodically try switching to Firefox and always end up back on Chrome for performance reasons.
Features wise I prefer Firefox (containers are amazing, extra privacy features/blockers by default nice, the new picture in picture for video is nice, dev tools have a lot of cool extra features etc), but Firefox often struggles and stutters for me, whereas I rarely have issues with Chrome.
The devtools are a good example, sometimes just having them open on a page I am developing slows everything down. Closing/re-opening the tab seems to fix it for a bit but then it just starts to happen again, Chrome's devtools are always snappy. Similary if I want to inspect an element on a page Chrome never has a problem doing it instantly, Firefox has to think about it for a while.
In general everything that uses the GPU I find far faster in Chrome. The other day for instance I was working on a d3 SVG visualization that strained Firefox but was no problem for Chrome, canvas performance is similar but not as bad in my experience. The most common offender is trying to watch a video: in Firefox everything slows down and the fans spin up, in Chrome no effect.
It is also true that all Google services that I am unfortunately tied to (GMail, Maps, Calendar etc) are snappy in Chrome and like molasses in Firefox...
My experience is exactly the opposite. When forced to use Chromium on Linux, I find it slow. The dev tools are lacking basic functionality that I came to depend on, and they are unreliable (missing requests in Network tab). When testing CSP protections about a year ago, Chromium allowed many of the requests that should have been blocked (and were, in Firefox).
I think it comes down to what one is used to. We probably learn to sidestep problems with any browser we use, but using a different browser inevitably leads to frustration.
Of course, this is not true if you are using Google services. Google seems to put an extra effort in making sure that their pages are as slow as possible in Firefox... I don't care much myself as I don't use them, but it's a good reminder of how (non)non-evil they have become.
Interesting... It has gotten a lot better over the past few years. The Google issue is not a deal-breaker for me, want to move away from GMail/Calendar for my personal account anyway.
Will probably try to make the switch again at some point.
Before I say this, I would just like to say I run Firefox as often as I can and love love love it, but chrome is better, due entirely to the market share of the Blink web engine. Microsoft Teams' webapp only works in Blink. Hulu bitches about widevine until Firefox gives me the notification to enable it. Canvas (that thing all us students use) gets random rendering errors on gecko that makes the UI malfunction a lot.
People mention that more people should use Firefox because google controls the Blink engine and Gecko is the only real competitor, and if Blink has too high of market share, than google can add exclusive functionality to blink for developers to use, edging all other browsers out of the market.
I would absolutely switch to Safari or Firefox except for the fact that I find Chrome's tab system to be so good.
Chrome tabs load and appear instantly. As you add more tabs they shrink down to a tiny but still usable favicon size. As you close tabs they resize well and with nice little UX behavior like the next close button always appearing directly under your mouse. They close and re-open instantly.
When you press enter in the address bar Chrome always loads your Search, unlike Safari, which often requires second press, or weirdly loses focus. Even doing simple web search feels so slow and broken in Safari tabs. When you have a moderate number of tabs Safari switches to this horizontal scrolling mode where half of your tabs are no longer visible. Basically unusable.
Chrome tabs are so good compared to the standard macOS tabs found in places like Safari and Finder. I wish I could have them everywhere.
Firefox with tree style tabs is on another level. Firefox also supports lazy loading tabs when restarted, so only pinned tabs and tabs you view consume resources.
I have nearly 1000 tabs open in Firefox and navigate them extremely efficiently. Does chrome do that?
I don't really want/need to use a new horizontal tab paradigm that supports thousands of tabs. I just want a really good implementation of the standard top-of-the-window tab bar.
I was lured in initially by the novelty. But it has become an amazing tool for organizing myself. When I browse HN, I shift-click on everything interesting, and it automatically gets organized into a nice collapsible tree.
When I am working on a project, I move search results, project pages, and relevant info to a single tree.
I leave other things open indefinitely, which effectively replaces bookmarks.
(To anyone who plans to try it, you may need to edit userChrome.css to turn off the top tabs; there's guides on this. I also recommend tweaking TST's settings, imposing a limit on nesting, and preventing auto switching tabs )
I tried to switch to Firefox, in large part because of the contianers, and I still use it in some places, but the thing that really got me was battery usage. I switched my desktop at work to Firefox, and it was great, and I still use it there, but then I switched by work laptop to Firefox, and (according to powertop and htop) the battery and CPU usage of Firefox was probably 3-4x Chrome. This is a total deal breaker on a laptop.
Firefox's JavaScript engine is still slower and large JavaScript apps become sluggish more quickly.
Also, I still haven't found a way to let users download files from JavaScript (e.g. through a button click that invokes a function) in Firefox, that wouldn't open another window and in the process close their websocket connection.
This was true (I no longer use Chrome and don’t know if it’s improved in the last two years). Chrome needed enormous amounts of memory compared to Firefox. Using RAM is not an issue as such, but it makes other applications running at the same time miserable (and the user too).
It's worth pointing out that Google cripples its apps in Firefox intentionally and since Edge is now basically Chrome, MS probably, (perhaps unintentionally?), follows suit.
Unfortunately the Firefox codebase itself is the only marketing tool at work here. I still use Firefox for everything else, not least due to the fact it isn't trying to swindle every last private detail out of me, but for work stuff and gaming (O365/GCloud) I don't really care about any of that.
Hiding behind lies and random personal attacks will not make Firefox better, but comparisons to its main competitor in key areas might help
I’m not seeing a lie. Google intentionally cripples/degrades software in Firefox. Hell, Google degrades YouTube viewed in iOS Safari because they want me to install the app.
I occasionally have Firefox freeze due to some site's script freezing, forcing me to end it through task manager as Firefox won't respond to any button clicks.
Also (unless I missed something) there's still no way to get hardware video acceleration on Firefox on Linux outside of Wayland. When I'm on Linux I'm also installing Chromium (with the VA-API patches applied) entirely for YouTube as my CPU usage ramps up immensely when watching videos. Watching videos in mpv (or equivalent software) is an alternative, but I like the convenience of watching videos directly through the browser.
Other than that though Firefox has been rock solid for me and has been my daily browser for years now.
I've notice a bit of this on Linux but it never bothered me, however, I feel like it's gotten considerably better lately. For example, I use to have to keep Twitch open in it's own browser instance or else the stream would become very low quality, or even die completely, when I went to another tab. That doesn't seem to be a problem anymore, but I don't know if it's anything FF did.
obviously untrue since you can't be "objectively better" on subjective criteria, of which there are many.
i'll tell you though, the #1 subjective thing i hate is that single-click in the Chrome (Chromium) omnibar doesn't honor OS selection convention. unless you're on windows, where single click selects-all. having this behavior forced on Mac is jarring and awful and no amount of pleading would convince the dictatorial "my way is the right way" attitude of the gatekeepers. they refuse to even have a hidden flag to control this behavior. it's actively hostile.
the incongruity can be seen by clicking in any form field. eg chrome://flags search field. Note that single-click merely places the cursor on mac in a form field. then go to the omnibar. single-click selects-all. ARGH!
I had hoped Edge would fix this, but alas, no. (Also Opera keeps this behavior.)
Fast forward to newer behavior of hiding the protocol from the omnibar, until you click on it. Now when you click (or click-drag, the "proper" way to select less than the entire URL in one stroke, thank you oh overlords), as soon as you release the text changes and your mouse is no longer at the selection point. ARGH!! so awful.
I can't tell you how much I hate this poor implementation choice. However it's not enough to push me to FireFox, which is subjectively very much worse than Chrome.
Firefox containers are amazing! Being able to have a work set of tabs that open all the google services correctly, and a set of personal tabs that open google services correctly is the best thing ever. Not to mention containing all the sites that try to track you into their own little playground.
I am spending more time in FireFox too and want to give it some love. It still has some catching up to do with Chrome. I find myself switching to Chrome for work related activities, namely developer tools, and that's pure habit. I didnt have time to experience the FF version. Is it comparable?? But since my company's only supporting Chrome I'll do development in Chrome. At home I am on FF and have been so far loving it. Some extensions have to do some catching up in FF as well.
When in FireFox I am still experiencing weird bugs but am willing to give her some slack and hope they get fixed. Come on Firefox, FireFix and Rock!
On my xubuntu laptop, I observe the opposite. Firefox leaks memory until I need to restart it, several times a day. However, Chrome reliably releases memory, and I never need to restart it.
On my Arch laptop, it's exactly the opposite - Chrome can become supper sluggish even as I have 16GB of RAM, while Firefox stays snappy.
I assume there are more factors that go into this than just the browser itself. For example, I find Ubuntu to be one of the more sluggish distros out there in general, while Fedora and Arch feel way snappier, but am sure this isn't the case for everyone.
Firefox is at least objectively equal from a perceptive performance perspective. Chrome still has some wins that mostly stem from market share and what benefits come with that (people test in chrome first). I still main FireFox and promote it to anyone that cares to listen but as a web developer even I test in Chrome first because usually that is where most of my audience is.
I use firefox and chrome today, and even after all the perf improvements firefox is still a slower browser. On my 2 core 2014 macbook, only chrome and safari run with enough performance. On my other 4+ core devices it runs fast enough to use as my main browser. On those 4+ core devices, chrome is still more responsive than Firefox :(
Is pinch-to-zoom native on Firefox yet? Last I attempted to switch, you had to install an extension to enable it, and even then it was far "choppier" than Chrome's pinch-to-zoom experience. I've got pretty bad eyesight and zoom consistently so it was a dealbreaker for me.
The only reason I have Chrome installed on any machine is for YouTube. I tend to get weird flickers when in full screen while watching on other browsers, and Chrome is the only one I’m aware of that will allow you to view YouTube content in HDR.
