Not to mention that the entire class of Markov Chain Monte Carlo techniques only form a subset of general uses for Markov chains.
Markov chains form the basis of n-gram language models, which are still useful today.
Markov chains are also the basis of the Page-rank algorithm.
Hidden Markov Models (which are just an extension of Markov Chains to have unobserved states) are a powerful and commonly used time series model found all over the place in industry.
In the pre-deep learning model Markov chains (and HMMs) in particular had very wide spread usage in Speech processing.
They are probably one of the most practical statistical techniques out there (out side of obvious example like linear models).
Not to mention, it was less than a decade ago that one could have said about neural networks "Decades pass and you realize they either have little to no application or are incredibly niche".
Being reputable and being a vanity division are not mutually exclusive.
You could argue that at it's peak Bell labs was a vanity division. That research may have changed the world, but very little of it likely ended up benefiting AT&T in any major way financially. It's telling that once AT&T was broken up Bell labs, while existing in some form for years after, was never reestablished.
Bell Labs still exists but is owned by Nokia, who now makes cellular infrastructure. AT&T Labs also exists.
But the days of telecom research making rapid and monumental advances peaked decades ago. Nokia or AT&T or Huawei or Ericsson could quadruple their R&D spending and it wouldn’t reestablish the impact of Bell Labs of last century, because it’s simply a much more mature field.
Whether it was a vanity division or not, it did serve one practical purpose for AT&T: it presented the company as a benevolent monopoly, spending its profits on developing technologies that benefitted the nation. This helped stave off anti-trust action for a long time.
Facebook/Meta stock lost 30% of it's value a few months back and may not recover any time soon.
Many of these people are high level, so I'm guessing (based on a quick skim of levels.fyi) about 50% or more of their TC comes from stock.
This amounts to an effective 15% pay cut in a year with record inflation.
I think most of us would leave our job over a 15% pay cut, especially if we were well established in the field. On top of this it loosens those golden handcuffs quite a bit for anyone who was on the fence about being employed by facebook, but couldn't say no to the comp.
A lot of folks get into a zone where the money no longer matters, as long as it’s in a range where you don’t have to think about it. Still they might leave simply because they’ve been there a while, or want a change, or want to work with some particular people who are at a different organization.
Not to say that some might be motivated by money, but at that level I would be surprised if it motivated many.
I find this is true for me, but I ultimately don't work for a FAANG because the money isn't that important to me and there are many things about FAANG companies I don't like. However I don't think this generalizes well for people who do aggressively pursue careers at FAANG companies.
My experience has been that most of the people I know that work for FAANG are extremely TC conscious, that's a large part of why they work at a FAANG in the first place and the source of a non-trivial amount of their self worth. If you look around the comments on Blind (a biased source for sure, but certainly a non-zero part of the community) you'll find plenty of people where TC is all that matters.
I interviewed at FB a few years ago and one thing that surprised me when I asked "what's your favorite thing about working here?" I got the same answer from everyone I talked to: the compensation/benefits. I thought for sure the answers would involve working on hard problems at scale, having incredible amounts of data etc. However without fail every interviewer immediately pointed out their comp and other perks as their primary motivator for working there.
In general I would say that people that make a lot of money, make a lot of money because making a lot of money is important to them.
This idea that "eventually you make enough where money doesn't matter anymore" is a meme that couldn't be more wrong. Its the kind of thing said by someone who has never made that much money, but who is extrapolating their current relationship with money to a compensation multiples higher; then saying "well if I can live on $X right now, then the Y in $X+Y wouldn't matter, so money must not matter (past some level) (and, coincidentally, that level is always somewhere around what I make) (weird how it always works out like that, right?)".
You can say that it shouldn't matter; that a $200k salary decrease to someone making seven figures shouldn't matter, because they're rich and they can take it. Maybe that's a correct assertion; that it shouldn't matter. But: it does.
Well, personally, I'm in a situation where I struggle to spend more than half of what I earn, including my mortgage repayment.
I'm not a cheap person, if I need to spend money on something, I will. It's just that my life style and affinities mean I'm not spending much.
I'm not much of a fashion guy, so no huge collection of expensive clothes.
Nor am I a car guy (I don't even currently own one, and if I were to have one again in the future, it would simply be a tool).
When I travel, the destination is often decided at the last moment, and I'm more of a backpack kind of guy.
When I buying something somewhat expensive (price ~$1000 or more), I'm always evaluating the usefulness of the thing, for example, I kind of want to replace my folk guitar but in fairness, I rarely play my current one, and it's not like this will change with a nicer one (I also have two very nice electric guitars I've not touched in years).
Simply put, partly out of how I was brought-up, partly out of some ecological-consciousness, I do not buy something simply because I can.
Also, regarding this part:
> Its the kind of thing said by someone who has never made that much money
When I make this statement, opposition comes far more often from people "who have never made that much money". My friends with similar or greater earnings get my point of view, even if they don't have the same views, but my friends with lower incomes generally strongly disagree with this kind of view.
A Principal Engineer at my company, could easily increase his salary 20-50% by moving to a FAANG, and can do Leetcode problems in his head, doesn't move. Because he has more independence here and can work on more interesting problems here.
He grew up in relative poverty. He already gets paid a lot. It's enough for him.
Well that's not true... at least not for me. I'm not even at the point of earning excessive amounts of money (research scientist at a german university E13, so about 60kEur). And already now I don't care enough about money to have it influence where I'm thinking about applying...
I went for years not thinking about money, not negotiating or worrying about comp, and making enough that I didn't need to worry or think about it, I just wrote code and did research. It is definitely a thing
I think this is to some extent personality-dependent and not as universal as you posit.
I somewhat fall into the demographic you describe but find myself not really caring about maximizing TC because I’m working on interesting and meaningful problems. (I would never be interested in working on adtech or infosec even for double my pay — I’m just not interested in those areas.)
I have met many in tech whose game is maximizing TC. They’re very vocal but I don’t know if they represent everyone.
Until the dot com era money really wasn’t much of a “thing” here (SV), and you’re right: even now it’s more of an SF thing, but has infected the Valley too.
I'm not sure, I can't imagine feeling that way. The jump from $500k to $650k, or 1mil to 1.2mil, would still feel pretty significant, in terms of cash hitting your bank account.
Honestly, many many people feel that way. I have been there most of my working life.
When you don’t make enough to make ends meet, or to do so comfortably, sure, fixing that situation is going to be very important to you.
But once you have what you need to live the life you want, why not simply live the life you want? One of my closest friends has been at google for almost 20 years. He doesn’t want to be promoted because he likes his job and his coworkers.
I don't think this is correct. I'm lucky enough to be on a level where turning down a 100k increase in salary is something that I have to routinely do because my current job ticks so many boxes and I genuinely love it and don't have to risk it for that amount.
I'm not saying these people wouldn't take a six figure pay cut for a job they like better. Just saying a six figure increase can still be pretty significant and material.
Once your basic needs are taken care of, inflation at current levels is irrelevant. I'm by no means rich. By HN standards I have an average income. If not for the daily news stories about it, I wouldn't even know there was any inflation bubble right now. The top AI people at Meta make a substantial multiple of what I make. They might be leaving for more money, but it has absolutely nothing to do with inflation.
People at that salary level usually leave for other reasons, like not wanting to work for such a morally bankrupt company whose sole purpose is selling ads.
Genuine question, when you shop for groceries do you not look at prices at all and just buy what you want and tap your phone at the checkout?
Because to me, it seems impossible to not notice the inflation in food prices over the past year. It hasn't really changed my life, but it's very noticable that prices are going up.
No, I almost never look at prices for food. If there were no prices printed on food, it wouldn't change my shopping habits one bit. I do sometimes notice deals that are prominently displayed, like "50% off!" or "2-for-1!" and take advantage of that. If I'm ordering online instead of in-person, the grocery store's UI helpfully points out when there are coupons, and I'll use that. If it's something where I don't care at all what brand I get, I'll buy the store brand knowing that it's cheaper. But I never intentionally look at prices for the purpose of saving money. I don't know what a gallon of milk or a loaf of bread costs.
I know this is "privilege", but it's emphasizing my point, which is that these top AI engineers making 3-10x as much as me certainly aren't worrying about inflation.
> Because to me, it seems impossible to not notice the inflation in food prices over the past year. It hasn't really changed my life, but it's very noticable that prices are going up.
I earn significantly less than a FAANG employee, and I agree with GP. From groceries alone, I'd never have known we had inflation. The only increase in prices that I did notice on my own is that eating out has gotten more expensive. And that's because many meals I would have for under $10 have magically hit $10+ - a notable boundary point, and the price point where I start thinking twice about "convenience" lunches.
Depends on the city you live in. Rent here has been going up for a long time - even when inflation was near zero. So for locals, the rent increase alone is not an indicator of inflation.
When people refer to "inflation" without any additional modifiers, they're generally referring to the Consumer Price Index numbers released by the government. When the news reports that "inflation is at a 40 year high" or "inflation is at 6%", that is based on the CPI. The CPI does not include housing costs. The most visible single component of the CPI for most people is the cost of gasoline. That's the only thing that I've personally noticed costing more.
That was stated surprisingly confidently. It's entirely wrong though, the CPI does include housing costs. I don't know why you would single out gasoline as "most visible", but I guess that gives you flexibility to claim that your real meaning is whatever you want. Gasoline prices actually (currently) impact CPI about half as much as food prices and one tenth as much as shelter costs.
I am fairly senior and make tons of money. The ratio of income to my annual grocery bill is stupid. Also, I eat mostly salads and meat is not my most common protein source. Groceries could go up 10x and fluctuation in my investments would still grossly dwarf .
But one of my earliest memories is my mother crying at the ATM because the checking account was over-drawn and there was no food at home (by the time I was in high school we were much better off than most, but the treatment effect stuck -- my younger sibling and I think about money in very different ways. Crazy what a $2K savings buffer can do). I then spent a good part of my early adult life stressing about affording food & choosing between rent and desired groceries.
I still stress over every single thing that goes in the grocery basket, for absolutely not rational reason. In particular, I don't stress about e.g. restaurant bills. Why? Because the times in my life when food was scarce do not overlap with times I was in restaurants as a patron.
The grocery store in particular is a source of psychological terror for many Americans at one point or another in their lives, and those memories are visceral.
It's not on the level of food insecurity, but I see a ton of comments in this thread by people who obviously are too young to remember the dot-com bust of 2001.
I was just out of school in 1999 and can vividly recall one of the senior engineers warning me, with mortal seriousness in his eyes, "it isn't always like this". I also remember having absolutely no fucking clue what he was talking about, except on the most superficial level. He was then just about the age that I am now.
>Facebook/Meta stock lost 30% of it's value a few months back and may not recover any time soon.
I think a significant driver of that was the tightening screws re: criticism and pending federal investigation.
The brand reinvention was in the context of changing perceptions that could culminate in existential threats to the company. There is no good way out of it, but changing the conversation from "Break up facebook" to "gee that Meta sure is weird", is a mixed success, and exchanging catastrophe for mixed success made quite a bit of sense as a strategy.
You've never had to any kind of factor analysis in your work or done any searching for latent variables that map to customer/stakeholder question? Given the number of people I've worked with that are interested in modeling "engagement", I find this hard to believe.
PCA is an incredibly valuable tool that I've used in most jobs I've had. It's just a terrible idea as a default part of a feature engineering pipeline (which is what the author is talking about in terms of "feature selection"), for reasons outline in this article.
I suggest you don't be quite so quick to dismiss important concepts in this area, and before criticizing this post, at least read through it (I noticed your comment about misunderstand what the author is discussing by "feature selection" is the top comment here).
Nope, never. I'm not dismissing PCA altogether; I'm sharing my experience and pointing out that some topics come up much more often in interviews than on the job.
It's very clear if you read the article that what the author is calling "feature selection" might be better termed "feature generation". He explicitly calls out what he means in the post:
> When used for feature selection, data scientists typically regard z^p:=(z_1,…,z_p) as a feature vector than contains fewer and richer representations than the original input x for predicting a target y.
I don't even think this is necessarily incorrect terminology, especially given the author's background of working primarily for Google and the like. It's the difference between considering feature section as "choosing from a list of the provided features" vs "choosing from the set of all possible features". The author's term makes perfect sense given the latter.
PCA is used for this all the time in the field. There have been an astounding number of presentations I've seen where people start with PCA/SVD as the first round of feature transformation. I always ask "why are you doing that?" and the answer is always mumbling with shoulder shrugging.
This is a solid post and I find it odd that you try to dismiss it as either ignorant or click bait, when a quick skim of it dismisses both of these options.
Am I alone in really disliking Towards Data Science?
While their articles always look nice, their content is all written quickly by data scientists wanting to polish their resume with the ultimate aim of rapidly generating content for TDS that will match every conceivable data science related search. This post clearly exists solely so that TDS can get the top spot for "Word2vec explained" (which they have). As evidence of this tactic you can see that there already is a TDS post "Word2vec made easy" [0], offering nothing substantially different than this one.
The problem is that content is almost never useful, it just looks nice at first skim through. The authors, at no real fault of their own, are just eager novices that rarely have new perspective to add to a topic. It's not uncommon to find huge conceptual errors (or at least gaps) in the content there.
I personally encourage everyone at every level to write about what they can, but the issue is that TDS has manipulated this population of eager data scientists in order to dominant search results on nearly every single topic they can cover related to DS, which has made searching for anything tedious.
Compare this post to the fantastic work of Jay Alammar [1]. Jay's post is truly excellent, covering a lot of interesting details about word2vec and providing excellent visuals as well.
I'm assuming TDS will fold as soon as DS stops being a "hot" topic (which I think we'll be in the relatively near future), and will personally be glad to see the web rid of their low signal blog spam.
I honestly think it all boils down to numpy being developed long before matrix libraries became a standard part of software development.
Ruby's early "killer app" (remember that term?) was Rails. Even to this day there is almost no major code out there built in Ruby that isn't ultimately related to building CRUD web apps. While Ruby may be losing popularity now, it moved the web-development ecosystem ahead in the same way that Python has moved the scientific computing world ahead.
20 years ago if you wanted to use open source tools to performant vector code there was Python and a hand full of oss clones of commercial products. Given the Python was also useful for other programming tasks in a way that say Matlab/Octave is not, it was the choice for more sophisticated programmers who wanted an OSS solution and need to do scientific computing. This creates a positive feed back that persists to this day.
Given that Python remains a decent language relative to it's contemporary peers and it has a massive and still growing library of numerical computing software it is extremely unlikely to be dethroned, even by promising new languages like Julia.
Even to this day there is nothing even close to numpy in Ruby. I do DS work in an org that is almost entirely Ruby, but we still use python without question because we know re-implementing all of our numeric code into Ruby would be a fools errand.
Had ruby had early support of matrix math, it wouldn't have surprised me if it would have replaced Python.
But that begs the question -- why did numpy develop in python and not ruby?
The rest of the thread offers some suggestions though. One is simply that python was born first, and got the numpy precursor before ruby 1.0 even happened. Which seems like a thing.
Ruby had a numpy style library since the early 00's, I forget exactly when. But it never got the kind of momentum numpy and the Python ecosystem surrounding it did.
Lots of comments in this thread from people who's Ruby experience is only from the post Rails era after ~2008, and don't understand that the post Rails culture wasn't really a thing when Python was first gaining momentum for scientific computing.
Are you seriously going to tell me that starting PayPal, having two divorces, 8 kids, and rolling that into Tesla and SpaceX makes someone the smartest entrepreneur? I thought entrepreneur included all parts of life. Elon is not a savior, he's smart, but he got lucky too.
No it wasn't a joke but also you are also blowing out of proportion my reasoning for having children. There have been dozens of reasons why I had kids. One of them, very small compared to the others, is the Idiocracy movie.
The idea that smart people have less kids is dangerous for society and I wanted to "be the change that you want to see in the world".
Now if you want to take that and blow it out of proportion and say THE ONLY REASON WHY YOU HAD KIDS IS WATCHING THE IDIOCRACY movie you are using Reductio ad Absurdum, you are enlarging my claims until they are absurd.
Try instead to watch the movie and then comment on whether Idiocracy is applicable to real world or not.
I'm surprised this argument is so hidden in the comments. All things being equal I would prefer permanent standard time, but it's pretty obvious that a very rational way to solve this problem is just choose whichever setting we use the majority of the time.
> They point to evidence, notably from the Nordic nations, that economies can continue to grow even as carbon emissions start to come down.
This a pretty laughable piece of "evidence" given that the most important export from the Nordic countries is petroleum.
But even if this wasn't the case, you can't talk about global systems on a local scale. There are countries that have continued to reduce fossil fuel growth and continued to increase economic growth, but at the end of the day these are just accounting tricks. The US has seen a bit of this, but we also exported a lot of our energy intensive manufacturing to other countries.
I'm sure someone will point out that some studies have tried to correct for this by measuring the carbon foot print of imports. But these studies are also flawed for several reasons. First the reduced cost of good is China is not only the lower labor cost but all of the other industrial services nearby that allow those goods to be created efficiently. It's very difficult to determine how far up the chain of production you need to go.
The other is that there's never an attempt to account for how much energy/fossil fuels go into the dollars that flood into an economy. If your finance sector is making a lot of money of foreign energy consumption, that counts.
The bottom line is that global consumption of energy has never decreased for any source[0]. We even burn more wood to produce energy than we did when that was one of of the major source of home energy!
We are already hitting the limits to growth, but the reality of that is too terrifying for most people to accept so we end up convincing ourselves that its not happening... while we're watching a war over fossil fuels develop in Europe.
>There are countries that have continued to reduce fossil fuel growth and continued to increase economic growth, but at the end of the day these are just accounting tricks.
The UK did this by switching from coal to gas powered electricity generation. It wasn't an accounting trick, it really did have an impact. The economy continued to grow.
Markov chains form the basis of n-gram language models, which are still useful today.
Markov chains are also the basis of the Page-rank algorithm.
Hidden Markov Models (which are just an extension of Markov Chains to have unobserved states) are a powerful and commonly used time series model found all over the place in industry.
In the pre-deep learning model Markov chains (and HMMs) in particular had very wide spread usage in Speech processing.
They are probably one of the most practical statistical techniques out there (out side of obvious example like linear models).