Some recruiters are seriously sketchy. Someone I know joined a company where the recruiter told her that they give all their employees at that level this same salary so there's no room for negotiation unfortunately. After she joined, she asked what other people were making, and found out that even a new grad had higher compensation than she did despite the fact she had several years of experience already.

Apparently, they tried to do the same thing to that other employee, but that other employee had another offer and just said that he would take the other offer, to which the recruiter apparently responded, "Oh wait, I'm sorry actually I think we can do something about that. I was looking at the wrong sheet."

Oh wow. this happened to me recently. I foolishly believed that it was true since I've heard of companies like Reddit that have policies like these.

It isn't hard to write a comment about it that will show up in a search engine, and while that is far from perfect, it will allow employees who have freedom of motion to steer clear of slimy firms and those who don't to at least be able to negotiate from a slightly less-weak position.

One commonly mentioned alternative explanation is simply that people will quite reasonably not risk negotiating aggressively when they perceive their negotiating power as being lower. Given the number of women or non-Caucasian/Asian men who've reported being paid less or held to a higher standard than peers, I would seriously consider the possibility that the data is showing us a symptom rather than the cause of the pay-gap.

One of the problems is that when women lack such knowledge, they come across as less confident because, in the statistical sense of the word, they literally are less confident. In other words, in an absence of hard information, you cannot be sure if the figure you are asking for is too high, too low or just right. But when they evince this lack of confidence, people usually try to give them pep talks rather than hard data. It gets treated like a lack of self esteem rather than a lack of information.

I do think you are correct that women simply cannot afford to negotiate as aggressively and this is a factor. But I also think they lack access to knowledge that a lot of men are exposed to casually simply because they are male and this makes it easier for them to hang with the right crowd and get anecdotes, etc. And that piece is what I was trying to capture with the word savvy.

I am a woman. I know a lot about negotiating. I remain poorly equipped to negotiate on salary because I lack information. Having general savvy about negotiating technique is insufficient. You cannot play hard ball if you do not have hard information particular to the problem space in question.

1) I don't need a job. You'll pay me what I want or no deal. In those cases things like what this recruiter pulled won't work.

2) I need a job. In this case it doesn't really matter to me.

But they also can't keep bringing in people above a company's ask. It's a balance that has to be managed.

A lot of large companies have tiered salary levels that are not in any way tailored for the individual. In Germany, we have a Tarifvertrag in virtually every large company, which fixes the salary.

I had a similar experience looking for my first job in Chicago a few years back.

I met with a fairly large general recruiting firm in the area just so they could feel me out.

I named my expectation of 50-60k to which the recruiter replied.

"You know... when I moved to Chicago I was living on 18k"

Not exactly the type of response I expected.

Everything is negotiable, including when you hear things like that and "that's not our policy", etc.

I was the only woman in an office. One of my co workers made a pretty horrific comment (not about me, but about women in front of me). Someone went to HR and we had sensitivity training...but everyone assumed it was me because I was the only woman. Maybe I SHOULD have gone to HR, but I didn't want to because I feared being the obvious complainant. Which I ended up getting the side-eye for anyway.

I realized I would much rather work somewhere with at least 2 other women.

I've worked in small teams where I was the only Asian person among a team of white men, and I neither noticed any hints of racism nor sexism.

I've also worked in teams where it's been highly diverse in ethnicity though not gender, and again, never noticed any hints of sexism.

Though I have certainly worked in a company with a number of women, including a female co-founder/CEO who was absolutely sexist... against women. I'm not suggesting women are bad CEO's or that diversity is bad, I'm saying that from my experience, it hasn't been about the diversity of the team so much as the type of people in the composition.

Also from my POV, I'm not sure how having a random % of women makes more or less sense than a random % of black people or Latino's. Like if a company is predominantly composed of white men, does adding a white female make it more diverse than adding a black man?

You're lucky nobody said something racist about Asians when you were the only Asian. Otherwise, you would have been in the same situation. But that's it: luck. It could happen to anyone and then the workplace could turn hostile.

Quantity matters. Everything is contextual. If one is in a "hypo-minority" situation, then one is more likely to stand out. Being unique can be nice, but being irrevocably marked as unique, with no respite can feel like being trapped. Being singled out is simply more likely if you are truly singular.

Like if a company is predominantly composed of white men, does adding a white female make it more diverse than adding a black man?

I think that very much depends on the people involved. So in this, you are right. But quantity does matter a lot.

At any rate, I've never been on a team that didn't have several Asians. But I've had many co workers tell me they've never worked with a woman before (which in itself is awkward to hear unprompted, haha).

Thanks for pointing out that race and sex are not comparable. It's true, they really aren't.

I've been a minority almost my entire life. Most of the issues with being a minority are not really present on software engineering teams. The demographics are too widespread.

There is no in-group. I would estimate 50-60% of software engineers are immigrants. That group is further segmented by country of origin. Everyone is different from each other.

Country of origin transcends both gender and race in terms of shared experiences and commonalities. American born people of all races and gender are more likely to befriend each other than to befriend a person who speaks broken English.

This diversity on software engineering teams minimizes the effects of being different because everyone is different from each other.

Software engineers also tend to be more introverted and immigrants are more polite (due to unfamiliarity with American culture).

For people to make inflammatory comments and get away with it, they need to have allies and be part of an in-group. These prerequisites are much more difficult to fulfill as a software engineer.

Software engineering is one of the best professions to be a minority in.

It's really easy to know when you are not part of the majority though: When you don't see any differences and everything looks cheery and happy, congratulations, you are not treated as a minority.

We can't really boats about the industry when we have 15% women and about under 10% African-American. I've worked at a place where we had a large architecture meeting: 25 architects, zero women. The company boasted 50-50 gender split, but with vert few exceptions, you could make a great guess of role and gender. Guess that men sitting in an engineering pod are developers or managers, and that women are either QA or systems analysts, and you'll get it right. The testers had the same CS background as the developers, except they were paid a good 30% less. The rest of the women came from recruiting and HR. You could also guess which department they worked on just by looks too.

I know four women that have dropped out of software engineering in the last year, just because the toll of being treated differently made them lose any love they had for the industry, and are now doing jobs that pay way less, but where they don't have to work twice as hard as a man to get half the recognition. One of them is rather unattractive by your typical standards. When she quit her last job, many people didn't even know she had been working there for years: She might as well have been socially invisible.

Maybe you've been very lucky in your career, and haven't seen the discrimination, or maybe you really are part of the in-group and don't know about it.

Disagree. We have amazing contributions from a variety of people from Europe, South America, and Asia. It's not perfect, and it should get better, but it's nothing to be ashamed about.

It's interesting how the NBA happily celebrates black culture and black people who represent their game despite the fact that they are overrepresented relative to the general population, but the tech industry basically never celebrates the contributions from a variety of immigrants and non-whites in its own industry.

Why is there always an in-group?

What is the in-group of a place with:

- 10% Asian Americans

- 10% Indian Americans

- 10% various white European immigrants

- 15% Indian immigrants

- 20% Asian immigrants (14% Chinese, 4% Korean, 2% other)

- 25% white Americans

- 5% Black/Hispanic Americans

- 5% Black/Hispanic immigrants

(With 10% females spread among those race/country lines)

What I see happen is that the various groups separately cluster based on country of origin. None of the groups are dominant, so no one person (even a leader of a group) will feel comfortable making inflammatory remarks.

Many times everyone on the team is introverted and no groups form at all.

>Maybe you've been very lucky in your career, and haven't seen the discrimination, or maybe you really are part of the in-group and don't know about it.

Ad hominem? I've been a minority in the most formative years in places where being different is tough and I understand the difficulties. I contrast the experience and demographics of work with those years.

I don't know how that's even a question -- one group composes 90% of the workplace, and its a group that enjoys special privileges.

Racial discrimination and in-grouping isn't the only kind of in-group.

I've definitely interviewed female candidates for a small startup where they decided not to proceed once they found out they'd be the first woman on the team. I certainly don't hold it against them, but it really does make increasing diversity a lot harder if nobody is willing to be the first one.

It's a great example of how hard it can be to change the trajectory of systems even when everyone earnestly works towards change.

I don't know how well it would work, but it's an idea...

I'd rather have a happy and productive career than an ugly sexual harassment and wrongful termination suit.

"Oh, you don't have any other women on staff? You need to know that thats going to be sort of rough for me, in complicated gross ways. I'll still do it, but only if you pay me an extra 10% compensation pay on top of what you were otherwise going to pay me. (Which you can stop doing as soon as you hire another woman)."

Like, it was violent and creepy and Elliot Rodgers-esque. I would have looked like a psycho to the other co workers if I laughed. As it stands, the offending co worker looked like a psycho.

(He got fired. But I was a bit scared for my safety.)

If "everyone assumed it was you" and then gave you "the side-eye" for the consequences of this nasty person's comment's getting reported, fired, then this seems like clear signal that that workplace was no good in that way and it's time to find friendlier waters.

How do you think this would have gone differently had there been two more female employees? The hateful one would have kept a tighter lid on their true personality? The side-eye would have been more them vs us, or again, left thought but not expressed?

Anyways, lack of diversity sucks.

For 90% of my co workers, it was just "we all have sensitivity training for sexist comment reported to HR and only one woman, soooo..."

This. Interviewers, stop asking questions that only a fresh college grad should have at the forefront of their mind.

One of the best interview's I've had, was actually for a job I didn't get. It was for a game development company, and they actually posed problems and questions one might face in their day to day work. For example, writing an efficient collision detection algorithm.

These aren't questions "only a college grad should know" they're often questions that are baseline knowledge for understanding programming vs just having a few rote cargo cult "skills".

I have been on both sides of the interview process at Google, and I remember when I was prepping to conduct my first interview, one of my teammates gave me some advice. He suggested I open with something along the lines of "merge two sorted lists". Someone who couldn't do this with a minimum of effort barely fits the definition of "knowing how to program" (let alone being a good engineer). And yet, a non trivial chunk of the candidates he had met with didn't clear this bar! Starting with an assumption that your candidate can handle complex questions when they can't even program can just muddle the issue, and there's very little advantage to skipping over the basic test.

Bear in mind that this was at Google, and these were candidates who had already made it past the first line filter.

And yet, I'd be surprised if almost all of those who didn't pass his "basic test" went on to be gainfully employed, and successful, building X app at X company instead of at Google.

I'm not even being particularly picky here. Big companies can be notorious for hiring CS grads from great programs and never having them do any actual computer science, and I agree that that bar is unnecessary for good engineering. But this filter isn't _unrelated_ to the job of being an engineer (or even a code monkey!), it's basic, basic programming. I'd imagine it does a great job of removing the kind of candidates who are capable of trivially avoidable bugs that become time bombs down the road.

That said, from what I've heard it isn't the questions themselves so much as the overall interview process that puts people off.

First of all, that's not what cargo cult means....https://en.wikipedia.org/wiki/Cargo_cult_programming. What you're describing (though it doesn't apply to what I was saying) is also a bad thing, but you can't just use random phrases to mean whatever you want.

More to the point: You seriously think that engineers won't have occasion to use loops, iteration, and data structures resembling lists? Because as I've mentioned multiple times, those are the basic, basic skills being tested by a question like "merge two sorted lists". One might have actually have occasion to write that algorithm on the job, but the real point is to test for a basic understanding of control flow and performance. I can barely imagine a programming job that would never have occasion to use those things, and most of the actual engineering you do in the simplest engineering jobs involve tasks of similar complexity. This isn't me blindly defending whatever interview policy Google happened to have; I'm talking about the adjustments I personally made to find what works over the course of a couple dozen interviews.

> Many of Googles products implode shortly after getting released making software quality irrelevant.

Now this is just embarrassing. Your argument is so bereft of anything resembling logic that you have to resort to completely irrelevant asides. I suppose they should just hire random 14 year olds to do their engineering since software quality is irrelevant.

Per the definition on the linked wikipedia page cargo cult programming is "ritual inclusion of code or program structures that serve no real purpose", "results of applying a design pattern or coding style blindly without understanding the reasons behind that design principle" and "organizations that attempt to emulate more successful development houses". It's of course based on the concept of cargo cult behavior, where you try to achieve success by imitating external properties without the substance.

I'm saying that Google by having things like a challenging admissions process, predictable career path, stark focus on credentials, all-inclusive perks and campus environment are trying to imitate an elite school which is an environment where many people at Google felt successful. By doing that they recruit computer science heavy coders which, at least when they are inexperienced, tend to overly rely on CS style programming (advanced techniques, optimizations, correctness) in favor of other concerns of software engineering (organizational friction, code flexibility, overall design). There's evidence that candidates that want to work at companies like Google often cram these types of questions without necessary having used them in real world projects. I would say both imitation of academia, over reliance of CS as a factor in recruiting and the risk of neglecting software engineering in favor of computer science techniques is relevant to the concept of cargo cult and cargo cult programming.

> Your argument is so bereft of anything resembling logic that you have to resort to completely irrelevant asides.

The whole point of the cargo cult concept is that you do things that in the end is ineffective. We all know that Google has a lot of smart people, with credentials and that can write code. Yet, that doesn't always seem to produce good result as far as shipping products. Yes, one could blame management but that is also a factor of company culture.

> I suppose they should just hire random 14 year olds to do their engineering since software quality is irrelevant.

Why would you accuse me of "irrelevant asides" and then bring up "random 14 year olds". Software quality is subjective. One could argue that writing, both as part of developing and documenting software, is essential for software quality. You seem to use a lot of rhetoric that isn't relevant to the discussion at hand.

> You seriously think that engineers won't have occasion to use loops, iteration, and data structures resembling lists?

That's not the point. There are other ways to find out if people can manage these things that doesn't favor people with a theoretical background. Other method might be looking up peoples portfolios, having them submit code they've written, pair programming or reasoning around example code.

Googles hiring process is adopted for how Google works. Having a common theoretical framework might be important at Google. But that doesn't mean that people who doesn't fit into that profile can't be good software engineers, nor that this process is even feasible for other companies.

Yes, but in this case, it's a cargo cult of over-qualification. Hardly a bad thing.

>Many of Googles products implode shortly after getting released making software quality irrelevant.

That is a function of leadership and poor product planning and has nothing to do with the quality of the engineers there.

Even with merging two sorted lists, remember that you're asking people to merge two sorted lists at a whiteboard while explaining what they're doing, under somewhat stressful conditions, which may trip up someone who could do this easily at a keyboard.

We (my team at Amazon) don't start with a softball, but we try to make sure every question can be down-leveled if someone is struggling. My current one degenerates to: "insert numbers in a sorted list, then find the index of a number in the list". A surprising number of people struggle with this.

I know lots of people get tripped up by graph and tree problems (myself included) because they don't use them in real life often, but simple lists are another story. At FaceGoogAzon scale you can't always rely on array.sort().

Not one did it successfully. Most didn't even try. The best one assumed five numbers given. But we had to hire someone, and it wasn't this one. The deciders just hired the nicest person.

I don't want to ask it of the one we hired because I don't want to cause bad blood. :-(

It's possible that what little programming they did in their 'information science' (or whatever degree, not straight CS) only involved Windows programming, and nothing on the command line.

Are you sure that Google's first line filter is actually good?

The correct answer would be to loop while neither list is empty and always put the smaller element into the new array and remove it from the original list.

Then you iterate over each of the lists individually (one of them will already be empty) and copy over the remaining elements at the end.

Edit: See http://stackoverflow.com/questions/5958169/how-to-merge-two-... for some solutions (in one of the answers there, they made the same mistake.)

By the way, I don't think that group includes you: I'm quite sure you'd be able to pass it in an interview context. You'd probably be paying more attention to the details of the question in an interview and I usually give sample input/output to reduce the chance of a candidate accidentally overlooking a word in the question.

I mean, I've gotten 3rd grade arithmetic wrong in my own HN comments before! ;)

I think people are assuming that I force the candidate to do this in C or something and munge about with next pointers and sentinel nodes. They're free to do so if they'd like, but usually they have a pretty high degree of control over what language they use and in this problem, what data structure it starts in (including arrays). As mentioned upthread, that means the problem literally reduces to something like loops, iteration, conditionals, collections. These abstractions are fundamental enough that again, any reasonable definition of coding includes them. Physics PhDs writing simulations occasionally have to deal with more than one particle (or whatever), and if they have no clue how to "do X for every Y", they won't get very far.

This isn't just hypothetical for me: I've interviewed PhDs (including non-CS ones), and pretty much all of them blew past this first filter. I've worked with people whose only prior programming experience was during their math PhD, and they generally were pretty great at programming (and pretty horrible at engineering). The latter is easy to teach (this guy became one of my best engineers) but the former is much more fundamental, much harder to teach, and generally something you want at time of hire.

Since you say that you will have CS degree "soon", I'll assume you are fairly new to programming. Making mistakes is something that you will do for your whole career. A mistake does not make you one kind of programmer over another. Even many mistakes does not. It just means more practice.

Having the courage to post code off the top of your head in a world-wide forum, means you can have the courage to transcend where you are now. Keep trying to improve -- whatever it takes, and don't unnecessarily limit yourself.

If someone is offended by being asked to code something like this then they are probably going to be offended by the work they'll have to do once they start employment.

Of course there are other parts of our interview as well, but I've found an initial interview with a difficult algorithms problem does a great job filtering out bad programmers. Once we've filtered out the bad programmers we'll do a second interview that involves code from a project we've used in production before.

I don't agree with you because it's not the same context. In a day to day job context, you can focus on doing it correctly while in an interview there are a lot of factors that can affect your focus/creativity/attention.

In the real process, you might on any given day also have to wrangle with your build system, meetings, the structure of existing code, the test setup, colleauges who need help, debugging tools.

Now that might be an argument for why a whiteboard interview is inadequate. But not for why an whiteboard interview is distracting.

I had a friend in "support" who had build a lot of a data normalization system that I often worked with, and we talked about the phenomenon a couple times over lunch. Basically, she was a little offended about the lower pay and the "support" title, but would rather keep her work / life balance and relatively powerful position within the infrastructure stack, than join the production team for a raise. In hindsight, someone should have fought for the whole L2 support department to be better paid; there was certainly money for that (we were in a wildly profitable niche), and the support engineers had the high ground. But we were both young technical types, and didn't have the business / legal skills or the confidence to confront management.

I'm not sure how things have played out there in the years since I left, but if they wanted to, that entire L2 support team could take off and form a devops / analytics consulting firm, and probably multiply their incomes by some small integers. Then they could hire from the product team too, if they needed web/UI ;)

I wouldn't respond to one of those truthfully - I give four or five stars on everything, and never write any complaints in the free-form text fields. If possible, I just avoid doing them - especially if it's a small company, but at larger companies they tend to be per-team, so it works out the same.

I submitted a comment in a similar office survey service and despite my submission being anonymous, shortly after submitting it, a superior had a meeting with me to address what the anonymous complaint was about, having determined the comment was from me.

In this case I was very glad they figured it out because it ended up being a low risk way of bringing up my concern, but had I not had such a great employer, the result could have been me getting fired or something similar.

The comment was only one short sentence and there were 50-100 people employed at the company, so if you want to be honest on platforms like this, I would limit what you say and obfuscate as much as you can. It doesn't take much to identify you.

> despite my submission being anonymous, shortly after submitting it, a superior had a meeting with me

I'm glad you had a good experience, but a great employer wouldn't have lied to you about the survey being anonymous.

If my employer told me a survey was anonymous, I would expect anonymity on the Rank/Rate/True-False type questions, but assume that any freeform comments would be identifiable to someone as being from me based on style, content or something else.

There can still be issues of group identification: If a manager gets far worse scores than their peers, nothing guarantees that higher management will actually do something about the manager, vs believing that they just have a team with bad attitudes, and tell the manager to fire the person that they think si the biggest troublemaker, but if that's the case, chances are you wanted to get out of Dodge anyway.

http://stephaniehurlburt.com/blog/2016/10/5/a-few-stories-of...

Giving a take-home exercise and having part of the interview be a collaborative review of it? Good idea, works well.

Developing exercises where interviewer and candidate pair on a problem, using real coding tools and looking things up as necessary? Good idea, works well.

"Here's the whiteboard, here's a problem, code it up" -- doesn't usefully distinguish qualified from unqualified candidates, doesn't give you meaningful information about job-relevant skills, serves only to make people squirm and serve as a "yeah, but I had to do it to pass the interview, too, so now you do". Get rid of it forever.

Also, any set of questions designed not to determine someone's programming ability but instead to act as a proxy for something else, like "went to Stanford", goes out the window first chance I get.

This is not implementing van Emde Boa search structure, but simple stuff any entry level programmer must comprehend for a general programming posistion.

If you think the only way to have a coding exercise is to pose a whiteboard problem, then my advice is get out of this industry, now.

But my point is that, as a team you need to be able to express ideas in a format that other developers can reason with you in a live setting. Nothing beats a whiteboard in that regard. You can use a projector and a computer, but what if you need to draw a graph or something else?

It's not a tall order to ask them to be able to do this. And please don't forget that I'm not asking people to implement RB search trees, but usually a simple loop and an if statement. Reverse a list using a for loop, transform this string into a palindrome, is this string a palindrone, remove all duplicates in a list - stuff like that.

Asking coders with 10 years of experience to write 10 lines of pseudoish code is wrong!! Try avoiding a coding exercise in interviews for coding positions at all costs. In the worst case make it a take home exercise. This is the only way to save this industry!!! Got it!

Also there's the self confidence issue, common for many people, even while not a lot of people speak out about it. Ask someone with low self confidence to stand next to a white board and "implement" solutions to random problems, and he/she will freak out. At the same time, these people might very well excel in a more nurturing, friendly environment. Yet at the same time, people who are actually self confident and able to solve problems without much context on a whiteboard, might start running in circles once faced with unfamiliar problems that do not directly map to the algorithms they already know.

I found that just talking about code or problems with someone, can tell you lots of things about the ability of a person. Also, most likely, the candidate will already have some code put online, and god forbid companies are too lazy to look at it, rather than stroking their own ego by asking out of context questions that they already know the answer to.

I don't think a company interviewing a candidate needs to be holding their hand during the interview process. I understand that someone might perform a lot worse when whiteboarding, but an interviewer should be taking that into account.

Google sees about half of their employees come from at least a second interview attempt, implying that qualified people fail at least 1/2 the time. Maybe you're claiming that their process is terrible? Or that most companies are significantly better at hiring than them? As the only data point I can find, though, it seems pretty damning.

> And if there are only three examples of rejections

It's unclear to me why only rejections count. 7 for 7 is a lot more concerning than 3 for 3.

FTR, out of the 6 interview processes I've gone through I can think of 2 that don't correlate in this way (one in each direction). So I think I have some basis for suspicion.

I wonder if this goes both ways. If a male was leading the interview, and they were interviewing a female, would the interviewee tend to talk to the female interviewer?

In my experience, yes, this definitely happens.

1). Is it correct that there is no consensus on say, the top three reasons that females are under represented in computer science? It's frustrating that so many lay people voice just opinions and anecdotes. Is the real research starting to agree on what are the most important factors yet?

2) Some people say genetics is one reason females are underrepresented. For example they say, maybe on average females simply tend to be less interested in the subject. Is there any data that actually supports, or excludes this notion?

I think the closest ones to the truth are those who explore the early years and how they relate to the careers they will choose.

For example this study[0] on neonatal babies (meaning not yet influenced by culture/society) concluded that males are more interested by mechanics and females ate more interested in faces; reinforcing the idea that females are more social oriented and males more exploration oriented.

By the way, one of the reasons there is no consensus is the controversy that generates reaching any conclusion that is not "it's just that society is sexist"

[0] http://www.sciencedirect.com/science/article/pii/S0163638300...

https://www.youtube.com/watch?v=E577jhf25t4 (it has english subs)

Or just check this part https://www.youtube.com/watch?v=E577jhf25t4&feature=youtu.be... if you don't want to watch the whole ~40min documentary

2) I read about some statistics that women are less likely to be engineers in countries with high standards of living(e.g. Sweeden) compared with countries with lower standards of living (e.g. India). If true, given that engineering jobs are paid better, that would imply that once money isn't the main motivator, women either show less interest in engineering, or prioritise other things (e.g. people-facing jobs, or work-life balance).

Edited to add: by the way, back in the 50s and 60s, computer programming was female-dominated.

[1] https://media.npr.org/assets/img/2014/10/21/womencoding_wide...

[2] https://media.licdn.com/mpr/mpr/p/7/005/06b/02d/33fe648.jpg

We get all the stress, but none of the protections, in tech. And we never really get to stop doing it.

I've been through the washing machine enough times that I am pretty resistant to it, though it does still get to me as well. Early career, I did feel pretty humiliated after some of these experiences. Sometimes, I think we lose sight of this. When I described what it was like to "interview" at google and other places, people outside tech kind of shuddered. Their interviews are more like conversations - yes, you need to know what you're talking about, but you don't stand in front of a committee, at a whiteboard, getting treated like a grad student going through quals (if only!) every single time you interview at a new company!

I've heard some questions about whether this is just an inevitable part of being a programmer. If it is, fine, but then please, let's stop scratching our heads about why there's a supposed "shortage" of programmers. If an extremely unpleasant quasi-hazing is a necessary part of applying for a developer job (every single time), then let's not be surprised when people decide they don't want to work in this field anymore, don't want to enter it in the first place, are reluctant to change jobs, or, alternatively, prefer careers in actuarial work, law, medicine, nursing, or other fields where they get to take their exams under conditions that are far more respectful of the candidate and result in a lasting, meaningful credential, respected in their field, so they don't have to do it over and over.

http://www.americanbar.org/groups/disabilityrights/resources...

http://ncees.org/exams/special-accommodations/ada-exam-accom...

http://www.usmle.org/test-accommodations/

Are we ok with allowing discrimination like this to continue?

While it's just one case, I think this is illustrative of how seriously certification, licensing, and other exams are taken in the rest of the world. Exams can be useful and perhaps even essential, but they aren't trivial. It's a big deal to put someone through this. It is important to have safeguards to ensure fairness, integrity, and consistency.

One of the reasons we don't have this in tech is that companies have no incentive to be transparent about their exams, because they fear lawsuits. They'd much rather reply "we've decided not to pursue your candidacy further at this time" than tell you specifically what you did wrong during their technical exams (they really aren't interviews, they are exams - calling them interviews just confuses people who aren't familiar with what goes on in tech).

The thing is, these exams really are formal affairs. My understanding is that if you applied to google, there is a database entry somewhere with your name, screen captures of what you wrote at the whiteboard, notes on your performance, and a score. However, the person who has been through the tests have more or less no rights to any of this information.

I suppose some people might say "well, that's their business, if you don't like it, don't apply." My response is, yeah, exactly. I believe tech culture turns off a huge number of people - these interview exams are a notable part of this, along with back visibility, open offices, micromanagement, and concerns about career longevity are other. I believe that people with choice, those who aren't obliged to work for a particular employer in a particular field as a condition of living in the US are choosing not to work in tech. There's a constant drum beat about this shortage, from a field that puts candidates through this constantly.

I do think that tests without any protections for the examinee is one of many things tech may need to reconsider to attract people with choice into this field. I see no reason for government to get involved in helping tech get workers without while going about business as usual.

What does this have to do with 'female'? This is the norm in high tech for everyone - sadly.

Am I under the impression that this girl thinks that she's being singled out as a girl on this? Because I got it most of the time during my career. Not saying it's right, but I don't think it's a gender issue.

Yes, this stuff happens to many people, not just women. But it happens to them on an alarmingly regular basis.

This is an easy mistake to make if you are a kernel hacker in an organization that has, say 20% female developers overall, but where only 2% of kernel hackers are women. Easy to make, not malicious, but not much less annoying because of it, specially if you are the 10th person to assume that. From your point of view, it is a very efficient cache prediction strategy, since you'll get it right 90% of the time[1], but from theirs, it means half their queries end up miss-predicted...

[1] Yes, I am assuming 50% of all developers are kernel hackers, you can switch to something more realistic like product vs infra, front vs backend, etc. Also, the point is not about "lower-level systems programming being real-er programming!", but about making assumptions based partly on gender.

In some circles, you will get people who think along the lines of "yeah, of course there is sexism in industry, I mean, look at Bob, he keeps cracking jokes about women and browses The Red Pill at work. But I am not like Bob, I don't have anything against women, I am trying to help, ergo I can't be sexist". Yet these people can be condescending without generally meaning to disrespect or make less on anyone. When confronted with a woman who is very technical and assertive about that fact, they quickly adjust to treat her as technical, but their default assumption still goes to 'relatively less technical than me' when meeting any new female developer (and doesn't when meeting a male college). I was trying to point out that sort of thing is a problem too.

I would suggest this has nothing to do with gender, and that wanting 'less condescending interviews' is merely a gripe about the current state of the entire economy. Not only is it not tech specific, it's not even industry or nation specific.

FUCK YOU PAY ME

It's all about an aggressive attitude towards compensation. It's why I make so much and others whom I believe deserve more than me do not. They just aren't aggressive enough. You have to be aggressive by default when it comes to compensation. It's not about the words per se, but the attitude.

This happens to men and women. Recruiters suck, it's not a gender bias in most cases, it's that the role attracts a certain kind of person.

A lot of this doesn't need to be viewed through the lens of gender. The fact is our industry has a lot of money in it and that attracts all kinds of jerks/sharks/shady people. Men deal with all of this on a daily basis as well. I'm just starting to think we don't spend as much time organizing as a group and instead just put our heads down and take it/fight through it in isolation.

By contrast, when Google found out that they had a gender skew in promotion rates, they actually looked at the data and realized that the issue was that women employees were promoted at the same rate but put them self up for promotion at lower rates. Instead of taking more control out of every employee's hands, they launched a drive to make managers pay more attention to talented employees and encourage them to self-nominate for promotion. At the time, this closed the promotion-rates gender gap at the relevant levels.

Criticism of gender equity initiatives in tech is often taken as criticism of the goal, but much of the time it's simply a criticism of the cynicism and stupidity involved in the implementation.

[1] Apologies for the inflammatory language on its face but I'm referencing a specific term that's somewhat less insulting. https://en.wikipedia.org/wiki/Useful_idiot

Why are companies with female % that is similar to the female CS graduate % considered sexist but a company that overhires female engineers (to such a degree as to be suspicious of sexism) is seen as diverse?

Meaning, they are hiring in proportions which are consistent with their applicant pools, those pools are just different because of friend networks.

People that study these things always take a macroscopic view and talk about sexism/racism in the industry in aggregate and rarely in any specific company.

if you don't know female engineers you might not see yourself doing this. especially if you parent neither see this as a fitting career path

Nobody said that except you, which betrays your agenda.

Good engineering talent is in short supply and has been for two decades now. It certainly won't do any harm to explore alternate recruiting channels.

There is always an opportunity cost. Exploring alternative recruiting channels means not utilizing recruiting channels that have been optimized to find talent regardless of race/gender.

>Nobody said that except you, which betrays your agenda.

Articles that state that software engineering companies are sexist because they don't have a 50/50 gender split are very commonplace.

My agenda is not being discriminated based on my race or gender fyi.

In the US?

If you're hiring from the whole planet, that's not the valid measure.

this is difficult for a variety of reasons.

If a great man comes along hire him. If a great woman comes along hire her.

If you have a quota of X you are going to hire less qualified people, thus creating a culture of quotas and not performance.

You also have to look at differences in the middle of the pipeline. Imagine you only give a phone screen to 5% of female applicants, but 10% of male applicants. You have to think very hard about why that happens. Maybe your ways of rating resumes have a built in bias.

For instance, imagine that I only interviewed new grads that had at least two internships in large tech companies. A rubric like that looks neutral, but you'll discover that the demographics of CS graduates vs those that have those two internships are very different (far fewer women, and a lot more people that will identify themselves as asian).

I agree with your approach, but the gender is not the only diversity metric. If you push this toward ethnicity, religion, sexual orientation, introversion, geekness level, you might end up in a difficult situation and your company will look like Noah's ark. Instead, what I would do is to specify the tasks that needs to be carried out and define metric to measure it, then do a blind hiring. Attributes like graduation or work experience is irrelevant if the candidate passed the test. During the probationary period, if the team doesn't like the new members or the other way round, you can let them go.

The right way is to look for qualified candidates from multiple backgrounds. For example, plenty of companies recruit from MIT but how many recruit from Spelman College?

Lol.. oops.

https://twitter.com/laggdotme/status/784502966728269824

But how else is the interviewer going to prove how smart they are, and signal their dominant status to you?

The most important part is that the candidate can say "pass" and that you don't force them to squeeze out a shitty answer.

There is simply no better way to find out the breadth of someone's knowledge than to quiz them over a broad range of things. Think about it: this is exactly what engineers do with each other naturally and in a fun way over time.

"Hey, do you know about X?"

Compare this with Google's idiotic idea of asking a single question and judging the candidate on that randomly selected question.

No one who knows their stuff will fail to answer a long list of pop quiz questions well...but anyone of any skill level might not know the answer to any particular question.

Bloom's taxonomy[1] is useful here I think. Assessment ideally shouldn't stall at 'understand' or 'remember'. I do agree it is important to know about a large variety of things. The more specialized the job is, the more important it is for a candidate to do a decent subset of it on day 1. But sometimes you can explore a candidate's knowledge through 'apply' (use X to do Y) or 'analyze' and 'evaluate' (do Y), such as in a system design question, to see the structure of their thought process rather than something more scattershot.

Finally, regarding skipping a question, sometimes you can sense that a candidate's knowledge is limited in some area, which is a good time to just move on and try to go deep on something else. Not everyone has been afforded the same opportunities and has the same amount of experience. Software is a job where the potential to grow can do you better than a fixed skill set, especially if you see hiring as similar to making an investment.

[1] https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/

Please note that I have never been called back after doing that.

A genuine technical interview can and will be cut short immediately once it becomes clear that you are not bullshitting on your resume. The person who still asks that 20th question after you have already answered the previous 19 correctly is just being an ass, and playing stupid, time-wasting games with your interview.

That's the only time I've been that intentionally rude. He really deserved it, though. And (consistent with my other comment in this thread) I'd name them, but (a) they went out of business a long time ago, and (b) I can't quite remember the name of the firm. Something "clever" ca. 2004, missing a vowel.

Is it rude to do that in an interview? How am I supposed to decide if I want to work for a company without some insight into those things?

  A: What does your competition look like in this space?
  B: What's your plan for overcoming Company Y's huge lead in market share?
  
  A: Tell me about your development process.
  B: So what percentage of your process overhead is cargo cult edicts from upper management?
  
  A: How much runway do you have?  How long before profitability?
  B: When the current owners look for their exit, how screwed will I be?
  
  A: How do you deal with technical debt?
  B: Show me the worst code in your repo.

If the company focus on trivia that much on interviews and let juniors run the show expect no good senior working in there. That's never going to end well.

If 'getting the job' in that instance is the metric of success, then it doesn't work well at all, but in quality-of-life terms it's served me well.

Is it genuinely offensive to be asked the complexity of bubble sort? If such a question causes you so much offense that you lash out involuntarily, then you have some serious personal problems to work out.

Offensive, no. Just trite and tedious.

And more fundamentally, simply not indicative of the high quality, (genuinely) cerebral environment the interview subject, back in the original article, is looking for.

Why be rude to the interviewer even if that is the case?

I get that it's generally better to be polite. But when the same kinds of tedious and disspiriting (and sometimes downright condescending, and/or simply time-wasting) behaviors come down the interviewing pipe, over and over again... I can understand the temptation to bypass decorum, and allow a gut-level, emotional response to surface.

Being as while we might generally prefer to keep things professional -- we're not potted plants, either.

(I should note that, as a consultant, nobody ever asks me to whiteboard some code for them, despite often committing into their own repos and doing work that is no less important to them as a company. Almost like the hoop jumping is just some power-move crap on the part of the interviewer.)

If a few more people were genuinely afraid of being humiliated for blindly cargo culting google's hiring practices it might go out of fashion quicker.

At least, that's my hope. Which is why I usually tease people for doing this (yes, that upsets them).

I think you need to pause, take a step back, and consider who is actually making an ass out of themselves in that situation.

It's more effective when you're over-the-top polite when you set the trap with leading questions though.

Trolling gives me a bit of mild amusement in what is otherwise a dud investment of my time. The only people who seem to have a problem with that are people who have an exaggerated respect for authority figures (always unhealthy).

If, on the other hand, the interview process is obviously relevant to the job at hand and thorough I go out of my way to compliment the interviewer to their boss even if I know I failed.

Politely leaving without saying anything doesn't improve the situation for anyone. These pop-quiz questions need to be stopped.

Tongue-in-cheek aside, of course passive-aggressive isn't the ideal path to improving interview processes. :P

If you do not know such things then how are you going to take them into account when you have to solve a problem?

That's the point of the "hate" behind these questions.

Point one, it checks if you understand the most primitive sorting algorithm out there. Pretty low bar assessment of your general CS knowledge.

Point two, it checks whether the candidate has understanding of computational complexity.

Now you might argue that you don't need any of that in the day job, but that's on you. If the interviewer wants to check you have fairly basic minimum of understanding of very basic CS concepts, that's a suitable question to ask.

"Has memorized the complexity class for one particular, indisputably marginal sorting algorithm"

is equivalent to

"Has an understanding of computational complexity"?

Now you might argue that you don't need any [understanding of complexity] in the day job

That is quite clearly not what what said.

Just wonder what would you suggest as an alternative, if you need to confirm basic algorithms proficiency and understanding of big-O? You inevitably arrive to an algorithm question, and bubblesort is as good as any.

Actually the blank stare means "What is this, sophomore year again? I can't believe anyone still cares about bubble sort."

Just wonder what would you suggest as an alternative?

Look at their GitHub/Mercurial account (which you've been ignoring all this time in your desperate search for something to nail them on, but if you would spend a second or two looking, you'd find is chuck full of algorithm stuff -- much of it way more intricate than bubble sort). And ask as many questions as you like based on some project you find there.

Or, pick a problem you're working on that's algorithm-related (but which you genuinely don't fully know how to solve). Use that as discussion material. What approach they'd suggest, given that the data are sparse / not evenly distributed, whatever.

You know, as if they were a peer. Not an interrogation subject.

I like how you cast me into some mean soulless generalized interviewer and started bashing down your pain points, while I actually don't interview people. But a simple algorithm question is not out of line on a programming interview; it's not whiteboarding or take home assignments. It doesn't take much time to answer, it shows you have some very basic CS knowledge at least. I did not use bubblesort outside of CS101 class some 25 years ago and had to actually think about its runtime complexity, but it was not hard and it did not take long.

Again if you think basic CS knowledge has nothing to do with your job that's on you. Should add that I'm not really comfortable being grilled in any way on the interviews, it's not a very good dynamic. But I don't see any general, dignified way to filter out non-performers. Unfortunately just credentials are not enough in our trade.

Actually, yes. But that's just a matter of personal taste.

Again if you think basic CS knowledge has nothing to do with your job that's on you.

Hmm -- you keep coming back to this supposition, for some reason. And again, that's not at all what I was saying.

Should add that I'm not really comfortable being grilled in any way on the interviews, it's not a very good dynamic.

At least we're on the same basic page, then. The main difference I guess is that (1) I see the issue of maintaining "dynamic" -- the overall tone of bilateral respect, in the interview process -- as not just important, but very important, arguably crucial in fact; (2) dynamic aside, the "quiz-show" approach is rife with methodological weaknesses (for example, it highly favors those who cram on the material - or were simply lucky, and happened to have heard your questions before in there interview process); and (3) I find it quite easy to assess ballpark competence in tech folks (and to find red flags for incompetence) just from unprompted, regular discussion.

But again, that's me, not you. You can apply whatever filters you like. Like Steve Jobs said (in a different, but related context) ultimately either they'll work or they won't -- and everything will sort itself out.

As a software engineer, there are things I do all the time: Reading unfamiliar code and third party documentation, do some debugging (often based on logs), and try to write code that is well factored and thoroughly tested. I want to know whether any prospective teammate is good at those things for their experience level.

Interestingly enough, the general algorithm questions are easier the newer you are, and the less you know about all the topics I mentioned, because more often than not those are learned on the jobs, while the basic algorithms are learned in college. I end up reading papers on algorithms at work, but they are not algorithms that I expect to be general knowledge, and that I'd expect people to read on the job when they are applicable: How many candidates are going to be able to explain hyper log log, Paxos or Lamport clocks on command? Is knowing all of those three by heart a sensible pass/no pass signal? I for one don't think so.

And hence, kind of silly to use as material to grill people on.

Bubble sort question probes the understanding about the complexity theory, hopscotch hashing about concurrent algorithms and CAP theorem about problems with distributed databases.

Naturally when these questions are not open for discussion and instead only a short answer is expected then there is in my opinion something wrong with the interviewer or with the company.

You have to understand that the other side knowns in fact nothing about you and I find these questions to be quite fair to improve the situation.

It's just that, again, you're picking a very marginal example to do that with.

And more fundamentally -- simply asking someone "What's the complexity of $foo"? doesn't tell you anything about whether they "understand" complexity. It only tells you whether they've adequately memorized that particular cell on their crib sheet.

As they are thoroughly incentivized to do, thanks to people employing interview techniques like these.

Naturally this question in isolation is not very informative and it is hard to differentiate if the answer to it comes from a more general understanding or is just memorized. It has to be supported with other questions and wider discussion.

What I want to say is that in a larger context this question is still not unfounded and it is hard for me to consider this or any other listed questions as a signalling a dominant status.

I think that these three questions showed are quite good and fair (for example I did not what is Hopscotch hashing, but it looks quite important after looking it up, so thanks for this).

Some of such questions might be possibly indifferent for the position you were applying, but seeing some hate behind these questions is in my opinion quite unjustified.

This is absolutely absurd. I have a CS degree from Berkeley (universally considered one of the best CS programs in the world), and I spent the last few years on an unusually CS-focused team within Google (specifically, artificial intelligence). Off the top of my head, I could easily imagine not remembering what exactly bubble sort is. What's the use of retaining the details of an algorithm that's literally held up as the "don't do this" solution to sorting? The most I can imagine being useful is knowing the theoretical best case average complexity of a sort. And my job is wayyy more CS-focused than the average engineering job at Google (let alone elsewhere). You heavily overestimate how useful rote memorization is to real-world productivity.

Whether a candidate _understands_ these concepts, on the other hand, can actually be illuminating. If you described bubble sort to someone and asked him to describe the complexity, then you're getting all the signals you're describing about CS understanding and education. (though there's controversy about the usefulness of these skills as well)

Though to be honest, Google uses those kinds of questions and apparently treats them as useful, so I'm not sure why you'd have wanted to work there.

Eventually got fed up with their process, told the interviewer what I thought of it and hung up.

(which, to point out at least one upside, is the only way I've ever discovered of getting a company I don't want to work for to stop bugging me with recruiter spam)

Ah ok, I guess I misunderstood you. Though this "absurd and pointless situation" does in fact happen: There definitely is a class of interviewer who will expect off-the-top-of-your-head detailed knowledge (like being able to remember bubble sort's details from just its name).

No, it means you're smart and have learned to sift out unimportant matters of detail ("Was bubble sort, which I only ever had to think about for a week or two the first half of my sophomore algorithms class, and was only ever brought up as a negative example anyway, O(n^2) or O(n^3)?") in order to focus on much more interesting and useful stuff.

not knowing or remembering doesn't preclude one from understanding that one approach or technique can be more (or less) costly than another.

Reading the Aphyr "Jepsen" series is what helped me learn it... when I was still in high school. And this was necessary because I was curious about which database I needed to use for a particular side project I was writing.

It's totally possible to just Google "bubble sort complexity" or "CAP" and learn about it.

Yes, you can today easily find quite a lot of information and learn about it, but this was not the situation the original poster lamented about.

The claim was that when faced with such question, one can just google it and read the answer. Problem solved. Because these are just some random facts like who is the president of France.

My point was that in my opinion these questions actually probe the general understanding about the CS field.

As a matter of practical use, most of the time details about algorithms and data structures, from implementation through complexity analysis, can easily be regarded as "mere facts." Those times when this doesn't apply are times when somebody applying formal academic level CS is surely required, but these are far less frequent than most programming tasks require--even at Google.

This is like knowing that lookup complexity of the hash table is usually O(1) (but could become O(n)) and one should use it instead of list when frequent lookups are necessary.

I am not an interviewer, but I think that the questions listed were actually quite good pruning questions and if you do not know them, well, there are the search engines and you can look them up for the next time instead of starting lamenting about the unfairness and showing the ignorance that is in in this case in my opinion demonstrating the serious negligence.

Instead of asking about Bubble sort, ask what is generally the complexity of a naive sorting algorithm and how good it can get for a general case.

I have investigated immutable hash tables but I do not know much about concurrent hash tables. This is my blind spot, so I do not know what the relevant question about the Hopscotch hashing would be.

Instead of asking what is CAP theorem, ask if you can make a distributed system with consistency and availability at the same time.

I think that this was the intention behind these questions after all and is the reason why I am trying to argue in support of them.

When I'm trying to evaluate a candidate, I want people who proactively search out knowledge in that middle zone. People who understand complexity theory, and CPU caching behaviour, and the C10k/C10M problem just because they know they might need it some day. Because they want to work on problems where this stuff is relevant.

This sort of knowledge is the difference between hearing "our website is slow - I think its the database?" and "our website is slow - I took a look and our database server has run out of open file handles. But I'm still confused - that shouldn't explain the latency we're seeing."

The CAP theorem is a great example of this sort of middleground might-be-useful knowledge. Understanding how your database will behave in the face of network partitions is exactly the sort of thing that separates good engineers from great ones.

And yes, I too keep forgetting this name but I could talk about the issue if they explained what real problem they have.

As for bubble sort, I'd just write

  for (i=1; i<n; i++)
    for (j=i; j>0; j--)
      if (X[j-1] > X[j])
        swap(j-1,j);

And insertion sort actually is better. It can be made to work online, keeping all elements seen so far sorted before receiving the next element. And I think it ought to be more cache friendly than bubble sort.

1. This isn't the sole and probably not even the dominant reason for such questions.

2. When people resort to such intimidation you can out-alpha them by threatening to leave as if they were not worth your time (has to be convincing, ofc). Intimidation is a specialty of the insecure so this trick can work, though you may want to consider whether such job is going to be fun.

Doctors go through years of schooling involving a substantial amount of memorization as well as board exams, residencies and so on before they're allowed to do much doctoring at all. This is the case because it is crucial that they not forget in an exam that neck pain combined with a headache could be a sign of meningitis, etc.

If an engineer needs to know which sorting algorithm is best, taking some time to research and google it is not going to cause a problem. That's why questions like this are bad, even offensive--they make people feel as though they're bad engineers when actually they're being put through an ineffective interviewing process.

Why is it crucial if it's just a "5 second google search"? And not every doctor is dealing with serious life threatening illnesses, just like not all developers are working on bootstrapping their uber for cats with Angular 2 and pouchdb.

Because you can't google every single symptom. And sometimes the symptom is only mentioned very briefly in passing. And sometimes the patient is hiding the symptom. And sometimes the important piece of information isn't a "symptom".

So, you need to have these in your head so that you can connect the dots to "I thought it was X, but new data Y suggests it might be Z. I need to go push on abdomen, put stethoscope to stomach, right now while patient is sitting in my office." A good example of a non-symptom that would go into your diagnosis--patient just got divorced and started dating again ... hmmm, STD's are probably back in the probability pool again, might want to check for one given the other symptoms.

Where computers are useful, but, sadly, we don't have good implementation is spotting localized trends. If one person comes down with Karposi's Sarcoma, that's an anomaly. If 10 people come down with it, that's a problem. It would be good to have databases with data from multiple doctors so that we could let the computers spot those things.

Not every illness is lethal, for most of them you can just make it worse. Even less requires quick decision. Also, software evolves and scales much quicker than humans which makes our industry less standardised.

> That is not something you can say with someone's code being written this exact second. There is always time to research, test, experiment, and change.

In my world, there is a risk assessment which defines if we have time to research, test, etc. or just fix it on the go, because it is not a big deal if it occurs. In other cases we have a cold sweat and dead silence during deployment, don't we?

http://www.ncpa.org/sub/dpd/index.php?Article_ID=23148

Not sure why doctors don't look after their patients. Is it because there is no incentive to? If they do bad diagnosis that means the patient comes back and the doctor gets more money. As long as they don't get sued there's really no harm to the doctor.

As a software engineers we are incentivized to perform well. A crappy system means time spent debugging issues, money lost for our employer. We also take pride in building good systems so we can take that experience to the next company and get s higher wage or position.

As it is, we can't even seem to find a program that can analyse an ECG better than an overtired and exhausted resident at 4 am, which is really quite a low bar to cross.

So, let's take heart attack diagnoses in emergency rooms--a fixed procedure laid down by following statistics is WAY better than human judgement.

Generalized abdominal pain--computers aren't so good. There are a lot of possibilities and most of them are about equal. The doctor is more likely to tease out the problem over time through conversational randomness.

Some obscure genetic disease--a computer is going to be FEROCIOUSLY better than a human. A computer can keep all of those .00001% options in mind as tests and diagnoses rule out the far more common problems until the low-probability things pop up into statistical relevance.

Also, it may surprise you, but there are many highly experienced and extremely capable programmers that didn't study computer science in college. Filtering those people out because they can't answer programming trivia questions is, at best, hugely short-sighted.

It's called being flexible and able to learn, and it's the single most important thing you're supposed to get out of your formal education.

Bittorrent is a great example. All the pieces had existed for a while, but Bram Cohen put them all together into one place. After that, lots of people created other BitTorrent clients. However, without that first, smart person, it wouldn't have appeared.

He finds that such trivia quizzes are strongly biased towards supporting younger candidates who are fresh out of college.