I let any of my interviewees know that it is ok to use google, I just want to know what they searched for. I'm not running a trivia contest, so if they forgot the name of something or need a little refresh, that's fine.

The way I combat this is by asking questions about their current jobs, then diving deep into the technical problems they faced. It requires a lot more preparation for each candidate, but it ensures it’s not a trivia test but a discussion.

One of my favorite interviews (as an interviewer) was when a SDET candidate provided a link to their website. When I visited it there was an issue loading some page. So I asked him about it and how he would troubleshoot, then asked him to come back on Monday with a solution. On Monday, the site was fixed, and he explained to me what happened in AWS, how he figured it out, etc. So he was hired, going on 2 years now and doing very well.

I don’t need robots who can recite what a b-tree is (ChatGPT can do that). I need people who will work hard, can understand the big picture and how to approach problems, while being a good personality on the team.

This.

My other favorite is very open-ended questions. I mostly do ops-y interviews, and my favorite question is "what happens when you type 'curl https://google.com' in a terminal and hit enter?"

The question is so broad there isn't a "correct" answer to Google, and it crosses enough domains that any article they find is going to be too long to skim. Then I ask them to really zero in on some aspect of it they feel comfortable with and give detail. What syscalls happen to start up curl? How does the OS know how to communicate with the local router? What's the entire flow to translate "google.com" to an IP?

It's also just fascinating to see which parts candidates latch on to. I had one person spend like 30 minutes just talking about TLS and PKI. Another person delved into kernel packet handling for a while.

That's what I do (instead of curl, with a browser).

MANY super cool convos spun out of this question (interviewed around 60 people). One of them never actually got to the network request part bc we went DEEP into event handlers in a GUI etc. Another candidate was all over the place with key exchange protocols and what and how can go wrong.

I usually don't ask further questions to "corner" them, let them go into any of the details they want.

Ah, I am so happy I am not the only one who invented this interview method :-)

This is a popular question, turns out.

https://github.com/alex/what-happens-when

Thanks for this. Looks like it's going deep into "client side".

Thing about "Google.com" (I used Facebook/Gmail) is that when server side stuff is of interest (as it was for this cloud engineer job), then I also want to hear about geo DNS, reverse proxies, LBs, CDNs, eventual consistency, distributed storage, etc, all the complexity that is happening once that cat video appears in the browser.

Then again we can go back to JS, CSS, JIT and others' space

This is the most common question for screening non-ICs like EM, TPM at Amazon.

I have a "greenfield" scenario I ask SRE candidates. I give them unlimited time, resources and money to build whatever systems they want to ensure that code is production ready before it goes to prod. The only constraint is that they will be the only person who ever has a pager, 24x7x365. Tell me how you make that work with high confidence that your life won't be ruined. It's not verbatim but that's the gist. So many paths to explore and I think it does a great job of leveling people.

I've only ever had to hire one person to work beneath me, but that sounds like how I ran the interview.

We were hiring students for a temp position. We're a Controls Engineering company, but my department is dealing with more traditional languages for supporting applications and needed extra help for a bigger project. I know the tech we use isn't standard in the university so the interview was asking students about how they approached their major projects and their methodology for learning new tools/languages.

The first guy who straight up said "I already know enough of C++ and Java, I suppose I'd just google how to do x in c# and branch from there" got the job. Because...yeah, that's about it. We talked about a 4th year project and what his responsibilities in the team where, problems faced, solutions found, etc.

This is how I was interviewed when I was a scientist/engineer and how I got interviewed at gov labs when I switched to programming. I still refreshed material for my interviews but they were focused on the actual job areas. I was really shocked that this was not common in the software world and feels weird that as an AI research people are asking me about leet code and not about mathematical formulas, limitations, and analysis of architectures or my own research. PhD scientist interviews (at labs and universities) are essentially a short form of people's discretions with a focus on Q&A. It's always appeared successful to me and I figured that the leet code was always because 1) momentum and 2) there's so many applications that an arbitrary filter has no realistic effect on outcomes other than reducing the number of candidates (due to the difficulties of measuring merit). (Similar to university admissions) But I think we all have to admit that meritocracy is not realistic and act under this belief. It's fine to have arbitrary filters if we recognize them as arbitrary but not if we go around and tout superiority for passing these. But I guess that's a corollary to Goodhart's Law

Because those fields are currently too limited and that would only work for researchers that are exactly in the field the company wants talent for.

Researchers' fields/interests are too narrow, by far, for companies to find enough candidates. Machine learning departments go from low thousands (FANG) to maybe a dozen individuals on the low end (small regional banks, ...). That adds up to let's say 40.000 jobs worldwide.

Plus there's the non-cheating cheating. The majority of conferences (including ICLR/NIPS/CVPR) are still presentations by companies about how they "innovated" by letting an intern use 10-year old techniques, in an established library (ie. not pytorch, but an "integrated" solution, sometimes going as far as an Oracle tool) to look at their own proprietary data (in medical, social sciences, sometimes chemistry). This then delivers them a "paper", goes into the conference proceedings, and they make sure this delivers dozens, sometimes hundreds of citations for all individuals involved.

Don't get me wrong. Delivering a major paper at those conferences is a major, incredible accomplishment that's beyond me, for example. But there's 20-30 people on a yearly basis that "really"/honestly do that and over 5000 total presentations at those conferences. And there's 10000 or more candidates needed to fill positions at companies.

And then the question is: who would you rather hire? A math phd, or frankly even a CS master with no relevant machine learning papers, or that intern?

The fact that most employers aren’t like you is keeping me from starting interviewing for a job after nearly 2 years of sabbatical. I simply refuse to participate in the typical interviewing process where I simply cannot show my skillset, 20+ years of experience and my diligence.

They are out there - when I was looking a few years back I would ask what the interview process is like. If whiteboard, I politely refused and tactfully said why. You won't be taking any FANGG positions, but there are plenty of positions out there that pay FANGG money minus the interview overhead. I've had some pretty great interviews when it turns into a conversation about a previous solution, or a project I'm working on, which have led to almost 100% offers. In contrast, I would do absolutely horrid in whiteboard or quiz-like interviews. Some people are built for it, but that's not me.

I had several bad experiences right out of school - I would regularly do programming exercises for recruiters and wasted a lot of my time I will never get back. Some of those exercises would take 3 to 5 hours, and I was told I was one of the faster candidates :/

Once I was able to build a resume and show off projects I told myself I would never go through that experience again.

I don't know if this comment will help you at all, but good luck to you! If you are in the Midwest area I know of a recruiter or two that are great people and can help get you pointed in the right direction. My email is in my profile.

Thank you, I appreciate you sharing your experience, it hits close home. I have moved back to Europe, but am actually looking at remote positions in the US. And thank you for your offer, I’ll take you up on that!

The problem is no matter how much training or systems a company has for its interviewing process, employees won’t read it and will inconsistently interview candidates (couldn’t remember List method == red flag; another interviewer rightly wouldn’t ding someone for this). I think interviewers really need to develop their critical thinking skills in how they judge talent. I’d rather have someone who wasn’t able to solve part three of an interview problem but high leveled some decent approaches that indicate, given more time, they’d be able to solve it. It’s interesting to me that we place this artificial time barrier on many interviews when in reality work is not chunked out that way and often takes quite a bit of thinking to come up with good solutions to things. On the other hand, I don’t think take home tests are a good solution for interviewing either. I think a lot of interviewers just end up copy catting whoever they trained with when watching interviews as a shadow (if they even had this opportunity).

I always gave open book exams to my students. They don't need to memorize details, but show understanding. Questions would range between "find the page in the book" to "can't answer it if you don't understand the subject matter."

Interviewing is a bit different, since there's no book, but when we looked for a junior dev, the applicants got a laptop with VS Code, the common browsers, internet access and an empty (macOS guest) account, and a few (short!) tasks to show if they understood the questions and were able to start formulating an answer. We also looked for a personality match (we're a small company) and signs of general intelligence. Worked out reasonably well.

I really dislike the more extreme approaches to interviewing that one sometimes comes across, like leet code scores. That's just too disconnected from the actual work. Unless you need your employee to grind leet code questions, of course.

I expect my interviewees to use Google and am disappointed if they don't. It's more important to me that they know how to find the right answer than it is that they just know it off the top of their head. 90% of their time is going to be spent figuring out how to solve all the known-unknowns they're going to come across.

As long as you tell them they can, that seems great. I think it's normal for people being interviewed to assume they can't use any outside assistance.

Yup! If I see them struggling to answer a question or admit to not knowing something I will ask them "well, how would you find the answer?", to which they always half-jokingly answer "I'd Google it". That's when I'll let them know that I want to see some of that too!

Why? You'll have access to google when you're working so I would assume you're allowed to google stuff during the interview.

Because people administering tests always have weird or arbitrary assumptions and restrictions. Part of getting through school is jumping through all the contrived hoops they place before you, even when it'd be much easier to just walk around the hoops. Job interviews are no different; different interviewers have different expectations, and many don't want you consulting outside information sources.

Ditto, 20 years on and I research and read articles when I start any new project because in one year there can be entire paradigm shifts in tooling and best practices.

yeah same. it's not a recitation / memorization challenge, it's an honest "if you ran into X at work, show me what would you do" exercise. It's hard to imagine an answer that doesn't start with "I'd probably google it." How do you evaluate the quality of the search results? How would you scan a page to quickly find the salient details? How would you test what you'd found to determine whether it was an effective solution? etc.

Tangentially related…

Life insurance companies (in the USA 20 years ago when I sold it) would require a medical exam for every life insurance application. If the applicant commits fraud by having someone else take the examination (usually a blood draw, height and weight measurement, and maybe a heart EKG for bigger policies), the policy is considered valid after two years. Even if the insurance company finds out fraud was committed, the benefit payout is guaranteed after two years.

The implication being that after 2 years the immediate risk of death the insurance company wants to protect itself against did not come to pass, so the policy holder should not have the risk of losing their policy to a post mortem determination of fraud from the insurer despite believing they were insured.

Taken back to the fraud in interviews, you could make the same argument that after some time working on the job and not gettimg fired, the risk of getting someone not actually qualified did not come to pass, so its more useful to look at the employees current performance than their original fraudulent interview.

But the bar for firing is below the bar for hiring.

This is true not only because of status quo bias, sunk cost etc., but also because employers will often try really hard and be very patient, in the hope that an employee's performance will improve.

Sounds similar to how (most? all?) life insurance policies will pay out for suicide after a two-year waiting period - the idea being that those who are dedicated enough to plan out that far in advance don't actually end up going through with it, or it's a rounding error.

So perhaps the medical exam stuff should be swapped around; everyone can get life insurance, but if you want payout for certain things to kick in before two years, you get a medical and pass.

I think any insurance company would have a lifelong suicide exclusion if they could. I’m pretty sure they are legally obligated to pay after two years, as it’s be too easy for nearly any insurance company to claim most deaths were possibly a suicide. The main point being that you can’t pay the mortgage on your house, which may be the sole reason you got life insurance, if the benefits aren’t being paid out.

Also, there are some policies that do have limited benefits the first two years. This is especially common for any type of insurance that doesn’t require a medical exam. They tend to be very expensive for the amount of insurance they provide, but maybe have a manageable monthly payment because the amount of coverage is relatively low. This type of insurance is usually “I can’t qualify for anything else” coverage, or it’s for a very specific purpose, like paying for funeral expenses when for some reason the person didn’t have the desire to just save that money in the bank.

> I think any insurance company would have a lifelong suicide exclusion if they could

I suspect you're right.

There certainly are desperate people who go to Vegas, lose their entire savings, and choose to end their life because they're not willing to put in the work to continue trying to survive for another, say, 40 years, starting with zero assets.

There are also people with severe mental illness (poor/no executive function, prone to psychotic episodes, imbalances in their brain chemistry) who make poor choices or don't even consciously make choices but their body is nonetheless doing things, who end up also choosing to end their lives.

There is overlap between these two groups. How could an insurance company decide which is which? If cancer can kill you and they pay out for that, why would they not pay out for deaths due to mental illness? Because they can. We're becoming more willing to talk about mental health, but there's still a ways to go.

There's a ton of stuff that can go wrong with the mind and I couldn't hazard a guess at what proportion of people are 100% okay.

There’s also probably some social bias. In the US, we have a strong bias towards self-determinism and saying someone did something because (in-effect) they are pre-wired to make that deterministic action flies in the face of social conventions about individual free will. There are always some policies that are shot down because they run against social normative ideas, even if those ideas are shown to be false.

(The free will vs determinism is obviously overly simplistic above)

The problem for the industry with the "no questions asked" plans is the only people who sign up for them are the ones who have no other option, which makes them more expensive, which makes the only people who sign up for them ... etc etc etc.

Yes. There’s a few factors:

- higher premiums for everyone because the pool of applicants aren’t that healthy

- the super sick people simply die in the first two years

- the semi sick people can hang on for years, but weak health may me limited ability to pay because of limited employment prospects

- the not sick people will find better prices or simply realize the insurance is overpriced and cancel it

I don't understand how #2 and #3 would be considered cheating?

If they can still answer the question and show evidence that they understand it, what's wrong with reference materials and googling?

There are different issues of "is this an honest interviewee?" and "is this a good interview?" and you're welcome to argue the latter all you like but the solution to a bad interviewer isn't just to cheat because you don't like it.

Context matters. If you're looking to see "have you really done security stuff?" by mentioning some OWASP Top 10 vulns and asking for a quick description, for somebody that has indeed done any security at all that will be trivially easy and anybody else will need to google it. Can they skim google and get you an intelligeable answer? Maybe but that's not what you're looking for, the question itself is a proxy for something else.

For most of my interviews I tell folk up front to google whatever they need to. But at the very very earliest stage of the interview loop we do a quick "programmer is a big field, what kind are you?" round so we know how to place people and the questions we use for it are all trivially easy for somebody even remotely familiar with the fields we're looking at and also trivially googleable for anybody else. The goal with that piece isn't to challenge people it's just to round their skillset to the nearest role that we have. I've caught people retyping code, pretending to be inventing it, from what's clearly the first article they found when they googled the keywords. It was never intended to be a hard problem, just something to quickly tell an infrastructure from a frontend engineer. It was more or less bitwise identical to the article they were retyping from and they couldn't explain the code or modify it. That doesn't even accomplish what they want, even if it worked we'd then go on to interview them for a job they couldn't do.

Again you can argue about whether that's a good practise or not but it's still not licence to simply cheat on it. You're conflating these and and I think it's not the right thing to do.

I lol’ed at the Paper Stack Overflow error.

I'm so curious about that last one. You interviewed and accepted one person, but a different person showed up to actually work? If so, that's incredibly bizarre.

There’s lots of stories of this happening. Especially once video interviews have become common making the grift easier. Example: https://www.askamanager.org/2022/01/the-new-hire-who-showed-...

Apparently this is moderately common especially during remote Covid work; there were some discussions of it on HN, even to where the person interviewed and accepted was not the person working, and apparently the work was being done by an entire group of people.

Been remote since 2015 and not gonna lie, thought about finding contractors on Fiverr or e-Lance (or whatever e-Lance is called these days) to do just that.

Basically become a PM for a team of four Vietenamese kids and pull in pay from 3 gigs.

I later did some contracting on the side of a full-time job in 2018 and it was hard -- confusion and burnout set in quickly. But still think about the offshoring approach sometimes...

There's any number of stories about it being done, but anyone who has tried working this way realizes that you either need very skilled underlings or you spend most of the time managing it anyway and could have done it yourself quicker.

Hey, I'm looking for remote work. Lets talk? Email in the profile.

How does this impact taxes and other liability overhead on your part?

Yup. We did a couple phone screens. Person we interviewed knocked it out of the park. Solid answers with historical background on it, we liked the way they solved problems. Most of the team was part of the interview process.

The new person started and we realized the person that came in had no more information on what/how we had talked about in the interviews than we did. What was done easily over a screen share was well outside of what this person was capable of. In the interview, solid yanking/putting in VIM - in person, unable to save/close, for example. He was present -- we did wonder if the recruiter we worked with had a paperwork mistake, so asked a few bits about our interview and while he could parrot bits of the answer and our discussion, he could not come close to actually hitting the why or next steps like he did in the interview.

That was the point where a camera was required for the interview, for us.

>interviewed one person and had a different one show up

I'm always amazed at the extent people go to to cheat. It gets to the point that you're putting in more effort than you would have to be successful without cheating.

I sometimes feel some people simply enjoy cheating too much. Its not about accomplishing a goal by any means necessary, its something about doing something wrong that certain people seem to enjoy.

In the pandemic when hiring was remote this was very prevalent in my country.

This became kind of a business where a person sells his services to help you cheat by appearing instead of you in the video interview and also helps you with the work after you got the job.

- Modern problems require modern solutions. This sounds creative and ballsy. I'd be intrigued to see more.

- If the answers are so basic and guessable that one can both find and then deal with a printer to commit it to slow-RAM physical legacy memory, then the problem is not with the candidate

- People with such little self awareness that fail to perceive this problem have demonstrated why you don't want to work with them. The problem is not the fact that they were hotfixing answers straight to prod

- This was just a reverse racial discrimination security test. By identifying the non-authenticated user, you both demonstrated exemplary awareness of identity phishing attacks as well as a base standard that 2 people aren't the same based on their protected identity criteria

What I don't understand is... these are pre-screening interviews at companies that get tons of applicants, right?

What happens during the actual in-person interview that follows ??

A lot of 'on sites' were remote during the pandemic, and a lot of roles now are remote, so may not have an 'on site' at all.

All the big tech companies I’ve spoke with over the past several months were still doing virtual “on site” interview loops.

Not even at the local branches ??

Many companies aren’t ginormous with branches everywhere.

Google Meet is cheaper than a flight & hotel room.

And yet all the examples so far have been of "ginormous" companies, haven't they ?

I hired on during the pandemic for what would be considered a ‘ginormous’ organization. I never went on-site for about 2 years. I believe my employer tried to mitigate it through more extensive reference checks and a very long probationary period.