This sounds great but Google has been making it harder and harder to run your own mail server even for personal use. I think they would be happy of email servers were only run by a few large companies. They make it sound like they are doing the right thing but really they are bully the industry to do it their way. So many people have Gmail accounts that you can't run an email server that cannot send email to Google.
I've run my own email server for about 15 years. Every now and then I have to drop everything and implement some new technology that Gmail demands I have. Granted SPF, DMARC and TLS are all great technologies but I take issue with Google making the decision that everyone is going to switch, now and with out sufficient warning.
I'm in the same boat as you - running a personal/friends/family mail toaster for 16 years now. But I disagree that Google have made the administration harder. Internet mail servers don't live in isolation, and context has always been changing. I have no issue with adapting configuration as expectations change, so Gmail's evolution has never inhibited my ability to exchange email with the world. This move is no different. Really, I feel that the elephant has been tremendously careful not to tread on the mice.
I don't believe this particular change materially improves security, since it isn't representing an end-to-end behaviour, but if it raises awareness and challenges mass surveillance then so much the better.
Personally I would like them to advocate for other necessary infrastructure advances and include, for non-IPv6-capable MTAs, an icon of a lurching monster that refuses to die.
That's not right, I host some email servers for small companies and private people who all have problems with Google accounts although supporting DKIM, SPF and a TLS connection. They block them as bulk mail and in their FAQ the option hosting your own server isn't even listed. Of course contacting them is impossible, so the only way left is creating a ton of fake accounts on gmail and add the addresses to their contacts or ask friends with gmail accounts to add them and mark them as no spam. Google definitely IS the email bully and there's not much you can do about it.
Maybe they have a reputation system in which I've gained a good standing and you haven't. Have you considered that your attempts to game their system have actually been regarded as abuse?
All major email providers do similar things because this is exactly the setup you'd have if you were a spam operation. Email is just something that you shouldn't try and do yourself on a small scale.
How long before 'The world wide web is just something that you shouldn't try and do yourself on a small scale.'.
Really that's bothers me greatly. Email is the example of a successful open protocol and by all means you should try and do it yourself. The less we are reliant on these giant companies the better and to abdicate email to google just because they've achieved critical mass is to wipe 25 years of internet history under the carpet.
Decentralization is good, not bad.
If installing and operating a compliant email server is too hard then we should make that easier, not push to get everybody to switch over to some corporation.
It very much is something that should stay decentralized. I'm was shocked when I learned that google mail has 1 billion accounts. It's starting to break email as a non-proprietary service.
Email remains the poster child (given its age, perhaps that should be poster grandparent) for how a lightweight federated communications protocol can service needs from micro to mega.
For comparison: the sidelining of XMPP by corporate interests was awfully disappointing; the centralisation of social networking feeds is downright heartbreaking. But none of that should put us off trying and trying again. Everyone with an interest in how the Internet protocol stack works in context should try running their own mail service, it's a fantastic and low-cost way to gain insight.
Huh, I've had more problems with other blocklists, but never with google. It certainly can be a pain. We need something simple, maybe sort of like EV SSL certs, to allow more painless establishment of email sending reputation on new IPs and domains.
I too have my own email server for some years now (7 maybe?). Only had problems when I mis-configured it.
I'm frightened as well that google is taking over email. As long as people rely on web based email, there's no hope for actually encrypting email. TLS is nice, but really, it doesn't mean much. Your email is stored plain text on Google's servers anyways.
The article is pretty clear that gmail users can keep emailing others who don't support TLS or authentication; they will just now see an additional icon informing them of that condition. Nobody is being demanded to switch anything.
> For now they will just warn on it. In a few months they will send it to spam
As a recipient of all kinds of mail, I'm okay with that. I'd rather not see that unencrypted email than have it QUANTUM INSERT'd or whatever the Chinese equivalent is.
My comment has nothing to do with spammers, and everything to do with the possibility of unencrypted email I receive (spam or not) being nefariously tampered with while in transit (see QUANTUM INSERT)
No, best practices are to avoid using any of that and trusting anyone's servers. S/MIME if you trust the CA system, PGP if you don't. Supporting either of those would be a real step forward for privacy - but of course if Google can't snoop on your email then it's not helping them target their ads.
but of course if Google can't snoop on your email then it's not helping them offer you a high-quality email service for free by showing you (hopefully) relevant and unobtrusive ads
Well, except they are. Before, yes, they were just best practices. It was great if you had them but by no means required and didn't really impact your experience much if at all. With this switch, though, they became requirements to getting a "normal" experience in Gmail. Even having a warning is a degraded experience at this point.
I'd actually consider that an improved experience. Sure, the sender using the improperly created personal mailbox will be inconvenienced if people ask about it. However, the actual user of gmail benefits from this.
Now I can just get a free cert and turn on TLS. What's the problem, exactly?
Most people are not capable of running their own mail server. The convenience of services like Google, plus the risk of turning your mail box into a spam machine, vastly outweighs the downsides for most people.
> Most people are not capable of running their own mail server.
I think that is a big part of the problem. It should be relatively straightforward for someone who isn't a full-time email server administrator to setup a mail server correctly, but it's not. At least, it wasn't easy last time I tried it with Postfix and (iirc) Courier on Ubuntu. All the cryptography options are disabled by default and you have to spend a lot of time figuring out which ones should be turned on, where to stick the certificate files and how they should be formatted, how to get Courier and Postfix to talk to each other, etc...
Maybe there's an easy solution (besides "pay someone a monthly fee to manage this all for me") that I'm oblivious to, but it seemed like I was on a well-travelled path and it was a lot harder than it should have been.
If it helps, I recently rebuilt my mail server and changed from FreeBSD+qmail+Courier to Ubuntu+Postfix+Dovecot. In doing so, I used this series from Ars Technica:
It shows how to set up SPF, DKIM, TLS, anti-spam filtering, Sieve, certificate-based authentication (I still haven't figured out how to do this with an iPhone), and so on. The only bolt-on it references but doesn't explore and I actually used is the Z-Push package to implement ActiveSync.
Yes. I periodically test with various recipients and mail goes through without a hitch. The only difference I might have versus people starting out fresh is that the domains I host are relatively aged. The newest is two years old and the oldest is nineteen. I also made sure that DNS is set up properly, both forward and reverse, and especially for IPv6.
maybe my sin is hosting at linode - for a new recipient at either of the behemoths, I seem to have 50% chance of going to spam. Thanks for your comment.
I host with linode, and have had generally good results sending mail to gmail. They're definitely not the worst. Granted, I have had SPF enabled for about 10 years.
Most ISPs don't allow inbound (or, often, outbound) connections on port 25, so the number of people who can run mail servers at home is pretty limited. I think even Cox stopped allowing it a while ago, and they were among the most lenient of the big ISPs. (Comcast hasn't allowed it for a while.)
That, I think, is why the distribution managers don't make the default configurations a little more friendly/sane; most users aren't interested in installing a MTA to do anything but local delivery or smarthosting, and the people who are, are probably going to tweak the configs to death anyway, so it's not worth spending the time making it run well and securely straight out of the box.
It wouldn't be too hard to build a "mailserver in a box" with (say) Debian + Postfix + LetsEncrypt + BIND that you could stand up in an hour and be reasonably secure (and I'd be kinda surprised if that doesn't exist in some form already), but I don't know how many people would want it who aren't running mailservers already, and have the capability of doing so.
So run it on AWS. In general, I think running mail servers off a residential, or even business ISP is very sketchy. E-mail is my primary line of communication for important matters, and I can't afford to have a snowstorm, busted hard drive, orange juice spill, or basement flood take out my mail sever.
I don't think it's "sketchy" to have different requirements. For my personal needs, email is definitively important, but it doesn't need HA, and its store-and-forward architecture means it copes fine with an hour or six of downtime (serves usually retry for several days until giving up).
As with so many things in AWS, it's left up to the customer to inform AWS that a) you're running a mail server, b) what the purpose/use case is and c) request they configure the reverse lookup associated with the elastic IP you've allocated.
Source: I've been running public facing SMTP servers in EC2 for years with no issues.
Still, unless you're running a server for a lot of people and you have tons of free time, you'll discover that it's more expensive than paying any of a bunch of people to take care of email for you.
You're absolutely correct. The cost of my AWS deployment in service of my personal email, and less than 5 other people comes in at ~$5.06/mo. The time, however, is the real cost. If/when I gain significant users, Fastmail would be at the top of my list of companies to consider.
If you make hard things easy to do, then people that don't know what they are doing will do it (poorly). They'll set up a server once, and never maintain or update it and it'll eventually be hacked and taken over by spammers. I set up and maintain our mail servers and work, and for my personal email, I chose to point my domain to Gmail to let Google take care of it, I don't want to expend the effort it takes to run my own mail server.
I tried it on Ubuntu 14.04, but unfortunately you get only pretty old versions, even via PPA. And no matter what I did, opensmtpd crashed each time it received a mail from localhost to a local receiver.
I'm having high hopes for Ubuntu 16.04 having a more recent version on board. Then I will try to switch from postfix again. Because as you've said, it's really straightforward to set up :)
Long before Let's Encrypt, SMTP transactions with STARTTLS have permitted self-signed and non-root-CA chained certificates.
The pervasiveness of self-signed certificates for SMTP servers means that rejecting them would drop large amounts of email. STARTTLS is basically useful for thwarting passive collection of network traffic.
Gmail's new rules on unencrypted e-mail don't support self-signed certificates though - you have to use an offcial CA-issued certificate from one of Google's approved CAs.
That doesn't bring a lot of extra security though, because there's no name verification. I can get a valid letsencrypt cert on anyrandomdomain.com, and if I can hijack your MX and point at it, it's "valid".
I don't need to control your domain. If I control my own domain, which could be any throwaway domain I just purchased, I can get an SSL certificate on it.
And if I can point your MX records there, via hijack or any other means, then I have a valid SSL certificate for receiving your email.
Yes, they recently enabled the DNS validation. Otherwise, it wants to use a webserver to validate ownership. It can spin up an embedded webserver if you don't have one already.
I would say the opposite: There is no excuse for cementing the role of CAs for SMTP. DANE doesn't need any CA, and there is no problem with legacy clients that require CA-signed certs.
It's been a long time since the cost of a cert was an inhibiting factor - at least for 1st world businesses. (I don't suppose I'd want to be paying for a US dollar priced TLS cert in Zambian kwacha or trying to get a petty cash reimbursement for one on a typical IT salary in South Sudan...)
My first concern is, if nowadays is so easy to get your certs signed by a CA how can "Authenticated/Encrypted" emails successfully prevent phishing attacks? In my modest opinion there are many week points in X.509 and how CAs are verifying identities, and even if this things were fixed you still have the problem of state-sponsored attacks that have no solution within the current www pki. I personally have no problem if Google wants to add some icon in the UI, but I share OP's concerns.
The use of TLS (which is what uses a CA-issued certificate) isn't to prevent phishing attacks, it's to prevent emails being read or modified in transit.
DKIM (which does not use a CA-issued certificate, it uses a public key published in DNS) is the technology that's intended to authenticate the email sender. It still wouldn't stop phishing attacks where the purported email sender is something like "admin@facebook-account-verification-2016.net" though, and I don't know that there really is a good technical solution to that sort of thing.
As a publicly traded company, it optimizes for GOOG. I have nothing against google, but I do think there is not enough skepticism our fear about them eating the whole stack.
* Receive gmail link from friend.
* Use Chrome Browser as gateway to internet.
* Use DNS to resolve that URL.
* Site built on Angular and has new SPDY tags.
* Libraries and Fonts served from CDN.
see something unclear at the url.
* use google search to find that link
* google analytics beacons on both websites report your traffic.
I am concerned about the temptation for google to get up to something naughty. Luckily, as we now, it is an undisputed fact that our data is 100% safe with google and there are no organizations trying to actively attack their network or intercede. We also know, that compromising an individual (either an employee or a customer of google) is 100% impossible, never has happened, and never conceivably could happen. So it isn't a big deal.
So, is the issue you can't run a mailserver? Maybe. Maybe not.
You can't crawl many valuable sites now, unless you are google bot or a big vendor.
2 (3 if we count microsoft) companies can ban you from competing on mobile.
etc.
The slight losses in some areas to optimize for what google wants are small. In aggregate, they are wildly irresponsible for us to accept and we shoudlnt.
I meant that the company does what is in its own best interest. It does publicly trade stock with voting rights as GOOGL which has a 1:1 share to vote. It does not allow the class B stock to trade publicly which is 1:10.
I meant, and you correctly pointed out was not clear or a particularly good way to verbalize, that google like most companies does what is best for itself. In googles case this is a combination of maximizing shareholder value and the vision various thoughtleaders in the company have.
I do not believe google is evil or bad.
I used to believe this.
I now believe that google is doing a lot of great things. However by achieving horizontal success in almost every area of such a powerful concept, it effects how the landscape develops and does cause many issues.
Even if we accept google is mostly good, and I mostly do, it is concerning. I hope that their voluntary meta structure will allow them to distribute fault tolerance and limit power consolidation.
Couldn't agree more. I run a number of my own services on my own servers but email is not one of them. I don't have the time or energy to keep up with it and it's imperative that I receive all emails sent to me and all my sent emails are received. Google apps does that perfectly for me and I don't have to worry about it.
As far as "I take issue with Google making the decision that everyone is going to switch, now and with out sufficient warning." it would be nice to get some notice on these changes no doubt but email is shit. It just is but we are stuck with it and at least google is attempting to drag it forward even if in doing so they leave some outdated servers/services behind. I'll take more secure of infinitely backwards compatible any day.
To play devil's advocate, why is email shit? It is pretty much a way to send someone a document in the original way Berners Lee defined html which was essentially sgml. I get not letting people mail active scripts, but every large mail provider also makes a browser.
So email is shitty because providers fuck with it at the mailserver level (i think this is the case, and have heard that about providers) as it doesn't seem to be a problem at the view level.
So this is sort of the issue. We couldn't get consensus on this early on, so no one was able to force standardization, now only a few players have the ability to define standards but they all own large scale messaging channels outside of email.
edit: meant to say that browsers obviously view html and css3 is pretty good as well. We could probably standardize this enough to allow people to write emails in this format and embed imagery. We would not allow JS for obvious reasons, but sending cool emails would be fun and we could also add new features. I mean, the web has developed somewhat shittily, but adding a few new features outside of like the 16 original tags would be cool. Not sure what percentage of tags and attributes actually work, but then again, no one is...
What does that mean? A mail server can be a small perl script running on an embedded computer, or it can be a hundred front-end SMTP servers using a database as a back end datastore, there's no real infrastructure limit other than being able to make and receive connections on TCP port 25.
And as to "sufficient warning," sure, I can see that it would be nice to have given people like you more lead time. But then again, "GMail has always supported encryption in transit using TLS," and if you care about running your own email server, it feels to me like the writing has been on the wall for that one for a long time.
This is not entirely true. Gmail (and other big email providers) blacklist blocks of IP addresses belonging to hosting providers who are perhaps not as proactive as they should be at detecting and removing spammers. If you unwittingly sign up at one of these providers, it doesn't matter how perfect your mail configuration is: anything you send to a gmail address is gonna go to spam. Gmail of course doesn't publish these lists, so the only way to find out about them is to run into one.
I understand gmail's problem, and I don't have a better solution, but it's not just about insecure mail servers.
I disagree - I run a personal mail server and dkim+spf just took a little bit of research to set up, so I'm very glad this brought them to my attention because I want to do what I can to make my emails trustworthy.
If there are any potential downsides to these things I'm interested to hear them, but it seems like they're generally agreed good practices, and hosting your own mail means you're signing up for some ongoing learning and maintenance anyway.
Maybe I'm in an odd position, but is email really such a big communication tool? I mean sure in business world, but for that companies have their own email servers anyway. I only receive email from 2FA-things and order confirmations/paypal recipes. I hardly ever actually send anything from Gmail or receive anything from actual people. All my communication with actual people is either through apps like Signal or WhatsApp or through IM like IRC
Main complain to Google or other big company. Stop putting email in spam that doesn't have reverse DNS set. Why should one IP be tight to one email domain?
Well, it would be much worse if they refused to implement those enhancements, or tried to force proprietary or gmail-specific alternatives.
They pretty much had to do something to improve e-mail's abysmal confidentiality and authenticity, and at least they used open technologies which can be configured on a Debian VPS in a couple of hours using a HOWTO.
I think as HNers, we're focusing too heavily on niche cases (like running your own email server). But, for general public who is still sharing their own (and more importantly, their clients') SSNs, passwords, and other very sensitive information on email, this may be the trigger that educates / trains them to be more careful. I am definitely looking at this as a positive.
Like why would I need to give my SSN to register for an account to take the GRE exam? And why would anyone ever make SSN an optional field? If its not required why would you ever ask for it?
> And why would anyone ever make SSN an optional field?
One reason might be that some states have laws on the books that prohibit requiring people to give you their SSN (unless you are actually required to collect it by some other law, of course).
> If its not required why would you ever ask for it?
What about "Your message was categorized fir Bayesian spam filtering and may have contributed to eventual upstream rules" for all those installations that historically ran SpamAssassin?
If you're using Gmail or sending to a gmail address[1], you know what you are in for, and if you don't you should at least know that anything you send to someone else is no longer in your control and you have very little control over who sees it.
1: Google Apps for business accounts are not scanned for ads.
What alternatives do you suggest, besides running your own mail server? I'm in the invite list for Protonmail and have also used the infamous cock.li for informal stuff, but I was looking for something a bit more established, that I can count on long term stability. It's bad enough to switch email addresses once, to be switching every time a service goes kaput is unacceptable.
What do I suggest in lieu of Gmail or some other large provider that may scan your email? If it really matters to you, your only choice is to run your own mail server. If you don't control your endpoint, then I think no matter what you profess, you don't really care. Personally I just use a combination of Gmail (because I don't care) and a POP/IMAP account at my local ISP (which is not free).
If you want free email, expect to pay in some other way. There's no such thing as a free lunch.
Obviously every third-party service is trusted, that doesn't mean one "doesn't care". By your logic even a private mail server isn't enough, you'd have to use PGP. If you don't have any recommendations just say so.
> Obviously every third-party service is trusted, that doesn't mean one "doesn't care".
No, I'm serious about this. I wasn't trying to be flippant. If you care enough about the integrity of your email content and it not being used to further a company's profit, the only way to be sure of that, to the extent that you can (which may not be much), is to run your own mail server. If that seems like it's way too much trouble, I think a you should take a close look at your motives for wanting a gmail alternative. Is it about the integrity of your email, or sticking it to Google? If it's avoiding Google because they specifically cause you concern, that's fine, and there likely plenty of choices, but I'm not sure what they are (as I said, I just use Gmail because I don't care).
> By your logic even a private mail server isn't enough, you'd have to use PGP.
Well, by my logic you have to do enough to make yourself comfortable. Depending on your reasons for avoiding some other companies that will be different things.
> If you don't have any recommendations just say so.
I don't have any recommendations for Gmail if you consider a good UI, responsively web based, and free as major components of that. If you are willing to give up one or more of those, there are options. The local ISP I mentioned is Sonic.net. By all accounts (including mine, I've worked there multiple times in the past), a great company, and with great EFF ratings. An email account there is not free though.
They don't appear to be scanned for Ads, but talk with the Google Marketing team and they will be able to tell you what domains are sending to your competitors if they use Google Apps.
The only source I have is a call with them when they told us they could do this. Down vote me all you like, I was shocked when I heard this as well. They suggested this to us, not something we would have ever asked for. Nobody else has ever had this experience?
You said 'using' or 'sending to' which imply that by my action I somehow consented to be tracked by using email as it was designed to be used. But there is also received from, i.e. I receive a message from a gmail user and now Google associates my address with some advertising keywords.
More worrisome, "Your message has been added to your permanent record at wholesale data storage and may be used against you, in perpetuity, by current and/or future regimes, partner corporations and other select criminal organizations (tax-funded or independent) for reasons including but not limited to financial or political gain, manipulation, incrimination, assassination and personal entertainment."
Which is different from any other email provider in any substantial way (including your local ISP or personal server) because... they have a reasonable hope for reliable storage?
Seriously: this is calling out Google in a way that's comical since it's equally applicable to your own computer.
> Which is different from any other email provider in any substantial way (including your local ISP or personal server) because...
Single point of failure.
Government entities have to go through physical work to seize multiple mail servers distributed geographically. This keeps the cost of fishing expeditions high enough that they won't just do it by default.
With everything at Gmail, Yahoo, and Microsoft, you only need to serve 3 entities, who already are known to roll over.
However, far more concerning to the HN crowd should be this fact: Do you want the companies most likely to buy you out for a large value to be the ones holding all of your internal emails?
At this point, Google probably knows more about the quality of business than the businesses do. It would be an interesting question as to whether Google could be considered a corporate insider for a vast number of companies.
> Government entities have to go through physical work to seize multiple mail servers distributed geographically. This keeps the cost of fishing expeditions high enough that they won't just do it by default.
Except they seem pretty seize-happy and the only thing protecting your house is the say-so of a judge.
> With everything at Gmail, Yahoo, and Microsoft, you only need to serve 3 entities, who already are known to roll over.
That seems quite unfair, as at least 2 have been very public about their expenditures to try and make such attacks impossible in the future, and have vocally fought subpoenas.
> Do you want the companies most likely to buy you out for a large value to be the ones holding all of your internal emails?
Yes. The lawsuit should I discover it would probably make me richer and more famous than all the buyout events I've experienced.
> Except they seem pretty seize-happy and the only thing protecting your house is the say-so of a judge.
You're being obtuse. Even if every judge rolls over, if you have to seize multiple email servers in multiple jurisdictions, the paperwork represents expense and time that law enforcement simply will not do unless they have a really strong reason. "People are lazy" is the universal constant. We fear computerization of things precisely because computers aren't lazy.
> That seems quite unfair, as at least 2 have been very public about their expenditures to try and make such attacks impossible in the future, and have vocally fought subpoenas.
That's what they say publicly. However, if they roll over for governments like China, they're going to roll over for the US who can genuinely affect their revenue stream.
> The lawsuit should I discover it would probably make me richer and more famous than all the buyout events I've experienced.
Your naivete is touching. Google wouldn't do anything actionable. They scan your email store and know not to invest. You'll never prove anything for a passive non-action like this.
Even for positive action failures, it's very difficult to prove. This is the whole point of "parallel construction". You dragnet to find something incriminating, and then build the legal path to what you now know to search for.
Your trivial additional inconvenience running what sounds like a non-trivial geographically dispersed non-cloud-service email system warrants not calling out services with poor mail transit security. So millions of customers improved security vs you figuring out how to use LetsEncrypt. Because Google subsidizes the free service with ads.
I do not follow this logic, but what's more:
> Google wouldn't do anything actionable. They scan your email store and know not to invest. You'll never prove anything for a passive non-action like this."
Yeah well having sold a few companies to a few mega-nationals, we try to be honest and deserve the acquisition, as opposed to trying to fleece people. Lame-duck acquisitions shit on employees for investor gain, often for investment clawback and exit.
But also, if you are a paying edu or org customer, they stop scanning for and serving ads.
So forgive me if I don't feel a ton of empathy towards your strong desire to be dishonest in a hypothetical google acquisition where they hypothetically do this.
> You can use full disk encryption on your own server. Not every country has laws that force you to forfeit your crypto passwords
It won't count for anything if the emails your server is sending/receiving are not encrypted, which exactly is what Google is advocating. I don't understand GP's smug rejoinder, as if encrypting emails in transit is a bad thing.
Well, no, it really isn't. None of that really occurs on a personal server, and local ISPs probably don't do wholesale data storage because that's not their business, but it is Google's.
Well, you have no reliable way to assert your personal mail server is not bugged and observed right now.
You're one judge's pen-stroke away from having personal property seized and then it's just a matter of how real your machine's physical security measures are.
> Well, you have no reliable way to assert your personal mail server is not bugged and observed right now.
Quite true. If you, personally, are a target of the NSA, you are totally fucked. We know that they will make up evidence if they cannot find some.
However, we put locks on our doors even though most of them can be picked very easily. Why?
Security best practice is "defense in depth". You defend at each level to make it more expensive for an attacker. The goal is to make attacks against your stuff more expensive.
If you make the government have to dispatch someone physically, there is a vast amount more friction. There was an inspector from Scotland Yard who once commented that "If your drives are encrypted such that it takes us more than 40 hours of work, your drives are not going to convict you. We will spend our time on gathering other evidence." Physically seizing a server is annoying paperwork.
So, the goal is to prevent them from being able to dragnet low-level offenses for cheap. If you're a murder suspect and they have good reason to come after you, then they're coming after you irrespective of the cost.
You have no reason to suspect it either. Yes, in this hypothetical scenario your mail server or your ISP's server might be compromised, and yes, in this scenario, there wouldn't be a big difference. But we don't live in hypothetical scenarios. Paranoia is great and all, but let's be real. An average Joe's personal mail server is more than likely not bugged nor observed, and if communication happens in TLS, then there'd be no reason to suspect the delivery either.
What, precisely, do we "know" Google/Microsoft/FUDCo is doing? Certainly not willingly collaborating with every quasi-legal search and seizure presented to them.
Do you even know what you originally replied to? Because it's right there. We already know Google stores a great big swath of data about you for various purposes, including advertising. We know that one of the many sources of this data is what is gleamed from scanning emails for advertisements. Don't act like it's not common knowledge.
Yes - it is common knowledge that Google (and others) use email for advertising.
But you (or perhaps upthread) are implying that Google willingly hands over data to government authorities.
The evidence would point to the contrary: Google (and Microsoft I might add) are complying with the law, but are not simply rolling over and handing out whatever is asked of them.
Their ability to fight back against unwarranted requests is probably much better than someone running their own mail server in their basement.
I doubt that there is a way to build a webmailer without processing the emails content at some point. And as it is processed anyway; using it to adjust your ads doesn't appear to me as something significant.
>I doubt that there is a way to build a webmailer without processing the emails content at some point.
It is disingenuous and/or ignorant to suggest that temporarily loading an email into memory for the purpose of displaying it on the users screen is the same as parsing and catagorising the text and storing the results of the analysis in a database for the purpose of manipulating the user.
> It is disingenuous and/or ignorant to suggest that temporarily loading an email into memory for the purpose of displaying it on the users screen [...]
I may be wrong, but I think they were referring to things like spam detection, malware detection, possibly search indexing, and such rather than just "temporarily loading an into memory for the purpose of displaying it."
How do you run an efficient server-side search without maintaining a parsed index? I have 40,000 emails in my inbox. You want this to be linearly scanned everytime?
You could do everything client side as well and just store encrypted indices on the server. Probably not a good idea, but not the end of the world.
I don't think users actually care though. Google's attempt to redefine "end-to-end encryption" to include decrypting something mid route bothered me. This... meh.
There's a lot of things that I question about Google, but forcing their Gmail customers to adopt more secure practices is worthy of praise. They are the biggest free email provider in the world, and they are owning up to a responsibility to ensure their users can work safely.
Those running their own email servers have a similiar responsibility to their own users, even if it's only themselves. You had time to set up the server in the first place, so you have time to make it work with TLS. Now that Let's Encrypt is here there's no excuse to be running an insecure email (or web) server.
> Not all affected email will necessarily be dangerous.
To me, it sounds like they're saying that most of the "affected email" WILL be dangerous -- just not ALL of it -- and that's highly misleading, of course.
Overall, though, I think this will be a good thing if it pushes more organizations ("mail senders") to implement opportunistic encryption for incoming mail and SPF/DKIM signing for outgoing mail.
That seems to be what they're referring to; that is, sending to an MX host that doesn't support opportunistic encryption) and/or receiving mail that doesn't have a valid DKIM signature. Did anyone else understand this differently?
This will be popcorn time for those using Gmail for business.
I have a plugin in Thunderbird that shows the DKIM status of incoming email as gray/red/green, meaning no DKIM/DKIM invalid/DKIM valid. Most of the email I get from business accounts (usually on Exchange), including those from the company I work at, have no DKIM. (Yes, I've complained to the ITsec dept. They don't even have SPF set up...) Of the rest, surprisingly many have invalid DKIM sigs.
>If you receive a message that can’t be authenticated, you’ll see a question mark in place of the sender’s profile photo, corporate logo, or avatar.
This makes it sound like I (the sender) can set the image displayed if I am using DKIM. Is that the case? Or is it only if I have DKIM and have a Google account with that email?
Gmail uses an associated Google+ profile for authenticated emails, so you need both for it to work going forward, I presume. To get started, check https://www.google.com/business/
There really should be some kind of standard or mail header though. :) Come to think of it, services could support a vcard mime-type attachment, perhaps? Except that likely wouldn't support URLs to profile photos... Maybe we've identified a missing feature of DKIM? ;-)
Gmail will prefer the image you include in the sender's contact info, if you manage your own address book, and it only falls back on Google profile pictures if you don't have one (which is the common case).
I use claws on some machines (it has a different set of annoyances than Thunderbird) and it has the feature to pull senders' images from https://www.libravatar.org/ . The occasional mailing list participant gets an image that way and, amusingly, the occasional spammer as well.
Interesting. I'm wondering if they also warn Gmail users if a mailserver has TLS enabled with a self-signed certificate. Because i think many mailservers actually support TLS but do ignore certificate verification, because of the self-signed certificates.
Unfortunately, this is the sort of a change that's a red herring for any actual improvements to email security.
The use of unencrypted or encrypted link to the receiving email provider's MX server doesn't change all that much in terms of who can read the email: it's still sitting in plaintext on the recipient's server (as well as the sender's server), and the group of actors who can sniff traffic on the backbone like that is probably just as easily able to get it from the servers.
The authentication feature is even worse. The problem of spam and phishing isn't that email claims to be from important-service@bigbank.com, it's that email claims to be from "Big Bank" <whoisthis@some.really.shady.ru>. It's been noted before that spammers tend to be the most aggressive at uptaking new "anti-spam" technologies like SPF and DKIM, and this sort of validation feature seems like a prime vehicle for exploitation by spammers.
You're arguing that we shouldn't do anything, instead of taking a step in the right direction. Email is an old ecosystem, so it's not possible to make big improvements all at once.
I'm not arguing that we shouldn't do anything. I'm arguing that the presentation of the data is at best meaningless and at worst downright harmful.
DKIM authentication is in no way an attestation that it was sent by its sender. Furthermore, from what I can tell, more damage is caused by spoofing that works on the "I use a very similar name which is hard to see the difference" level (e.g., animenewsnetwork.com versus animenewssnetwork.com). Finding and testing solutions to that problem is something that doesn't require changing or deploying anything to new to the ecosystem, and it would arguably bring more benefit than deploying end-to-end encryption.
> I'm not arguing that we shouldn't do anything. I'm arguing that the presentation of the data is at best meaningless and at worst downright harmful.
What do you think should be done to make progress that's better than Google's proposal? Personally I think encouraging all senders to adopt DKIM, transport layer TLS, DMARC, SPF, etc. by displaying auth results from those protocols in the UI is a good first step. It's similar to the push for HTTPS on the web.
> DKIM authentication is in no way an attestation that it was sent by its sender.
Google didn't mention DKIM in the blog post we're discussing. Are you referring to a related effort? The blog post was about encryption in transit with TLS.
> I'm arguing that the presentation of the data is at best meaningless and at worst downright harmful.
Google's DKIM solution is as close as one can reasonably get given modern technology. If I send a DKIM-signed email from example.com that passes validation, then it means the following is true: I sent an email through a server managed by the domain owner, that had access to the DKIM key and chose to sign my email. It's not authenticating a person, but it's authenticating that the domain's mail servers sent the message. The attestation is at the domain level, not the sender level. This attestation is still useful though: if the domain owner did not want to allow you to send email from jcranmer@example.com, then it would not accept that email from you or DKIM sign it.
It's a good step, but a very small step. It's been a long time since the Snowden leaks. You're pretty much naked to state surveillance without:
a) An open source client that encrypts your mail
b) A public key resource with a web-of-trust
All the major Web portals/ecosystems could lock the spooks out of email payloads very effectively. That would still leave other issues to solve, but securing payload in flight and at rest seems like the first real milestone that makes a difference.
There's a lot of supposition in your post and no evidence in favor. First of all what makes you think Gmail stores user data in plain text at rest? Secondly think about your definition of "backbone" networks. Google's own network is extremely large in scope, and generally user traffic lands in Google's network at a point very close to the user. For example if a user in Japan sends mail to Google, that connection will be terminated in Japan, and backhauled across Google's network to wherever the actual servers are located. Google intentionally shrinks the scope of network surveillance by minimizing the network distance between the user and Google's frontends.
> The use of unencrypted or encrypted link to the receiving email provider's MX server doesn't change all that much in terms of who can read the email: it's still sitting in plaintext on the recipient's server (as well as the sender's server), and the group of actors who can sniff traffic on the backbone like that is probably just as easily able to get it from the servers.
That's not at all obvious to me. And this sort of backbone sniffing is exactly what we've seen state actors do. What possible downside could rewarding link integrity have on the story for user privacy? A false sense of security? Quite the opposite: that's what we already got in spades.
A state actor who can sniff traffic can just as easily serve a secret order to the email server operator to compel them to hand over the data. In fact this is already SOP for governments when dealing with communications which are encrypted in-flight but not end-to-end.
Except that's demonstrably not what the NSA was doing; they were sniffing the backbones at the switches, presumably because they didn't want to create any sort of paper trail or get warrants, even secret FISA ones, against particular people.
From Google's perspective, it means they have to be involved, however non-consensually, in the process, rather than having it done to them transparently. So I can see why it's a win for them.
Whether it really matters to Joe Random User is arguable. What Joe User really ought to demand is end-to-end encryption rather than transit encryption, but he's unlikely to do that because key management across multiple devices (and who wants to read their email only on a single device?) is a pain and the companies that would be able to make it less of a pain aren't exactly incentivized to do so.
Good initiative. Question is; what does the TLS icon indicate; is it just opportunistic TLS, or do they do any verification? What, if so? Some private consortium where only members can get a "green lock"?
DANE comes with its own complications: it means you need DNSSEC, and that's a bit of a pain to set up at the best of times, especially with there being no way to currently automating DS/DNSKEY record updates to maintain the trust chain.
There have been some mutterings in the IETF ProvReg WG on ways of allowing registrants some ability to automate the process, but it's still early days.
This is what GMail is doing: making sure that sending an email is like sending a letter (instead of a postcard). TLS email is not 100% secure (private, authenticated, etc), but not 100% insecure either.
Verifying TLS for email is an easy step in making email a bit less insecure, and it requires no intervention from users. If you need something secure, then yes, go for GnuPG or other forms of end-to-end encryption (if only I could name a few).
Yes, TLS protects the information in transit but it does not prevent a malicous intermediary from reading or altering your mail. Sort of like an envelope could be opened at the post office, read, and then resealed before you get it.
Which kind of medium do you have in mind? If you want a federated communication medium in wide use that doesn't require you to use proprietary software or a centralized server, there isn't much choice. This is why I'd say it's a worthwhile endeavor to make it possible to communicate securely and privately with email.
I wholly agree regarding email hosted in the United States, especially for any emails stored over 180 days on servers you don't own (see Electronic Communications Privacy Act of 1986).
+1 for fastmail, here, too. Amazing service, really good communication during rare downtime, contributes heavily to open-source/community, decent prices, can heavily customize filters/etc, and as far as I'm aware, probably the most mainstream email provider that won't give into the NSA.
I'm sure that Australian Federal Police and Victoria law enforcement would be able to exercise search warrants on Fastmail's servers if they needed to.
Since you're in Tennessee (and thus a U.S. Person), you're actually ineligible for collection under FAA 702. Gmail/Hotmail/Yahoo, etc. would actually be the safest place for your information.
Of course that assumes that you believe the NSA follows U.S. law. If you don't have that assumption, that also implies that the NSA wouldn't respect Australian sovereignty enough to keep it from infiltrating Fastmail's servers.
I'm saying this to illustrate that there's no silver bullet for secure transmission and storage of information.
> I'm sure that Australian Federal Police and Victoria law enforcement would be able to exercise search warrants on Fastmail's servers if they needed to.
Nobody is arguing against search warrants. People are concerned about warrantless searches.
I like Fastmail (paying customer here) but let's not kid ourselves that they are anymore safe from the NSA or the Five Eyes. That's wishful thinking and is the same argument as if you host your email in Switzerland, it's magically untouchable from mass surveillance.
We've played with it in the past - it was on beta for a while - but too much legit email would have been marked as invalid. We're not big enough to force something like this through. Google is :) Good for them. I'm 100% in agreement with them on this work, and we'll definitely be doing something similar.
Google's not particularly marking them invalid though?
The not-being-authenticated indicator is just replacing any avatar with a question mark ("we're not sure this person is this person") and a tiny indicator that warns you that outgoing mail is sent in the clear.
As long as you're not sending mails to spam for this (which Google doesn't seem to be doing, unless of course a DMARC policy tells them to - I imagine) then I don't see the issue?
meh. This doesn't seem very interesting.
What would be really interesting is gmail support for public key encryption. They're perfectly positioned to roll out a user-friendly key management system.
Webmail struggles to offer security reassurances when utilising public key encryption. For example if Google allowed you to upload your private key and email would be transparently decrypted, that would be a great user experience, but now you have no way of knowing if the NSA/GCHQ/etc forced Google to give them your private key.
If you didn't upload your private key to Google now you need your browser to decrypt your message in-line but preventing the website with the encrypted version from stealing the resulting decrypted version from the secure container. So that requires every browser to add these secure containers, for content to be marked for decryption, and for key storage.
If you aren't worried about usability then you could just write a Word document, encrypt it and attach it to an unencrypted email. No browser support, or website support needed.
I don't know what they might do in the future to encourage people to use this, or if they feel that there's a point at which it would be sensible or useful to actively promote it.
It's still not ready for production use. One problem with e2e is that it's JavaScript based and runs in he browser, so there is a certain attack vector present there. To defend against this, ideally e2e needs to work with a smartcard (such as the yubikey neo) so that the private key cannot be stolen.
There was an issue I was tracking a while back to integrate this support, but it's still a work in progress.
It doesn't even need the hardware part in most cases. OS keyrings support pkcs11 interface with signing exposed. That means you can just send data to be encrypted for example by gnome-keyring and the browser never sees the actual key.
In the short run, they're simply a smaller and more diverse attack surface. There's pretty valid "security through obscurity" reasoning to apply here (though that is, of course, not a train of thought one should bet the whole farm upon).
More importantly, end-to-end encryption means that your desktop/laptop/tablet/phone/TI-83 is doing the crypto, especially signatures.
On a somewhat related note, I remember reading some documentation on a one-time password scheme, which I can't find anymore. It briefly mentioned something about using DES calculators to handle crypto signatures (for what reason i cannot remember). For our purposes, a PGP/GPG hand-held device would be neat, though perhaps cumbersome to actually user.
I don't think that phones are secure, but people do use them for things like 2-factor authentication. I suppose that it is the form-factor that is more relevant, since most phones are smaller than a TI-83, so much so that people are willing to carry them around everywhere.
Ultimately, what I envision would be a cross between a Blackberry (size, QWERTY keyboard) and the TI-83 (focus on being a simpler, lower-end computer). Heck, one may as well give it a unix-like OS and leave out wireless communications and sophisticated graphics. I think this is feasible, but the market would be quite small.
> Yep surprised they haven't rolled one out yet. Ideally placed.
In fact, they will fight tooth and nail against it since their business model relies on having access to the plain text of the message. It's the very reason they are releasing this technology: a user sees a padlock icon that's confusing similar to other "encrypted" mail offerings, for example OpenPGP. For Google, it's essential that end-to-end encryption does not become the norm and that plain text emails are kept inside their walled garden.
That being said, as a form of opportunistic encryption, TLS on the SMTP layer is great and there are really no excuses not to use it in 2016. Google could push it aggressively and add an (user optional) delay against servers that try to deliver email over a plain connection: "472 Try again in 30 sec or use STARTTLS"
With encrypted indexes maintained on the client end. Web search is a complex problem, but searching a single user's mailbox is child's play, you don't need Google for that. Client end processing could work for advertising too, but at that point you have an information leak that severely weakens security, the client will ask for adds relevant to terms found in the message.
For anyone wishing to run their own mail server, check out Mail-in-a-Box[0]. It's relatively easy to set up, comes with a nice Web GUI and they recently added support for automatic provisioning of TLS certificates with Let's Encrypt.
I'll tell you what I want: I want Google to help identify non-DKIM-compliant forwarders. As the operator of (yes, I know) a vanity e-mail domain with DKIM, SPF, and DMARC records, I have no problem sending mail to gmail directly; in fact, my outgoing mail uses postmark, so I'm not even directly responsible for my sending reputation.
BUT! It drives me crazy that many of my recipients get e-mail at hosts (schools, mostly) that forward the e-mail
with differences (encoding changes, subject changes, etc.) that invalidate the DKIM signature. Since they're forwards, the SPF check is going to fail, too, so the end result is that google shoves it into a spam folder.
I claim that google definitely has the data to be able to identify these bad forwarders--heck, even mail sent from gmail to these hosts will presumably fail DKIM checks on the way back into google--and I'd love to see them contact these domains, or even publish a list of known bad forwarders, so that I can push them to make changes.
I'm glad that companies are starting to display warnings about insecurity. It used to be that the insecurity was only advertised if something that was supposed to be secure was broken. However not having any security in the first place is often worse. I would like to see this trend continue and start warning everyone about systems that aren't secure.
I think that's the point---they should assume the worst.
It's part of a general trend of moving the needle from "Internet services are insecure and if you really want to send something secure, you should be sure your channel is encrypted" to "These channels should always be secure; if they're not, here's a big red flag to warn you that they are not."
See also the process of bypassing the "This site is insecure" alarm interstitials in Chrome and Firefox these days for sites with bad secure TLS credentials. The frog has been boiling from "We warn the user with a tiny icon they'll probably ignore" to "Users have to know secret words or convoluted config flows to bypass this inescapable error panel."
I wish someone made a better (non-SMTP) messaging standard which is secure, efficient, easier to understand/implement and without a clutter of all legacy stuff (and hopefully with a decent reference implementation!). Spams would still remain, but at least that would free us from worrying about the transportation layer security. (But then we might need to reinvent something like DNS on the way, so maybe that's why that no one is trying this.)
How disappointing, when I read the headline I thought Google was supporting PGP signing and encryption of emails. They could easily do so in their web and mobile clients, keeping emails safe from prying eyes.
Though that would also prevent analysis of emails for ad targeting, so they'd have to do it as some sort of paid project.
Its great that Google wants all the lines carrying data from their servers to be secure and tamper proof. It would be interesting to see if they ever support end to end encryption which would lock them out of scanning the data as well.
Then how would searching your mail work? That breaks the product on a fundamental level and makes it worse than all competing products for all but a few users with specific needs. The existing end-to-end browser extension is a reasonable compromise.
It would work in the same way as opening any encrypted mailbox works. Your password is used as the key to decrypt your mailbox when you login. The same key could also be used to search the email.
Right now they're already telling you that they scan every email. So yeah, you would have to trust them that they do in-fact discard they keys and don't allow decryption in other scenarios. But large companies with tons of cash to lose if they're sued, will rarely blatantly lie about what they're doing.
Want to run your own mail server with TLS, SPF and DKIM? Check out sovereign on github. Makes it easier and includes a lot of other useful stuff such as your own calendar and contacts server.
question about this: if i send mail to my IMAP or POP server over TLS, it may still travel to various spots on the journey to its final destination unencrypted using SMTP, right?
It's what the email server that you are making the IMAP or POP connection to does next that matters. If you send mail to a Gmail account it should make a TLS connection to Google's servers directly.
> Gmail has always supported encryption in transit using TLS, and will automatically encrypt your incoming and outgoing emails if it can. We support industry-standard authentication to help combat email impersonation.
> 1. If you receive a message from, or are about to send a message to, someone whose email service doesn’t support TLS encryption, you’ll see a broken lock icon in the message.
> 2. If you receive a message that can’t be authenticated, you’ll see a question mark in place of the sender’s profile photo, corporate logo, or avatar.
AKA, it operates like the lock in my URL bar right now except in reverse. By the way Google has explained this feature, it seems fairly good to me. At least it doesn't say "this is secure" because, among other vulnerabilities, the remote server admin can still read what is received (as has always been the case with email).
I'm gettin tired of seeing 'mail is hard'. No, it's not.
You need to learn it, and there are a lot of knobs and buttons, indeed, but it's not hard, especially not with the plethora of tutorials around.
Sysadmining was never that easy and was never intended to be done by the general public by clicking on a few 'continue' buttons and as the web is evolving, so is mail. Deal with it.
This is interesting but as pointed out by other comments there is great danger of Gmail abusing their position to make life harder for small email providers.
I would be more positive towards this if they gave precise, technical details of their notion of "supporting TLS" and "being authenticated", ideally with a service allowing me to test easily whether my mail server is fine according to them (rather than having to sign up for Gmail to test it).
I wonder how long it will be before Google fully cuts Gmail off from the outside world, much like how they turned gchat from a member of the federated xmpp/jabber ecosystem into hangouts and cut it off from the everyone else.
I guess if nothing else, it would probably reduce spam!
I've run my own email server for about 15 years. Every now and then I have to drop everything and implement some new technology that Gmail demands I have. Granted SPF, DMARC and TLS are all great technologies but I take issue with Google making the decision that everyone is going to switch, now and with out sufficient warning.