Well, now we have it in the news that the US Govt is openly and officially hoovering up all the email they can get and snooping in it. The IRS has also said they don’t need a warrant to collect email as “there is no expectation of privacy”. In other words, we’ve been had so long and so much we don’t have the right to expect moral treatment any more… (And folks wonder why I actively discourage things like Facebook, text messages, email, etc… and keep it to the bare minimum…)
It’s been that way “for a while”, but now it’s not just “suspects” being pulled out of the vault for inspection. This article is from 2007:
By Z. BYRON WOLF (@zbyronwolf)
Nov. 7, 2007
It would be difficult to say whose e-mail, text messages or Internet phone calls the government is monitoring at any given time, but according to a former AT&T employee, the government has warrantless access to a great deal of Internet traffic should they care to take a peek.
As information is traded between users it flows also into a locked, secret room on the sixth floor of AT&T’s San Francisco offices and other rooms around the country — where the U.S. government can sift through and find the information it wants, former AT&T employee Mark Klein alleged Wednesday at a press conference on Capitol Hill.
“An exact copy of all Internet traffic that flowed through critical AT&T cables — e-mails, documents, pictures, Web browsing, voice-over-Internet phone conversations, everything — was being diverted to equipment inside the secret room,” he said.
Has an interesting video.
Finally, We Get Some Answers About How The Government Gets Data From Facebook, Google, Etc.
Henry Blodget | Jun. 8, 2013, 9:57 AM
There has been an uproar over the past 36 hours after two news organizations reported that nine of the country’s biggest technology companies are partnering with the government in a massive spying program in which the FBI and National Security Agency have been given “direct access” to the companies’ “central servers” and allowed to monitor any user at any time.
This direct access, the initial reports implied, allows the government to follow the communications of any of the companies’ hundreds of millions of users in real time, with no legal oversight.
One of the stories quoted a career intelligence officer as saying that this surveillance program was so powerful that, “They quite literally can watch your ideas form as you type.”
The impression these stories created was that Google, Facebook, Apple, Yahoo, Microsoft, and other companies had voluntarily opened their servers to government spies and allowed the intelligence agencies to do whatever they wanted.
Importantly, every company in the stories immediately denied that they had given the government “direct access” to their servers.
The companies confirmed, as they have many times in the past, that they provide specific information to government investigators in response to specific requests — when they are required to do so by law. But they emphatically denied that they they had opened their servers to the government. Most of the companies also said that they had never heard of the spying program, PRISM, that they were supposedly partnered with.
Such is the general fear of privacy violations by the big tech companies that, upon hearing these denials, many people accused the companies of lying. Others parsed their denials, looking for ways to square the carefully worded language with the assertions in the news stories. Still others focused their skepticism on the document upon which the assertion that the NSA had direct access to the companies’ servers was based, which struck many people as misleading.
According to Claire Cain Miller’s article, what is going on between the government and the technology companies is basically discussions about how the companies will provide the specific information the government requests.
Importantly, the transfer of this information appears to follow the normal procedure:
The government requests specific information.
The companies’ lawyers review the request.
The companies lawyers approve the information transfer.
The companies make the information available to the government electronically.
According to Miller, in deciding how to facilitate the fulfillment of these requests, some of the companies have had discussions with the government about creating a storage server that the government has access to — a “dropbox” of sorts.
Importantly, any information placed on this server would still be reviewed by the companies’ lawyers. And the information placed on these servers is not, say, “all the information generated by all Facebook users every day” (Facebook has explicitly said this.) Rather, it is likely much narrower requests for information about specific users, all of which have to be legal under the Foreign Intelligence Surveillance Act (FISA).
So a TLA (Three Letter Agency) sends a request to a company who’s very existence depends on FCC approval, corporate structure approval, SEC approval, etc. etc. and we can all rest comfortable knowing that they will stand up to the very government that gives them life to protect our privacy, right? I’ll believe that when we have mandatory notification of people who are being monitored and a public listing of how many requests are made, and what percentage are denied, and with a large denied percentage.
OK, that sent me off looking into “peer to peer” email that would be encrypted “end to end”. I figured someone had likely already done it, but if not, basing off of the Bitorrent model would likely be a good starting point. It would take a few more folks willing to be active at “sharing” file transport to more or less anyone, and more dependence on “magnet link” like facilities (to say “I want all email for me to show up here”…). There would also need to be an automatic “key exchange”, such that you would hold a small library of “public keys”. Your client, when joining the hive, would auto-generate a key pair, and share it’s public key with the hive, while accepting a library of other public keys. In that way, email to anyone in the hive would be encrypted automatically at the source. (This gets around the present problem of folks not bothering…) It would not hide the “contact trace” per se, since mail for you does come to you, but via “whoever” is on at the time.
That was the mental model of my “architecture”.
But first, a bit of a look around…
These folks made one using Java. (Unfortunately, Java isn’t particularly secure, so would likely require a re-write to be truly industrial strength, but likely “good enough” for ordinary folks discussing soccer games and weather). Source code is available and I’m downloading it now. No idea how good it is.
ePOST Serverless Email System
A Peer-to-Peer Platform for Reliable, Secure Communication
ePOST is a cooperative, serverless email system. Each user contributes a small amount of storage and network bandwidth in exchange for access to email service. ePOST provides
A serverless, peer-to-peer email service
Secure email emong ePOST users
An organically scaling service that requires no dedicated hardware
Very high availability and data durability
Compatibility with POP/IMAP clients, SMTP mail servers
Why Did We Build ePOST?
Peer-to-peer systems have gained wide popularity, partially due to their self-scaling properties and their resilience to failures. However, most existing peer-to-peer systems provide best-effort services, whose availability is not critical to their users. A question is whether peer-to-peer systems can provide service that users depend on in their daily lives and work. We deployed ePOST to show that a cooperative peer-to-peer system can provide availability, reliability and security that matches or exceeds that of server-based solutions, while reducing hardware cost and administrative overhead.
How Can I Use ePOST?
The ePOST project is no longer under active development. You can still download the ePOST sources and setup your own ring by following the directions on the Download page. ePOST supports Microsoft Windows, Mac OS X, and Linux running on java versions 1.4.2 or greater.
It is not under active development, which can mean that either it didn’t get much interest, or that it’s pretty stable and works OK as it is.
There are a couple of others, and I’m just posting up links and a ‘first blush’ impression. They are presented in the order listed in the search engine, not in any kind of value weighted order.
Has several interesting bits that show up ( it looks like a search engine in its own right).
BigSpeed Peer-to-Peer SDK 2.0 – Agent/Hub components for private peer-to-peer file sharing and instant messaging.Sample applications are included in Visual Basic .NET and Delphi 7. BigSpeed Peer-to-Peer SDK is a set of two COM components (Agent and Hub) that lets you set up a virtual private peer-to-peer (P2P) network for secure file sharing and messaging.
Along with several bitcoin ‘peer to peer’ links. Unfortunately, Bigspeed.net looks to be a parked domain name and the download link fails. Still, it might be out there somewhere on some other site.
An interesting I-EEE Link:
Conventional e-mail systems are prone to problems that affect their dependability. E-mail systems operate following a “push-based” approach: the sender side server pushes the e-mails it wants to send to the corresponding receivers’ servers. This approach may impose processing and storage overhead on the receiver side. This paper presents a peer-to-peer e-mail system in which messages are sent directly from senders to receivers using a “pull-based” approach. The sender stores locally all e-mails it intends to send, and notify their receivers using a global, distributed notification service. Receivers can then retrieve such notifications and decide if they want to receive the corresponding e-mails. If so, e-mails can be retrieved directly from their senders. This proposal is inspired from file sharing peer-to-peer systems, in which users locate and retrieve the contents they are looking for. A prototype was built to show the feasibility of the proposal, and experimental results show its viability.
So looks like it worked. But doesn’t mention encryption. They want $19 to buy the article, or one can be an IEEE member. So “Go Fish”…
Microsoft was looking at it in 2003:
[Paper] Secure and Resilient Peer-to-Peer E-Mail: Design and Implementation (Citations: 20)
Jussi Kangasharju, Keith W. Ross, David A. Turner
E-mail is a mission-critical communication function for virtually all institutions. Modern e-mail employs a server- centric design, in which the user is critically dependent on her mail server. In this paper we present a peer-to-peer (P2P) email architecture that eliminates the need to rely on a single server and boosts the resilience of email against any kinds of attacks. Our architecture also provides con- fidential communications for all users. We present how the basic mechanisms of sending and reading email are implemented in our architecture. We also consider additional schemes to improve anonymity in our architecture. We present our prototype implementation and discuss the future of P2P communication architectures.
Conference: Peer-to-Peer Computing – P2P , 2003
The F.B.I. is wanting to be helpful by telling people that peer to peer is risky business. (One can only wonder how much “riskier” it can be than the known 100% probability that the government has their snout in the email trough… It’s a very sad day when one starts to think that the Mafia is more honor bound than the government and that “shady characters” might be more reliable and safe than people with badges, and that trusting strangers is safer than trusting the government. We are “on the cusp” of that now. How the news of government over reach is handled will show which way we fall from that fence… )
Quoted in full so you won’t need to “print” a visit on their server to see it…
Risks of Peer-to-Peer Systems
The FBI is educating and warning citizens about certain risks and dangers associated with the use of Peer-to-Peer systems on the Internet. While the FBI supports and encourages the development of new technologies, we also recognize that technology can be misused for illicit and, in some cases, criminal purposes.
Peer-to-Peer networks allow users connected to the Internet to link their computers with other computers around the world. These networks are established for the purpose of sharing files. Typically, users of Peer-to-Peer networks install free software on their computers which allows them (1) to find and download files located on another Peer-to-Peer user’s hard drive, and (2) to share with those other users files located on their own computer. Unfortunately sometimes these information-sharing systems have been used to engage in illegal activity. Some of the most common crimes associated with Peer-to-Peer networks are the following:
Copyright Infringement: It is a violation of federal law to distribute copyrighted music, movies, software, games, and other works without authorization. There are important national economic consequences associated with such theft. The FBI has asked industry associations and companies that are particularly concerned with intellectual property theft to report to the FBI—for possible criminal investigation and prosecution—anyone that they have reason to believe is violating federal copyright law.
Child Exploitation and Obscenity: The receipt or distribution of child pornography and unlawful obscenity over the Internet also is a serious federal crime. The FBI cautions parents and guardians that, because there is no age restriction for the use of Peer-to-Peer services, pornography of all types is easily accessible by the many young children whose parents mistakenly believe they are only accessing music or movies. In fact, children may be exposed to pornography—and subsequently lured by sexual predators—even though they were not searching for pornography, as some network users deliberately mislabel the names of files for this purpose.
Computer Hacking: Peer-to-Peer networks also have been abused by hackers. Because these systems potentially expose your computer and files to millions of other users on the network, they also expose your computer to worms and viruses. In fact, some worms have been specifically written to spread by popular Peer-to-Peer networks. Also, if Peer-to-Peer software is not properly configured, you may be unknowingly opening up the contents of your entire hard drive for others to see and download your private information.
The FBI urges you to learn about the risks and dangers of Peer-to-Peer networks, as well as the legal consequences of copyright infringement, illegal pornography, and computer hacking. For more information about the law, visit http://www.usdoj.gov/criminal. The FBI takes seriously its mission to enforce the laws against those who use the Internet to commit crime. To report cyber crime, please contact your local FBI Field Office, or file a complaint through the Internet Crime Complaint Center at http://www.IC3.gov.
This guy seems to have a patent on some aspect of it:
United States Patent Application 20090144380
A peer-to-peer email system and methods are provided for distributed email distribution, prevention of SPAM, and efficient email storage. Each email client also serves as a node in the peer-to-peer system, relaying email messages and/or attachments. Large attachments may be transmitted directly from sender to receiver, and if the receiver is not online at the time the sender sends the attachment, the receiver can request the attachment from the sender at a later time.
And this one:
Peer-to-peer email messaging
US 7849140 B2
System and method for facilitating communications between peers in a peer-to-peer environment and network email clients. In one embodiment, network nodes including peer nodes may host mail transfer agents. The mail transfer agents may act as bridges between peer-to-peer protocols and email communication protocols. The mail transfer agents may communicate with peers according to peer-to-peer protocols and with email clients according to email communications protocols. Peers may communicate with mail transfer agents to send peer-to-peer messages to email clients. Email clients may communicate with the mail transfer agents to send email messages to and receive email messages from other email clients via the peer-to-peer network and to obtain peer-to-peer messages from peers.
Publication number US7849140 B2
Publication type Grant
Application number 10/231,225
Publication date Dec 7, 2010
Filing date Aug 29, 2002
Priority date Aug 29, 2002
Also published as US20040064511
Inventors Mohamed M. Abdel-Aziz, 5 More »
Original Assignee Oracle America, Inc.
Patent Citations (61), Non-Patent Citations (9), Referenced by (10), Classifications (20)
External Links: USPTO, USPTO Assignment, Espacenet
As the early Unix based mail systems were effectively peer to peer, I have a hard time seeing how it can now be patented…
Sidebar on DNS
I was having some DNS “issues” here at Starbucks. ( I typically set my DNS servers by hand to ones I trust). Had to use the DHCP supplied server to get things to work. Downloaded an app to test DNS speeds (thinking maybe my West Coast optimized were not optimal here).
An interesting tool, that I’ve not fully vetted, but the chain of links that brought me to it seemed reasonably likely to be a clean thing to run. Interesting to note that it rapidly identified that Starbucks is “intercepting and redirecting my DNS requests”. So looks like that VPN tunnel to the home DNS server might be of interest ;-) (So I can do a VPN to a known IP number, then use MY DNS servers inside an encrypted protected tunnel and avoid whatever they are doing…)
But that will be “for another day”. It seems to be a common feature of various “free” internet connections to molest the DNS to force you through various advertizing screens and / or logins and / or “accept terms”… that then also causes ongoing sloth of DNS lookups, sporadic failures, and generally doesn’t let you do nice things like use DNS servers that pre-block various advertizing and high risk sites… Oh Well…
Yet Another Project for “nights on the road”. Alternatively, I can also load up a batch of “things I care about” directly into the laptop DNS table. (It is already loaded with a large “block list” of things that are evil, so crowbarred to ‘localhost’ and grounded…)
This does tangentially relate to the email issue, in that one of the ways to “capture your traffic” is to spoof your DNS lookups and route your traffic through a snoop box that then forwards the requests / traffic on to the correct destination (a “man in the middle” attack). So knowing your DNS servers “matters” and having known DNS lookups “matters”…
To some extent, we’re entering a space where having direct and personal control over key bits of the communications path is essential to prevent folks, official or otherwise, from snooping or being a royal PITA. It is now less of a PITA to run your own DNS, have your own email server, and encrypt all traffic; than it is to accept the “free” public services that come with a snout up your skirts… Not something I really wanted to be doing (as I’ve done it for pay for too many years) but “you do what it takes”.
There were not a lot of “This is a final product” links. Yes, I didn’t go through dozens of pages looking for something rare. I figure folks can do that a bit on their own.
Old Style (very early days) email was largely “peer to peer” in basic structure. UUCP or Unix to Unix Copy has been in use since the ’70s for transfer of a variety of things, from files to email, between peer systems.
UUCP is an abbreviation of Unix-to-Unix Copy. The term generally refers to a suite of computer programs and protocols allowing remote execution of commands and transfer of files, email and netnews between computers. Specifically, a command named uucp is one of the programs in the suite; it provides a user interface for requesting file copy operations. The UUCP suite also includes uux (user interface for remote command execution), uucico (the communication program that performs the file transfers), uustat (reports statistics on recent activity), uuxqt (execute commands sent from remote machines), and uuname (reports the UUCP name of the local system).
Although UUCP was originally developed on Unix in the 1970s and 1980s, and is most closely associated with Unix-like systems, UUCP implementations exist for several non-Unix-like operating systems, including Microsoft’s MS-DOS, Digital’s VAX/VMS, Commodore’s AmigaOS, classic Mac OS, and even CP/M.
Given that, any patent on peer to peer email will of necessity be limited to some particular technique, and NOT a general patent over the whole turf.
Still, it is encouraging that some folks are looking into the idea of peer to peer email and care enough (i.e. see a product potential enough) to do a patent.
OK, from my perspective, there isn’t a simple “click and install” product, and certainly not a ‘thriving community’ like the bitorrent crowd. Too bad. Would be nice to have it all wrapped up and working…
Still, not much stands in the way. First off, anyone can set up an old style UUCP link between two systems and just shovel mail back and forth inside a private group. Anyone doing anything they wish to keep private needs to learn to run their own mail server. (It isn’t that hard. I’ve run many. While the fine points of SPAM filter maintenance and mail header parsing are a bit tough, a basic “send mail to @myfriend.com via smtp.myfriend.com” is pretty easy.)
It also looks like a “quick path” likely exists via taking a known working mail reader (even some of the web browsers understand email these days) and configure it to do opportunistic encryption (i.e. put an ‘exit’ in the code to look in a database of known public keys and always encrypt if possible. If no key exists, issue a ‘request for key’ to the destination and delay delivery of the email for ‘a reasonable time’ to see if a key is presented). Then take some of the torrent code and convert it to send email instead (that ought to be near trivial as a process – put email in an encrypted container, send torrent file for that container to the destination, let them pick it up whenever they want… )
I’m “way rushed” right now, what with coming up to speed on a new job and having 8+ hours a day at work, then another 2 or so of “overhead” for things like finding dinner and doing “laundry while on the road” and such. So it is unlikely that I can get to this “soon”. Still, it looks both doable and necessary. With luck “the open source community will provide”… faster than I can find time to do it myself.
There are many SMTP (Simple Mail Transfer Protocol) agents available, so simply setting up your own mail server is a reasonable “first step” for many folks / groups. Simply put: You don’t NEED a central mail service like google or AOL or whatever handing over all your “stuff” to the government. You can make your own server.
Free Mail Servers
The free mail servers (sometimes called Mail Transfer Agents, or MTA for short) on this page allow you to transmit email from one computer to another, using something called the Simple Mail Transfer Protocol, or SMTP for short. For example, it can receive email from an email client (software) and send it to another system. They can also allow email clients to retrieve its stored messages using either the Post Office Protocol, or POP3 for short, or the Internet Message Access Protocol, or IMAP. The email servers can either deliver the messages directly to the destination (end-to-end delivery) or relay them to another mail server for further transmission.
Please note the following:
The programs mentioned on this page are not for the ordinary email user. If all you want is a computer program that allows you to read and write email, please see the Free Email Clients page instead. On the other hand, if you want free email services (like webmail or the like), please go to the Free Email, Webmail, and Email Forwarding page.
This mail server for Windows, hMailServer, supports the SMTP, IMAP and POP3 email protocols. It also includes a score-based spam filtering system (SpamAssassin) and can be integrated with antivirus software (to scan incoming and outgoing email). Other features include support for server-side rules, SSL, multihoming, virtual domains, routing, built-in backup, etc.
Mercury Mail Transport System (Windows, Novell NLM)
Mercury supports the following protocols: SMTP, POP3, IMAP (IMAP4rev1), SSL (for SMTP, POP3 and IMAP), PopPass, HTTP (for mailing list management), finger and PH (for directory lookups). It can handle multiple domains on one system, supports aliases, autoresponders, forwarding, filtering, Bayesian spam filtering, mailing lists, multiple queues, domain mailboxes (that receives all mail sent to a particular domain), customisable (“customizable” in US English) notifications (such as delivery failure messages), greylisting, relaying, killfiles, blocklists, full session logging, the ability to listen on multiple ports, the ability to relay messages via SMTP or use full end-to-end delivery, the ability to restrict which connections are allowed (based on address range), etc. (This is just a subset of the list of features available: the full list is too long to mention here. Please see their site or documentation for details.) This mail server is only free for private or non-profit use.
Apache James (Windows, Mac OS X, Linux)
Apache James is a mail transfer agent (“MTA”) that implements POP3 and SMTP. IMAP support has been added to the version 3 series of the software, which, at the time I write this, is still under development. The software can also function as an NNTP news server. The software is written in Java, which means that it can run on any system that has a Java Virtual Machine installed.
Citadel is actually a groupware program (with instant messaging, bulletin boards, shared calendars, mailing list server, etc), but it is listed here because it includes an email server. It supports ESMTP, POP3, and IMAP, and also includes a webmail interface. Access to your email over TLS/SSL is also supported. It has built-in spam filtering with support for things like blacklists, SpamAssassin, and an antivirus. Binary packages for Debian/Ubuntu Linux are available. For all other Linux users (or perhaps also users of other Unix type systems, including Mac OS X), you will need to compile the program yourself from the source code.
Postfix (Unix-type systems) (Source code only)
Postfix is a mail transfer agent with support for SMTP, DKIM, DomainKeys, SenderID, TLS encryption and authentication, junk email filtering, etc.
qmail (Unix-type systems) (Source code only)
This software, qmail, is an SMTP server for Unix-based systems like Linux (and presumably also Mac OS X). It was written to be a secure replacement for sendmail (another mail server). It also provides a POP3 service.
Sendmail (Unix-type systems) (Source code only)
Sendmail supports SMTP, ESMTP, UUCP, etc. It is known (or perhaps more accurately, notorious) for being hard to configure, the result of its attempt to provide the administrator with the ability to control many things.
Qpopper (Windows [with cygwin], Mac OS X, Linux) (Source code only)
Qpopper supports POP3 and TLS/SSL. It also features authentication via login name/password, APOP, Kerebos and PAM. As far as I can tell, it does not implement any support for SMTP, which means that you can send outgoing mail through this email server from your email software. It is released as source code, so you will need to compile the program into an executable before you can use it.
“Sendmail” is a royal PITA. Period. Avoid it unless you are a masochistic Unix Sys Admin with a death wish… or can convince your boss it is important and the best; so as to assure job security …. ;-)
Of those on the list, the ones with TLS Transport Layer Security (i.e. encrypted communications links like SSL) offer a bit more security against folks with sniffers on the wire. Citadel looks like the most interesting one at this time.
As I’ve got Linux in a virtual Machine on the laptop, and packed one of my Raspberry Pi boards, I’ll likely take a shot at configuring Citadel or something like it in a VM or on the RPi, just to see how clean it is.
THE big exposure here, though, is that you need a DNS entry so folks can find you, and that means a registration of a domain, and that means things like name and contact information in the public domain… so a sub-text to all this is finding a way to do that WITHOUT publishing your name, address, phone number, etc via the DNS / Domain reg process. It may be that a ‘dark net’ is the way to go here for true privacy and security. (i.e. a ‘roll your own’ DNS of ‘illegal’ domains like .onion and then mail transport in TLS / SSL tunnels via that darknet under the covers of the “legal” internet). That way you can DHCP a “real” IP address, then enter the darknet and find other folks for the email exchange.
I likely need to look more at Onion and TOR to see if they have already done this. If not, it’s right up their ally.
OK, I’ve about finished my Espresso, and this is pretty much all I can do on this topic in one session. It’s not high on my priority list (for the simple reason that I don’t put ANYTHING in Email that I do not want read by 20 government employees, 6 systems admins, my spouse, my boss, and have on the front page of a newspaper. You are encouraged to act in the same way… IFF you want to communicate something in private, put it in a file, encrypt the file, and send it as an attachment (while “public key” is best, since then there is no ‘key exchange’ exposure; it is also possible to have a ‘protocol’ with someone like a spouse, where you just know that the password is, for example, “The dog’s name concatenated with the day sent and your full street address and ending with moms maiden name”… the longer the key the better…)
I encourage folks to encrypt any and all messages sent, even the “irrelevant ones”, as the more the world “goes dark” and encrypted, the more The Powers That Be will get the message that we don’t like them in our bedrooms, our bank accounts, or our mail. We have left behind the days when innocent people could depend on their innocence to protect them from intrusion and abuse ( the IRS scandal shows that – having “the wrong” political opinions can be enough…) so it’s time to be defensive.
Even if all you do is encrypt a bit of trash and attach a bogus encrypted file to random email messages ( to folks who know they are not real and will not ‘spill the beans’ in an open email…) that tends to ‘clog up the system’ of monitoring. Adding a few more petabytes of “trash” will fill the various servers and snooping agency budgets with yet more crap. Maybe then an audit or two will show them not worth the cost. Yes, a mild form of “passive aggressive” complaining.
With that, I’m on to my next topic.