For a few years now I’ve had a “muse”. How to build the infrastructure portion of a Skynet or the communications system used by a Borg Queen cluster. (As there are multiple Queen copies, they need some way to coordinate).
I’m pretty sure I’ve worked out most of the ‘hard bits’ (but there are a few I’ll not be sharing here – don’t really want a “Skynet Moment” in our future…)
However, that same musing provides a basis for a distributed, secure, encrypted sharing of files and compute functions. One that if properly done would be very hard to stop, shutdown, or really even detect. Open, yet hidden.
The ideas behind a Darknet are not new. This just automates much of it and suggests directions to make it more universally available. Some organizations promote such things, for example: http://www.darknet.org.uk/ Much of what is presently done is as a “peer to peer” network of friends: http://en.wikipedia.org/wiki/Darknet_(file_sharing)
The term darknet refers to any private, distributed P2P filesharing network, where connections are made only between trusted peers — sometimes called “friends” (F2F) — using non-standard protocols and ports. Darknets are distinct from other distributed P2P networks as sharing is anonymous (that is, IP addresses are not publicly shared), and therefore users can communicate with little fear of governmental or corporate interference. For this reason, they are often associated with dissident political communications, as well as various illegal activities. More generally, the term “darknet” can be used to describe all non-commercial sites on the Internet, or to refer to all “underground” web communications and technologies, most commonly those associated with illegal activity or dissent.
But such techniques are not generally ‘user friendly’ enough for the average non-technical person. It would work better to have a bit of ‘open source download’ software that could just be installed on a computer and automagically join a darknet. Yet that has opportunities for Agency penetration and attack, so it needs some infrastructure behind it.
Recently several world governments have decided to shutdown various parts of the open internet. We have the USA attempting to pass legislation that allows the recording and movie industries to shutdown websites at will. We have China with internal controls, working to gain even more access to what is shared, and greater blocking of what is not “approved”.
There are more (including Megaupload being sued by the Department of Justice and the Megaupload suit against Universal, with Google on the side via YouTube). Basically, BOTH the Rapacious Capitalists and the Oppressive Socialists / Communists don’t like free people doing free things. The use of Twitter during recent Arab Spring events did not escape Government Notice either. The US Government has issued a request for software to dredge through ALL such social networking communications for patterns that might indicate incipient “problems”.
It’s all about control, and all big governments and big corporations want control.
The notion that every single Tweet and potentially every single email will be snooped, logged, recorded, and used against the involved individuals, is in the process of implementation. That means it is time to “go dark” and move those communications inside of encrypted systems.
So, IMHO, the time has come to share an idea. To turn it free and let it go where it will go, come what may.
It is NOT a finished product, nor even a finished specification. It is an architectural vision. There are many design details to work out, and a boat load of programming to make it real and running. The ‘good news’ is that large parts are already coded, so mostly just need integration.
One of the major areas needing ‘improvement’ is the area of vulnerability testing. There are very large forces opposed to free people doing what they wish, so you can be assured that any tool that facilitates that will be attacked. And this IS just such a tool. It can be used for secure computing and information sharing by ANY GROUP. That would include Mafia, Terrorists, oppressive governments, secret services, you name it.
That has been my major reason for not “sharing” until now. But it can also be used by oppressed peoples yearning to be free, by folks in a new holocaust to show their plight, and even by Free People just wishing to be free and communicate private things in private.
Once it exists, there will be opportunities to create anew all the things, like email providers and twitter and google and…) inside this new Dark World. So there will be plenty of opportunities to make money off of the usual scams of spam and advertising, but the problem is that all those thing will need to be recreated.
Yin meet Yang…
At the same time, times have changed from when I first thought of this. Encryption and distributed computing are nearly universal now. High speed encrypted networks (such as VPN Virtual Private Network) are common. Even browsers have built in secure connections (httpS type connections). It is “only a matter of time” now, and IMHO not a very long period of time, before someone else publishes anyway.
OK, with that preamble, what IS this idea?
Conceptually, it is pretty simple. I could be described from top down or bottom up. The first is easier to ‘vision’, but the second is easier to see how it work. So I’m going to blend them a bit.
At the top level, it is a distributed network based virtual machine cluster, running on many shared systems, and communicating over a hidden Virtual Private Network, with data stored on a distributed cryptographic file system.
No part lives on any ONE machine. Most parts are located on many machines. You can shut down parts, or break connectivity, and the virtual machine just shifts where it is doing its distributed computing and where it gets any particular encrypted data blocks. (Think of it as a giant RAID – Redundant Array of Inexpensive Disks).
Yes, there would be a large amount of encrypting and decrypting, and a fair amount of network traffic. In the days of a 386 CPU and dial-up networks, it wouldn’t work fast enough for most folks. (I’d originally thought about this, then, and figured it would take machines inside organizations with T1 type internet connections 1.5 Mbit. Now we have faster than that speed coming into private homes.)
So we can have many private computers, all contributing some compute resources and some storage space to a Virtual Machine that exists nowhere, and everywhere, that does the actual file sharing.
The individuals can then have their real physical machine “ask” the Distributed Virtual Machine for services. The “open” communication all happens between your laptop / desktop machine and the PART of the DVM that lives on your box. The only communication that is ‘in the clear’ and can be intercepted is that from your keyboard / screen to and from the DVM on your machine.
Your DVM node then communicates, via VPN connections to other nodes. What is visible on the public service network is a series of encrypted block transfers that seem to originate from random places and go to random places. What is more, cutting ANY of the connections does not stop the DVM from running nor does it stop the data from existing nor from moving. It is like losing a disk in a giant RAID, it just keeps going. Or losing a node in a giant High Availability Computing Cluster, that also just keeps on going.
Nonstop computing. Nonstop Data.
You can kill it by shutting down ALL the nodes. However… When they come back up, they reconnect and start running again. In essence, you must PURGE all the machines that contribute resources. Since the individuals who own those machines are not inclined to purge them, that will ‘be a problem’…
To prevent ‘polluting the compute pool’ via a broken or deliberately compromised node, several “Drone Nodes” can be told to store or compute the same things. Their results get compared inside the DVM and if one of the nodes is divergent, it can be flagged as suspect (or perhaps. in a bit of puckishness, assigned to a ‘rogue pool’ that gets spun out to let the Agencies go “play with themselves”…)
In theory, you can kill it by blocking all file / data block transfers that are encrypted ( i.e. not ‘clear text’) but there is a counter to that (it consumes about 80% more resources, so it not a preferred mode of operation, but is very possible). The problem with blocking all encrypted block transfers is that it also ends up stopping all https web browsing and all private use of VPNs. An ‘area for development’ would be assuring that any inter DVM block transfers did not present an obvious ‘signature’ allowing selective identification.
The ‘work around’ for blocking encrypted transfers or ‘open transfer only’ is to simply embed the encrypted data in steganographic blocks. Shipping what look like personal photographs, or compiled programs, but have hidden data inside them in a way that is not visible nor provable. This adds overhead, so is not the preferred method, but as Moore’s Law doubles compute capacity every 18 months, it’s really just a 3 to 5 year “issue”…
We have at core a Distributed Cryptographic Networked Machine ( DCNM is in use by CISCO to mean something else, so my use here would be ‘idiosyncratic’. I also like DCM that echoes the Department of Civilian Marksmanship for Distributed Compute Machine. Folks will need to decide what to call this beast. For now I’m using DVM Distributed Virtual Machine that is also a Doctor Of Veterinary Medicine. Sigh, we’re running out of TLAs – TLA being Three Letter Acronym or Three Letter Agency, depending on context )
At the bottom of this DVM is the use of Domain Name Service DNS Most attempts to shut down services depend on changing DNS entries to ‘erase’ the offender. (DNS is one of the few places where there is any ‘central control’ on what is an official Domain Name, but it is not FORCED to be central, so we exploit that distributed ability)
The Individual Private Machine (IPM) node must look to the DVM node running on it for the first level of DNS (at least, for any files or data that are ON the DVM network). As part of the installation of the DVM, the DNS of the IPM host is pointed to that Virtual Machine as top level DNS. This lets the DVM have an internal DNS that knows where all its parts are located without resorting to the external (and thus government controlled) DNS and it lets the IPM use that DNS to gain access to those blocks.
One open issue is how to segment the DNS name space for DVM private use, IPM use, and specific shared services. A way for “anonymous publication of membership” needs to be worked out that is both distributed, but resistant to spoofing. (Likely involving a private key verification for any publication. Basically, I could ‘join’ and ask for .emsstuff to be under my control. I’d give a key to the DNS server (once it found that .emsstuff qualifier available) and gave me authority. Any FUTURE changes would require me to present my key again, perhaps with a public/private key redirection in the middle to prevent captured key attacks. At that point any .emsstuff would be from my key (though not necessarily my public IP address nor machine) and, if found to be bogus, it all tracks back to my key. So intrusions and spoofs get self limited (or handed over to the spoofnet machine with all the other attackers…)
WHATEVER IP number your machine is assigned, gets communicated to your DVM and thus gets incorporated into the Virtual Cluster. (I’d prefer to avoid using the MAC address, if possible. In theory a snoop would at most find out what machines were participating in the Darknet by MAC address, but it would be nice to find a way to obscure even that information as it is one of the few bits that map to particular hardware). There will be some issues to work out involving NAT Network Address Translation and how that gets handled, but it really ought not to be much of a problem. (It provides a ‘spoofing’ opportunity in that the Virtual Cluster if it picks up your Real IP number from the public network side of the communications can then have a government machine ‘slide in’ in front of you and take your IP to try to participate ‘inside’. I think that’s not much of an issue, but needs proving.) If need be, one could implement a new level of DNS that only exists on the private side inside the DVM such that it could not be spoofed or intercepted.
OK, at that point you have a DVM node, it has secure private DNS, and it wants to join others. Part of any DVM node is storing shared data blocks in a secure form. This is done via an encrypted file system. Blocks are only decrypted inside the memory of a local machine and “open copies” of data are only stored in the IPM private computer open space after the individual chooses to download them from “wherever” they are stored physically.
That is, if you download Beethoven’s 9th it only is a visible MP3 file after the download is decrypted and presented to your Open Side desktop. There are many cryptographic file systems available, and most likely it will take an integration of a couple of them to make things robust enough for resistance to wide spread attack. XtreemFS has built in cryptographic and distributed file support. An older one that I’ve used as the base of my ‘muse’ is TCFS the Transparent Cryptographic File System But actual choice can be implementation dependent.
The key point here is that multiple copies of any block of data must be kept on multiple machines. That way any time a file is accessed, the particular blocks in question may come from any of several machines. No individual knows what is stored in the encrypted blocks that they contribute to the DVM Cluster when they join ( they just provide ‘free space’ to the cluster that decides what gets stored where). Because of this, no individual can choose to ‘pollute’ or ‘corrupt’ any particular product, nor detect where any particular file is actually located. This provides both “denyability” to the participating individuals along with more robust file service.
In an ideal case, at least 2 copies of each block would be retrieved from 2 different locations and compared. As long as they agree, they could be accepted (one might want the ability to set a higher count number when under attack) and if there is a mis-match, more block copies are retrieved and compared until it is determined which blocks / DVM Node is compromised, then it can be isolated.
Computing is also distributed to various nodes. As all this happens INSIDE the DVM, it is hidden from inspection and even which node is doing that fetch / compare process can be ill defined. Your individual DVM Node might well be providing compute functions to OTHER downloads or messages, not just your own.
The code for distributed computing is already in existence, but would likely need enhancement and some customization. MOSIX is an example of a self configuring distributed Linux, Beowulf Clusters are another with more manual assemble that could be automated. Some work would need to be done to move them to a Virtual Machine base and layer that on top of an encrypted block transfer cryptographic file system. This is the place that, IMHO, will take the most work. Initial versions could be brought up ‘in the clear’ then moved to ever deeper levels of virtual and encrypted distributed function. PVM Parallel Virtual Machine is another part of the base code available.
In essence, we already have JAVA as an existence proof of a downloaded Virtual Machine, we just need to make one of those that Clusters like MOSIX and that works on top of encrypted layers (much like Java inside an httpS link).
Once the Virtual Machine is running, it stores data “everywhere and nowhere” in particular.
Some of the technical issues involved in keeping encryption secure can be found in the documentation at the TrueCrypt site: http://www.truecrypt.org/docs/
TCFS has the interesting feature of a ‘quorum’. Each node can have its own ‘key phrase’ and the total file system can only be decrypted when a quorum of users connect. By defining a few initial nodes as “Queen Darknet” nodes, they could form the initial DVM Machine and the repository for the Virtual Machine operating system. Then they could have a ‘shared encrypted secret’ and identify other Darknet Queens via an open exchange.
The candidate Darknet Queen would encrypt her copy of The Secret and present it to a detected DVM (via their public key). That cluster would then decrypt it (using their private key) and IFF The Secret Key matches, they could admit that Candidate Queen into the DVM cluster and share the OS file system. This level of implementation is optional, especially for early instantiations, but it would be nice to have something like this eventually. If a node were found to be “suspect” or “compromised”, it could be told a ‘secret’ that assigned it to a ‘special’ cluster, the “spoofnet” referenced above. This behaviour could also be based on a quorum system, but needs to have a way to protect against an attack via a government hoard of machines being presented as ‘mutually trusting’ and thus taking over traffic. I think this can be done with a pre-encrypted private secret, but there are likely better ways.
I could even envision a few Darknet Queens that would form a core cluster DVM which would then “anoint” Lesser Nodes as compute servers or storage servers. That is, your laptop might be ranked as “probably OK” when you install the applications, so you could contribute a GB of disk to the Virtual Filesystem that stored shared user data, but not the OS codes… and you could provide compute services to the user level workload (encrypting or decrypting OTHER Lesser Node shared blocks, for example).
This would make the whole system more robust to “spoofing” via Government Agencies putting up a bunch of Nodes and then inspecting memory contents and / or communications. Basically, they only get to see the user level, not the private level. I think that with proper design, this need could be avoided, but for initial instantiations especially it would be beneficial for a Darknet Queen Candidate to only have a Coronation via a trusted host mechanism that helps filter out spoofing attacking nodes. ( A derivation of this system leads to Skynet and a Borg Queen system via parasitism of involuntary nodes, so the direction of ‘involuntary recruitment’ is to be discouraged.)
I think it is time for an Open Source Project (or perhaps a few private Shared Source Projects) to start building Darknet Queens, and start making a file sharing system built on top of Distributed Cryptographic File Systems, on a Distributed Virtual Machine, made from nodes that voluntarily self-recruit and provide a share of their disk space and compute power to this Virtual Machine. All communications over VPN Virtual Private Network links that are established as needed between the nodes. DNS first level provided by the DVM itself (perhaps as a unique system and probably with “illegal” qualifiers – that is, not just the .com and .net “approved” ones, but things such as .dvd or .download as desired)
All the parts exist. It ought not to take long to integrate them. Then would begin the long hard work of proving security and assuring continuity.
But such is the price of liberty.
Postscript: It is always possible that this system is already built and running. That’s part of the “problem” faced by darknets. How to ‘recruit’ new membership while staying invisible. There are brighter and better hacker than me out there, so this could easily be a ‘done deal’ and I’m just coming to it late. But I’ve not seen one like this, yet…