Hacks, Leaks, Investigations, Evidence – Leaps…

The Dimocrats are still pushing the meme that “Russia Hacked the USA Election!!!!”.

This suffers from the bold and obvious lie that you can NOT hack an election. Computers can be hacked. So right out the gate, they offend common sense. So we dig. Such is the way of giant Neon “Dig Here!” signs.

So grant them a bit of slack in their rope. Maybe they are just incredibly sloppy thinkers and are using that as ‘short hand’ for “Hacked the DNC and used that to change the election.” OK… that, then, leads to an examination of what the DNC had, what evidence of a hack was found, and how to finger Russia with it.

Others have already examined the actual election process, backend servers, and voting machines and found nothing. Not even the Democrat vote diddling (though there were a few heavily democratic precincts that seemed to have a democrat overvote for Hillary…). So I’m not going to be looking at the VOTE, or the voting process, as those have been vetted by others.

A note on Hacking vs Cracking: In proper jargon use, breaking into a system is a “Crack”. Hacking can just be writing some quick and dirty code for your employer to get the job done. Sadly, in common use, Hacking has become a synonym for Cracking, so I’ll use the common “hack” and “hacking” below. But you know it isn’t right, ok? ;-)
(Pedants of the world unite!)…

But First:

What is the link between {stuff} taken from the DNC and “influence the election”?

The {stuff} was a long list of historical emails. Similarly, to conflate two unrelated issues…, the Madam Clinton Emails were a very long historical set.

A critical point about both sets of emails: NO ONE HAS CLAIMED THEM FAKE.

We were not regaled with endless hours of pleading Democrats and hangers on sobbing about how wronged they were by this faked up bag of lies. We were not faced with Democrats on the House and Senate Floor speaching and ranting for hours about the Evil Of Fraudulent Emails. No, we were met with silence on the issue of provenance and purity. All the ranting was about possession of private emails.

So, down in the very belly of this beast, the core complaint is not that someone is slandering them with fraud. No, the core complaint is that someone has let the truth be seen.

So the horrible evil Oh My God! HORROR!! at the core of the complaint is that the Truth damaged the Democrats and shifted the election.

And the truth shall set you free… Congratulations President Trump, and thanks for the freedom…

So, Dear Democrats, some helpful advice:

1) Do not put evidence of crimes, corruption, media coordination and influence pedaling in your email.

2) Do not expect email to evaporate once sent. Expect it to be forever.

3) Do not expect email to be private. It is subject to subpoena, to leaking, to hacking, to carelessness, to a vindictive spouse or staff member, or even just a janitor reading a screen you left unlocked. Oh, and it is also subject to the evil “Reply All” button…

4) DO learn to use encrypted email. While not perfect, it does tend to keep email archives free from interlopers and it does keep the contents secret from all the hands that carry it from you to recipient. (Subpoena can still force the password from you, and Agencies with $Billion budgets can still crack many ciphers, and there is always the risk of a Brute “Mafia” like “offer you can’t refuse”… But for 99% of risks, it’s the answer. I suggest PGP based open source mail handlers. At all costs, avoid Microsoft and any other PRISM influenced products.)

The Nature Of The Emails

This is a modest, but important point. I’m again going to conflate DNC emails with Hillary emails.

In both cases, relatively long histories were dumped. To me, this shouts “Insider” more than “hack”. Why?

A typical email server doesn’t keep an archive of email going back years. It fills them up and crashes the machine, especially in large shops. The email server is a transfer agent.

Pick up here, drop off there. IFF email goes to other organizations, it may pass through many ‘hands’ on the way. Your desktop. Your server. Your ISP (Internet Service Provider), perhaps another intermediary transfer agent (like USA ISP to International Carrier to EU ISP), to the recipients ISP, to their email server, to their desktop.

Now multiply that by 4 Billion folks sending email, by a dozen a day, by 365, by years… Stew heavily in Nigerian Prince and Work From Home SPAM in the dozens per person per day, then season lightly with a half dozen email addresses per person (home, work, sock puppet for high class things, sock puppet for illicit things, …) Rapidly you are talking more disk space than even the NSA can cough up. (Though they are trying…)

Now most of those steps have folks desperate to just move the mail along, and then forget it. Each desktop is at the whim of the owner, but I’m sure many are familiar with the I.T. Department nag to “clean up your email folders”. Frankly, part of why I’m so slothful about servicing email now is how much of my life was spent doing it for money… Shoveling e-shit into the compost pile… Electronic stable boy.

Now for many of the originators, there is now SARBOX to worry about. For Companies and Lawyers it specifies a long retention interval. ( 5 to 7 years depending). Network Appliance makes a nice big file archiving appliance for just that purpose. Similarly, many Government Agencies are subject to legal retention requirements. This is very very poorly served by DIY (Do It Yourself) management on the desktops. For SARBOX, even if the Chief Information Officer didn’t order the loss (or even if he ordered the loss NOT to happen) he gets prison for failure to comply. So folks use those nice NetAp devices at the central email server. Even the System Admin can’t erase them.

But that kind of archive is usually “center of the empire” and very well guarded. Locked down very tightly and surrounded by layers of protection. I’ve never heard of one of them being hacked. ( I’m sure, somewhere, it has happened, but the rate is so low as to not rise to awareness). Furthermore, the ‘evidence of the hack’, is on peripheral boxes. Desktop machines and one or two poorly guarded servers. This necessarily implies that either the DNC and Hillary had among the sleaziest, poorest security, and ‘walk in friendly’ configurations on the planet AND were being used for Email Archives (which, in the case of Hillary, looks to be true…) or The Hack wasn’t getting the archives…

At this point the DNC and Hillary Email diverge.

For Hillary we have a Hacker in custody who said he did it, where there is evidence he did it, where a law enforcement agency caught him in the act and where he was hauled in by the FBI. He said it was a trivial hack technique based on knowing personal details to make a custom dictionary (names, family and pet names, addresses, place of birth, etc.) then using it in a Dictionary Attack on some folks or in a “I forgot my password / Tell me your last name and DOB and I’ll send it to you.” spoof. There is also evidence (weak, but extant) that many TLAs (Three Letter Agencies) and other actors had hacked into her home brew server by other means.

Given what I’ve heard of the set-up, it would be a nearly open book to anyone with skilz. First off, it was built on PRISM infested equipment (so the NSA was in, and potentially the CIA), second, it was Microsoft, so if you didn’t patch daily, you were hacked with known zero-days, and if you DID patch daily, you were hacked by ‘non-fixable’ hacks.

AtomBomb – The Unfixable Bug:


Bold bits by me.

AtomBomb: The New Zero-Day Windows Exploit Microsoft Can’t Fix?

Celestial writes:

There’s a new zero-day Microsoft Windows exploit in the wild by the name of AtomBomb, and Microsoft may not be able to fix it.

Ensilo security researchers have discovered a new zero-day exploit in Windows that attackers can make use of to inject and execute malicious code. The researches call the exploit AtomBombing because of its use of a Windows function called Atom Tables.

What’s particularly interesting about the exploit is that it does not rely on security vulnerabilities in Windows components but native Windows functions. This means, according to the researchers, that Microsoft won’t be able to patch the issue.

So at this point, we can largely dispose of Hillary’s Hack. It was an open book to all comers and at least one was Romanian (and sharing with friends) and not Russia. However, I’d say it was almost certain that at some time a Russian intrusion happened. The name of the server was obvious. The location insecure. The operating system and protective layers a joke. Frankly, I’d expect them to be “in” the same day they first looked at it. Which means something like 8 years ago. So why didn’t things leak then?

Because the Russians Are Not Stupid. A fundamental of spycraft is you don’t expose sources and methods, you use them to collect intel for your use, not publication. I suspect they enjoyed a near real time email feed from the Secretary Of State for years, in silence. This argues for email dump to be someone other than them. My personal muse would be an NSA guy, aghast at what was in evidence. Like a Snowden, but not willing to give up the $1/4 Million salary… He (or she…) would have all the requisite skilz to pull it off and leave no finger prints, access to PRISM, and lots of neat toys to work with. Though more likely would be the underpaid I.T. guy Hillary had set it up who was making a backup one day and dropped a load… But I digress.

The bottom line on Hillary is we know she kept a full copy (found on Huma’s Laptop with the Wiener…) and that it was around until she had her lawyers erase it. We know it surfaced in full at the time the laptop went to the FBI, and in parts before that. We know at least one of her hackers was found (though he had likely not leaked it) and that he said he had a doomsday copy for safety. He wasn’t a very good hacker, so that shows lots of good ones walked right in and snagged copies. Assigning source of any Hillary leaks is going to be an exercise is “ME ME MEE!!! PICK MEEE!” with a dozen hands up in the room…

For the DNC:

We know Podesta fell for a phish. That, alone, is enough. Yet we also have evidence that the box wasn’t that well run and secured, and ample evidence that the privilege escalation path once in was easy. Privilege escalation is when you get in with weak powers, you find ways to raise your powers. Moving from “user” to “admin” to “root”.

How many others fell for a phish? How many other bugs, holes, unpatched zero-days? Was it PRISM? Were they on Microsoft? (Almost certainly…though I haven’t bothered to verify).

Once you are this far into the pants-down party, you know you will never know which of the hundreds of actors trying to get in, made it in. You may never even know how many made it.

Now in this case the server was better run than Hillary’s (but still had issues) and we know it was compromised by a very simple and direct path (so likely other paths exist and other compromises happened).

BUT, in all these cases we know two things about the nature of the emails:

1) They were undisputed as truth.

2) They were a long history.

That last item argues for one of two cases:

a) Folks were packratting email online. For Hillary known true.

b) The server was an archive (typically not true, but sometimes done even if poor practice).

Which leads to another bit of advice for the Democrats: IFF you keep an email archive, keep it on a USB drive in a locked vault, not on-line. If it doesn’t NEED to be online, don’t PUT it online.

The long historical mail set is modest evidence for “leak” over “hack” as the leaker has access to the archives, even if protected, while the hacker must break through many layers (and pass many detectors in a good shop) to get to them. Then the leaker just copies to a thumb drive at high speed while the hacker must ‘exfiltrate’ or copy outbound the huge data volume over days to weeks and not be noticed. It is remotely possible that the hack happened 8 years ago and the emails were exfiltrated and accumulated over that period without being detected.

So the very nature of the email dump (archives) argues for “leaker” over “hacker”. Doesn’t prove it, but points that way. Just something to tuck in the investigation hat while looking at the rest of the process.


Leaks happen when someone inside decides they just can’t stand it any more and their allegiance shifts from “to the patron” to one of two things:

1) Money. Sadly, many folks can be bought.

2) Morality. Morals vary, so which morality is variable.

Once we had a security sweep of a very secret building and project. At that time, the hired expert also gave a tutorial on risks. The one that I didn’t know, and floored me, was the “insider bought” risk. We were asked “What do you think the going price is to be compromised?” I guessed based on the expectation that I’d never be able to work again in the industry, and would need enough to continue the present life style indefinitely after legal costs and pegged it about $2 Million. (In an annuity, that would be about enough) The answer? $1000 to $2000. Now even if you inflate that to today and call it (way over inflated) $10k, that’s trivial money for LOTS of actors and agents. That was the going rate to get hands on copies of backup tapes.

Morality is actually a much higher barrier, as most folks driven by it are greatly pained to get over that hump of betrayal. It is far easier to do nothing, and suffer the angst of moral indignation than take action and suffer the angst of seeing yourself as a betrayer of trust and thus immoral yourself.

Examining the DNC and Hillary, it’s pretty clear that the physical security was so-so at best, and who knows what the staff level of vetting was. That puts #1 as an easy approach. Pay a janitor $10k to swap his badge for the day. Pay a ‘just above minimum wage’ night operations person $10k to run a command. Hard to protect against that, and harder to catch.

On the morality front, there was so much incredibly offending material in the emails that lots of folks would be morally offended by it. I could easily see a higher end Systems Admin type doing maintenance on the system, seeing something pass by that offends, taking a look for more, and having an OMG! moment. Now these folks would also be good enough to make sure they left no finger prints themselves, and, if needed, leave behind fake ones of a ‘hack’. Heck, for many folks in this class, it becomes a game of sport to do it so well nobody can tell. There are whole conferences for this kind of stuff, with “White Hats” doing their best to block “Black Hats” and with everyone having a grand old time sharing methods and warez. (tools). IF it was done by an insider admin of medium to high skill, you would likely never know and it would be fingering someone else.

Substantially all security is set up to keep bad guys out and assumes insiders are good guys. There are ways to set up security that doesn’t assume that, but outside a few paranoid companies (justifiably paranoid) and some government agencies, that typically isn’t done. Setting all kinds of credentials and levels of access and such is a royal PITA (Pain In The…) and you are asking the folks setting it up to do it to make their own lives harder… so it usually is only done in large shops where those are two different groups of folks.

IF the leak was an insider with technical skills, it would be damned hard to distinguish it from an outside hack IFF (If and only If) you knew it happened at all.

Investigations and Evidence

Near as I can tell, all the investigations have been after the fact. None of them “in the act”. This is a huge point.

With an active hack in progress, you can watch packets on the network in real time. Where do they go? What is in them? What personal style is acting? What tools show up, what do they look like in operations, what log files and such do they erase? It is usually possible to at least find the first link back (the exit node) of the attack. Sometimes you can ‘crawl upstream’ and find more redirection nodes. IF you are very good and very lucky and / or the attack lasts a long time, you can sometimes get all the way back to the originating node. At the same time, you can log ALL the actions real time, sending logs to a system NOT under attack and NOT subject to modification of the evidence.

Sidebar: We once caught an attack on our honey pot (at Apple) due to a paper syslog printer. At the time, it was ‘trendy’ to put all logs on disk to ‘save paper’. I insisted we keep our ratcha-ratcha TTY printing out 128 column green paper… Well, a night operator, tasked with maintaining that paper, noticed something on the printout (as we hired smart operators, not dumb paper changers) and called in the admin staff. The result was we caught an attack in progress that was clueful enough to have doctored the syslog FILE and remove any evidence of their entry… but they couldn’t get into the locked, remote, high security computer room and manually erase the paper. (Later we went to all electronic logs… on a WORM drive aka optical disk that could not be erased.) I’m certain the DNC and Hillary had nothing like that set up.

For a postmortem forensics review, much of that information is gone. You have log files that have likely been erased or doctored. You have what is left on the disks, that can also be doctored if not erased. You might have some logs of network activity. And, IFF you have an IDS / IPS (Intrusion Detection System / Intrusion Prevention System), you will have whatever data you configured it to collect (assuming IT was not hacked and compromised… which is why I like to put their OS on a CD ROM and their logs to write once media…) Now that can be a lot of information, or it can be a very little, or it can be completely WRONG information.

Really good hackers get in with a set of warz, immediately start changing any log files and IDS systems to erase evidence of the attack, and exfiltrate what is highly interesting, erase those logs, then lay low with long duration backdoor kit. If possible, picking up additional bits over long periods of time. This is a skill set that takes years to understand, so I’m not going into it here. If you want to know more, attend one of the many hacker conferences for a few years.

Excellent hackers leave indirection evidence that is hard to find (so either you don’t find it and don’t know you were hacked or if you DO find it, since it was hard to find, think yourself sooo smart it must be real…) and deflect any search elsewhere. IMHO, that’s the hardest to properly find. All the real evidence was erased, and what you are working from is the McGuffin. (Thing in a story line everyone is searching to find, that may not be real. See The Maltese Falcon as example.)

So what we know publicly about the investigation is that it was a postmortem, it found some forensic evidence, that evidence was an old Russian warz, and thus the conclusion is:
“Russia Did It!”

The flaws in this are many.

The BIGGEST flaw:

You don’t know how many hacks happened. It may well be that the Russians hacked in 6 or 8 years ago and have been sniffing data ever since. That does not at all prevent an Admin dumping a tape and leaking it. It does not at all prevent a Chinese team sucking out the data and erasing their tracks. It does not at all prevent an NSA guy from dropping a USB drive on Wikileaks. It does not at all prevent the local ISP Night Shift Operator, who is bored silly, from piping a router feed of email to their laptop as it goes by and collecting a set (though good ISPs have systems to prevent that). It does not at all prove that only Russia is to blame for the hack / leak, and not some Fat Bastard in the basement of his Mom’s house using downloaded Russian warz (commonly available) to do the hack.

Assigning the Data Public Dump to the Russian Hack is a leap of faith.

Assigning the hack with Russian Warz to Russia proper is a leap of faith.

Assigning the Data Exfiltration to the Russian Warz is a reasonable, but still, leap of faith.

Until and unless you get more evidence to support those leaps, you are stuck at this point right out the gate.

Further Flaws:

Now you continue to investigate. You use those leaps to direct your search for more information and evidence. But you simply can not depend on them.

At this point, in support of the Assertion Russia Did It: The NSA and other TLAs have access to pre-hacked from the factory PRISM program warz that we just can’t see. They may well have recorded all the data flow in transit from Hillary and the DNC servers as it went to Russia and just didn’t have the staff to look at it until now. (The problem with collecting everything is that you can look at nearly nothing…) They may well have hacked the hackers and have inside intel that they did it, but can’t divulge that publicly without compromising their hack. Welcome to “Spy vs Spy”. Yet you can NOT just take the word of the CIA or NSA. The CIA is by definition in the disinformation business. The NSA typically is in the ‘be silent’ business, but when they do speak, it is rarely unvarnished. (They once advised a better cypher box -grid of numbers – for DES IIRC – only years later was it found that this helped protect a then newly discovered attack algorithm. Basically the NSA new years before about that hack, but could not and would not tip the hand.)

So just at the first step, we can only know “The Russians likely were in the box and had the data too”. But can’t assign causality to the dump, and can’t even say for sure it wasn’t someone using known old Russian hacks that anyone can download from the internet. (Only tracking the traffic as it left would really tell you that. Finding a Russian assigned IP hard coded into the hack would only prove the hacker was leaving it as a false flag until they turn it on again and change the IP or that they pwned (took over) a router along the way and routed that IP to their box, likely via a VPN).

Now what we SEE in public is just the assertion that these old Russian warz left a ‘fingerprint’ on the machine, so it must be the Russians. Hardly convincing and at best a Noob level of analysis. I dearly hope they have a lot more real evidence behind the curtain and are just not sharing it as they aren’t stupid.

We don’t know that any Russian Destination for the hack was Russian Government. Now maybe the NSA has some way to show where the packets went, and can finger them, but the public intel doesn’t show that. Even if you believe a Russian IP address in the hack actually got to Russia, you can’t know where in Russia it went after that nor who was sitting at the terminal (nor if the router at that end routed the packets back to lower New Jersey…) That’s why you route hacks though places that are not cooperative with USA / EU TLAs. Russia doesn’t send the NSA their router log files.

As I pointed out before, I used a Romanian VPN to bounce my IP off of Romania just to get past geography restrictions on watching a news broadcast. Took me all of 5 minutes to set up, and on an Android Tablet no less. Not exactly hacker kit. Want to know who was watching that news? It points to Romania. Beyond that, you need to get a Romanian search warrant for their server and / or routers… except they advertise that they don’t keep logs… Now that’s for use by The General Public for things like watching TV. What do you think professional hackers do? Hmmm?

Were I doing this hack, I’d use the best tools available, then dirty the trail with evidence of someone else doing it (likely China, but their kit leaves little behind, so easier to do Russia) and route packets through a non-cooperating jurisdiction. Presently, that’s Asia… (Ecuador has promise, but not much traffic to hide in and they are ‘picky’ about foreigners…)

Think it is hard to get a Russian VPN? Here’s the result of a duckduckgo search. I’ve bolded one bit:

Russian VPN – Secure VPN Service For Russia | Golden Frog
Russian VPN Service Secure Your Connection With Russia’s Fastest VPN. Fast and easy VPN access from anywhere in the world; Unlimited server switching and IP addresses

[Search domain http://www.goldenfrog.com] goldenfrog.com/vyprvpn/russia-vpn
Russia VPN | Russian VPN Service to Get a Russia IP Address
Russian VPN service. In Russia and around the world, Internet users are becoming more aware of the risks they run every time they go online over an unsecured network.

[Search domain http://www.le-vpn.com] le-vpn.com/vpn-russia/
Russia VPN Service – Free And Paid Russian VPN For PC/Mac/iPhone
Using our Russia VPN service you can view sites that are available only for Russian IPs, unblock Skype, access poker/gambling/adult sites anonymously.

[Search domain http://www.supervpn.net] supervpn.net/russia-vpn.html
ThreatConnect follows Guccifer 2.0 to Russian VPN Service
ThreatConnect determines Guccifer 2.0 is using the Russia-based Elite VPN service to communicate and leak documents directly with the media.

[Search domain http://www.threatconnect.com] threatconnect.com/blog/guccifer-2-all-roads-lead-russia/
Russian VPN – get an IP address in Russia – Simple VPN service
Russian VPN. Connect to the Internet with an IP address in Russia. Access Russia-only sites and services from anywhere in the world.

[Search domain zenvpn.net] zenvpn.net/en/vpn-locations/russia/
Best Russia VPN Service – vpncoupons.com
If you are looking for a VPN service to use in Russia to unblock content from overseas or security measures, you can’t go wrong with the following choices:

So anyone can be “from Russia”, and it looks like Guccifer has been found to use them. So the guy we KNOW was arrested for doing the Hillary Hack, and how admitted to many many more, was known to use a Russian redirection node. Golly… (I’ll leave it for you to connect the dots…)

In short, the publicly available information about the Hillary and DNC “hacks” is grossly insufficient to finger the Russian Government. We have a confessed hacker for Hillary (and a list of others he did) who used a Russian cover. We know the hack warz used are publicly and widely available. We DON’T know what happened in real time. We DO know that postmortem forensics on the boxes will be subject to all sorts of False Flag efforts. We DO know that there could easily be (most likely were) many actors who gained access to the DNC and Hillary servers as they were wide open and known good attack targets.

We also know if the NSA has more, they are not going to talk.

So the Public simply can not know what actually happened and certainly can’t know who did it.

In Conclusion

Were I to guess:

I’d guess the Russian Government was “in” from the start, exfiltrated emails, but kept it all for themselves. That’s what they do.

I’d guess Guccifer got in, got caught, and spilled the beans about having hacked these folks.

I’d guess that China was in, leaving no fingerprints, as that is what they do and if Russia got in with old warz, China can walk in with their current stuff (that includes pre-hacking the firmware of some chips widely used. I’ve been part of a recovery effort on one such.)

I’d guess Israel was in, as they would be watching what China, Russia, and Hillary were doing.

I’d guess the NSA was recording it all, but looking elsewhere most of the time. I’d speculate that maybe one of their folks took a peek and was horrified and did a ‘leak’ as a very possible scenario.

That much is pretty much certain, but speculative. After that, there are many other actors who are “up for the job” and may have gathered intel from Hillary and / or the DNC. By their actions, the DNC and Hillary look to be owned by {The House Of Saud | Some Muslim Group}. So a modest speculation on a place to investigate would be their intelligence agencies. I’d also look for evidence the UK got in (though they would say nothing and leak nothing, but are very good and curious..) or that the EU got in (maybe why the DNC and Hillary work so hard to make us look like the EU and what Soros wants… he has enough money to buy the information with his coffee money for one day…)

In short, once you know a phish worked on the DNC and that Guccifer hacked Hillary, it means the door was wide open for folks who are actually good at hacking and they could stroll in whenever they wanted. And they wanted.

So I expect the reality is that somewhere in that long list of actual actors in the box, someone saw an opportunity or an offense, and acted on it. And, once again, what did they do?
They let the Truth be known.
Oh, the humanity… /sarc;

Subscribe to feed


About E.M.Smith

A technical managerial sort interested in things from Stonehenge to computer science. My present "hot buttons' are the mythology of Climate Change and ancient metrology; but things change...
This entry was posted in Political Current Events, Tech Bits and tagged , , , , , . Bookmark the permalink.

21 Responses to Hacks, Leaks, Investigations, Evidence – Leaps…

  1. philjourdan says:

    If the Russians did hack Hillary or the DNC, then the value of the data would be increased had she won. So their bets would have been to make her win! Countries do not hack useless data (which the DNC hacks now are).

    And WORM. Paper is too bulky, but yes, you need them on WORMs

  2. p.g.sharrow says:

    The latest from the talking heads about “Proof” that the Russians did it is that someone used a Russian language keyboard in the hack!…pg

  3. cdquarles says:

    On that usage, I am on the pedant bandwagon. If you’re talking about breaking into systems, you’re cracking them, not hacking them. Why dirty a good name for a good practice with the implications of wrong doing where none exist. Sheesh. If you’re getting people to leak stuff, that’s neither hacking (building a working computer system) or cracking (breaking into a computer system for nefarious purposes, like a safe cracker looking for stuff to steal).

    I ran across a wit the other day saying that the emails revealed couldn’t be proven to be real. Sure they could, but getting into message digests and checksums in order to get it across to them that the emails were real, was a lost cause. Sure, they could have been spoofed by someone inside with access to the relevant machines, but that is another issue. As I recall, none of the relevant parties ever offered up that defense. The emails are real and from the named parties’ machines.

  4. Larry Ledwick says:

    Interesting support for your thesis E.M.


    Looks like the CIA is trying to soften the meme that the Russian’s did it.
    “its more complicated than that”

  5. Graeme No.3 says:

    The obvious initial, if not believable, defence would be to claim that the e-mails weren’t genuine (while frantically removing any copies you can find). To adopt a “How dare anybody read our dirty mail?” approach immediately makes them all true, and subject to adverse interpretations.
    With Hilary’s campaign always struggling that attitude may well have strengthened the vote against her.
    There is also the question as to why someone didn’t check up on the security of Hilary’s communications given her known carelessness. philjourdan’s comment about those with the data being able to influence the (expected) incoming President doesn’t apply only to enemies.

  6. Larry Ledwick says:

    Related to electronic voting security from the UK


    I think the important issue that everyone is missing is that electronic systems don’t have reliable tamper proof logs like hard copy logs.

    Perhaps you could create one using something like block chain technology where the systems produce a check sum hash for each vote entry hashed with some unique value that can’t be easily spoofed (ie machine MAC address plus a one time pad code string sent from a central high security device and a high precision time string like NTP).

  7. p.g.sharrow says:

    The DNC complaint that the Russians hacked the vote count is just their cover story for the massive over count in certain precincts caused by their manipulations. Democrats always point at others to cover for their own transgressions…pg

  8. I watched Julian Assange being interviewed this evening. He bends over backwards to avoid compromising his sources but he did say that his information did not come from Russian government sources. While he did not say that Barack Obama lied………..he did say that BHO used “Lawyerly Language”.

    Given that nobody has ever shown Assange to be wrong about anything pertaining to these emails it seems likely that BHO is lying as usual.

  9. Pingback: Hacks, Leaks, Investigations, Evidence – Leaps… | gwfenimore

  10. E.M.Smith says:


    At the time, WORM was new and expensive, paper was common and cheap. Now it’s different ;-)


    It’s a constant battle. Choose to be accurate, but hard to understand, or work with the common usage and be somewhat inaccurate, but better communicated. I’ve spent a lot of time and effort looking for a good solution. My conclusion is that the language is driven by the average user, and the average user is of average or lower IQ and ability, and many of the smarter users just don’t care, so languages always decay.

    So I fight a rear guard action in retreat, and don’t expect to win that battle.

    @Graeme No.3:

    I think they had enough clue to know there were other folks with copies (or reply with copy or replay to forward with copy.. or) and it would only take one of them surfacing showing the email valid after shouting “FAKE!” to completely ruin them. Once that settles into the brain, folks don’t shout “FAKE!”…

    The sHillary was known to be rude, berate folks, and generally be a Royal PITA to work with. Folks like that don’t get any hand-holding or help above what is needed to avoid having yourself whacked (organizationally, not, oh, nevermind…)

    So who is going to step up to ‘help’ her? If I’m her I.T. guy, all I want to do is get out of the room as fast as possible and assigned to something else. Once she says she wants her own server, I’m seeing salvation, pass it on to the next schmooo.

    If I’m having her in my face, the last thing on my agenda is “Protect Hillary” and especially not “Protect her from herself” since you known all you get from it is insults, attacks, and derision. I’ve “been there, done that” with executives who didn’t like what the I.T. guy had to say. In many ways, that is why over the whole of the USA most companies have crappy security and use Microsoft. It’s the easy and cheap path and executives don’t get a bonus for having no problems on their watch. On more than one occasion I’ve seen the head I.T. guy get a promotion for ‘after action’ fixes and recovery from disaster while in my shop I had to fight to keep the staff that kept those things from happening in the first place. “No Problem” means “Cut Staffing” to most execs.

    So after you internalize that (and darned near every I.T. shop now has), the exec who wants to do something stupid causes you to think “How can I distance myself from this” rather than “How can I stop them” and almost never “How can I show them this is a bad idea”… I know a couple of high end I.T. guys who observed the same thing. Two of us have adopted the standard of “I will tell you 3 times, then comply”.

    1) That has issues because of {FOO}.

    2) I recommend not doing that because of {FOO}.

    3) You have instructed me to do that, I will do it. This is the 3rd time I’m going to say “That is a bad idea because of {FOO}” and you will not hear it from me again. I’m now going to build {BAR} as instructed. (AND you document the conversations…)

    Even that raises ire in some folks, so use judiciously and with the best suck-up submissive posture you can manage. With someone like Hillary, her staff will rapidly learn to NOT do that, and instead say “Here is {BAR} as you ordered! Sign the payment chit here please!” (and document the orders…)

    IMHO, that social aspect is THE main reason why the USA has crappy security, poor dominant operating system (Microsoft) for desktops, and PITA applications (mostly Microsoft). Oh, and a flood of H1B visa folks from a very suck-up prone culture…


    That point (zero wrong) is a huge one. Assange guards that strongly. They have some way of vetting the ‘leaks’ they get are real, and not bogus, but I can’t figure out what all it might include. Whatever it is, so far it has been perfect. (And you know bad actors will be desperately trying to spoof him, tarnish Wikileaks, or just put a bogus slam on someone else).

    His background was in computer security, so he knows the tricks… When he says the source wasn’t the Russian Government, he knows how to prove that.


    I’ve found it oddly effective at characterizing the inside of the Democrat Machine to just take their talking points du jour and swap [the target] out and {Democrat Machine} in.

    “Vast Right Wing Conspiracy” means “Vast DNC Conspiracy” (now in evidence via emails)
    “Racist Right Wing” means “Racist Democrats” (In evidence in their history and emails).
    etc. etc.

    Used a Cyrillic keyboard? How would they know? Unicode in the typing stream? That can come from any keyboard. (Just takes more escape sequences…). The WashPo assertion of Russian hack of the Electric Grid (now shown bogus and retracted) was traced back to a commercial Ukrainian software product. Guess what alphabet is used in Ukraine…

    And who else uses Cyrillic? And knows Russian? ANYONE in the Former Soviet Union and any country they dealt with a lot. ALL the TLAs of the world (CIA was big on it.) Heck, in the ’70s when in college and thinking of being a CIA guy (filled out the application but chickened out on sending it…) I even took a Russian class. Now faded to where I can barely make out “Протон.” Proton on the side of rockets or :Спасибо, что это хорошо.” (Thank you, that’s good.)

    It is as likely it was a Ukrainian working to tilt away from the diaster under Obama or a CIA guy acting on orders (or acting on his own…) as it is that the Russians would be involved, if based only on the use of Cyrillic.


    That guy has clue:

    Sir John Sawers told the BBC that casting a ballot with pencil and paper was “actually much more secure”.

    He warned: “The more things that go online, the more susceptible you are to cyber attacks.”

    My mantra: “If it doesn’t NEED to be online, don’t PUT it online.” -E.M.Smith

    Votes don’t need to be online. Paper, mark sense pencil. Full audit of EVERY election by a third party. (i.e. the government can run the paper through their counters, with observers from both sides, then the paper goes to an accountancy agency and run through their readers. The two vote tallies need to be inside the published error bounds of the machines or a recalibration of the machines is done and the paper counted by hand…)

    On the Woolsey quote: I always liked that guy. Smart and has his head on straight. I’d love to work for him, but I think he’s retired now. Very gratifying to see him reach the same conclusion (and he says the TLA insiders are whispering in his ear… so they likely have clue too, prior to the political management filter…)

  11. John P Miller says:

    Brilliant! What a great subject for you to combine your IT chops, ability to find/ collate great material, and story-telling skill! You really ought to send this to a media outlet (and to Trump..although you’d think he’s already gotten the equivalent from someone — he does have access to pretty smart folks). More people need to hear what “might have” happened.

    So ironic that Hillary got a “pass” from her own government on stupid IT tricks that really matter to national security, but then — no doubt by someone with that blatant failure of government to police itself in mind — gets taken down by stuff that’s venal and political, rather than of true national security concern. It’s beyond delicious…

  12. Larry Ledwick says:

    Interesting, apparently the FBI never asked for acess to the “hacked” DNC servers.
    Wouldn’t that be considered good practice in any major intrusion situation, to examine logs and server configurations etc.?


  13. E.M.Smith says:

    @J.P. Miller:

    Um, I think the media outlets can decide to follow here if they like what I write… I’m happy if they pick up a line of reasoning and use it. Similarly, Trump looks to have clue on this. Don’t know if it is from prior experiences securing his businesses (that I.T. budget item…) or someone whispering in his ear; but it looks like he knows this stuff already. Who knows, they might even be followers of my blog ;-)

    Still, thanks for the compliment and glad you found value in it.


    Yeah, just saw that on RT (what comes on the TV automatically this time of day for me… yes, I have the channel swapping automated ;-)

    Some company (CrowdStrike) was hired by the DNC and the FBI only asked for their report, not the hardware.

    This is just so wrong…

    Chain of custody lost.
    Independence lost.
    Methods and legally vetted procedures in doubt.
    3rd agency beholden to the client (DNC) determining outcome. (Validity lost).

    I know nothing about CrowdStrike and their reputation or skill level, but it is not a law enforcement agency and not subject to their forensic legal standards. They might be great and highly trusted, but then the hardware ought to have been sent over for confirmation of their findings.

    FWIW: In good forensic practice, the hardware is immediately quarantined.

    (IF you can, you do some dramatic things on first approaching the machine, like either not shutting it off while you do some explorations, or powerfail it immediately and stuff the memory chips into freezing liquid so you can extract the contents. That was likely not available here… so ‘moving on’…)

    The powered off cold hardware is taken apart. First step is a physical inspection for any added things inside. Is there any evidence of physical compromise. Then the hard disk is put into a special cloning rig. The disk is copied, bit by bit, to a duplicate. The firmware is extracted and examined. This isn’t your ‘copy the file system’ but is more like a “dd if=disk of=newdisk” that copies every bit on the device.

    From that point forward, that original disk goes into an evidence bag and is never touched again in this examination. It is kept pristine. Specifically so it can be handed over to TLAs with the statement “We made a clone, but no bits on this were changed”.

    Now on the clone, you start looking for all sorts of things. A list too long to put here; but it includes things like looking at the exact bits in the MBR (Master Boot Record). What is in the free list (blocks technically ‘unused’ but sometimes things are hidden in it). Is there a set of blocks NOT in the file system and NOT in the free list? Essentially every byte on that disk needs to be in the usual and customary place and holding the proper bits, or it is likely a compromise. Notice that NONE of this involves what you see as the file system contents. MANY hacks use things outside the regular file system as their own storage area…

    Now you also have someone looking AT the file system. That includes extracting and examining all the available log files AND comparing every executable binary or script with what is known to be the manufacturers version. Every non-executable is also examined. Was something hidden beyond the EOF (End Of File) marker used by Word, but inside the available data blocks? This continues until EVERY file system file has been examined and all apparent contents validated AND any non-apparent content checked.

    Somewhere along the line, you find something out of place or wrong. That gets documented and written up. Sometimes it is nothing, sometimes it is everything. But you do not know until the last of it is examined. Then comes the hard parts… figuring out which is true, which a false flag distractor, which is real but irrelevant (maybe they picked up a Nigerian Prince email but didn’t respond). Then finally work to some conclusions.

    All of this can take months, but you try to finish in weeks. At some point (sometimes even hours or days depending on the size of your team) you have enough to call in Law Enforcement. Usually not the Local LEOs (Law Enforcement Officers) but sometimes, if it is a big city with a separate cybercrimes unit. If it is Federal Crime related, the FBI is the usual contact. If it looks like it’s international spying, a call the the CIA or NSA can be in order (if Nation as actor and not clearly just commercial criminal) but usually that is left to the FBI to decide. They, then, look at your goods. IF they think it warrants a full criminal investigation, that quarantined and chain of custody marked disk is handed over and their forensic team re-does everything you did (plus anything they can do that you don’t know exists…)

    Now, did CrowdStrike do that? Do they have those procedures in place? Is their history and reputation impeccable? One hopes so…

    “But hope is not a strategy. -E.M.Smith”

    So the level of work of CrowdStrike now needs a big inspection by the FBI… to cement the case.


    I have nothing against 3rd party agencies. I worked for one in the I.T. Security Department of Charles Schwab. Our group did just that kind of analysis on a hard drive of one employee of Schwab when {something} infected their machine. We had Schwab badges, desks in the department, and did all our work on site in their San Francisco building. We reported to a Schwab department manager. BUT, our paycheck was from a company with a contract to Schwab, so technically we were a 3rd party. We had fairly strict forensic procedures and everything was documented. All hardware work was done in a special secured lab. Frankly, that’s part of why I’m happy to have my accounts at Schwab. They have very good security.


    So it could easily be that CrowdStrike is just as good, and almost certainly they are better than the DNC staff. Given that, hiring them was likely the right thing to do. BUT, when the investigation is punted to Law Enforcement, their ought to be a second investigation by them. Especially in something this big and important. Similarly, their reputation and procedures ought to be closely examined. Finally, was there a cloned / quarantined disk? If so, why didn’t it go to the confirming examination by Law Enforcement?

  14. cdquarles says:

    Exactly right, EM. In another life, I would on occasion have to collect evidence for law enforcement purposes. Given where I was, often I was the only person who could sign off on that evidential chain of custody, necessitating my sitting on it personally, doing nothing else, until the sheriff or his deputy arrived to take custody of it and sign off certifying that the chain was properly maintained (or not). [Yeah, that’s why some folk get paid big bucks (rhetorically in some cases), for their necks are on the line legally and held to a higher standard than certain politicians.]

  15. E.M.Smith says:


    Oh yeah… the “Availability Charge” and the “Responsibility Fee”… Been paid that many times.

    IIRC, at Schwab there was a special vault where the hardware could be placed, and it was monitored for access. When out of the vault, things (even the clone) had 2 people present. The doer and the observer. Things signed with “I did FOO” and “I observed FOO”… (very hard to arrange buying off two folks from different angles… doer and client staff…) When it was in the vault, you could take a potty break… and the systems did the watching. At other places, you sat there, looking at the “stuff” until it was handed over.

    I was once paid a Responsibility Fee for attending a killer party… The company needed a “Manager in Charge” (i.e. sucker to take legal liability) and I was elected, being lowest manager in the management food chain. Got paid to drink beer and dance and be liable if something bad happened…

    I was once paid full boat hourly rate (something like $100+/hr) for a 48 hour period to sit in the ops area of a datacenter while everyone else was at an offsite for the weekend. Most of it nothing happened. Had one disk fail and got to figure out their byzantine backup system, replace the disk, and restore. That broke the monotony… and delayed my nap ;-) The client was very happy to know “No Bad Thing Will Be An Issue.” for that duration.

    For Chiron (now gone, so I can discuss it…) I got paid to watch someone else work. It was an FDA “Qualified Installation” at an IBM CoLocation Facility near Denver. A weekend spent in a computer room, about every 10 minutes signing my name and noting date / time confirming that the installer had done what he signed / described he did above. “I pushed the red power-on button. Joe Admin. I saw it. Me….” But that’s what the FDA demanded. Drug testing data archive, so literally lives might depend on it. But mostly the FDA requires a written script an idiot can follow to build the same system to validate it and ANY step that doesn’t work perfectly on their copy is a fail for your $Billlions drug test data… so we had to painfully detail write up how to install a Network Appliance Server. Harder than you might think. You can’t say “Turn power on” for a button as buttons don’t turn, nor “push the red button” as in the example above, since they might change the color on the next one shipped… so you write things like “Apply power using the power on / off setting of the machine.” Yeah, Government regulations… The admin checks the box for ‘Apply power using…’ and both sign.

    While I love getting paid the Availability Charge and Responsibility Fee , the “work” is usually very dull and there is usually little joy in it. The perils of a high energy brain forced to run at near zero speed… I’m sure it is better now in the era of smart phones, tablets, and other gizmos… then again, when you are paid to be “eyes on FOO”, you must have eyes on FOO even if it just sits there doing nothing…

  16. Pingback: TT – 2 Weeks (and counting the days) | Musings from the Chiefio

  17. G. Combs says:

    E.M. Says “….I got paid to watch someone else work. It was an FDA “Qualified Installation”…”

    Brother does that bring back memories!

    I got to sign the ‘I watched FOO’ many many times in an FDA regulated facility and also got to write up the “Walk up to the XXX machine. On the right side as you….. turn yellow knob…..” All done in short sentences at the sixth grade level so anyone with a high school degree could easily follow the directions. I did that for an entire manufacturing process on more than one occasion.

  18. E.M.Smith says:

    @G. Combs:

    I feel your pain…. I never knew just how difficult it can be to do nothing until then. Days in a locked computer room doing nothing and you can’t leave and their is no e-signals (secure facility) and you can’t jack-in to the network and you Must Watch someone typing at a keyboard… Can’t even read a book as then you didn’t see what happened and can’t legitimately confirm it.

    Like watching paint dry for an hour, then checking the box that says “watched paint dry”…
    sign and date…

    Worse, I was the guy who had written up the procedure, so I was 100% aware of very single step, what it did, and what the result would be, as I had done it all in the lab to make the procedure… so even watching it wasn’t in any way novel… and I’d seen it all before a couple of times.

  19. Gail Combs says:

    E.M. Makes you kind of wonder about all the drugs made in China now….

    I got a batch of diphenhydramine (Benadryl) that made me throw-up and have diarrhea.

Comments are closed.