An early report on Suricata

I’ve gone ahead and installed Suricata (an IDS / IPS system that does packet inspection) onto the Debian Raspberry Pi chip I use as my Daily Driver.

Why?

Because I kind of stalled out on doing the config work for Snort on the Alpine based DMZ box, I wanted something running fast (partly as I’ve gone too long without one to be comfortable), and I wanted to play with Suricata already. Besides, it does work on the box that is your daily driver too, it doesn’t need to be installed as an ‘on the side’ packet inspector…

To install I just did the usual for Debian: “app-get install suricata”. It complained. I had to do a “app-get update” first, then it installed fine. Just accepting the defaults, it collects a lot of information. I’ve done NO tuning (yet). It has a forest of tunable parameters, so that is going to “take a while”.

I did run it with:

suricata -c /etc/suricata/suricata-debian.yaml -s /etc/suricata/rules/dns-events.rules -i eth0

And it is running fine at the moment. Now in this one line you can see a few interesting things. First off, the default configuration file that suricata expects is /etc/suricata/suricata.yaml yet here it is suricata-debian.yaml. Why the gratuitous name change? Who knows… But you either need to call out that path name, or copy the default file to suricata.yaml file.

Next note that there are some pre-made rule sets available in /etc/surciata/rules. I choose one that, from the name, looks like it would inspect DNS things (and right now I have the hots to cut back gratuitous DNS traffic to places I didn’t request… so I’m hoping this helps.) Finally, I specified which interface to inspect. There’s LOT of other things you can ‘call out’ or speicifiy or configure or… It will likely be days before I have a set I like, and it is auto-started as a service or daemon, and and and… thus this early posting 1/2 done config… What I do from here on out will diverge ever more from what “other folks” may want to config, and it is important to point out that “out of the box” it does something sort of useful, so no reason to wait to play with it.

When launched, it immediately started to complain about a BUNCH of missing rules files. I’d thought I was saying “just use the one” but who knows. This is yet another “Dig Here” for me this evening.

19/11/2016 -- 14:24:21 -  - This is Suricata version 2.0.7 RELEASE
19/11/2016 -- 14:24:22 -  - [ERRCODE: SC_ERR_OPENING_RULE_FILE(41)] - opening rule file /etc/suricata/rules/botcc.rules: No such file or directory.
19/11/2016 -- 14:24:22 -  - [ERRCODE: SC_ERR_OPENING_RULE_FILE(41)] - opening rule file /etc/suricata/rules/ciarmy.rules: No such file or directory.
19/11/2016 -- 14:24:22 -  - [ERRCODE: SC_ERR_OPENING_RULE_FILE(41)] - opening rule file /etc/suricata/rules/compromised.rules: No such file or directory.
19/11/2016 -- 14:24:22 -  - [ERRCODE: SC_ERR_OPENING_RULE_FILE(41)] - opening rule file /etc/suricata/rules/drop.rules: No such file or directory.
19/11/2016 -- 14:24:22 -  - [ERRCODE: SC_ERR_OPENING_RULE_FILE(41)] - opening rule file /etc/suricata/rules/dshield.rules: No such file or directory.
19/11/2016 -- 14:24:22 -  - [ERRCODE: SC_ERR_OPENING_RULE_FILE(41)] - opening rule file /etc/suricata/rules/emerging-activex.rules: No such file or directory.
19/11/2016 -- 14:24:22 -  - [ERRCODE: SC_ERR_OPENING_RULE_FILE(41)] - opening rule file /etc/suricata/rules/emerging-attack_response.rules: No such file or directory.
19/11/2016 -- 14:24:22 -  - [ERRCODE: SC_ERR_OPENING_RULE_FILE(41)] - opening rule file /etc/suricata/rules/emerging-chat.rules: No such file or directory.
19/11/2016 -- 14:24:22 -  - [ERRCODE: SC_ERR_OPENING_RULE_FILE(41)] - opening rule file /etc/suricata/rules/emerging-current_events.rules: No such file or directory.
19/11/2016 -- 14:24:22 -  - [ERRCODE: SC_ERR_OPENING_RULE_FILE(41)] - opening rule file /etc/suricata/rules/emerging-dns.rules: No such ... 
[...]

There are places with loads of pre-made rule sets, so I likely just need to find them and do a big download, not make them all from scratch.

What is there out of the box?

root@R_Pi_DebJ_DD:/# ls -l /etc/suricata/rules
total 56
-rw-r--r-- 1 root root 13512 Mar  4  2015 decoder-events.rules
-rw-r--r-- 1 root root  1498 Mar  4  2015 dns-events.rules
-rw-r--r-- 1 root root  2872 Mar  4  2015 files.rules
-rw-r--r-- 1 root root  8339 Mar  4  2015 http-events.rules
-rw-r--r-- 1 root root  2380 Mar  4  2015 smtp-events.rules
-rw-r--r-- 1 root root 11879 Mar  4  2015 stream-events.rules
-rw-r--r-- 1 root root  4084 Mar  4  2015 tls-events.rules

So either I need to find all those other rule sets somewhere and add them, or figure out how to say “don’t look for those”, or both. These rules will likely keep me busy inspecting packets for a good while.

Now further down, there are more error messages. These are about duplicate rules, so I’m suspicious that there is some flag to say “Do Not Take The Default Rules” that I’ve left out, and that my specific call out of the DNS rules is a duplicate of the defaults. Note to self: RTFM… (Read The, ah, Friendly Manual) … on Suricata and figure out what command to really issue…

19/11/2016 -- 14:24:22 -  - [ERRCODE: SC_ERR_DUPLICATE_SIG(176)] - Duplicate signature "alert dns any any -> any any (msg:"SURICATA DNS Unsollicited response"; flow:to_client; app-layer-event:dns.unsollicited_response; sid:2240001; rev:1;)"
19/11/2016 -- 14:24:22 -  - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert dns any any -> any any (msg:"SURICATA DNS Unsollicited response"; flow:to_client; app-layer-event:dns.unsollicited_response; sid:2240001; rev:1;)" from file /etc/suricata/rules/dns-events.rules at line 2
19/11/2016 -- 14:24:22 -  - [ERRCODE: SC_ERR_DUPLICATE_SIG(176)] - Duplicate signature "alert dns any any -> any any (msg:"SURICATA DNS malformed request data"; flow:to_client; app-layer-event:dns.malformed_data; sid:2240002; rev:1;)"
19/11/2016 -- 14:24:22 -  - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert dns any any -> any any (msg:"SURICATA DNS malformed request data"; flow:to_client; app-layer-event:dns.malformed_data; sid:2240002; rev:1;)" from file /etc/suricata/rules/dns-events.rules at line 4
[...]
19/11/2016 -- 14:24:22 -  - [ERRCODE: SC_ERR_NO_RULES(42)] - No rules loaded from /etc/suricata/rules/dns-events.rules
19/11/2016 -- 14:24:22 -  - [ERRCODE: SC_ERR_PCAP_CREATE(21)] - Using Pcap capture with GRO or LRO activated can lead to capture problems.
19/11/2016 -- 14:24:22 -  - all 7 packet processing threads, 3 management threads initialized, engine started.

Note at the bottom dns-events.rules had no rules loaded, which further supports the notion that I’ve duplicated a default; then it starts with 7 threads for packets and 3 for management. Probably way more than needed on a small Pi board…

So the rest of the evening will be spent figuring out what config options I want in the config file and what command options I want and how to invoke the command (or many copies if it?) ongoing… Tuning, configuration, and administration.

The output all goes into the /var/log area, so that needs to be somewhere with some space.

root@R_Pi_DebJ_DD:/var/log/suricata# ls -l
total 1000
-rw-r--r-- 1 root root 252127 Nov 19 14:43 eve.json
-rw-r--r-- 1 root root  78354 Nov 19 14:24 fast.log
-rw-r--r-- 1 root root   2922 Nov 19 14:27 http.log
-rw-r--r-- 1 root root 578678 Nov 19 14:43 stats.log
-rw-r--r-- 1 root root  99786 Nov 19 14:24 unified2.alert.1479594262

Already headed for a MB and I’ve only just launched it a couple of minutes ago on a system with an idle browser open and not much else going on.

What’s the stuff in there look like?

Here’s the smallest one:

root@R_Pi_DebJ_DD:/var/log/suricata# cat http.log
11/19/2016-14:24:58.389413 sr.symcd.com [**] / [**] Mozilla/5.0 (X11; Linux armv7l; rv:38.0) Gecko/20100101 Firefox/38.0 Iceweasel/38.7.1 [**] 10.1.1.13:55394 -> 23.5.251.27:80
11/19/2016-14:26:02.789491 weather.unisys.com [**] /gfs/gfs.php?inv=0&plot=1000&region=eu&t=9p [**] Mozilla/5.0 (X11; Linux armv7l; rv:38.0) Gecko/20100101 Firefox/38.0 Iceweasel/38.7.1 [**] 10.1.1.13:53909 -> 50.206.172.197:80
11/19/2016-14:26:02.900399 weather.unisys.com [**] /css/WMAX.v1.0.2.css [**] Mozilla/5.0 (X11; Linux armv7l; rv:38.0) Gecko/20100101 Firefox/38.0 Iceweasel/38.7.1 [**] 10.1.1.13:53910 -> 50.206.172.197:80
11/19/2016-14:26:02.881379 weather.unisys.com [**] /css/amstabs.css [**] Mozilla/5.0 (X11; Linux armv7l; rv:38.0) Gecko/20100101 Firefox/38.0 Iceweasel/38.7.1 [**] 10.1.1.13:53911 -> 50.206.172.197:80
11/19/2016-14:26:02.930470 weather.unisys.com [**] /javascript/animatedcollapse.js [**] Mozilla/5.0 (X11; Linux armv7l; rv:38.0) Gecko/20100101 Firefox/38.0 Iceweasel/38.7.1 [**] 10.1.1.13:53913 -> 50.206.172.197:80
11/19/2016-14:26:03.001182 weather.unisys.com [**] /images/unisys_logo_syt_2016.png [**] Mozilla/5.0 (X11; Linux armv7l; rv:38.0) Gecko/20100101 Firefox/38.0 Iceweasel/38.7.1 [**] 10.1.1.13:53914 -> 50.206.172.197:80
11/19/2016-14:26:03.110205 weather.unisys.com [**] /images/setupbutton-blue.png [**] Mozilla/5.0 (X11; Linux armv7l; rv:38.0) Gecko/20100101 Firefox/38.0 Iceweasel/38.7.1 [**] 10.1.1.13:53917 -> 50.206.172.197:80
11/19/2016-14:26:03.123673 weather.unisys.com [**] /images/WRN_Ambassador_logo_small.png [**] Mozilla/5.0 (X11; Linux armv7l; rv:38.0) Gecko/20100101 Firefox/38.0 Iceweasel/38.7.1 [**] 10.1.1.13:53916 -> 50.206.172.197:80
11/19/2016-14:26:03.192461 weather.unisys.com [**] /javascript/jquery.min.js [**] Mozilla/5.0 (X11; Linux armv7l; rv:38.0) Gecko/20100101 Firefox/38.0 Iceweasel/38.7.1 [**] 10.1.1.13:53912 -> 50.206.172.197:80
11/19/2016-14:26:03.000360 weather.unisys.com [**] /images/gobutton-blue.png [**] Mozilla/5.0 (X11; Linux armv7l; rv:38.0) Gecko/20100101 Firefox/38.0 Iceweasel/38.7.1 [**] 10.1.1.13:53915 -> 50.206.172.197:80
11/19/2016-14:26:04.133119 weather.unisys.com [**] /gfs/9panel/gfs_1000_9panel_eur.gif [**] Mozilla/5.0 (X11; Linux armv7l; rv:38.0) Gecko/20100101 Firefox/38.0 Iceweasel/38.7.1 [**] 10.1.1.13:53918 -> 50.206.172.197:80
11/19/2016-14:26:04.741269 weather.unisys.com [**] /favicon.ico [**] Mozilla/5.0 (X11; Linux armv7l; rv:38.0) Gecko/20100101 Firefox/38.0 Iceweasel/38.7.1 [**] 10.1.1.13:53919 -> 50.206.172.197:80
11/19/2016-14:27:51.602170 ocsp.digicert.com [**] / [**] Mozilla/5.0 (X11; Linux armv7l; rv:38.0) Gecko/20100101 Firefox/38.0 Iceweasel/38.7.1 [**] 10.1.1.13:47224 -> 72.21.91.29:80
11/19/2016-14:27:51.882760 ocsp.digicert.com [**] / [**] Mozilla/5.0 (X11; Linux armv7l; rv:38.0) Gecko/20100101 Firefox/38.0 Iceweasel/38.7.1 [**] 10.1.1.13:47224 -> 72.21.91.29:80
root@R_Pi_DebJ_DD:/var/log/suricata# 

Does any of this matter? Likely not too much. There are some things I’d wonder about and want to ask myself if I wanted to block it, whatever it is. Most of it is from weather.unisys.com (I’ve bolded one of them) and is because I have opened a page with a bunch of images on it at http://weather.unisys.com/gfs/gfs.php?inv=0&plot=1000&region=eu&t=9p

Similarly the digicert (also bolded) is likely just checking than an https: cert is valid.

But that top line has “sr.symcd.com” and I have no idea who or what they are, so needs some investigation to decide “I asked for this in the page and want it” vs “this is crap and put a block in the DNS server”.

http://sb.symcd.com.ipaddress.com/

shows an address at Akamai

We found that the organization hosting sb.Symcd.com is Akamai Technologies in Cambridge, Massachusetts, United States.

A more detailed IP address report for sb.Symcd.com is below. At the time you pulled this report, the IP of sb.Symcd.com is 23.61.187.27 and is located in the time zone of America/New_York. The context of sb.Symcd.com is “Symcd” and could reflect the theme of the content available on the resource. More IP details of sb.Symcd.com are shown below along with a map location.

IP Address of Symcd is 23.61.187.27
Hostname:	sb.symcd.com
IP Address:	23.61.187.27
Host of this IP:	a23-61-187-27.deploy.static.akamaitechnologies.com
Organization:	Akamai Technologies
ISP/Hosting:	Akamai Technologies
Updated:	11/19/2016 11:56 AM
City:	Cambridge
Country:	United States
State:	Massachusetts
Postal Code:	02142
Timezone:	America/New_York
Local Time:	11/19/2016 05:54 PM

Now since they are a web cache service and cache all sorts of companies web pages for faster delivery web-wide, there are a normal and legitimate thing. So I’m not interested in blocking or diverting them. But at least now I know my system IS talking to them…

Believe it or not, you get to do something like that process for Every Single Thing that shows up and isn’t clear what it is doing or why it showed up. It can be a full time job… (Now you know why I’ve been slow about setting one of these things up… it is the start of the workload, not the end…)

There’s a nice helper package that sorts a lot of this out, called ‘barnyard’, that I’ve not installed yet. It’s next on my install list. I think it sorts through the “unified” alert file that’s binary data:

root@R_Pi_DebJ_DD:/var/log/suricata# file unified2.alert.1479594262 
unified2.alert.1479594262: data

The file fast.log is full of these:

root@R_Pi_DebJ_DD:/var/log/suricata# more fast.log
11/19/2016-14:24:44.645879  [**] [1:2200075:1] SURICATA UDPv4 invalid checksum [**] [Classification: (null)] [Priority: 3] {UDP} 10.1.1.
213:64294 -> 192.168.1.1:53
11/19/2016-14:24:44.674654  [**] [1:2200074:1] SURICATA TCPv4 invalid checksum [**] [Classification: (null)] [Priority: 3] {TCP} 10.1.1.
213:56508 -> 50.18.192.250:443
11/19/2016-14:24:44.701003  [**] [1:2200074:1] SURICATA TCPv4 invalid checksum [**] [Classification: (null)] [Priority: 3] {TCP} 10.1.1.
213:56508 -> 50.18.192.250:443

To me, it looks like something is making garbage packets (invalid checksum). Who is it at 50.18.192.250?

root@R_Pi_DebJ_DD:/var/log/suricata#  nslookup 50.18.192.250
Server:		127.0.0.1
Address:	127.0.0.1#53

Non-authoritative answer:
250.192.18.50.in-addr.arpa	name = ec2-50-18-192-250.us-west-1.compute.amazonaws.com.

Hmmmm… Amazon “aws”.com that AWS is Amazon Web Services IIRC and is their cloud server farm. Likely I’ve got an advert somewhere (probably from Amazon itself) or cookie beacon that’s making invalid packets (or some valid page is running on AWS? Wonder if WordPress runs on AWS?…) That’s particularly interesting since the only pages I have open now are a management window into my interior router (that I’m on…) and 6 panels of my WordPress site (where I don’t have ads on them…) 3 of them relate to comments and making this posting. Two others are “boiler plate” like the TV link. One is the Hillary thread. Perhaps something in one of the comments on it?

root@R_Pi_DebJ_DD:/var/log/suricata# grep -v checksum fast.log 
root@R_Pi_DebJ_DD:/var/log/suricata#

So looking for lines without checksum in them is a null result. OK…

Coming back a bit later, there’s another interesting one shows up:

11/19/2016-15:27:38.850012  [**] [1:2200029:1] SURICATA ICMPv6 unknown type [**] [Classification: (null)] [Priority: 3] {IPv6-ICMP} fe80:0000:0000:0000:0489:f034:d3aa:2639:143 -> ff02:0000:0000:0000:0000:0000:0000:0016:0
11/19/2016-15:27:38.850012  [**] [1:2200094:1] SURICATA zero length padN option [**] [Classification: (null)] [Priority: 3] {IPv6-ICMP} fe80:0000:0000:0000:0489:f034:d3aa:2639:143 -> ff02:0000:0000:0000:0000:0000:0000:0016:0

All the more interesting as I’ve got IPv6 shut off on my network… (Perhaps it is the spouse on her Mac… or maybe the Pi is trying to run IPv6 while the network doesn’t… At any rate, some IPv6 thing needs to be found and told to shut up…)

Moving on…

The eve.json file has what looks like a log of packets in it. One is the same as the Akamai IP. Another looks like it is going to the RPi DNS server, but on an odd port.:

{"timestamp":"2016-11-19T14:24:44.729821","event_type":"alert","src_ip":"10.1.1.13","src_port":56508,"dest_ip":"50.18.192.250","dest_po
rt":443,"proto":"TCP","alert":{"action":"allowed","gid":1,"signature_id":2200074,"rev":1,"signature":"SURICATA TCPv4 invalid checksum","ca
tegory":"","severity":3}}
{"timestamp":"2016-11-19T14:24:44.627565","event_type":"alert","src_ip":"10.1.1.13"," src_port":52305,"dest_ip":"192.168.1.1","dest_po
rt":53,"proto":"UDP","alert":{"action":"allowed","gid":1,"signature_id":2200075,"rev":1,"signature":"SURICATA UDPv4 invalid checksum","cat
egory":"","severity":3}}
{"timestamp":"2016-11-19T14:24:44.627565","event_type":"dns","src_ip":"10.1.1.13","src_port":52305,"dest_ip":"192.168.1.1","dest_port
":53,"proto":"UDP","dns":{"type":"query","id":23074,"rrname":"duckduckgo.com","rrtype":"A"}}
{"timestamp":"2016-11-19T14:24:44.627565","event_type":"dns","src_ip":"10.1.1.13","src_port":52305,"dest_ip":"192.168.1.1","dest_port
":53,"proto":"UDP","dns":{"type":"answer","id":23074,"rrname":"duckduckgo.com","rrtype":"A","ttl":84,"rdata":"54.215.176.19"}}

Checking on that UDP (no need for reply – hope it gets there no retransmit) type packet shows it to be a time stamp error packet:

https://redmine.openinfosecfoundation.org/issues/1715

Updated by Marcel de Groot 9 months ago

I’m also seeing this. Strange thing though:
On two machines (Debian Stretch), both with the same Suricata 3.0 compiled from source with NFQUEUE enabled, and the same kernel, also compiled from source (4.5.0-rc5), one writes the timestamp correctly and the other not:

On the one machine where this symptom plays Suricata is listening via iptables mangle on the FORWARD chain
On the other machine it listens via NFQUEUE in the mangle table on the INPUT and OUTPUT chain. Here the timestamps are correct.

00.000000 [**] [1:2200075:1] SURICATA …etc
vs
02/25/2016-14:31:10.631435 [**] [1:2522390:2498] …etc

I’ll try to check whether changing the NFQUEUE entry makes a difference.

So perhaps the “usual” issue of Time being a pill on the Pi as it has no hardware clock, or maybe I’ve got another option to set right for it…

Only one file left, the Stats.log file. Boy that sucker is growing fast.

-rw-r--r-- 1 root root 2604875 Nov 19 15:49 stats.log

2.6 Meg already… Wonder what’s in it (and how to prune it back…)

-------------------------------------------------------------------
Date: 11/19/2016 -- 14:24:30 (uptime: 0d, 00h 00m 09s)
-------------------------------------------------------------------
Counter                   | TM Name                   | Value
-------------------------------------------------------------------
capture.kernel_packets    | RxPcapeth01               | 0
capture.kernel_drops      | RxPcapeth01               | 0
capture.kernel_ifdrops    | RxPcapeth01               | 0
dns.memuse                | RxPcapeth01               | 0
dns.memcap_state          | RxPcapeth01               | 0
dns.memcap_global         | RxPcapeth01               | 0
decoder.pkts              | RxPcapeth01               | 0
decoder.bytes             | RxPcapeth01               | 0
decoder.invalid           | RxPcapeth01               | 0
decoder.ipv4              | RxPcapeth01               | 0
decoder.ipv6              | RxPcapeth01               | 0
decoder.ethernet          | RxPcapeth01               | 0
decoder.raw               | RxPcapeth01               | 0
decoder.sll               | RxPcapeth01               | 0
decoder.tcp               | RxPcapeth01               | 0
decoder.udp               | RxPcapeth01               | 0
decoder.sctp              | RxPcapeth01               | 0
decoder.icmpv4            | RxPcapeth01               | 0
decoder.icmpv6            | RxPcapeth01               | 0
decoder.ppp               | RxPcapeth01               | 0
decoder.pppoe             | RxPcapeth01               | 0
decoder.gre               | RxPcapeth01               | 0
decoder.vlan              | RxPcapeth01               | 0
decoder.vlan_qinq         | RxPcapeth01               | 0
decoder.teredo            | RxPcapeth01               | 0
decoder.ipv4_in_ipv6      | RxPcapeth01               | 0
decoder.ipv6_in_ipv6      | RxPcapeth01               | 0
decoder.avg_pkt_size      | RxPcapeth01               | 0
decoder.max_pkt_size      | RxPcapeth01               | 0
defrag.ipv4.fragments     | RxPcapeth01               | 0
defrag.ipv4.reassembled   | RxPcapeth01               | 0
defrag.ipv4.timeouts      | RxPcapeth01               | 0
defrag.ipv6.fragments     | RxPcapeth01               | 0
defrag.ipv6.reassembled   | RxPcapeth01               | 0
defrag.ipv6.timeouts      | RxPcapeth01               | 0
defrag.max_frag_hits      | RxPcapeth01               | 0
[...]
-------------------------------------------------------------------
Date: 11/19/2016 -- 15:51:50 (uptime: 0d, 01h 27m 29s)
-------------------------------------------------------------------
Counter                   | TM Name                   | Value
-------------------------------------------------------------------
capture.kernel_packets    | RxPcapeth01               | 30614
capture.kernel_drops      | RxPcapeth01               | 0
capture.kernel_ifdrops    | RxPcapeth01               | 0
dns.memuse                | RxPcapeth01               | 5779
dns.memcap_state          | RxPcapeth01               | 0
dns.memcap_global         | RxPcapeth01               | 0
decoder.pkts              | RxPcapeth01               | 30614
decoder.bytes             | RxPcapeth01               | 13760162
decoder.invalid           | RxPcapeth01               | 0
decoder.ipv4              | RxPcapeth01               | 30497
decoder.ipv6              | RxPcapeth01               | 13
decoder.ethernet          | RxPcapeth01               | 30614
decoder.raw               | RxPcapeth01               | 0
decoder.sll               | RxPcapeth01               | 0
decoder.tcp               | RxPcapeth01               | 27484
decoder.udp               | RxPcapeth01               | 2980
decoder.sctp              | RxPcapeth01               | 0
decoder.icmpv4            | RxPcapeth01               | 28
decoder.icmpv6            | RxPcapeth01               | 12
decoder.ppp               | RxPcapeth01               | 0
decoder.pppoe             | RxPcapeth01               | 0
decoder.gre               | RxPcapeth01               | 0
decoder.vlan              | RxPcapeth01               | 0
decoder.vlan_qinq         | RxPcapeth01               | 0
decoder.teredo            | RxPcapeth01               | 1
decoder.ipv4_in_ipv6      | RxPcapeth01               | 0
decoder.ipv6_in_ipv6      | RxPcapeth01               | 0
decoder.avg_pkt_size      | RxPcapeth01               | 449
decoder.max_pkt_size      | RxPcapeth01               | 1514
defrag.ipv4.fragments     | RxPcapeth01               | 0
defrag.ipv4.reassembled   | RxPcapeth01               | 0
defrag.ipv4.timeouts      | RxPcapeth01               | 0
defrag.ipv6.fragments     | RxPcapeth01               | 0
defrag.ipv6.reassembled   | RxPcapeth01               | 0
defrag.ipv6.timeouts      | RxPcapeth01               | 0
defrag.max_frag_hits      | RxPcapeth01               | 0
tcp.sessions              | Detect                    | 531
tcp.ssn_memcap_drop       | Detect                    | 0
tcp.pseudo                | Detect                    | 106
tcp.invalid_checksum      | Detect                    | 445
tcp.no_flow               | Detect                    | 0
tcp.reused_ssn            | Detect                    | 0
tcp.memuse                | Detect                    | 10028632
tcp.syn                   | Detect                    | 531
tcp.synack                | Detect                    | 481
tcp.rst                   | Detect                    | 384
dns.memuse                | Detect                    | 0
dns.memcap_state          | Detect                    | 0
dns.memcap_global         | Detect                    | 0
tcp.segment_memcap_drop   | Detect                    | 0
tcp.stream_depth_reached  | Detect                    | 0
tcp.reassembly_memuse     | Detect                    | 73457184
tcp.reassembly_gap        | Detect                    | 0
http.memuse               | Detect                    | 725196
http.memcap               | Detect                    | 0
detect.alert              | Detect                    | 470
flow_mgr.closed_pruned    | FlowManagerThread         | 545
flow_mgr.new_pruned       | FlowManagerThread         | 206
flow_mgr.est_pruned       | FlowManagerThread         | 1161
flow.memuse               | FlowManagerThread         | 6205704
flow.spare                | FlowManagerThread         | 10000
flow.emerg_mode_entered   | FlowManagerThread         | 0
flow.emerg_mode_over      | FlowManagerThread         | 0

Golly, looks like one of these status reports every 15 seconds or so… I think I need to find how to shut that off unless I’m curious…

The End

Well, that’s what I’ve got so far. Lots of stuff to sort through.

Hopefully this gives you an idea what kind of information you can get out of a system like this. All kinds of traffic analysis on what’s talking, to whom, and even some ideas about why.

Lots of work goes into just pruning out most of that information so only the “important bits” flow to the display point. I think the program named ‘barnyard’ does that. This “how to” guide installs all sorts of things (including barnyard and mysql and more…) so clearly more is expected. IIRC, barnyard takes the suricata output and puts it into the mysql database for easier report and alert preparation.

https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Suricata_Snorby_and_Barnyard2_set_up_guide

For me, what all this says is that I need to put the Suricata Logs and active areas onto a patch of ‘real USB disk’ and not on my SD card. First, it will fill the SD chip way too fast. Second, the wear rate on those bits will be mighty high. Third, it will take a performance hit from many small writes where an SD card really does a giant ‘write and refresh’ for each of them.

So, with that, I’m going to shut down Suricata, move the log location to a patch of Real Disk, and study up some on the options at launch and in the config file. (Probaly a day or three in the future as Sunday is not my most productive day). Heck, I might even read some of the manual pages and the install and configure guide; now that I’ve installed it and played with it a bit. ;-)

With that, I now return you to the world of political chaos and disorderly minds that is the bulk of the world…

Subscribe to feed

Posted in Tech Bits | Tagged , , , , , | 7 Comments

What Now In Climate Wars?

This is just a very short bleat and to some extent lament about future unknowns.

I was pondering “what to post next” and thinking about how the whole Global Warming thing had gone from “New, What-Is-It Shiny Thing!” to “Disappointing abuse for asking rational questions” when I first ran into the Warmer Advocacy sites, to the joy of discovering WUWT (partly brought on by the 10 year anniversary posting…) which lead to doing a bunch of number crunching computer work (a lot on GIStemp), which resulted in this blog…

THEN, just a year or so back having the realization that “It isn’t about the Science” and that the Science was just window dressing on a Political Agenda. That only a fool brings a science knife to a political gun fight… and I started doing some of my first politically oriented postings…

To now with a Trump win and BREXIT and the world seems to have made the right political choices.

Where to now?

All of which leaves me wondering: Where to now?

Is the science still relevant enough to “do more” on it? Is the number crunching model / data analysis really of any remaining value? With the politics likely resolved for 4, perhaps (hope hope) 8 years, what relevance does flogging it have now?

I know, I know, “They” will not go quietly into that good night, so neither ought we…

But still, I’m in a contemplative mood, contemplating what fork to take next, and finding only shadows down the various paths…

Perhaps a wander down “memory lane” with a book about “What is wrong with the temperatures as Global Warming indicator?” that catalogs all the things wrong from thermometers to math to analysis to models?

Perhaps a book about resources and how they do NOT ever run out?

Perhaps just ongoing day by day blogging of “what happened today” even if not so important? (Bringing up an intrusion detection system on a small computer is fun for me, but really not so much for most folks…)

Wait for the “other side” to start their “reactionary response” and react to the reaction?

Frankly, I’d love to just “Declare Victory and go home”, but I’m already AT home! ;-) And somehow I doubt the global cabal sucking down $Billions in Climate Cash will just walk away quietly and fly off to their Swiss Bank Account…

So, in the end, I’m left with: What now? Like that moment at a great party when you notice the punch bowl is 3/4 empty, the band is taking a long break, and the crowd has thinned out a bit. Is it time to think of “what now?”, or just wait until the bowl is refilled and the band comes back from break?…

I’ve looked at doing a Climate Model port to a small cluster (The Average Joe DIY Climate Station), but what’s the point if they are “known to not work” and Trump is busy putting them out of business anyway? Similarly, I could spin a “GIStemp on a chip” for the Raspberry Pi in about 5 hours of work; but who would want it? (I actually ran it once on the Pi as a proof of concept, but that was a couple of years ago so I’d likely need to do a re-port of the newest code). Does any of that matter? In Hillary’s terminology: “At this point, what difference does it make?”…

Kind of like the dog chasing cars and biting at the tires: Once you catch one, what do you do with it? Well, Trump is in, and he’s a skeptic looking for places to cut costs. I suspect “right quick” NASA will be getting back to space issues and NOAA will be looking more at “weather this week” than “unknown catastrophe in some future century”… Teeth are in the tire, now what?

Well, I’m open to suggestions. Just don’t go all worried on me. Not planning to change anything any time soon. Just wondering which oar to pull on the most and what direction is land… and where did I put that little brown jug of grog? 9-)

While Contemplating

Here’s a little mood music for while you all are contemplating.

Dedicated To Madam Hillary, Hillary dear, Are you leaving soon?

Subscribe to feed

Posted in AGW and GIStemp Issues, Human Interest | Tagged , , , , , | 62 Comments

Canned Crackers? Dry Canning? Say what?

I’m pretty well versed in some things. Storing food for an emergency is one of them. Dad came from an Amish / Irish mix family, on a farm, so lots of DIY food prep and storage. I grew up in a Mormon town, so lots of cultural emphasis on having a year of stored food in case of crop failure (since they had almost died out in one such famine year after heading to Utah, IIRC the story). Folks canning, salting, drying, freezing, etc. etc. just about everything.

Given that, it is a bit unusual for something to surprise me. This did, it’s a ‘bit new’ to me… You can can crackers.

Seems there’s a technique called “Dry Canning”. Not “water bath”. Not “steam pressure”. Just dry.

FWIW I’ve stored dry goods like grains and crackers in jars for many years. Part of my “Quake Kit”. One “dry pound” of food per person per day is about right. Think it will take a month for ‘relief’ post disaster? Family of 4? You need 30 x 4 = 120 lbs of dry food. Beans, rice, sugar, flour, whatever. If storing ‘wet’ food, like canned chili, canned stew, or frozen meat, it will take about 4 to 8 times that much (depending on how much of the food is water).

The best way I’d found “so far” was in 1/2 gallon Mason jars with a small vacuum pulled via a device made mostly for freezing food in vacuum pouches. I have one of these things that came with an adapter to suck the air out of canning jars:

http://www.foodsaver.com/

Dry goods stored that way last longer than in ‘just air’. Not enough for me to regularly use it since things rarely stay in storage for ‘a few years’ to have the staling show up, but in testing it was significant. (IIRC, it was about year 4? that it was different for noodles and such. Salt and sugar just don’t care ever…) It is just a plastic cover over the regular canning lid that sucks the air out, then when you remove it, the usual ‘suck down’ of the canning lid happens and you tighten the ring.

Well, seems there’s another way. Drive the air and moisture out of the food in the jar by heating in your oven (and kill a lot of bugs / bacteria / whatever too) then tighten the lid down. Let it cool to pull a partial vacuum.

Well Duh…

I hate that “well duh” moment… OTOH I really like learning new tricks… I guess overall it’s a win ;-)

Ran into it here:

(Had a polite popup, but seems to go away on reload, so I’m not seeing one now)
http://louspennypincherspantry.blogspot.com/2011/04/canning-crackers-say-what.html

THURSDAY, APRIL 7, 2011
Canning Crackers! Say What?

A couple of days ago, a friend dropped by and I was busy canning crackers. Say what? That was exactly her reaction. They had a big sale on Premium Saltines and I stocked up. Before I share with you the easy method for preserving food this way, let me introduce to you how and why I know such things.

Well, that was my reaction too!

As long as I can remember, I have been preserving and raising food. As a child, I spent long hours with my mother and grandmother picking, snapping and canning beans and lots of other things too. As a member of the Church of Jesus Christ of Latter Day Saints, I have been taught that is both frugal and provident to store and preserve food, water and other commodities for times of trial.

Golly, similar background too ;-)

[…]
Dry pack canning is the method I used for the crackers. Living in the Heart of Dixie has it’s perks, but heat and humidity are not among them and items like crackers have a very brief shelf life in this climate.

My first experience with dry packing in my oven was when I found oyster crackers on sale a few years ago. We opened up the first jar about a year later and they were as crispy and crunchy as if I had just opened the bag.

OK, has done duration testing. Good.

You can also preserve any other type of cracker and most cookies using this method. Other food items that dry pack well include: pasta, grits, cereals and other grains, beans (except for pinto beans and I don’t know why that is), rice, popcorn, cornmeal, flour, instant potatoes, powdered milk, and many more items. You can also use Macaroni & Cheese, Hamburger Helper, Rice-A-Roni, etc. by putting the pasta or grain in first, placing the seasoning packet on top and also include the directions.

Now here is how easy it is. Fill clean canning jars with the food. Place the lid on top but do NOT tighten it. Place in a cold oven and do NOT let the jars touch. Use the middle rack if possible. Set the oven to 225° F. and use the following time table:

Start timing when the oven reaches the desired temperature.

Pints – 20 minutes
Quarts – 30 minutes
Half Gallons – 45 minutes

Tighten the lid on each jar, being careful not to get burned, and set on a towel to cool. The shelf life of these items is about 10 years if stored in a cool, dry place.

I’m just slightly dumbfounded. So simple. So direct. So logical. So… why didn’t I ever think of it?

Essentially a drying and pasteurizing process for already dried goods, with a ‘small vacuum due to cooling’ removing some of the oxidation from air, finally a sealed glass container preventing future oxidation (once the oxygen inside is used up) and acting as a vermin barrier.

What’s not to like?

As the family has shrunk (kids moved out) I’m running down the storage. It also needs some amount of ‘turn over’. As I refresh and restock it, this method will be used on about 1/2 the jars. Likely those things that kept least well in storage without oxygen absorbers (regular air) at room temperature and pressure…

Note that this will likely kill living seeds. Don’t expect to store wheat this way and then plant any of it. It may also have an effect on things with volatile oils (spices, brown rice – rice oil is fragile, then again, it goes rancid in air storage pretty quick) It will likely take a little experimentation to find all the best details.

FWIW, the site looks interesting and likely a good place to spend some hours. I plan to as soon as I have some spare…

Doing a search on “Dry Canning” showed lots of folks know about this, so I guess I’m just being a bit slow about it all ;-)

In Conclusion

I’m adding some links to other pages here. I’ve not read all of them, some have popups and those must wait for my pop-up killing no-script browser (on Knoppix) isolated from anything I care about. (Clicking the ‘close box’ can activate scripts doing things you Do Not Want, so I never click the close box on a pop-up for sites I don’t know well…)

I’ve not done more than a cursory look at these pages, but note them for future investigation:

Yeah! Polite and without a pop up! Talks about using oxygen absorbers (that would be a good thing but I’ve not done it yet):
http://www.simplyprepared.com/dry-pack_canning_in_jars.htm

Nice polite site with no pop-up at the moment, well, after a while a small one in the lower corner out of the way shows up, but you can ignore it. Seems a bit more cautious than the others, and lists a 200F oven temp that seems too low to me:
http://www.budget101.com/content.php/3975-how-to-can-using-Dry-Oven-Canning

Has an obnoxious pop up that keeps returning on reload, after a while, but looks complete:
http://americanpreppersnetwork.com/2012/10/dry-canning-the-easy-way.html

Another one with an obnoxious pop up that won’t go away on reload. I’ll look at it later with my secure “no scripts” browser. For now, I can’t really tell what it says as the pop-up won’t stay gone long enough to see (and I don’t click on things as this is permission to ‘do things to my machine’… and I’m not giving that permission).
http://theprepperproject.com/oven-canning-for-long-term-storage/

Here is a contrary point of view:
http://www.preparednesspro.com/why-i-loathe-oven-canning-method
Advocating 5 gallon pails for dry goods and (rightly) points out sensitive things like nuts and brown rice are not going to take to this well. However, a bit ‘preachy’ IMHO in that things like white rice and crackers can benefit from some added drying especially in humid places, and their complaint about jars is exactly wrong, IMHO.

For one thing, open a 5 gallon pail of rice for 2 people and you have a brand new storage problem as you will NOT eat it all before it goes bad. I have enough trouble using a 1/2 gallon. I’ve used plastic pails, plastic jars and tubs, plastic bags, etc. etc. I’ve also had squirrels, rats, insects, and God Only Knows What chew through them if stored in, say, a garage or basement or on the patio. I’ve had their seals give out (more often than I liked), and I’ve had water get into them (including some seeds in baggies inside a freezer.) What is the ONE thing that I’ve never had fail? 1/2 Gallon jars. Even came through a 7 ish quake (in boxes with crumpled newspaper around them) and a minor water “problem” (that took the box, but not the jar). In any case, nice to know the ‘con’ point of view.

Subscribe to feed

Posted in Emergency Preparation and Risks, Food | Tagged , , , | 19 Comments

Alpine Lunix IDS – Snort no Suricata

So I was trying to get directions on how to install and configure Suricata on the Alpine flavor or Linux on the Raspberry Pi B+ model that is my DMZ services platform. I’d settled on Suricata as somewhat better than the older Snort. For some “odd reason” my searches were coming up empty. (Well, really, full of irrelevant and orthogonal crap…)

Eventually I just tried it.

dnspi:/home/chiefio# apk add suricata
ERROR: unsatisfiable constraints:
  suricata (missing):
    required by: world[suricata]

Oh Dear, no target for “suricata”. It isn’t built on Alpine. Digging further, I found a reference to a problem with one of the musl libraries causing a build fail.

http://forum.alpinelinux.org/forum/installation/problem-compiling-software-because-musl-toolchain-cpusett-issue

Sat, 2014-07-26 13:37

I am compiling suricata; I get error at reputation.c using util-affinity.c util-affinity.h, the cpu_set_t typedef is not set or correct #include or something;

cpu_brio_iniit
cpu_prime_init

#define cpu_set_t cpuset_t

#include

I beileve it is because of use of msul usage in Alpine. I need the correct compile time falgs, and libraires to configure and complie suricata.

Second, I compiled xombrero browser from github, It compiles alright but I get segmentation fault as it opens and closes.

Can somebody make suricata and xomberero for Alpine Linux x86-64

Thanks.

This is a relatively common kind of error. Programmers work in a context, a ‘programming environment’. Once the thing works, they feel they are done. Management asks “Does it work?” and when the answer is “Yes”, they move resources to another task. Lost in the shuffle is “Work in ALL environments?”.

I’m particularly sensitive on this one as, at one time, I ran Q.A. for a compiler tool chain that ran in 72 different platforms and crosses (compile on one machine, target another). That meant 72 different QA environments to assure things really worked everywhere. It also meant some of the code was “ugly” as it either had to avoid a ‘neat feature’ on one (or a few) of the environments that was missing in others, OR have paths around the variations (so longer and more complex).

In this case, either of three things is the most likely issue:

Suricata uses a ‘feature’ in glibc that isn’t in musl (and the Suricata programmers didn’t care or didn’t test on musl). That “feature’ can be real and intended or an unintended side effect. If ‘real’, Suricata needs a programming path around it for musl, if unintended, it’s using a ‘bug’ as a ‘feature’ and Suricata needs to not do that.

The musl library writers may have missed that real ‘feature’ in their coding. (It is considered horrid practice to code in ‘side effects’ or ‘bugs’ of the other code just to make things work the same…) This can be an omission, a coding error (bug) in musl, or failure to detect a ‘side effect’ in glibc that ought not to be used and often isn’t documented. If it is an omission or failure in musl, they need to fix it.

Finally, it can also be a ‘bug’ in glibc that just happens to let Suricata work with it despite something being ‘wrong’. While unlikely, given the long use of glibc, it is still a possible.

There will be finger pointing between glibc, Suricata, and musl developers until resolved… “Who owns the bug” only comes after agreeing “what is the bug?” It looks like it has been festering for about 2 years now…

Ah, the joys of the Free Software Culture… /sarc;

Now if I really Really REALLY cared, or had a salary from a company that really cared, I could join up, take that bug as mine, and work it to a solution. Figure out who owns it (musl library issue? Suricata issue? Or glibc doing a stupid thing?) and then either work THAT issue, or put a ‘patch’ or ‘work around’ either in musl or Suricata to make it run anyway since I didn’t want to wait for [whoever] to fix [whatever].

There are never enough hands and eyes to fix everything fast… especially in ‘specialty’ or ‘minor’ Linux flavors.

Now I’m more interested at the moment in just getting an IDS running, so I’m going to swap over to Snort (that IS on Alpine Linux musl) instead and “press on” for now. Later, I’ll give a shot at Suricata on a different OS (likely Debian since it has the most folks whacking it into shape; perhaps Ubuntu as “Debian with more features & fixes” and worst case Red Hat as ‘commercially manicured if obtuse and haughty’).

Why not take on the bug? Well, I’m already about 2 x over committed, and I’m not particularly skilled in either glibc or musl libraries (and those things are rather critical and subtle in obscure ways some times). I also don’t have a ‘test rig’ with enough hardware variations to be sure my ‘fix’ didn’t break some other configuration. I could likely write a ‘shim’ between Suricata and whatever library was the problem (to supply what was missing) but that’s just a short term hack to avoid the problem, not a real fix. ALL of those costing way more than a $25 Pi (that is currently not doing anything in my desk drawer) brought up on Debian…

So for now, at least, no Suricata on the Alpine base system. This opens a longer term question of “Abandon Alpine for the DMZ server?” but I’m not there [yet] as there is a lot to like about Alpine. As it matures this kind of thing will eventually get fixed [if enough people use and support it]. So I’m going to place myself in that band of folks “using and supporting” and not in the group of “debugging code with musl compatibility issues”. At least for now.

Snort

Snort is an older, but still quite capable, IDS. It basically sucks up any network traffic that goes by and inspects it / lets you look at it. I’m OK with swapping to Snort for now. I’m also OK with putting Suricata on a different Raspberry Pi card. I may do both. For now, though, it is ‘Snort on a Pi’:

dnspi:/home/chiefio# apk add snort
(1/9) Installing libdnet (1.12-r6)
(2/9) Installing libpcap (1.7.4-r0)
(3/9) Installing daq (2.0.6-r0)
(4/9) Installing keyutils-libs (1.5.9-r1)
(5/9) Installing libverto (0.2.5-r0)
(6/9) Installing krb5-libs (1.14.3-r0)
(7/9) Installing libtirpc (0.3.2-r1)
(8/9) Installing libuuid (2.28-r3)
(9/9) Installing snort (2.9.8.2-r2)
Executing snort-2.9.8.2-r2.pre-install
Executing busybox-1.24.2-r11.trigger
OK: 46 MiB in 78 packages

I’ll need to configure it, and after I do that, I’ll add the config here. Since I’d been running after Suricata for the last week, I wasn’t expecting to be doing a Snort config instead, so that part is going to be “light” until sometime later in the day (or night…)

The Alpine boards are helpful with lots of good advice. Unfortunately, as a young port, some of it is less than authoritative:

https://wiki.alpinelinux.org/wiki/Intrusion_Detection_using_Snorthttps://wiki.alpinelinux.org/wiki/Intrusion_Detection_using_Snort

This material is work-in-progress …

Do not follow instructions here until this notice is removed.
(Last edited by Fab on 10 Jan 2013.)

So 3 years of “WIP” and…?

(Bold by me)

This guide will set up (list subject to change):

Snort
Barnyard (maybe)
BASE

This guide will assume:

You have a knowledge of your network setup (at least know which subnets exist).
You have Alpine 2.0.2 installed and working with networking setup.
You have had at least three cups of coffee this morning. And not decaf.

Well I’ve only had 2 cups of caf coffee, but I also had a large Earl Grey Tea that I find focuses the mind better than coffee… so I think I’m up to the task…

But that page is mostly about unpacking a tarball and doing a make install and such long hand. I’ve just installed via apk, so I think I don’t need to do those steps. “We’ll see”…

https://wiki.alpinelinux.org/wiki/Intrusion_Detection_using_Snort,_Sguil,_Barnyard_and_more

Points to the prior one as a merger candidate, then goes on…

This material is work-in-progress …

Do not follow instructions here until this notice is removed.
(Last edited by Dubiousjim on 1 Jun 2012.)

4 years ago? Really? Maybe my “highest and best use” will be signing up to update the ‘docs’ after I get it running…

With that, I think I’m going to wander over to other systems docs on Snort and see if they are more enlightening, then take on the config tasks without a custom ‘how to’ guide (i.e. seat of pants…)

http://www.aboutdebian.com/snort.htm

Looks like it might be helpful.

As completion, here’s the Snort home page:

https://www.snort.org/

Oh, and remember to do an ‘lbu commit’ after installing or changing the configs in Alpine…

Subscribe to feed

Posted in Tech Bits | Tagged , , , , | 1 Comment

Democrats – Lords Of Chaos Choose Defiance

Well, it looks to me like we’ve got the early returns on what Soros and his Lords Of Chaos team have decided for the Democrats. “Double Down On Illegal” is what it looks like to me.

First off, we have the ongoing Soros funded street theatre riots (and just where is the prison full of said rioters the next day? Hmmm? Oh, right, the Democratic mayors saying we need to let folks break things and harm people to let off some steam…)

Second, we now have the Parade Of Sanctimonious Cities.

https://pjmedia.com/trending/2016/11/12/nyc-la-mayors-learn-nothing-promise-to-continue-as-sanctuary-cities/

NYC and L.A. Mayors Learn Nothing, Promise to Continue as Sanctuary Cities
BY STEPHEN KRUISER NOVEMBER 12, 2016

Alternative, more realistic headline for the article: “Mayors of NY and Los Angeles pledge to keep Democrats in extended minority.”

Officials in New York and Los Angeles on Thursday said they hoped President-elect Donald Trump would not follow through on a campaign promise to withhold federal funds from “sanctuary cities” that shield people who are in the country illegally.
The nation’s two largest cities have sharply limited their cooperation with U.S. immigration authorities seeking to deport undocumented immigrants.
[…]

Both media and Beltway types haven’t quite figured out that a good portion of America isn’t in sync with them on “comprehensive immigration reform” and the coddling of people who aren’t in the country legally, many of whom are violent. It was the original engine propelling the Trump machine forward in the early primary days, and remains an issue of great disconnect even now that Trump has won.

Fox had a map with a half dozen others. You know the drill. All the usual Democratic Strongholds. Portland, San Francisco, NYC, Newark, Boston, etc. etc. Those are just the ones already “got the memo” and staging “defiance” pressers with the compliant Democratic suck up media.

The total of Sanctimonious Cities is likely up around 200, and more of them will make such Street Theatre In A Suit presentations too; as soon as the Soros check clears…

http://www.washingtonexaminer.com/map-over-200-sanctuary-cities-in-32-states-and-d.c./article/2567880

Has a nice map of about 200 of them.

Map: Over 200 ‘sanctuary cities’ in 32 states and D.C.

By PAUL BEDARD (@SECRETSBEDARD) • 7/9/15 8:54 AM

There are over 200 “sanctuary cities” in 32 states that give safe harbor to illegal immigrants, even violent ones with felony records like the man accused of killing a San Francisco woman last week, according to a new analysis.

RELATED: GOP leader: ‘You could hear from Congress’ on sanctuary cities

The Center for Immigration Studies on Wednesday posted a map of the cities. On their website, they reported:

“More than 200 cities, counties and states across the United States are considered sanctuary cities. These state and local jurisdictions have policies, laws, executive orders, or regulations allowing them to avoid cooperating with federal immigration law enforcement authorities. These ‘cities’ ignore federal law authorizing U.S. Immigration and Customs Enforcement (ICE) to administratively deport illegal aliens without seeking criminal warrants or convictions from federal, state, or local courts.”

Hit the link to see the map. To my eye it looks like most of them are in the BosWash Corridor, Oregon, California, Colorado, and for some odd reason a cluster in Iowa / Illinois area. Chicago and Illinois I can understand, but Iowa? I thought they were smarter than that…

In any case, it doesn’t hit 80% or so of the “Flyover Country” that voted for Trump, so hurting those cities will NOT hurt his constituency and the Street Theatre In A Suit will be all preaching to the (illegal?) choir.

Dear Donald

I’m going to have some sporadic advice for President Donald. (No, I don’t think I need ‘elect’ in there, he’s the Pres now. Obama is off on his final vacation in Europe…) I know they have a web site where you can submit ideas, but I’m just going to put it in Dear Donald postings. If it has merit, it can advance on its own. ( I create ideas and things, but I’m not so interested in promotion of them, and self promotion just seems sleazy to me…) So here’s my Dear Donald on this one:

Dear President Donald,

The Democratic (Soros funded) machine met recently to decide how to torpedo your administration. From the looks of it, Street Theater Protests are to continue (isn’t there a law about incitement to riot? Doesn’t that apply to funding from organizations and from Soros?) Also on the cards is Public Rebuke by the mayors of Democratic Strongholds. They are “playing to their base” of illegal voters…

So they are taking the Defiance Of Authority route (as all good babies try) and are having a public tantrum. It is my belief that you simply need to do what all good parents do: Give them a swat on the rump, no dinner, and send them to their room to cry.

1) The Swat: Direct that any funding withheld from Sanctuary Cities be directed to the I.C.E. budget to enhance immigration enforcement.

2) No Dinner: Suspend Federal money from any city that doesn’t comply with Federal Law. This will give you $Billions more to disburse as you like and will be especially useful for building walls and hiring I.C.E. agents. Make sure each such redirection is tagged as to the city of source (so folks can ‘thank’ them) and maybe even dedicating parts of the wall with a dedication of “These 20 miles funded by The City Of Chicago”…

3) Off to bed: Hold your press conferences without any mention of their politicians. None. They are in their room… DO state that “$Billions have been added to the budget of {agency} for border enforcement thanks to The City Of {wherever} this year.” A particularly nice touch might be having some of the money fund a Federal Criminal Alien Task Force to go into those cities and find the criminals. Oh, and while they are at it, enforce the crime of harboring them against anyone who tries to help hide them.

That’s my idea and advice. If you find it useful, please take it gratis.

E.M.Smith
Deplorable At Large

Subscribe to feed

Posted in Political Current Events | Tagged , , , , , , , | 176 Comments

RT – A Remarkably Clear View of America & Trump

This is a video of “Crosstalk” on RT Russia Today. It runs about 24 minutes.

One of the guests is Jim Rogers, a legendary investor who has been out of American TV lately (as he moved to Singapore IIRC, to escape American taxes and our growing debt bomb…) He has a very nice clarity about him. Two other guests, one in the EU and one Russian, have a better understanding of America, our media, and our politics, than anything seen on our news shows. IMHO, well worth the time to watch, just to see how we look from ‘outside’ if nothing else.

This kind of show is why I watch RT. (They also have some loony side of left shows, so be advised… it varies by show host and what their political bent might be. So be selective and critical while you get to know the shows.) I’d love to see this kind of show on American TV, but it isn’t going to happen.

Subscribe to feed

Posted in Political Current Events | Tagged , , , , | 15 Comments

Alpine Pi – dnsmasq, lighttpd & squid (DNS, web server & proxy server)

I’m still ‘working out the kinks’ in my Alpine based DMZ server. I’ve got DNS via dnsmasq running (well), lighttpd web server running (seemingly well), and squid (proxy to web) running (OK, with https config issue).

This is the core set of services needed for general web browsing / ad blocking / efficient pipe (telco) bandwidth use.

Before we start, there are a couple of things I find annoying about the ‘busybox’ on the Alpine. While it gives you ‘most’ of the common Linux tools, some of them are more limited. All ‘livable’, but a couple of my work habits need modification. Searching for text inside the ‘vi’ editor is dodgy, for example. A staple in my life… Similarly, using !! for “do the last command again” is missing in ‘ash’ (the default shell). God I use that a lot. Nothing horrid, but not a ‘Daily Driver” desktop. Probably a security feature on a DMZ box as there is less to attack.

The DNS services let me quash sites that just spy on me or send trash to me. It can also let you block entire domains. For example, you could send *.cn to the localhost address ( 127.0.0.1 ) and prevent any name that resolves to a China high level domain name from getting contacted.

The web server, as of now, just lets me put out a simple “Hi!” if anyone contacts it. This is useful (at least in theory, I’m still working on the practice) as a responder to those DNS re-routes of web requests. So instead of letting ads.down.yourthoat.com send you a MB of crap on your “by the byte WiFi” link, you can instead have your local web server respond with “Hi!”. (Many folks set up a 1 transparent pixel reply, so you never see anything. I’m going to do that… just after I get the “Hi!” response to show…). Why do this? Well, waiting for timeouts and error messages is not as fast as a local Ethernet speed response. Essentially, it is better to have a landing space for those website redirects via the DNS entries.

Then there is squid. I’ve covered it before when I installed it on Debian. The purpose is mostly to just cache frequently used web pages and images so you only need to download them once. Saves wire time and cost. Saves you waiting at the keyboard while the same old same old crap downloads the 1000th time today…

There are also incidental security benefits from all of these. DNS blocking of evil actors is clearly a win. Avoiding “whatever” is in those bits of website crap via your own server is a win if any of them are malware. Squid is less of a protector, but it does still protect some. Attacks against “you” via web page ports will tend to land on the squid server instead. It can be made much more heavily protected. (That is, you can remove software and commands that could be used by an attacker, it can run a hardened kernel, you don’t have your personal information on it, etc. etc.)

That’s the “why” and the “what” of where I’ve gotten so far.

Evaluation

The DNS server, dnsmasq, is working great. DNS lookups are faster, and many sites are blocked. Essentially it is a full win, with little in the way of ‘grief’ from it. The “open issues” mainly are that I’ve got a fixed list of domain names to block, and that ought to be an automated created list. (Or, have a fixed list and add an automated update list…) I just point it to my Telco router as the first bring up stage, so it is just a filter and buffer in front of the regular DNS chosen by my Telco ISP. After that’s working, then I change the list of DNS servers to put an ‘ad blocking’ one first, then the Telco boundary router (as it is sitting on my desk and I think it, too, may cache a bit), and after that some alternatives in case of DDoS attacks. How many and which ones to list is personal choice. I tend to avoid the Google DNS servers (that everyone else suggests to use) simply because of their information harvesting behaviour. I don’t need Google tracking every DNS lookup from my IP address to be used in DNC targeting. (Yes, Google was ‘caught’ offering services to the DNC to profile every voter in the country…)

The lighttpd server seems to also be working fine. When I connect to the server, it responds with “Hi!”. What is not working is to get “Hi!” when a 404 page missing error is triggered (or perhaps some other error). Using Opera on the Tablet I get a long message about how Opera is sorry but it can’t fetch that page… so some more work needed. (Other browsers just silently display nothing, but that is still taking a timeout lag instead of a “Hi!” near instant service…) But basic web server function is clearly up and working well.

Then there is squid. It works FINE for http: requests. It refuses connections for https: requests IF I activate the cache service. By default, https: requests are just ‘passed through’ without any cache services. (This lets you have more complete end to end encryption and privacy, but with the loss of cache services). If left that way, it works fine. Turn on the cache services (that need a ‘man-in-the-middle’ encrypted tunnel TO the server and a second tunnel server To Website)) and it denies the connect. “Has issues” and it is likely my configuration not being tuned right.

I was going to wait until I got the wrinkles ironed out, but then got nothing done over the weekend. OK, so that means I’m going to air the dirty configs and add caveats. (A minor punishment for sloth…) Just realize these are NOT finished polished just dandy configs. They need more done to reach ‘finished and polished’. But they can get you started.

DNS via dnsmasq

First off, realize that a squid server has some DNS (Domain Name Service) server abilities too. DNS being critical to a lot of things, and folks wanting ‘control’, and DNS historically having been a PITA and “often poor”, it now gets diddled with all over the place. The latest example of this is that SystemD wants to take over DNS too… Part of why I like doing this on Alpine is I don’t have to second guess what SystemD is doing to thwart my wishes.

The dnsmasq service is fairly trivial to set up. At first, it does look a fright, but it isn’t. Mostly you can just ignore the boiler plate. As usual, you install it with [local package manager name] [add|install|whatever] dnsmasq and configure a configuration file. For Alpine, the package manager is ‘apk’ and the command is ‘add’.

dnspi:/home/chiefio# apk add dnsmasq
OK: 42 MiB in 69 packages

The config file is in /etc:

dnspi:/home/chiefio# cd /etc
dnspi:/etc# ls -l dnsmasq.conf 
-rwxr-xr-x    1 root     root         27443 Nov  7 22:32 dnsmasq.conf
dnspi:/etc# wc -l dnsmasq.conf 
687 dnsmasq.conf

Notice that it is 687 lines long. That’s why I said you can ignore most of it. 90% ish of it is comments and things commented out as examples you might need for some bizarre case. A few lines matter (as is usually the case for simple home systems). Those lines starting with a “#” are comments and don’t do anything.

Here’s the whole thing, but I’m changing some of the IP numbers to protect from too much information leakage… I’m also going to point out the lines I changed with BOLD and comments.

dnspi:/etc# cat dnsmasq.conf 
# Configuration file for dnsmasq.
#
# Format is one option per line, legal options are the same
# as the long options legal on the command line. See
# "/usr/sbin/dnsmasq --help" or "man 8 dnsmasq" for details.

# Listen on this specific port instead of the standard DNS port
# (53). Setting this to zero completely disables DNS function,
# leaving only DHCP and/or TFTP.
#port=5353

# The following two options make you a better netizen, since they
# tell dnsmasq to filter out queries which the public DNS cannot
# answer, and which load the servers (especially the root servers)
# unnecessarily. If you have a dial-on-demand link they also stop
# these requests from bringing up the link unnecessarily.

# Never forward plain names (without a dot or domain part)
domain-needed
# Never forward addresses in the non-routed address spaces.
bogus-priv

# Uncomment these to enable DNSSEC validation and caching:
# (Requires dnsmasq to be built with DNSSEC option.)
#conf-file=%%PREFIX%%/share/dnsmasq/trust-anchors.conf
#dnssec

# Replies which are not DNSSEC signed may be legitimate, because the domain
# is unsigned, or may be forgeries. Setting this option tells dnsmasq to
# check that an unsigned reply is OK, by finding a secure proof that a DS 
# record somewhere between the root and the domain does not exist. 
# The cost of setting this is that even queries in unsigned domains will need
# one or more extra DNS queries to verify.
#dnssec-check-unsigned

# Uncomment this to filter useless windows-originated DNS requests
# which can trigger dial-on-demand links needlessly.
# Note that (amongst other things) this blocks all SRV requests,
# so don't use it if you use eg Kerberos, SIP, XMMP or Google-talk.
# This option only affects forwarding, SRV records originating for
# dnsmasq (via srv-host= lines) are not suppressed by it.
filterwin2k

# Change this line if you want dns to get its upstream servers from
# somewhere other that /etc/resolv.conf
#resolv-file=

# By  default,  dnsmasq  will  send queries to any of the upstream
# servers it knows about and tries to favour servers to are  known
# to  be  up.  Uncommenting this forces dnsmasq to try each query
# with  each  server  strictly  in  the  order  they   appear   in
# /etc/resolv.conf
strict-order

# If you don't want dnsmasq to read /etc/resolv.conf or any other
# file, getting its servers from this file instead (see below), then
# uncomment this.
no-resolv

# If you don't want dnsmasq to poll /etc/resolv.conf or other resolv
# files for changes and re-read them then uncomment this.
no-poll

# Add other name servers here, with domain specs if they are for
# non-public domains.
#server=/localnet/192.168.0.1
server=127.0.0.1
server=23.253.163.53
server=192.168.1.1
server=208.67.222.222
#nameserver 23.253.163.53   (ad-blocking-dns2)
#nameserver 192.168.1.1     (Telco router)
#nameserver 208.67.222.222  (Open DNS)
#nameserver 68.94.157.1     (dnsr2.sbcglobal.net)
#nameserver 209.244.0.4     (resolver2.level3.net)
#nameserver 4.2.2.4         (d.resolvers.level3.net)
#nameserver 8.8.4.4         (google-public-dns-b.google.com)
#nameserver 8.8.8.8         (google-public-dns-a.google.com)

Note that I have comments listing a bunch of known DNS servers. When some DDos (Distributed Denial of Service) attack is taking down your regular servers, it is nice to have a list handy, and what could be more handy than the place you will be going to change your server settings?

I also do ‘strick order’ so it looks to itself (and cache) first, then the ad.block.server, and then goes on to try others only if needed.

# Example of routing PTR queries to nameservers: this will send all
# address->name queries for 192.168.3/24 to nameserver 10.1.2.3
#server=/3.168.192.in-addr.arpa/10.1.2.3

# Add local-only domains here, queries in these domains are answered
# from /etc/hosts or DHCP only.
local=/chiefio.homebase/

# Add domains which you want to force to an IP address here.
# The example below send any host in double-click.net to a local
# web-server.
#address=/double-click.net/127.0.0.1

address=/.tradedouber.com/127.0.0.1 
address=/doubleclick.net/127.0.0.1 
address=/ads.google.com/127.0.0.1 
address=/googlesyndication.com/127.0.0.1 

I put a couple here as a proof of concept. Most are in an external file.

I also don’t do any IPv6. Any IoT device using IPv6 on my network will get nowhere.

# --address (and --server) work with IPv6 addresses too.
#address=/www.thekelleys.org.uk/fe80::20d:60ff:fe36:f83

# Add the IPs of all queries to yahoo.com, google.com, and their
# subdomains to the vpn and search ipsets:
#ipset=/yahoo.com/google.com/vpn,search

# You can control how dnsmasq talks to a server: this forces
# queries to 10.1.2.3 to be routed via eth1
# server=10.1.2.3@eth1

# and this sets the source (ie local) address used to talk to
# 10.1.2.3 to 192.168.1.1 port 55 (there must be a interface with that
# IP on the machine, obviously).
# server=10.1.2.3@192.168.1.1#55

# If you want dnsmasq to change uid and gid to something other
# than the default, edit the following lines.
#user=
#group=

# If you want dnsmasq to listen for DHCP and DNS requests only on
# specified interfaces (and the loopback) give the name of the
# interface (eg eth0) here.
# Repeat the line for more than one interface.
#interface=
# Or you can specify which interface _not_ to listen on
#except-interface=
# Or which to listen on by address (remember to include 127.0.0.1 if
# you use this.)
#listen-address=
# If you want dnsmasq to provide only DNS service on an interface,
# configure it as shown above, and then use the following line to
# disable DHCP and TFTP on it.
no-dhcp-interface=eth0 

# On systems which support it, dnsmasq binds the wildcard address,
# even when it is listening on only some interfaces. It then discards
# requests that it shouldn't reply to. This has the advantage of
# working even when interfaces come and go and change address. If you
# want dnsmasq to really bind only the interfaces it is listening on,
# uncomment this option. About the only time you may need this is when
# running another nameserver on the same machine.
#bind-interfaces

# If you don't want dnsmasq to read /etc/hosts, uncomment the
# following line.
#no-hosts
# or if you want it to read another file, as well as /etc/hosts, use
# this.
#addn-hosts=/etc/banner_add_hosts
addn-hosts=/etc/hosts.block

# Set this (and domain: see below) if you want to have a domain
# automatically added to simple names in a hosts-file.
expand-hosts

# Set the domain for dnsmasq. this is optional, but if it is set, it
# does the following things.
# 1) Allows DHCP hosts to have fully qualified domain names, as long
#     as the domain part matches this setting.
# 2) Sets the "domain" DHCP option thereby potentially setting the
#    domain of all systems configured by DHCP
# 3) Provides the domain part for "expand-hosts"
domain=chiefio.homebase

# Set a different domain for a particular subnet
#domain=wireless.thekelleys.org.uk,192.168.2.0/24

# Same idea, but range rather then subnet
#domain=reserved.thekelleys.org.uk,192.68.3.100,192.168.3.200

# Uncomment this to enable the integrated DHCP server, you need
# to supply the range of addresses available for lease and optionally
# a lease time. If you have more than one network, you will need to
# repeat this for each network on which you want to supply DHCP
# service.
#dhcp-range=192.168.0.50,192.168.0.150,12h

# This is an example of a DHCP range where the netmask is given. This
# is needed for networks we reach the dnsmasq DHCP server via a relay
# agent. If you don't know what a DHCP relay agent is, you probably
# don't need to worry about this.
#dhcp-range=192.168.0.50,192.168.0.150,255.255.255.0,12h

# This is an example of a DHCP range which sets a tag, so that
# some DHCP options may be set only for this network.
#dhcp-range=set:red,192.168.0.50,192.168.0.150

# Use this DHCP range only when the tag "green" is set.
#dhcp-range=tag:green,192.168.0.50,192.168.0.150,12h

# Specify a subnet which can't be used for dynamic address allocation,
# is available for hosts with matching --dhcp-host lines. Note that
# dhcp-host declarations will be ignored unless there is a dhcp-range
# of some type for the subnet in question.
# In this case the netmask is implied (it comes from the network
# configuration on the machine running dnsmasq) it is possible to give
# an explicit netmask instead.
#dhcp-range=192.168.0.0,static

# Enable DHCPv6. Note that the prefix-length does not need to be specified
# and defaults to 64 if missing/
#dhcp-range=1234::2, 1234::500, 64, 12h

# Do Router Advertisements, BUT NOT DHCP for this subnet.
#dhcp-range=1234::, ra-only 

# Do Router Advertisements, BUT NOT DHCP for this subnet, also try and
# add names to the DNS for the IPv6 address of SLAAC-configured dual-stack 
# hosts. Use the DHCPv4 lease to derive the name, network segment and 
# MAC address and assume that the host will also have an
# IPv6 address calculated using the SLAAC alogrithm.
#dhcp-range=1234::, ra-names

# Do Router Advertisements, BUT NOT DHCP for this subnet.
# Set the lifetime to 46 hours. (Note: minimum lifetime is 2 hours.)
#dhcp-range=1234::, ra-only, 48h

# Do DHCP and Router Advertisements for this subnet. Set the A bit in the RA
# so that clients can use SLAAC addresses as well as DHCP ones.
#dhcp-range=1234::2, 1234::500, slaac

# Do Router Advertisements and stateless DHCP for this subnet. Clients will
# not get addresses from DHCP, but they will get other configuration information.
# They will use SLAAC for addresses.
#dhcp-range=1234::, ra-stateless

# Do stateless DHCP, SLAAC, and generate DNS names for SLAAC addresses
# from DHCPv4 leases.
#dhcp-range=1234::, ra-stateless, ra-names

# Do router advertisements for all subnets where we're doing DHCPv6
# Unless overriden by ra-stateless, ra-names, et al, the router 
# advertisements will have the M and O bits set, so that the clients
# get addresses and configuration from DHCPv6, and the A bit reset, so the 
# clients don't use SLAAC addresses.
#enable-ra

# Supply parameters for specified hosts using DHCP. There are lots
# of valid alternatives, so we will give examples of each. Note that
# IP addresses DO NOT have to be in the range given above, they just
# need to be on the same network. The order of the parameters in these
# do not matter, it's permissible to give name, address and MAC in any
# order.

# Always allocate the host with Ethernet address 11:22:33:44:55:66
# The IP address 192.168.0.60
#dhcp-host=11:22:33:44:55:66,192.168.0.60

# Always set the name of the host with hardware address
# 11:22:33:44:55:66 to be "fred"
#dhcp-host=11:22:33:44:55:66,fred

# Always give the host with Ethernet address 11:22:33:44:55:66
# the name fred and IP address 192.168.0.60 and lease time 45 minutes
#dhcp-host=11:22:33:44:55:66,fred,192.168.0.60,45m

# Give a host with Ethernet address 11:22:33:44:55:66 or
# 12:34:56:78:90:12 the IP address 192.168.0.60. Dnsmasq will assume
# that these two Ethernet interfaces will never be in use at the same
# time, and give the IP address to the second, even if it is already
# in use by the first. Useful for laptops with wired and wireless
# addresses.
#dhcp-host=11:22:33:44:55:66,12:34:56:78:90:12,192.168.0.60

# Give the machine which says its name is "bert" IP address
# 192.168.0.70 and an infinite lease
#dhcp-host=bert,192.168.0.70,infinite

# Always give the host with client identifier 01:02:02:04
# the IP address 192.168.0.60
#dhcp-host=id:01:02:02:04,192.168.0.60

# Always give the Infiniband interface with hardware address
# 80:00:00:48:fe:80:00:00:00:00:00:00:f4:52:14:03:00:28:05:81 the
# ip address 192.168.0.61. The client id is derived from the prefix
# ff:00:00:00:00:00:02:00:00:02:c9:00 and the last 8 pairs of
# hex digits of the hardware address.
#dhcp-host=id:ff:00:00:00:00:00:02:00:00:02:c9:00:f4:52:14:03:00:28:05:81,192.168.0.61

# Always give the host with client identifier "marjorie"
# the IP address 192.168.0.60
#dhcp-host=id:marjorie,192.168.0.60

# Enable the address given for "judge" in /etc/hosts
# to be given to a machine presenting the name "judge" when
# it asks for a DHCP lease.
#dhcp-host=judge

# Never offer DHCP service to a machine whose Ethernet
# address is 11:22:33:44:55:66
#dhcp-host=11:22:33:44:55:66,ignore

# Ignore any client-id presented by the machine with Ethernet
# address 11:22:33:44:55:66. This is useful to prevent a machine
# being treated differently when running under different OS's or
# between PXE boot and OS boot.
#dhcp-host=11:22:33:44:55:66,id:*

# Send extra options which are tagged as "red" to
# the machine with Ethernet address 11:22:33:44:55:66
#dhcp-host=11:22:33:44:55:66,set:red

# Send extra options which are tagged as "red" to
# any machine with Ethernet address starting 11:22:33:
#dhcp-host=11:22:33:*:*:*,set:red

# Give a fixed IPv6 address and name to client with 
# DUID 00:01:00:01:16:d2:83:fc:92:d4:19:e2:d8:b2
# Note the MAC addresses CANNOT be used to identify DHCPv6 clients.
# Note also the they [] around the IPv6 address are obilgatory.
#dhcp-host=id:00:01:00:01:16:d2:83:fc:92:d4:19:e2:d8:b2, fred, [1234::5] 

# Ignore any clients which are not specified in dhcp-host lines
# or /etc/ethers. Equivalent to ISC "deny unknown-clients".
# This relies on the special "known" tag which is set when
# a host is matched.
#dhcp-ignore=tag:!known

# Send extra options which are tagged as "red" to any machine whose
# DHCP vendorclass string includes the substring "Linux"
#dhcp-vendorclass=set:red,Linux

# Send extra options which are tagged as "red" to any machine one
# of whose DHCP userclass strings includes the substring "accounts"
#dhcp-userclass=set:red,accounts

# Send extra options which are tagged as "red" to any machine whose
# MAC address matches the pattern.
#dhcp-mac=set:red,00:60:8C:*:*:*

# If this line is uncommented, dnsmasq will read /etc/ethers and act
# on the ethernet-address/IP pairs found there just as if they had
# been given as --dhcp-host options. Useful if you keep
# MAC-address/host mappings there for other purposes.
#read-ethers

# Send options to hosts which ask for a DHCP lease.
# See RFC 2132 for details of available options.
# Common options can be given to dnsmasq by name:
# run "dnsmasq --help dhcp" to get a list.
# Note that all the common settings, such as netmask and
# broadcast address, DNS server and default route, are given
# sane defaults by dnsmasq. You very likely will not need
# any dhcp-options. If you use Windows clients and Samba, there
# are some options which are recommended, they are detailed at the
# end of this section.

# Override the default route supplied by dnsmasq, which assumes the
# router is the same machine as the one running dnsmasq.
#dhcp-option=3,1.2.3.4

# Do the same thing, but using the option name
#dhcp-option=option:router,1.2.3.4

# Override the default route supplied by dnsmasq and send no default
# route at all. Note that this only works for the options sent by
# default (1, 3, 6, 12, 28) the same line will send a zero-length option
# for all other option numbers.
#dhcp-option=3

# Set the NTP time server addresses to 192.168.0.4 and 10.10.0.5
#dhcp-option=option:ntp-server,192.168.0.4,10.10.0.5

# Send DHCPv6 option. Note [] around IPv6 addresses.
#dhcp-option=option6:dns-server,[1234::77],[1234::88]

# Send DHCPv6 option for namservers as the machine running 
# dnsmasq and another.
#dhcp-option=option6:dns-server,[::],[1234::88]

# Ask client to poll for option changes every six hours. (RFC4242)
#dhcp-option=option6:information-refresh-time,6h

# Set option 58 client renewal time (T1). Defaults to half of the
# lease time if not specified. (RFC2132)
#dhcp-option=option:T1:1m

# Set option 59 rebinding time (T2). Defaults to 7/8 of the
# lease time if not specified. (RFC2132)
#dhcp-option=option:T2:2m

# Set the NTP time server address to be the same machine as
# is running dnsmasq
#dhcp-option=42,0.0.0.0

# Set the NIS domain name to "welly"
#dhcp-option=40,welly

# Set the default time-to-live to 50
#dhcp-option=23,50

# Set the "all subnets are local" flag
#dhcp-option=27,1

# Send the etherboot magic flag and then etherboot options (a string).
#dhcp-option=128,e4:45:74:68:00:00
#dhcp-option=129,NIC=eepro100

# Specify an option which will only be sent to the "red" network
# (see dhcp-range for the declaration of the "red" network)
# Note that the tag: part must precede the option: part.
#dhcp-option = tag:red, option:ntp-server, 192.168.1.1

# The following DHCP options set up dnsmasq in the same way as is specified
# for the ISC dhcpcd in
# http://www.samba.org/samba/ftp/docs/textdocs/DHCP-Server-Configuration.txt
# adapted for a typical dnsmasq installation where the host running
# dnsmasq is also the host running samba.
# you may want to uncomment some or all of them if you use
# Windows clients and Samba.
#dhcp-option=19,0           # option ip-forwarding off
#dhcp-option=44,0.0.0.0     # set netbios-over-TCP/IP nameserver(s) aka WINS server(s)
#dhcp-option=45,0.0.0.0     # netbios datagram distribution server
#dhcp-option=46,8           # netbios node type

# Send an empty WPAD option. This may be REQUIRED to get windows 7 to behave.
#dhcp-option=252,"\n"

# Send RFC-3397 DNS domain search DHCP option. WARNING: Your DHCP client
# probably doesn't support this......
#dhcp-option=option:domain-search,eng.apple.com,marketing.apple.com

# Send RFC-3442 classless static routes (note the netmask encoding)
#dhcp-option=121,192.168.1.0/24,1.2.3.4,10.0.0.0/8,5.6.7.8

# Send vendor-class specific options encapsulated in DHCP option 43.
# The meaning of the options is defined by the vendor-class so
# options are sent only when the client supplied vendor class
# matches the class given here. (A substring match is OK, so "MSFT"
# matches "MSFT" and "MSFT 5.0"). This example sets the
# mtftp address to 0.0.0.0 for PXEClients.
#dhcp-option=vendor:PXEClient,1,0.0.0.0

# Send microsoft-specific option to tell windows to release the DHCP lease
# when it shuts down. Note the "i" flag, to tell dnsmasq to send the
# value as a four-byte integer - that's what microsoft wants. See
# http://technet2.microsoft.com/WindowsServer/en/library/a70f1bb7-d2d4-49f0-96d6-4b7414ecfaae1033.mspx?mfr=true
#dhcp-option=vendor:MSFT,2,1i

# Send the Encapsulated-vendor-class ID needed by some configurations of
# Etherboot to allow is to recognise the DHCP server.
#dhcp-option=vendor:Etherboot,60,"Etherboot"

# Send options to PXELinux. Note that we need to send the options even
# though they don't appear in the parameter request list, so we need
# to use dhcp-option-force here.
# See http://syslinux.zytor.com/pxe.php#special for details.
# Magic number - needed before anything else is recognised
#dhcp-option-force=208,f1:00:74:7e
# Configuration file name
#dhcp-option-force=209,configs/common
# Path prefix
#dhcp-option-force=210,/tftpboot/pxelinux/files/
# Reboot time. (Note 'i' to send 32-bit value)
#dhcp-option-force=211,30i

# Set the boot filename for netboot/PXE. You will only need
# this is you want to boot machines over the network and you will need
# a TFTP server; either dnsmasq's built in TFTP server or an
# external one. (See below for how to enable the TFTP server.)
#dhcp-boot=pxelinux.0

# The same as above, but use custom tftp-server instead machine running dnsmasq
#dhcp-boot=pxelinux,server.name,192.168.1.100

# Boot for Etherboot gPXE. The idea is to send two different
# filenames, the first loads gPXE, and the second tells gPXE what to
# load. The dhcp-match sets the gpxe tag for requests from gPXE.
#dhcp-match=set:gpxe,175 # gPXE sends a 175 option.
#dhcp-boot=tag:!gpxe,undionly.kpxe
#dhcp-boot=mybootimage

# Encapsulated options for Etherboot gPXE. All the options are
# encapsulated within option 175
#dhcp-option=encap:175, 1, 5b         # priority code
#dhcp-option=encap:175, 176, 1b       # no-proxydhcp
#dhcp-option=encap:175, 177, string   # bus-id
#dhcp-option=encap:175, 189, 1b       # BIOS drive code
#dhcp-option=encap:175, 190, user     # iSCSI username
#dhcp-option=encap:175, 191, pass     # iSCSI password

# Test for the architecture of a netboot client. PXE clients are
# supposed to send their architecture as option 93. (See RFC 4578)
#dhcp-match=peecees, option:client-arch, 0 #x86-32
#dhcp-match=itanics, option:client-arch, 2 #IA64
#dhcp-match=hammers, option:client-arch, 6 #x86-64
#dhcp-match=mactels, option:client-arch, 7 #EFI x86-64

# Do real PXE, rather than just booting a single file, this is an
# alternative to dhcp-boot.
#pxe-prompt="What system shall I netboot?"
# or with timeout before first available action is taken:
#pxe-prompt="Press F8 for menu.", 60

# Available boot services. for PXE.
#pxe-service=x86PC, "Boot from local disk"

# Loads /pxelinux.0 from dnsmasq TFTP server.
#pxe-service=x86PC, "Install Linux", pxelinux

# Loads /pxelinux.0 from TFTP server at 1.2.3.4.
# Beware this fails on old PXE ROMS.
#pxe-service=x86PC, "Install Linux", pxelinux, 1.2.3.4

# Use bootserver on network, found my multicast or broadcast.
#pxe-service=x86PC, "Install windows from RIS server", 1

# Use bootserver at a known IP address.
#pxe-service=x86PC, "Install windows from RIS server", 1, 1.2.3.4

# If you have multicast-FTP available,
# information for that can be passed in a similar way using options 1
# to 5. See page 19 of
# http://download.intel.com/design/archives/wfm/downloads/pxespec.pdf


# Enable dnsmasq's built-in TFTP server
#enable-tftp

# Set the root directory for files available via FTP.
#tftp-root=/var/ftpd

# Do not abort if the tftp-root is unavailable
#tftp-no-fail

# Make the TFTP server more secure: with this set, only files owned by
# the user dnsmasq is running as will be send over the net.
#tftp-secure

# This option stops dnsmasq from negotiating a larger blocksize for TFTP
# transfers. It will slow things down, but may rescue some broken TFTP
# clients.
#tftp-no-blocksize

# Set the boot file name only when the "red" tag is set.
#dhcp-boot=tag:red,pxelinux.red-net

# An example of dhcp-boot with an external TFTP server: the name and IP
# address of the server are given after the filename.
# Can fail with old PXE ROMS. Overridden by --pxe-service.
#dhcp-boot=/var/ftpd/pxelinux.0,boothost,192.168.0.3

# If there are multiple external tftp servers having a same name
# (using /etc/hosts) then that name can be specified as the
# tftp_servername (the third option to dhcp-boot) and in that
# case dnsmasq resolves this name and returns the resultant IP
# addresses in round robin fasion. This facility can be used to
# load balance the tftp load among a set of servers.
#dhcp-boot=/var/ftpd/pxelinux.0,boothost,tftp_server_name

# Set the limit on DHCP leases, the default is 150
#dhcp-lease-max=150

# The DHCP server needs somewhere on disk to keep its lease database.
# This defaults to a sane location, but if you want to change it, use
# the line below.
#dhcp-leasefile=/var/lib/misc/dnsmasq.leases

# Set the DHCP server to authoritative mode. In this mode it will barge in
# and take over the lease for any client which broadcasts on the network,
# whether it has a record of the lease or not. This avoids long timeouts
# when a machine wakes up on a new network. DO NOT enable this if there's
# the slightest chance that you might end up accidentally configuring a DHCP
# server for your campus/company accidentally. The ISC server uses
# the same option, and this URL provides more information:
# http://www.isc.org/files/auth.html
#dhcp-authoritative

# Run an executable when a DHCP lease is created or destroyed.
# The arguments sent to the script are "add" or "del",
# then the MAC address, the IP address and finally the hostname
# if there is one.
#dhcp-script=/bin/echo

# Set the cachesize here.
cache-size=15000
# Note this is a VERY big cache size, 10 x the default

# If you want to disable negative caching, uncomment this.
#no-negcache

# Normally responses which come from /etc/hosts and the DHCP lease
# file have Time-To-Live set as zero, which conventionally means
# do not cache further. If you are happy to trade lower load on the
# server for potentially stale date, you can set a time-to-live (in
# seconds) here.
local-ttl=42300

# If you want dnsmasq to detect attempts by Verisign to send queries
# to unregistered .com and .net hosts to its sitefinder service and
# have dnsmasq instead return the correct NXDOMAIN response, uncomment
# this line. You can add similar lines to do the same for other
# registries which have implemented wildcard A records.
bogus-nxdomain=64.94.110.11

# If you want to fix up DNS results from upstream servers, use the
# alias option. This only works for IPv4.
# This alias makes a result of 1.2.3.4 appear as 5.6.7.8
#alias=1.2.3.4,5.6.7.8
# and this maps 1.2.3.x to 5.6.7.x
#alias=1.2.3.0,5.6.7.0,255.255.255.0
# and this maps 192.168.0.10->192.168.0.40 to 10.0.0.10->10.0.0.40
#alias=192.168.0.10-192.168.0.40,10.0.0.0,255.255.255.0

# Change these lines if you want dnsmasq to serve MX records.

# Return an MX record named "maildomain.com" with target
# servermachine.com and preference 50
#mx-host=maildomain.com,servermachine.com,50

# Set the default target for MX records created using the localmx option.
#mx-target=servermachine.com

# Return an MX record pointing to the mx-target for all local
# machines.
#localmx

# Return an MX record pointing to itself for all local machines.
#selfmx

# Change the following lines if you want dnsmasq to serve SRV
# records.  These are useful if you want to serve ldap requests for
# Active Directory and other windows-originated DNS requests.
# See RFC 2782.
# You may add multiple srv-host lines.
# The fields are ,,,,
# If the domain part if missing from the name (so that is just has the
# service and protocol sections) then the domain given by the domain=
# config option is used. (Note that expand-hosts does not need to be
# set for this to work.)

# A SRV record sending LDAP for the example.com domain to
# ldapserver.example.com port 389
#srv-host=_ldap._tcp.example.com,ldapserver.example.com,389

# A SRV record sending LDAP for the example.com domain to
# ldapserver.example.com port 389 (using domain=)
#domain=example.com
#srv-host=_ldap._tcp,ldapserver.example.com,389

# Two SRV records for LDAP, each with different priorities
#srv-host=_ldap._tcp.example.com,ldapserver.example.com,389,1
#srv-host=_ldap._tcp.example.com,ldapserver.example.com,389,2

# A SRV record indicating that there is no LDAP server for the domain
# example.com
#srv-host=_ldap._tcp.example.com

# The following line shows how to make dnsmasq serve an arbitrary PTR
# record. This is useful for DNS-SD. (Note that the
# domain-name expansion done for SRV records _does_not
# occur for PTR records.)
#ptr-record=_http._tcp.dns-sd-services,"New Employee Page._http._tcp.dns-sd-services"

# Change the following lines to enable dnsmasq to serve TXT records.
# These are used for things like SPF and zeroconf. (Note that the
# domain-name expansion done for SRV records _does_not
# occur for TXT records.)

#Example SPF.
#txt-record=example.com,"v=spf1 a -all"

#Example zeroconf
#txt-record=_http._tcp.example.com,name=value,paper=A4

# Provide an alias for a "local" DNS name. Note that this _only_ works
# for targets which are names from DHCP or /etc/hosts. Give host
# "bert" another name, bertrand
#cname=bertand,bert

# For debugging purposes, log each DNS query as it passes through
# dnsmasq.
log-queries

# Log lots of extra information about DHCP transactions.
#log-dhcp

# Include another lot of configuration options.
#conf-file=/etc/dnsmasq.more.conf
#conf-dir=/etc/dnsmasq.d

# Include all the files in a directory except those ending in .bak
#conf-dir=/etc/dnsmasq.d,.bak

# Include all files in a directory which end in .conf
#conf-dir=/etc/dnsmasq.d/,*.conf

Here is a bit of my /etc/hosts.block file. It is 15,655 lines long, so can’t post all of it ;-). I’ve got a couple of sources glued into it, but the top is from one of them and has were to get their part:

dnspi:/etc/squid# head /etc/hosts.block
# This MVPS HOSTS file is a free download from:            #
# http://winhelp2002.mvps.org/hosts.htm                  #
#     
[...]                                                     #
# [Misc A - Z]
10.10.1.122  fr.a2dfp.net
10.10.1.122  m.fr.a2dfp.net
10.10.1.122  ad.a8.net
10.10.1.122  asy.a8ww.net
10.10.1.122  abcstats.com
10.10.1.122  a.abv.bg
10.10.1.122  adserver.abv.bg
10.10.1.122  adv.abv.bg

Where “10.10.1.122” would be the address of your web server inside your shop, or would be “127.0.0.1” if you have it running on your personal workstation and want to land there. (That is the default as shipped from the maker).

Web Server – lighttpd

Works well, added with:

apk add lighttpd

Configuration done here:

dnspi:/etc# ls -l /etc/lighttpd/
total 32
-rw-r--r--    1 root     root            27 Nov 13 02:05 error-handler.html
-rw-r--r--    1 root     root          8241 Nov 13 02:27 lighttpd.conf
-rw-r--r--    1 root     root          3436 Jun  3 16:33 mime-types.conf
-rw-r--r--    1 root     root           869 Jun  3 16:33 mod_cgi.conf
-rw-r--r--    1 root     root           683 Jun  3 16:33 mod_fastcgi.conf
-rw-r--r--    1 root     root           488 Jun  3 16:33 mod_fastcgi_fpm.conf
dnspi:/etc#

Note that only ‘error-handler.html’ and ‘lighttpd.conf’ have changed to the install date. Even there, the ‘error-handler.html’ file is a failed attempt to get “Hi!” on errors. The only thing that really is changed is the lighttpd.conf file. I spent a ‘long time’ looking for where error-handler.html was stored. Eventualy I found that there’s a path name setting that goes on in the conf file. Since it is “user configurable” it isn’t in the web pages… even though just about everyone will take the default.

Here’s parts of my (very limited and maybe not quite right) conf file:

dnspi:/etc/lighttpd# wc -l lighttpd.conf 
323 lighttpd.conf

dnspi:/etc/lighttpd# cat lighttpd.conf 
###############################################################################
# Default lighttpd.conf for Gentoo.
# $Header: /var/cvsroot/gentoo-x86/www-servers/lighttpd/files/conf/lighttpd.conf,v 1.3 2005/09/01 14:22:35 ka0ttic Exp $
###############################################################################

# {{{ variables
var.basedir  = "/var/www/localhost"
var.logdir   = "/var/log/lighttpd"
var.statedir = "/var/lib/lighttpd"
# }}}

# {{{ modules
# At the very least, mod_access and mod_accesslog should be enabled.
# All other modules should only be loaded if necessary.
# NOTE: the order of modules is important.

So 323 lines in it, most of them boilerplate or comments. I note in passing that it is the “Gentoo” default… Gee, I wonder what base release is used by the Alpine folks for their build? ;-)

Note that this is where “things are located” is defined. Log files in /var/log/lighttpd with state information in /var/lib/lighttpd and most important, you put your web pages in /var/www/localhost. Except you don’t… Next is a long section of “modules” all of which are commented out except for the two they tell you to use. Then follows a lot of things to do, or not do, where I just accepted all the defaults. Note this one, though:

# {{{ server settings
server.username      = "lighttpd"
server.groupname     = "lighttpd"

server.document-root = var.basedir + "/htdocs"
server.pid-file      = "/run/lighttpd.pid"

server.errorlog      = var.logdir  + "/error.log"
# log errors to syslog instead
#   server.errorlog-use-syslog = "enable"

Notice that line “server.document-root”? that tags “/httdocs” onto the path you thought you were using from the top…

dnspi:/etc/lighttpd# ls -l /var/www/localhost/htdocs/
total 12
-rw-r--r--    1 root     root            27 Nov 13 02:03 error-handler.html
-rw-r--r--    1 root     root             4 Nov  7 22:27 index.html
-rw-r--r--    1 root     root             4 Nov 13 02:28 status-404.html

The “index.html” says “Hi!” and works on connection to the server. The other two are attempts to get the missing page handled… that are not working yet. It just doesn’t respond.

On this error handler one I tried, and failed, to just get a simple web page displayed.

# error-handler for status 404
server.error-handler-404 = "/index.html"
# server.error-handler-404 = "/error-handler.php"

No, I have no idea why it didn’t work. Hopefully it isn’t tied to type .php files…

It then goes through a lot of server cache settings that I left at default along with some other setup for services I’m not using like veritual machines.

FWIW, here’s the entire file if you remove all the comments and blank lines:

dnspi:/etc/lighttpd# grep -v "^#" lighttpd.conf 

var.basedir  = "/var/www/localhost"
var.logdir   = "/var/log/lighttpd"
var.statedir = "/var/lib/lighttpd"
server.modules = (
    "mod_access",
    "mod_accesslog"
)
include "mime-types.conf"
server.username      = "lighttpd"
server.groupname     = "lighttpd"
server.document-root = var.basedir + "/htdocs"
server.pid-file      = "/run/lighttpd.pid"
server.errorlog      = var.logdir  + "/error.log"
server.indexfiles    = ("index.php", "index.html",
						"index.htm", "default.htm")
server.follow-symlink = "enable"
server.error-handler-404 = "/index.html"
server.errorfile-prefix    = var.basedir + "/error/status-"
static-file.exclude-extensions = (".php", ".pl", ".cgi", ".fcgi")
accesslog.filename   = var.logdir + "/access.log"
url.access-deny = ("~", ".inc")

Pretty small active set. You would think I’d have figured it out by now…

The squid proxy server

Yup, you guessed it, installed with:

apk add squid

Configured in /etc/squid:

dnspi:/etc# ls -l squid
total 360
-rw-r--r--    1 root     root           692 Nov  4 12:59 cachemgr.conf
-rw-r--r--    1 root     root           692 Nov  4 12:59 cachemgr.conf.default
-rw-r--r--    1 root     root          1817 Nov  4 12:58 errorpage.css
-rw-r--r--    1 root     root          1817 Nov  4 12:59 errorpage.css.default
-rw-r--r--    1 root     root         12077 Nov  4 12:59 mime.conf
-rw-r--r--    1 root     root         12077 Nov  4 12:59 mime.conf.default
-rw-r--r--    1 root     root          5069 Nov  7 22:40 squid.conf
-rw-r--r--    1 root     root          4233 Nov  7 18:42 squid.conf.NoSSL
-rw-r--r--    1 root     root          2315 Nov  4 12:59 squid.conf.default
-rw-r--r--    1 root     root        289464 Nov  4 12:59 squid.conf.documented
-rw-r--r--    1 root     root          5089 Nov  7 18:48 squid.conf.withSSL
-r--------    1 root     root          5435 Nov  7 18:37 squid.pem

Note that I’ve cot a ‘mime.conf.default’ that is a vendor supplied backup of the installed one, and a ‘squid.conf.NoSSL’ along with a ‘squid.conf.default’ and a ‘squid.conf.withSLL’. These are my attempts to make https: work with caching and saving copies of the ‘without’ that works. The squid.pem file is my private key for the https: encryption process (there’s a procedure to create that). FWIW, the bulk of what makes my squid.conf so much bigger than the squid.conf.default is a higher level of comments. I copied it from my prior squid server on Debian and they were more wordy.

As this posting is already long, I’m not going to put my entire squid config here. (For one thing, WordPress spell checking has an exponential slowdown with posting length and on the Pi M3 it reaches a ‘too slow to live with’ about this long. For another, it isn’t working right yet…) Here’s the “with SSL” with all the comments and spaces stripped out (most of it the defaults):

dnspi:/etc/squid# grep -v "^#" squid.conf.withSSL 
acl localnet src 10.0.0.0/8	# RFC1918 possible internal network
acl localnet src 172.16.0.0/12	# RFC1918 possible internal network
acl localnet src 192.168.0.0/16	# RFC1918 possible internal network
acl SSL_ports port 443
acl Safe_ports port 80		# http
acl Safe_ports port 21		# ftp
acl Safe_ports port 443		# https
acl Safe_ports port 70		# gopher
acl Safe_ports port 210		# waiss
acl Safe_ports port 1025-65535	# unregistered ports
acl Safe_ports port 280		# http-mgmt
acl Safe_ports port 488		# gss-http
acl Safe_ports port 591		# filemaker
acl Safe_ports port 777		# multiling http
acl CONNECT method CONNECT
acl QUERY urlpath_regex cgi-bin \? asp aspx jsp
cache deny QUERY
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
http_access deny to_localhost
http_access allow localnet
http_access allow localhost
http_access deny all
http_port 4353
cache_dir aufs /var/cache/squid 1024 16 256
access_log none
cache_mem 256 MB
maximum_object_size 10 MB
read_ahead_gap 64 KB
forwarded_for delete 
httpd_suppress_version_string on
shutdown_lifetime 30 seconds
request_header_access User-Agent deny all
request_header_replace User-Agent Mozilla/5.0 (Windows; MSIE 9.0; Windows NT 9.0; en-US)
visible_hostname The_Shadow 
refresh_pattern ^ftp:		1440	20%	10080
refresh_pattern ^gopher:	1440	0%	1440
refresh_pattern -i (/cgi-bin/|\?) 0	0%	0
refresh_pattern .		0	20%	4320
always_direct allow all
ssl_bump server-first all
sslproxy_cert_error allow all
sslproxy_flags DONT_VERIFY_PEER
http_port 4353 ssl-bump cert=/etc/squid/squid.pem key=/etc/squid/squid.pem generate-host-certificates=on options=NO_SSLv2

I did set cache dir and made max objects and memory use larger, perhaps they need to be the defauls on the pi… I made my testing Host Name “The Shadow” just for grins and chose to have it present a Mozilla ‘user agent’ since I don’t really want the “mobile” pages just ’cause I’m on an ARM chip.

Well, for what it’s worth, that’s what I’ve got so far.

In Conclusion

Hopefully that’s helpful to someone and it can save you searching web pages for a few hours trying to find out where lighttpd puts web page html files…

It isn’t hard to install any of these and get them basically running. It’s the niggly little things like making https: pages cache too that takes time and head scratching.

With that, Back to work for me…

Subscribe to feed

Posted in Tech Bits | Tagged , , , , , , | 7 Comments