Pi-Hole: Ad blocking at the network level

[spacein_vader]

A new thread after I got a bit off topic regarding drill bits and ended up talking about a Raspberry Pi powered ad-blocking solution that you can setup once and covers your whole network. Interested? Read on:

Get a Raspberry Pi, stick a minimal install of Raspbian/DietPi on it and install Pi-Hole using literally one line of code:

curl -sSL https://install.pi-hole.net | bash

Essentially it works as a global ad-blocker for your entire network by becoming your DNS server. If a client requests a domain on its blacklist (lists can be added, or sites black or white listed individually, with wildcard support,) it instead answers the request itself with a 1x1 pixel white jpg. Curated blacklists are auto-updated once a week and the GUI reports on most blocked, most allowed etc domains so you'll soon notice anything untoward appearing. Once it's up and running just alter your routers DNS setting and you're done.

The big advantages over traditional ad-blocking are:

1. It affects all clients on the network, even those (printers, TVs etc.) that don't usually support ad-blocking software, or when your wife/husband/kids let their friends put their devices on your network.
2. Most websites don't detect an ad-blocker in situ as there isn't one on the device, minimising the 'please turn off your ad-blocker' stuff.
3. It can block things the OS either can't or won't, like Microsoft telemetry.
4. If you install Unbound as well, it becomes a recursive DNS, keeping a local DNS cache and only referring to authoritative name servers.
5. It's all open source.

The only downside I've found is that as it gets more popular some device makers/providers (Google, I'm looking at you in particular,) have started hard coding in their own DNS to the hardware. If your router supports it (anything commercial grade or capable of running DD-WRT or similar,) you can force it to redirect any DNS traffic to the Pi anyway.

(By Admin - I have moved and merged the old post, but the chronological order may be odd - but it’s worth it!)

Oh tried that Audiobacon site but it doesn't like my adblocker, greys/fades out the site and puts a "please consider removing" message over about 80% of my screen.

Sadly, the "close" button doesn't work either, so it's add them to blocker whitelist, or beggar off. I beggar'd off.

They're, of course, entitled to prevent adblockers. But I use one by default, everywhere, and I'm entitled to decline to use their site, as I do elsewhere when this happens, rather than change my principles.

I do not want adverts, period. It'd have to be a site I desperately wanted, or needed to get me to unblock, and it hasn't happened yet. I even pre-record any and all TV I want on ad-carrying channels, and either skip ads, or more likely delete then entirely, before watching. That's how much I loathe adverts.

I may already be preaching to the choir here, but if not:

Strongly consider getting (or repurposing an existing,) Raspberry Pi, sticking a minimal install of Raspbian/DietPi on it and installing Pi-Hole. Essentially it works as a global ad-blocker for your entire network by becoming your DNS server. If a client requests a domain on its blacklist (lists can be added, or sites black or white listed individually, with wildcard support,) it instead answers the request itself with a 1x1 pixel white jpg. Curated blacklists are auto-updated once a week and the GUI reports on most blocked, most allowed etc domains so you'll soon notice anything untoward appearing. Once it's up and running just alter your routers DNS setting and you're done.

The big advantages over traditional ad-blocking are:

1. It affects all clients on the network, even those (printers, TVs etc.) that don't usually support ad-blocking software, or when your wife/husband/kids let their friends put their devices on your network.
2. Most websites don't detect an ad-blocker in situ as there isn't one on the device, minimising the 'please turn off your ad-blocker' stuff.
3. It can block things the OS either can't or won't, like Microsoft telemetry.
4. If you install Unbound as well, it becomes a recursive DNS, keeping a local DNS cache and only referring to authoritative name servers.
5. It's all open source.

The only downside I've found is that as it gets more popular some device makers/providers (Google, I'm looking at you in particular,) have started hard coding in their own DNS to the hardware. If your router supports it (anything commercial grade or capable of running DD-WRT or similar,) you can force it to redirect any DNS traffic to the Pi anyway.

I may already be preaching to the choir here, but if not:

Strongly consider getting (or repurposing an existing,) Raspberry Pi, sticking a minimal install of Raspbian/DietPi on it and installing Pi-Hole. Essentially it works as a global ad-blocker for your entire network by becoming your DNS server. If a client requests a domain on its blacklist (lists can be added, or sites black or white listed individually, with wildcard support,) it instead answers the request itself with a 1x1 pixel white jpg. Curated blacklists are auto-updated once a week and the GUI reports on most blocked, most allowed etc domains so you'll soon notice anything untoward appearing. Once it's up and running just alter your routers DNS setting and you're done.

The big advantages over traditional ad-blocking are:

1. It affects all clients on the network, even those (printers, TVs etc.) that don't usually support ad-blocking software, or when your wife/husband/kids let their friends put their devices on your network.
2. Most websites don't detect an ad-blocker in situ as there isn't one on the device, minimising the 'please turn off your ad-blocker' stuff.
3. It can block things the OS either can't or won't, like Microsoft telemetry.
4. If you install Unbound as well, it becomes a recursive DNS, keeping a local DNS cache and only referring to authoritative name servers.
5. It's all open source.

The only downside I've found is that as it gets more popular some device makers/providers (Google, I'm looking at you in particular,) have started hard coding in their own DNS to the hardware. If your router supports it (anything commercial grade or capable of running DD-WRT or similar,) you can force it to redirect any DNS traffic to the Pi anyway.

I had to read that twice to be sure I got it, but it looks to be VERY interesting.

I've never-owned a Pi, but it is something I've kept meaning to get around to, if only for a plag-around. This might well be just the place and reason to finally get around to it.

Off-topuc perhaps, but a wholehearedly appreciated off-topic, and one I am grateful for. Thank you. It is an approach I hadn't considered.


Damn. It seems you really can teach an old dog new tricks, after all.

[QUOT=spacein_vader]Although it's designed for a pi it'll run on any Debian derivative so you could put it on a PC or VM to evaluate. I decided against that as it's tested on the Pi and because it needs to be always on and a pi only pulls a few watts.

.....

Starting to think I should have started a seperate thread...

Excellent idea.

Tell you what -- create a thread somewhere and point me (or another admin) at it I/they will copy or move these posts to it. I could do it all myself now, but then it'd show as my thread and, y'know, credit where it's due.


Also, good points about Pi power usage, etc. On an existing server might be better if you already have one running 24/7 anyway, but mine aren't. I turn them on when I need them and sometimes that's days, even weeks apart. Pi is definitely the route for me.

Just thinking I could run it on my Fedora 28 based server...

A lot of organisations use it like that, it helps them minimise bandwidth use by multiple clients and there's nothing to stop them blocking *.facebook.com or whatever to minimise staff personal use.

Be prepared to get addicted to the stats screen at least for the first few weeks. Initially to see just how much junk is out there (my Pi currently has 591,999 domains on the blocklist and has blocked 88.4% of all requests it receives!) but also to see the top permitted domains to see what is happening on your network.

For example I've got 2 Roku boxes I use as end points for an Emby media server that lives in my garage, I like them and they're good devices. Turns out that as well as ads on the home screen both try to phone home to various logs.roku.com domains every 30 seconds. I've now blocked both the ads and the phoning home without compromising the usability (all the apps work and it can download software updates.) Next was something pinging netflix every 60 seconds. I don't use netflix. Tracked it back to the dormant app on my 7 year old 'smart' TV in the spare room.

Aside from the default blocklist I use a selection from a contributor called Wally3k. They're coded with a tick for those that won't break sites, a cross that might block multiple useful sites and a > if they are reported to occasionally block something useful. I've only got the tick list ones installed but it depends how willing you are to add things to the whitelist and how much those who live with you will complain if the odd site won't work. SWMBO was slightly irked to see Google shopping suggestions don't work any more but soon got over it.

Starting to think I should have started a seperate thread...

[philehidiot]

It has been a long time since I did any real notworking - does this adhere to your router via a LAN connection into it's rear?

[spacein_vader]

It has been a long time since I did any real notworking - does this adhere to your router via a LAN connection into it's rear?

The pi uses a standard RJ45 connector so you can connect it to the network however you like. You can even do it via WiFi but that's not recommended.

[peterb]

I guess any of the later version Raspberry Pis will do ( with the built in ethernet) and as DNS requests are low bandwidth, 10/100 Mb/s would be fine.

[spacein_vader]

I guess any of the later version Raspberry Pis will do ( with the built in ethernet) and as DNS requests are low bandwidth, 10/100 Mb/s would be fine.

If you aren't using Unbound for recursive DNS then even a Pi Zero has enough grunt, the problem with using one of those is that they're Wi-Fi only. As you point out, DNS requests are low bandwidth but they are sensitive to latency which Wi-Fi isn't great for.

That said, if you don't already own a Pi you might as well get a 3+ anyway given the price differential between it and earlier versions. They all have the same network speed as the ethernet port is connected over USB internally anyway but for this use case it isn't a problem. Most people just seem to find a 1m patch cable and sit the Pi next to the router. Many modern routers also have USB ports on them as well, which can often be used to power the Pi. Works on my FritzBox anyway.

If you do have Chromecasts or some brands of smart TV that use a hard coded DNS server regardless of what the router tells them you can forward all port 53 traffic from the network to the Pi's IP address. Most higher end routers or those using Open-WRT, Tomato or similar can do this.

[Bagnaj97]

Note this doesn't have to run on a RPi either. I'm using it on my home server running Ubuntu 18.04.
It's quite interesting to see what's being allowed or blocked, for example regular traffic from my TV to log-ingestion-eu.samsungacr.com. I've blocked that now, god knows what my TV was reporting back...

[CAT-THE-FIFTH]

Note this doesn't have to run on a RPi either. I'm using it on my home server running Ubuntu 18.04.
It's quite interesting to see what's being allowed or blocked, for example regular traffic from my TV to log-ingestion-eu.samsungacr.com. I've blocked that now, god knows what my TV was reporting back...

https://phyks.me/2017/12/stop-networ...amsung-tv.html
https://www.consumerreports.org/priv...ping-features/

[peterb]

Note this doesn't have to run on a RPi either. I'm using it on my home server running Ubuntu 18.04.
It's quite interesting to see what's being allowed or blocked, for example regular traffic from my TV to log-ingestion-eu.samsungacr.com. I've blocked that now, god knows what my TV was reporting back...

Yes, that was my first thought, running it on my Fedora powered server, but Pis are not expensive and separating a DNS function - which is pretty critical for the network - from the server seems a better solution.

Edit - and my Pi and a case has been ordered! I'm picking it up from my local RS Trade Centre this morning.

Just one thought

Many web sites rely on adverts to keep running. That includes HEXUS. Long term, the widespread use of these may result in more paywall websites and a reduction in content.

With regard to HEXUS, the advertising is not particularly intrusive, so you might like to whitelist the HEXUS adservers.

[peterb]

And picked up the Pi and a case - and got it working.

Installation was reasonably straightforward - my PSU for the Pi was a bit underpowered so it rebooted on loading the OS, and a DNS issue prevented on of the Pi-hole dependencies from downloading - but after that it was plain sailing.

It was necessary to renew DHCP leases from the router after I had changed the DNS settings (caused a couple of minutes head scratching) but seems to be up and running (and not blocking ads on HEXUS)

Seems to be blocking a few more things too - but analysis will have to wait.

Saracen - highly recommended.

[spacein_vader]

And picked up the Pi and a case - and got it working.

Installation was reasonably straightforward - my PSU for the Oi was a bit underpowered so it rebooted on loading the OS, and a DNS issue prevented on of the Pi-hole dependencies from downloading - but afte that it was plain sailing.

It was necessary to renew DHCP leases from the router after I had changed the DNS settings (caused a couple of minutes head scratching) but seems to be up and running (and not blocking ads on HEXUS)

Seems to be blocking a few more things too - but analysis will have to wait.

Saracen - highly recommended.

It's worth keeping an eye on traffic for a week or two, particularly the top domains allowed/blocked and Google any allowed ones you don't recognise to see if you should block them. Just discovered the laser printer I bought last week tries to talk to HP every hour or so.

In the drill bits thread (could someone merge the relevent posts into this one?) I linked to Wally3ks curated list of block list which is quite a handy resource.

It's worth knowing that some domains will increase in regularity when you block them. So my Rokus try to phone home every 5 minutes, but when it's blocked it tries again every 30 seconds so looks worse on the logs.

Did you set up unbound or are you still using external DNS servers? If so I'd recommend one that isn't Google and supports DNSSEC then enabling it on pi hole. Cloudflares 1.1.1.1 seems to be the current favourite for this.

Finally if you have any Android/Google devices I strongly suggest forcing all DNS traffic to the Pi.

[peterb]

Problem with the old drill bits posts is that they pre-date the first post here!

I didn’t set up unbound, but I use Zen’s DNS servers for the upstream DNS. I have now got my router (which is my DHCP server) serving the pinhole local IP address as the DNS server (which caused the head scratching until I renewed the DHCP lease on connected devices)

It was actually a lot simpler than it sounds!

I am now wondering whether to install it on my main server rather than leave it on the Pi.. I can see pros and cons for either!

[spacein_vader]

Problem with the old drill bits posts is that they pre-date the first post here!

I didn’t set up unbound, but I use Zen’s DNS servers for the upstream DNS. I have now got my router (which is my DHCP server) serving the pinhole local IP address as the DNS server (which caused the head scratching until I renewed the DHCP lease on connected devices)

It was actually a lot simpler than it sounds!

I am now wondering whether to install it on my main server rather than leave it on the Pi.. I can see pros and cons for either!

I'm also with Zen, didn't even check if their DNS supports DNSSEC but I'm using Unbound for my DNS now anyway. First time you go to an address there is a slight delay but once it's cached it's very quick.

I kept mine off my server, I see DNS as important enough to be independent. Occasionally I take the server (Ubuntu Server 18.04,) down to upgrade or tinker and I wouldn't want to pull down internet connectivity with it.

[peterb]

I’ve moved and merged the posts from the other thread - it’s a bit clunky but probably encapsulates everything (and cleans up the other thread )

[peterb]

I'm also with Zen, didn't even check if their DNS supports DNSSEC but I'm using Unbound for my DNS now anyway. First time you go to an address there is a slight delay but once it's cached it's very quick.

Yes, I might add that later - one step at a time!

I kept mine off my server, I see DNS as important enough to be independent. Occasionally I take the server (Ubuntu Server 18.04,) down to upgrade or tinker and I wouldn't want to pull down internet connectivity with it.

Yes, I’m tending to that PoV

[Saracen]

I’ve moved and merged the posts from the other thread - it’s a bit clunky but probably encapsulates everything (and cleans up the other thread )

I had planned to sort this today, so thanks for sorting it for me.

Now, if I leave out a bucket and some shampoo, the car needs a wash. It's outside HEXUS towers, between DR's Ferrari and Zak's Porsche Turbo. Can't miss it, the '04 Polo.

Don't forget the wheels.



/Ducks to avoid flying brick, then runs and hides.

Seriously, thanks Peter.

[Saracen]

Also, while I'm thanking people, a big thankyou, hug and a kiss for spacein_vader.

This is a two-fer for me.

a) Looks like a damn good idea.

b) Been meaning to get a Pi to pkay with since, well, when they came out. I just haven't gotten a round TUIT. This is a perfect excuse ... I mean, reason, yeah reason, I need to get one of these. Plus maybe a spare.

Thanks, Mr Vader.