PFSense Router with old bits :)

[Noxvayl]

After watching this:
https://youtu.be/ledv33t6SNE

And reading these:
https://arstechnica.co.uk/gadgets/20...speed-testing/
https://arstechnica.co.uk/gadgets/20...-linux-router/

I've been using a PFSense router and loving it. After researching and spending money on fancy DrayTek equipment before I thought I needed a change of pace

So at the moment I have a Vigor 130 feeding the router and then a Homeplug network feeding all the devices in the home including a Ubiquity WiFi AP. In comparison to the SkyHub it is SOOOO much better, and I'm sure better than anything you can purchase bar enterprise stuff.

Anyone else doing the same, or something similar?

[Dashers]

I've been doing this for quite some time. I originally intended to just use openSUSE and iptables, but there was a bug in the version which was stopping PPPoE from authenticating. I slapped on a copy of pfsense to see if that worked and it did. I was so blown away with how easy and yet advanced the configuration was I shifted over to that in a few hours.

My install is virtualised on VMware ESXi hypervisor. I've got a second nic in my "server" (cheap, low power PC) which is the upstream for a virtual switch which in turn has the virtual WAN interface connected to. I was slightly suspect that PPPoE authentication wouldn't work through a virtual switch, but it does.

The only required network gear is the openreach FTTC modem. Much faster than the dedicated buffalo unit I had before.

All running off the same hardware that does my media centre, email and lab stuff. Weighs in at about 50W. The next thing to do is buy a wifi card or dongle and pass that through to a virtual openWRT install, but I haven't got round to that yet.

[watercooled]

I ran m0n0wall (later forked to smallwall) for many years on a PCEngines ALIX, followed by one of their updates APU boards. Properly rock solid, both hardware and software.

We recently changed from Virgin to BT however, and I'm (just barely) putting up with the 'Smart Hub' for now, but I do have a VDSL modem standing by if I either get some time to tinker, or if it annoys me again. The WiFi is pretty good on the Hub though so can't knock it too much.

If you want some dedicated hardware for firewall software like this I can really recommend the stuff by PC Engines: https://www.pcengines.ch/apu2.htm
With that (well, the older version) and m0n0wall, I frequently got uptime of months, mainly interrupted by having to unplug it to move stuff about (I don't recall the ALIX ever crashing, though the APU did lock up just the once I think - LAN LEDs blinking and no response, perhaps a power supply problem, but it didn't happen again so I didn't really investigate it. That was also the first gen version - the new one is out now, which incidentally uses Intel NICs vs the Realtek ones on my first-gen model). They're not cheap at first glance, but really not much when you compare them against a half-decent consumer router!

I also measured power consumption of the APU board at around 5W - less than most consumer routers! To be fair, that doesn't include a modem, and I didn't install a wifi card which might have added an extra watt or two, but I'd been using one of those portable WiFi access points which seemed to perform about as well as the AP I had been using at the time, and used so little energy (USB powered) it didn't even register on my 1W resolution power meter.

Although I haven't yet put it through its paces, I ended up picking up one of these Netgear modems on offer: https://www.amazon.co.uk/d/Modems/NE...dem/B01GL3YPHI
Reviews seem solid enough, it's cheaper than many of the alternatives, and it's also reasonably low power IIRC.

You could also use one of these in bridge mode, as recommended by Thinkbroadband: https://www.amazon.co.uk/TP-Link-Wir.../dp/B013OXVA6M

m0n0/smallwall is a more minimal OS than pfSense and similar - it aimed at being a rock solid firewall and did it extremely well. I also preferred the traffic shaper over those in pfSense (at the time I tried at least) and its performance was somewhat better too, but probably not a huge concern unless you're pushing >500Mb/s. m0n0's traffic shaper was very simple to set up and, like everything else, performed flawlessly - you could simultaneously saturate upload and download, and still play an online FPS game none the wiser. I never quite figured out why, but pfSense's shaper never seemed to work as well, and dropped a ton of packets when links became saturated - it's one of the reasons I just preferred m0n0.

If you like playing with them (I've messed about with various distros on either a spare PC or in VMs), another more recent one to try out is OPNsense which I also find to be very good.

[Noxvayl]

Get an external PoE WiFi access point and a homeplug with PoE support. So much easier. Can install it in the exact position you want it, and get water proof ones for outside

[Bluecube]

@watercooled Can monowall run a VPN and also redirect certain traffic not to go via the VPN? (Hope that was clear!) I've been considering pfSense for this purpose but if monowall can do this while maintaining better performance than pfSense then I'd consider it.

[Noxvayl]

I'm definitely going to try SmallWall... will it work ok installed onto a USB drive?

Currently got my NASBox on a USB drive and it is doing perfectly so would really like to switch over to a USB.

[watercooled]

@watercooled Can monowall run a VPN and also redirect certain traffic not to go via the VPN? (Hope that was clear!) I've been considering pfSense for this purpose but if monowall can do this while maintaining better performance than pfSense then I'd consider it.

m0n0wall itself is defunct now, but the project was forked into Smallwall and run by one of the former m0n0wall devs, who's always happy to answer questions on the forums.

It does support VPNs, but unfortunately OpenVPN is not yet implemented. If I understand what you're trying to do, then yes it should be possible, with e.g. traffic from some hosts forwarded through the VPN, and others straight out onto the Internet. Documentation is here but I'm happy to have a play with it if it's any help? http://smallwall.org/docs/handbook/index.html

WRT performance, it's not night and day IIRC, but m0n0/smoothwall use ipfilter (not to be confused with ipfw) which seems to be a bit less resource intensive than pf, which pfSense uses (as the name implies). IIRC OPNsense uses a combination of software in the name of performance, so while I've not put it to the test, OPNsense might offer better performance too. But if you're using anything like a modern desktop CPU, I don't think any of them would struggle even with symmetric gigabit. I'm almost sure I did some comparisons with iperf but I can't seem to find them, I've probably either written them down somewhere, saved them, or posted them on one of the forums (or a combination) - I'll post if I manage to find them!

I'm definitely going to try SmallWall... will it work ok installed onto a USB drive?

Currently got my NASBox on a USB drive and it is doing perfectly so would really like to switch over to a USB.

m0n0/Smallwall are designed to write very little to the boot device - it's almost read-only apart from saving the xml configuration file, so booting from USB should be fine. Just make sure you use the boot image rather than the 'LiveCD' one, or configuration won't save and it will ask you to install every boot. The PCEngines ALIX booted from a CompactFlash card, and the APU1C from an SD card. pfSense also has an 'embedded' version that functions similarly, but doesn't support some features because of this.

[peterb]

Smallwall dev doesn't look very active - last release was two years ago and the latest beta was a year ago.

[watercooled]

Smallwall dev doesn't look very active - last release was two years ago and the latest beta was a year ago.

That's not necessarily a bad thing - it's a very lean/minimal distribution by design and tends to adopt only stable and proven software, therefore minimising the risk of bugs. You only need to fix something if it's broken after all!

New features are occasionally rolled out, it does what it does very well but isn't really targeted at the same range of features offered by larger distributions.

Lee Sharp, the project leader (I'm assuming that title BTW) is still active on the forums and responds to questions quite quickly. It's still very much actively supported, don't let the infrequent releases fool you.

BTW there's also another fork of m0n0wall which I've not really looked in to, but might be of interest: http://t1n1wall.com/
I'm not sure who's behind this fork, Lee Sharp (Smallwall) was heavily involved with the original m0n0wall though.

[DanceswithUnix]

I am sort of doing something like that. I have a VDSL router feeding the house, but that goes to a PC with two network interfaces which acts as secondary router and handles email etc. That lets me run a DMZ between the consumer router and the PC router for Internet facing services.

[watercooled]

Warning - rather long post about software politics, feel free to skip if it doesn't interest you! However there will be some bits about future hardware requirements that may interest you (check the bold bits).

I've just been having a scan through pfSense news/development, and it seems like the project is surrounded by aggression and toxicity lately! It's really quite a shame as I don't remember it being anything like that when I was posting on their forums a few years ago. Bear in mind this is just what I've picked up from what's turned up in Google searches, if anyone knows more, please don't get offended one way or the other (it's certainly not intentional and I've done my best to look at it objectively, before starting reading I was exactly neutral, I'm now not so sure given where most of the aggression seems to originate from) - I would like to learn more about it though!

There's some childish arguing/borderline bullying going on between pfSense and OPNsense developers and associated fanboys taking it way too far. You even have nonsense in the form of Twitter posts poking fun at decisions made by the fork - seriously? And people implying (or explicitly claiming) that a fork is a 'ripoff' of pfSense - what, in the same way that pfSense is a 'ripoff' of m0n0wall? Get real... They're all open source projects - isn't forking a project you don't agree with the direction of, exactly within the spirit of open source? Maybe I'm missing something but everywhere I've looked for an explanation (though it would need to be a heck of an explanation to justify some of what I've seen written) is just filled with the same toxicity. People have even asked the same question on forums and Reddit, and are met with the very same.

Bear in mind both companies are backed/funded/supported somehow by network hardware companies which may or may not influence decisions (decide for yourself). Before OPNsense's creation, AFAIK their hardware 'parent' were selling hardware running pfSense, but pfSense also have a competing 'parent' selling their own hardware. It's not unreasonable to think that conflicts of interest are a possibility in this situation.

Apparently (hearsay follows) some people behind pfSense were hostile towards this other company, and also seem to be (or have been) publicly hostile towards OpenBSD (with an apology mentioned in the following link), which as this thread mentions, was heavily relied upon for pfSense's creation, even 'borrowing' the project's name to form part of their own copyrighted name. It's one thing to do that, it's quite another to become incredibly toxic towards another project you claim to be doing much of the same. Hypocritical perhaps?

On to some stuff I've learned about the direction of pfSense (and IMO they constitute perfectly valid reasons to want to fork a project like OPNsense have done). There's the bit about the competing companies (above), but more than that, there are some IMO very bizarre decisions being made about future releases, and justifications that don't seem to make sense (DanceswithUnix and any other devs might have some more knowledge about these bits).

Firstly, in the 2.5 release they will be abandoning support for i386 and NanoBSD images entirely. That's kinda justifiable given how long AMD64 ISA CPUs have been standard, and I understand that dropping a legacy uArch could make some financial sense if some features require e.g. extensive testing on each platform, or extensions only present in newer CPUs, but on the surface at least, OK I'll take that at face value. However, they've then gone on to support ARMv7 in order to support, AFAICT, just the one appliance they sell. It doesn't seem like the ARM software is publicly available either. Why not ARMv8 and one of the remotely recent Cortex cores? That A8 box is clearly CPU-limited, the page claiming throughput 'exceeding 100Mbps' - that's hardly exotic performance even for a home broadband connection, and this is $150! For something with considerably less CPU grunt than a RasPi. They do claim >400Mbps forwarding without a packet filter in the footnote i.e. not actually operating as a firewall, which might just be down the the Ethernet ports being 'switched'. Even cheap SOHO routers can do better than that. It's a shame as it's a decent concept - the CPU choice just seems weird. Market segmentation perhaps?

WRT the NanoBSD images, I'm not too sure what that means for embedded installations - one of the big advantages of the m0n0wall-derived distros is their ability to run in-memory and write very little to disk, making them particularly well-suited to installing on e.g. memory cards. Once it's loaded, even if that takes a while due to a slow card, it's running and you can pretty much ignore disk performance (and write endurance/consistency of memory cards). I think I'll have to read up some more on this one.

Further down the line, with the 3.0 release, they're dropping support for any hardware without AES-NI support. Like, it literally won't even boot. Reddit post 'explaining' this here: https://www.reddit.com/r/PFSENSE/com...aesni/dh0qi53/
None of that makes sense to me. I'll address the reasons I've read:
* Software AES-GCM sidechannel attacks. Say for a minute I accept that's a valid concern, ignoring the various libraries hardened against exactly this, if you're hellbent on avoiding software AES implementations, why not offer additional ciphersuites for systems without AES-NI support, much like web browsers do (also because hardware implementations tend to be faster and and more power efficient)? Ciphers like ChaCha20 are inherently not vulnerable to these attacks. Oh but the devs complain about not being RFC compliance, which is a) untrue (it's a recommendation, they're freely available to read for yourself) and b) hardly matters anyway - being RFC compliant doesn't magically make you safe.
*Alternative ciphers don't have hardware offloads - so... this is a problem for whom exactly? ChaCha20 in software is generally faster than AES-GCM in software - so if this is *not* about performance as they claim, and software AES-GCM was not a problem, why is something faster suddenly a problem?
*Their cloud thing - apparently the choice has something to do with their cloud management platform, which I sure *everybody* wants to use - OK so apparently there are performance reasons for AES-NI support on their servers - there is absolutely no reason this needs to be extended to client software, it makes no difference which implementation is used at either end of the connection.
*Lastly, unless you're pushing a lot of traffic through a VPN, why does any of this even matter for a standalone firewall, which will have zero traffic going near their cloud platform, and the only encryption likely to be done will be for the confuguration GUI - hardly high-throughput, and likely on LAN anyway!
*Having to develop/maintain a software implementation. That's... kinda already done in the likes of OpenSSL, LibreSSL, BearSSL (just a few off the top of my head). Unless they're talking about rolling their own crypto, which is a catastrophically bad idea anyway.

So... is this really a valid reason for breaking compatibility with a large amount of hardware pfSense already runs on, for no real benefit, or is it more to get you to 'buy this shiny new appliance made by our parent'? The tone on their forums seems to be along the lines of, any hardware not supporting AES-NI is ancient and needs replacing anyway. Sorry but that's just nonsense - e.g. Intel only started supporting AES-NI on their Celerons etc with Skylake - it's bonkers to suggest Haswell is ancient and not capable of running even a gigabit firewall. xD

And that's not to mention the embedded hardware without AES-NI support, which is perfectly fine performance-wise. It's just a fallacy to suggest that hardware being old (in someone's opinion) = unsuitable for its purpose. The Typhoon's Motorola 68020 processors should all be torn out and replaced with MOAR POWAHHHHH!!! it seems. After all, the latest-and-greatest Atom C2000 networking gear is completely flawless isn't it? Oh wait... (BTW that's not a knock at Netgate hardware in the slightest, rather the mentality that old=bad, new=best)

And again, a perfectly valid reason for a fork IMHO!

Edit: Just to be clear, if you're thinking about the crypto stuff being important for SSL web traffic flowing through the firewall, it's not - it's forwarded through the firewall untouched, just as any non-SSL traffic would be. Also, sorry, that ended up being a monster post.

[watercooled]

Sorry, I did get a bit carried away there, it's just sad to see a project plagued with emotion like this. Granted the devs aren't responsible for a lot of it, but when people are putting so much effort into getting defensive, publicly poking fun and general vitriol, it's maybe not going in to development. That, and it seems they're clamming up about roadmaps etc for fear of it being 'ripped off', it's just not like how I remember it.

If someone thinks they've gone too far towards the commercial side, even if not everyone agrees, what's wrong with forking? It happens all the time - LibreOffice, OpenSolaris forks, forks of Chromium/Firefox, and the list goes on. As I said, pfSense itself is a fork of m0n0wall because the devs didn't agree on its minimal nature - I think both have their own place, and agree with m0n0wall's desire to stay as a minimal firewall distro.

[Noxvayl]

I've been trying for two days to get SmallWall installed, the damn thing won't install to a hard drive

I'm wondering if I've created the wrong CD image... there are only 3 downloads from the site.

[watercooled]

I've found FreeBSD can be a bit picky which hardware the auto-installer will work on. What exactly happens, or what error messages do you get?

[Noxvayl]

No error message, it completes and then fails to boot if the installation media is not present.