zaterdag 17 september 2011

Setting up a Virtual Firewall using VMWare

Last week I was setting up a virtual firewall to kee my homenetwork safe. Now, my old setup looked like this:

The server is actually a dual core machine with 4 gigs of memory, still performance was a little slow at times. Turns out ISA might be responsible for that. So, my annoyance  with ISA server hogging all resources was the main reason to find a better solution. (I think that the SQL server that get's installed with ISA interfered with the CSS dedicated server). In addition, ISA is incompatible with IPv6, which is a deal breaker.

But there was another problem. I have a 120 MBit connection and I often back up > 1 TB  from my domain controller (which is also a file server) to my backup machine. So as you can imagine, if I am downloading (since the server is also the gateway) while backing up, both processes go slower as they share the single LAN NIC. The best option would therefore be a solution where I would use a single physical machine (hey I do have to pay for the power I use), but having 3 NICs. 1 as a WAN NIC, 1 as a LAN NIC for the firewall and 1 as a LAN NIC for the domain controller.

So I decided to go with a more elegant aproach, but I did want to hold on to server 2003 as a DC. I downloaded vmware server 2.0 (should do the job) and created a linux virtual machine. Now the plan was to bridge 1 virtual NIC to the physical WAN NIC and bridge one virtual NIC to the physical LAN NIC. At the same time the 3rd NIC, to connect the DC, should be on it's own.

It should look like this:

Where the small server represents the virtual machine with two dedicated NICs.

Now I thought this should not at all be difficult, and it turns out it isn't but you should be aware of what VMware actually means when they say bridging. When you bridge a virtual NIC to a physical NIC, you do not bridge NIC to NIC. You bridge a virtual switch, to which the virtual NIC is connected, to the physical NIC. Therefore, the physical NIC can still have it's own IP address and be reached from the network, parallel to the virtual NIC. Now this is a security risk. :)  The solution is to disable the IP protocols on the physical NICs if you want them to be available only to the virtual NIC.

So here is the layout I used in the virtual network editor. Note that I did not change the default nets, VMNET0, VMNET1 and VMNET8. If you don't need more than 7 networks (which is not very likely in a home setting) I recommend letting them be. :)

Here the Realtek adapter is the WAN adapter (or NIC1) and the PRO/100S is the LAN adapter (NIC2). (after this initial test I did replace it with a gigabit adapter)

Also, to prevent any unwanted bridging, I turned of all automatic bridging:

The funny thing is that you can only do this by excluding all adapters and keeping the automatic bridging tickbox ticked. If you untick it, any non bridged network adapters will automatically be bridged to VMNET0. I know, weird... The default VMNETs I disabled in the host OS (simple and quick):

And I created a virtual machine (Firewall) , bridging 1 virtual NIC to VMNET2 (WAN) and 1 virtual NIC to VMNET3 (LAN exclusive for firewall)

Originally I had hoped to use PFsense. However, it turns out that PFsense is incompatible with the UBEE modem provided to me by Ziggo, and only when running PSense on VMware. Now as I could not change the virtual solution (see above), nor my provider or my modem, I changed to Firewall OS to IPfire, which is also quite resource efficient. 

After I disabled the IP protocols and Client for Microsoft Networks. Keeping only the VMware bridging protocol enabled:

Fianlyl, the host only saw 1 NIC and everything worked (because I was running in a test environment, the link is 100Mbps :P )! 

The final result is that my server is faster (running a dedicated CSS server next to it's other tasks without breaking a sweat) and I am able to backup @ full speed while downloading! 

I never figured out why a virtual pfsense install is incompatible with Ziggo + UBEE, but I am not the only one who has come across this problem. The strange thing is that the WAN adapter on pfsense get's an IP address by DHCP but there is simply no traffic possible (no http, no icmp nothing).