Recently, I enabled HTTPS Decrypt and Scan for Web Filtering profiles on a Spohos UTM 9.3 at a client site. This was done in order to see all of the domains and addresses being accessed via HTTPS (which these days is almost everything). Without enabling this feature, sites like https://www.facebook.com and https://mail.google.com do not show up in Web Usage reports. Turning this on provided immediate benefit. However, it also caused a few issues. Namely some pages displaying what looks like the standard Content Block message listing ‘Broken Pipe’ or ‘Timeout’ as the error.
What This Means
When these errors are displayed, it basically means the client sent a a request to the server and the server terminated the communication without a response. This started happening for the client because all of the https traffic was now being inspected. Which means it is also getting scanned for viruses. Which some end servers didn’t appreciate very much.
Since the sites that were causing issues were mostly financial and banking sites, these domains are trustworthy. The solution is to add them into the Transparent mode skiplist. But, it’s not as simple as adding www.megabank.com into the list. Most of these banks utilize no less than 10 different hosts when you process banking transactions. Each one of them has to be added individually. The skiplist does not allow for wildcards unfortunately. Another “gotcha” is the fact that some of the hosts can point to multiple IP addresses. One of the banks I had to add utilized 19 different hosts (all under the same main domain) and 10 of those hosts resolved to more than 2 IP addresses. Here are the steps I went through to set this up.
The best way to figure out what hosts you need to add is to review the Sites report under Logging & Reporting > Web Protection. Then click on the site. This will bring up the list of Domains for the site. You will need to make an entry for each of these so make a note of all of them.
I’m a big fan of groups, so the first thing I do when I set up anything like this is create a group for the particular service. For bypassing the proxy, I call the highest level group
1. Navigate to Definitions & Users > Network Definitions > New Network Definition
2. In the Name field type “No Proxy”
3. Set the Type to “Network Group”
Now, I’m going to add another group for each of the entities (banks) I want to grant this permission to.
4. Click the + in the top right of the Members box. Another Add Network Definition dialog will appear
5. Enter the entity’s name in the Name field. For this example, I’ll enter “Mega Bank”
6. Set the Type to “Network Group”
7. Save. You will now be back to the “No Proxy” definition.
8. Double-click the “Mega bank” group and you will be presented with a dialog to make changes to it. We are now going to add the hosts.
9. Click the + in the top right of the Members box. Another Add Network Definition dialog will appear.
10. Enter the hostname in the Name field.
11. Set the Type to “DNS Group”. This is critical! Most major sites use some kind of load balancing/fail over that allows for a hostname to point to multiple IP addresses. If you set this to “DNS Host”, and you get redirected to a different IP than the DNS Host record has cached, the errors will come back. Setting the option to “DNS Group” makes the entry store all of the possible IPs the host can resolve to.
12. Enter the hostname again in the hostname field.
Repeat steps 9-12 for all of the hosts in your list.
12. Save the “Mega Bank” group.
13. Save the “No Proxy” group.
Now we need to add the No Proxy group to the skiplist
14. Navigate to Web Protection > Filter Options > Misc.
15. Add the group to the destination hosts/nets list under the Transparent mode skiplist.
The downside to doing this is HTTP/S communication to these hosts will no longer be scanned for policy violations or viruses. So be sure you know what you are risking by allowing connections to these hosts