Like a lot of IT people over the past couple weeks I've been mostly fighting VPN issues and I thought it might be handy to go through a few of the issues I have seen and what (if anything) we have been able to do to fix them.
While I do not want to go into any details about the VPN software we run (for hopefully obvious reasons), I will explain that the VPN solution itself has two options - split tunnelling (the default) which sends specific routes down the VPN and the rest out to the internet and send all where all traffic is tunelled to the datacentre where the VPN terminates.
The datacentre itself has a pretty standard shared 1gbit internet line. This line is used for traffic in and out as well as being the link over which all VPN traffic flows. We do offer services over that link as well.
Because of the numbers of people now working from home and concerns around potential accidental flooding of the link thanks to someone using not using split tunelling and then deciding to stream a high def movie, the decision was made to disable the send all tunnel option so now everyone has to connect using split tunelling only and the same time additional bandwidth limitations were in put in place to prevent any one person from saturating the link by doing something silly like downloading half the file server.
With these mitigartions in place we still saw a few issues and I thought it might be worth listing them:
- Whitelisted Sites
I suspect that, like a lot of companies, we have site to site VPN's to specific partners. Those partners whitelist our external IP addresses which is fine for when people are in the office or when they use the 'send all' tunnel down to the datacentre. The problem we have seen here is that with the send all option disabled, we have had to add a lot of additional routes to the VPN so that those whitelisted sites now travel down the VPN. While relatively easy to work around with the routing fix, the sheer number of them was a surprise.
- Bandwidth, especially with remote sites
We have a number of remote sites, those sites are linked via a site to site VPN. Often, those sites are too small to have their own hardware and so they do not have a VPN endpoint. This means that when they work from home they VPN into the datacentre which can introduce a large amount of latency for them. While far from ideal, for the moment it is the only option we have. We have looked at cloud for this but there are cost concerns.
- Legacy apps
Not exactly a VPN issue but we discovered a few internal legacy apps that are very sensitive to bandwidth. This has meant that when connecting to them, they'll sometimes time out because they are expecting the user to connect at pretty much the full 1gbits like they would in the office. In some cases, providing a desktop they can RDP to has resolved the issues, in other cases, devs have had to take a look under the hood of some seriously old apps.
For the most part, the VPN solution has had basic monitoring - is it up? Does it have enough disk space, etc, etc. Now that we have a lot more people working from home, infosec have requested a lot more monitoring which has come with its own challenges. We've largely been able to exploilt an API And some text manipulation to get the data we needed.
Overall, the user base has adapted well to remote working and the VPN solution has coped admirably. There have been challenges and frustrations but things are starting to head the right way.
Subscribe to Ramblings of a Sysadmin
Get the latest posts delivered right to your inbox