chrisp

Mac OS X and VPNs

This page tries to summarize the available VPN protocols and solutions for Mac OS X. I make no claim of completeness or correctness. Note that there were a lot of changes between Mac OS X 10.1 and 10.2, and the information here applies to 10.2 unless noted otherwise. If you find errors or know of a new solution, please mail me.

IPSec

IPSec is the IETF standard for encrypting Internet traffic. It can be used with IPv4 (the protocol currently used on the Internet) and is mandatory for IPv6 (the future Internet protocol). IPSec is very flexible and actually something of an umbrella standard. This means that vendors can make proprietary extensions to the standard. So, while in theory different IPSec implementations should be interoperable, they often are not, for example when a custom authentication method is used.

Since Mac OS X 10.2, Apple ships built-in IPSec support. It is based on the KAME code also used by FreeBSD. There is no graphical user interface. Instead, one has to use the setkey command and the racoon key exchange daemon.

Third-party GUIs for the native IPSec are starting to become available. Among them are:

In addition, there still are third-party implementations of IPSec for Mac OS X; see below.

For further information about IPSec, check out the Understanding IPSec resource page put together by Hologuard.

PPTP

PPTP is a tunneling/VPN protocol originally designed by Microsoft. It is basically PPP with a special authentication scheme, encrypted and encapsulated in IP packets. PPTP has had serious security flaws in the past and some people claim that even the current implementation still has problems. Since I never had to use it, that's all I can say on the subject.

Mac OS X 10.2 ships with support for PPTP. It is configured through the Internet Connect application. Several third-party solutions still exist, including:

  • DigiTunnel, a commercial PPTP client with some nice features.
  • PPTP-GUI, a GUI frontend to the pptp-linux port. A bit old now, but reportedly works on 10.2.2.

The Cisco VPN Client

Cisco offers a wide range of VPN appliances, not all of them compatible. However, most of them – VPN 3000 Concentrators, IOS systems, and PIX firewalls – now support IPSec and are supported by the generic "Cisco VPN Client" (also refered to as the "unity client"). Although based on IPSec, these Cisco devices seem to speak a proprietary authentication protocol, at least in most real-world configurations.

Version 3.6 of the client works from the command line and installs a network kernel extension (NKE) to do the actual encrypting, even on Mac OS X 10.2. The client is not publicly available, but is licensed together with the Cisco hardware and then made available by the organisation that uses it.

Version 3.7 of the client is said to come with a GUI, although I haven't seen it yet. For the meantime, there are two unofficial graphical frontends: VPNConnect and CiscoVPN Frontend. VPNConnect allows you to store your password.

OpenVPN

OpenVPN is an open source VPN program with a custom protocol. Current releases support Mac OS X and compile out of the box as long as real-time compression is disabled. OpenVPN requires the tunnel kernel extension (see below).

OpenVPN sends all traffic encapsulated into UDP. This has a slightly larger overhead than IPSec, but is easier to get through firewalls and still has the desired reliability characteristics. OpenVPN can be configured to use static encryption keys or to dynamically negotiate keys with the peer. The dynamic key exchange uses the proven TLS protocol (the successor to SSL v3), and authenticates both sides using SSL certificates.

OpenVPN is tuned for maximum security and robustness, especially in mobile / dynamic IP situations. I have used it myself and can recommend it.

You can find some Mac OS X specific tips on my OpenVPN page.

VTun

VTun is an open source VPN program with a custom protocol. It runs on a range of Unix flavors, including Linux and Mac OS X. It requires the tunnel kernel extension (see below).

VTun can use UDP or TCP to encapsulate the traffic. This analysis of the protocol revealed some weaknesses in VTun 2.5; the authors are working on improving it.

Until the VTun protocol is improved, I'd recommend to turn VTun's encryption off and use a SSH tunnel to provide security.

Kurt Werle has a page with more information about VTun on Mac OS X.

PPP over SSH or SSL

When no tun device or similar is available, a common technique is to run PPP (which is available almost universally) over some sort of encrypted TCP connection. Various scripts and programs to set this up are available on the net.

The tunnel kernel extension

To use certain VPN programs, you need a so-called tun driver. Simply speaking, this driver gives the program access to the operating system's network stack through a virtual network interface. While Mac OS X "inherited" such a driver from BSD, it is not present in current shipping versions. Stefan Arentz ported FreeBSD's driver to Mac OS X as a loadable kernel extension. I have made some fixes and improvements to get OpenVPN running. See my tunnel page for downloads and more information.

Sep 27, 2009 Copyright © 2002-2011 Christoph Pfisterer