Collax Business Server seems to be a great alternative to Microsoft Small Business Server.
CBS is a Linux-based system which installs like an appliance and is configured from a simple web console.
Collax Business Server received an incredible review from VAR Business in their showdown article
Review: Can Linux-based Collax Replace Microsoft Small Business Server? Collax Business Server Features Networking
ISDN, ISDN channel bundling, xDSL (PPPoE, PPtP), Ethernet, analog modem, link failover, traffic shaping, tagged VLAN Support, RAS, RAS callback, DNS, DHCP
Mail Server
Protocols include: POP, IMAP, ESMTP, Multidrop, Secure (TLS), and SMTP authorization. This also offers optional virus protection, spamfilter, quotas, extension filtering (e.g. *.exe, *.mpg), external mail relaying, administration of multiple mail domains, customizable polling intervals, and mailing list management.
Fileserver
Supports Windows file sharing (SMB), FTP, NFS, adminsitration of user and user groups, quotas, optional virus protecion, data synchonization, and LVM
VPN
IPSec with X.509 or PSK, PPtP, DynVPN, CA for PKI, CRL
Web-Proxy
Authenticated or transparent, administration by time, user, or user groups; URL filtering, cache up to 8 GB, logging
Webserver
http and https, WebDAV, http put support, virtual hosts, scripting includes PHP4, CGI, and Perl
Firewall
Stateful Inspection, masquerading, NAT, DMZ, graphical network matrix
Fax Server
Receiving and sending on multiple MSN, SMS (send), mail-to-fax and fax-to-mail, fax client, spam protection
User Page
Web mailer, comprehensive filtering, out-of-office notification, contacts, calendar, access to fileserver
User administration
central user administration & LDAP right, local or external server, PDC, ADS connection, Kerberos
Back-up
via SMB, NFS, tape, incremental or full backup, scheduled or manual, full restoration or file based
Other functions
USV support, NTP server, instant messaging, monitoring, configuration data base, comprehensive online help