We have also explicitly decided that certain types of software are intrinsically insecure, especially when inputs arrive from outside the firewall. For example, in our research environment, users often develop experimental network protocols that they wish to test over the Internet. Expendable machines provide an environment in which we can deploy such experimental protocols. Testing behind the firewall is permitted only if the tester can control all inputs and no packets cross the firewall.
Our treatment of electronic mail delivery represents a broader example of this approach to insecure software. Based on sendmail's long history of security problems [3,4,6], we have decided not to trust sendmail to deliver electronic mail originating from outside the firewall. The fact that four more security holes [9,8,10,11] were found in sendmail after we deployed our firewall reinforces the wisdom of this decision. Consequently, sendmail processing and delivery of all externally-originating mail must occur on an expendable mail server outside the firewall. In this way, all external mail is completely processed by the insecure sendmail before ever touching a protected machine. Internally-originating mail, on the other hand, is processed by sendmail on a host inside the firewall. This separation of internal and external mail is entirely consistent with our policy of physically separating public and private data.
We accomplish the mail separation by publishing different MX records to protected and unprotected hosts. Protected hosts are only aware of internal mail servers, while external hosts only see the external mail servers. If a research group does not have its own DNS domain, it can implement the mail separation by associating each host with multiple mail servers: the internal mail server is the primary and the expendable mail server is the secondary. The packet filter blocks incoming TCP connections, so all outside mailers fall back to the secondary (external) mail server. Internal mailers, on the other hand, can contact the primary (internal) mail server directly. This alternative design has the drawback that if the primary mail server fails, all internal electronic mail flows to the untrusted external host, where it might be snooped or corrupted. Providing multiple primary mail servers behind the firewall reduces the chance of internal e-mail leakage, but ultimately, preventing electronic mail leakage requires end-to-end encryption.
Normally, this separation of internal and external mail would force users to access two separate spool files, thereby hurting usability and transparency of the firewall. To address this problem, the internal mail server uses NFS to mount the spool directory of the external mail host. A ``Magic Carpet'' process runs as a cron job on the internal firewall at one minute intervals. For each authorized user on the internal system, it checks for mail in the external server's spool directory. E-mail for those users is appended to their internal spool files, and an appropriately formated ``Received'' line  is added to the mail header to reflect the transfer. Biff is notified of each delivered message. To the user, therefore, internal and external mail are indistinguishable, even though they are initially processed by sendmail processes running on different machines. By only transferring mail for internal users, we can support ``guest'' accounts on the external mail server.
The Magic Carpet program does not provide a general-purpose rdist capability. Instead, it is written to operate only with inert data files. For example, we do not preclude the possibility of electronic mail ``bombs'' that exploit bugs in mail reading software. Such errors manifest themselves in user space, however, so the potential damage is less severe. Moreover, the mail bomb problem is a larger issue that the research community must address, particularly as electronic mail is embedding a growing variety of data types.
The expendable mail server is reconstructed nightly from information stored inside the firewall. Its aliases file is reconstructed behind the firewall by taking the internal server's aliases file and appending information collected from user .forward files. This technique provides additional security by ensuring that mail enters the firewall only if it is locally deliverable. It also makes the mail forwarding process transparent to users. We support logging of mail to mailing lists by introducing specially tagged ``dummy'' user names into the expendable server's aliases file. For example, the Magic Carpet delivers mail for users of the form LOG-foo on the expendable server to a logfile foo inside the firewall.
Our design for electronic mail processing offers two significant advantages over alternatives such as Post Office Protocol (POP) . First, except for a slight delay in arrival of mail from outside the firewall, local and remote delivery is transparent to users. Moreover, standard mechanisms for mail forwarding, aliases, and logging are naturally extended to the firewall environment. Our polling model also supports standard mail notification mechanisms such as biff. Second, because internal mail never leaves the firewall, our system allows internal mail delivery to continue and remain secure even when outside network connectivity is lost. As soon as the network is restored, delivery of external mail resumes as usual.