Websense Esga V76 201 Student Guide 18205o
This document was ed by and they confirmed that they have the permission to share it. If you are author or own the copyright of this book, please report to us by using this report form. Report 2z6p3t
Overview 5o1f4z
& View Websense Esga V76 201 Student Guide as PDF for free.
More details 6z3438
- Words: 30,523
- Pages: 202
Webs ens e® TRI TON™Sol ut i on
Techni calEnabl ementPr ogr am
E S GAv 7. 6Cour s e201 S t udentGui de Rev . 691267
©2011 Websense, Inc. All rights reserved.
Websense and the Websense logo are ed trademarks of Websense, Inc. in the United States and other countries. TRITON is a trademark of Websense, Inc. in the United States and other countries. All other trademarks are properties of their respective owners.
Table of Contents Websense Email Security Gateway Initial Setup. . . . . . . . . . . . . . . 7 ESG Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 ESG Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Personal Email Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 Policy Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Advanced Encryption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Clustering Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 Archive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 ESG Log and Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 Troubleshooting Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
ESGA v7.6 201 5
6 ESGA v7.6 201
1
Websense Email Security Gateway Initial Setup
This chapter focuses on the initial setup and configuration of the Email Security Gateway (ESG) module in a Websense V-Series appliance. Prior to the physical and logical deployment of the appliance within the network, you need to understand the fundamental concepts behind email security. Much of the initial setup has to do with your email domains and ensuring only authorized s can send and receive email through the ESG appliance. During the initial setup of ESG, the wizard requires you to define: 1. Email server IP address: The IP address of the mailboxes server, which stores, receives and sends email to and from local s. You can add additional email server after the initial setup. 2. Protected domain address: This is the email domain where the mailboxes reside. For example, if the email is [email protected] then the protected domain is websense.com. You can add additional protected domain after the initial setup. 3. Trusted IP address: In this field you enter the IP address of the hosts which are allowed to send email (inbound connection) through ESG and not be subject to IP filtering rules. 4. Data security server IP address: In this field you enter the IP address of the DSS server so you can take advantage of the DLP integration offered by ESG. 5. Log server IP address: In order for ESG to be able to generate reports, you need to configure it to talk to an off-box log server. The log server should be installed before you commence the ESG initial setup. 6. Notification email address: A valid email address is required which will serve as the default address for all notifications from ESG. This email address should be a shared mailbox that is available to multiple level individuals. This ensures maximum ability for your environment.
ESGA v7.6 201 7
Websense Email Security Gateway Initial Setup
Slide 1:1
ESG Initial Setup Before you begin the configuration and installation of the ESG appliance you need to gather some key information needed during the initial setup process.
Email server IP address
Protected domain
Trusted IP address
IP address and information for the DSS server
IP address of the log server
Shared mailbox (or ) where ESG can send all email alerts and notifications
Steps of ESG Initial Setup 1. Install the Websense V-Series appliance, including setting up the rack, connecting network cables, connecting the input device for command line interface, etc. 2. Run the first boot wizard through command line interface. Security mode and Appliance Controller interface (C) are configured during this step. a. Security mode - the appliance performs email security protection (ESG mode), or web security protection (WSG mode), or both (ESG+WSG). We’ll talk about security mode with more details later.
8 ESGA v7.6 201
Websense Email Security Gateway Initial Setup
b. Appliance Controller interface - IP address of Appliance Manager logon portal. Note This is NOT ESG configuration wizard. 3. Access Appliance Manager via browser to configure ESG DomU IP address and other information. 4. Install TRITON Unified Security Center, also called TRITON console, including Email Security, Data Security, etc; install log database (if not pre-installed) and log server. 5. to TRITON console, switch to “Email Security” tab, then run the ESG configuration wizard. Email server, protected domain, trusted IP, IP addresses of DSS server, log server, and shared mailbox for notifications are set during this step.
ESGA v7.6 201 9
Websense Email Security Gateway Initial Setup
Slide 1:2
Protected Domains The concept of a protected domain has a slightly different meaning when you refer to inbound messages and outbound messages. An inbound message is defined as a message sent from outside your company domain to an within your company domain. The filtering of inbound messages is fairly simple. The ESG reads the recipient address and if it does not match any of the configured protected domains, the email is immediately rejected. For example, if an ESG appliance configured with websense.com and surfcontrol.com as protected domains receives an message sent from [email protected] does not match either websense.com or surfcontrol.com. In ESG, there are four directions defined:
Internal: From protected domain to protected domain
Inbound: From other domain to protected domain
Outbound: From protected domain to other domain
Open Relay: From other domain to other domain
10 ESGA v7.6 201
Websense Email Security Gateway Initial Setup
Slide 1:3
Email Relay During the ESG configuration wizard, you have entered some key data, which makes the ESG a closed relay. Only the following email messages are routed (relayed):
From any sender (coming from any IP) to the protected domain
From permitted IP (typically the internal email servers) and from a sender in the protected domain to any recipient
The idea is to ensure the ESG only routes valid email and does not become an open relay. Hackers and spammers often use open relays and misconfigured email servers to launch an attack or send out massive amounts of emails, in relative security. All traffic points back to the compromised email server.
ESGA v7.6 201 11
Websense Email Security Gateway Initial Setup
Slide 1:4
Required Components Two domains inside ESG appliance Any appliance running ESG has at least two domains configured:
ESG domain: contains an SSL interface for communicating with TRITON Management Server and the Personal Email Manager (PEM) module for end s to manage their isolated email according to the policies defined by s. Both TRITON console and the PEM module store data on Log and Report Server.
Appliance Manager: provides the facilities for s to connect and configure the appliance.
We’ll bring more details about the ESG domains in the ESG Architecture chapter.
Two types of s There are two types of s who can access the various modules inside the two domains:
can connect and configure the appliance directly
can’t access ESG domain directly, but through TRITON console running on an off-box server to set ESG configuration and policies
12 ESGA v7.6 201
Websense Email Security Gateway Initial Setup
End with a valid mailbox
can access the PEM module in ESG domain directly without TRITON console
Important There is no direct istrative access to ESG module. s have to install TRITON console off-box to be able to configure ESG domain.
Let’s assume the following scenario:
ESG domain has IP address 172.16.90.100
Appliance Manager has IP address 172.16.90.101
TRITON Management Server has IP address 172.16.90.200
The following statements are correct:
The logs to the TRITON manager, on IP address 172.16.90.200, to configure the ESG domain. You need to point your browser to the IP address 172.16.90.200
The TRITON Management Server can communicate directly with the ESG module at IP address 172.16.90.100
The can configure the appliance settings by pointing the browser the IP address 172.16.90.101
An end can access the PEM module by pointing the browser directly to the IP address of the ESG module at 172.16.90.100
Tip Any appliance running ESG needs at least two interfaces with two separate IP addresses: one bound to ESG domain, the other to Appliance Manager.
ESGA v7.6 201 13
Websense Email Security Gateway Initial Setup
Slide 1:5
Network Interfaces The V5000 appliance has 4 NIC and the V10000 has 6 NIC. The Appliance Manager domain always binds to the C interface.
On the V10000
N interface is used for the Network Agent domain.
E1 and E2 interfaces are bound to the ESG domain. You can use one or both interfaces depending on your environment. You could only use E1; and you could use E1 as internal interface and E2 as external interface too.
P1 and P2 are bound to the WSG domain. Tip On a V10000 you can run both WSG and ESG at the same time.
On the V5000 (running ESG only)
P1 and P2 interface are bound to the ESG domain.
C interface is bound to the Appliance Manager.
14 ESGA v7.6 201
Websense Email Security Gateway Initial Setup
On the V5000 (running WSG only)
N interface is bound to the Network Agent domain.
C interface is bound to the Appliance Manager.
P1 and P2 are bound to the WSG domain. Tip On a V5000 you can run either WSG or ESG.
ESGA v7.6 201 15
Websense Email Security Gateway Initial Setup
16 ESGA v7.6 201
2
ESG Deployment
ESG is typically deployed in DMZ zone. While other deployment options are possible, this is the most common and the one that Websense recommends. By design, ESG can process both incoming and outgoing messages. Some companies may decide to process only incoming messages; however it is most common to process messages in both directions. When processing inbound messages, the ESG appliance receives the email from the Internet (or possibly the Intranet, for large scale complex deployments), scans the messages, and forwards them to company internal mail server. The incoming messages can be directed to the ESG appliances in a variety of ways, depending on the customer preferences and final objectives:
MX Record re-configuration
Firewall routing policies
In order to process outbound messages from inside the company, there are two main possible configuration options. The options mostly depend on the mail client settings:
Clients not using SMTP - the mail configures the SMTP server to relay all outbound traffic to the ESG appliance. This is the most common configuration.
Clients with configured SMTP server - the mail can configure all clients to point to ESG directly. Then the ESG routes the messages as necessary. This setting, while not very common enables filtering of all internal email messages. Typically, it is easier to assign the IP address of the SMTP server to the ESG appliance and then assign a new IP address to the SMTP server.
Warning All configurations require firewall re-configuration.
ESGA v7.6 201 17
ESG Deployment
Slide 2:1
ESG Deployment The simplest deployment for an email appliance is to put in on premise and configure the firewalls or the DNS (or both) to direct incoming mail to the newly deployed appliance. You then configure the existing email server (or the client, in rare cases) to send all the outbound email through the ESG appliance. In addition to the on premises deployment, you can integrate ESG with the cloud solution and you can pre-filter all incoming email in the cloud; this reduces significantly the amount of email that the ESG has to process on premises. To scale to any size environment, the ESG s clustering. This advanced feature also ensures that email filtering is always up and running in your network.
18 ESGA v7.6 201
ESG Deployment
Slide 2:2
Standard Network Deployment This network diagram represents a very simple network configuration. Regardless of the complexity of the actual network where you plan to deploy ESG, you can always reduce it, at least conceptually, to the one shown above. Inbound email messages flow according to the following steps: 1. The email messages arrive from the Internet to the perimeter firewall. Permitted traffic is allowed through. 2. The firewall permits the messages to reach the V10000 appliance in the DMZ. Tip Typically you should install the V-Series appliances running ESG in the DMZ. On appliances configured in dual mode, you can have the E2 and P2 interface in the DMZ and the E1 and P1 interfaces in the LAN. 3. The ESG processes the messages and matches them against the running policies. Permitted messages flow to the email server.
ESGA v7.6 201 19
ESG Deployment
4. The email server receives the messages which ed the ESG filtering. This email server can have actual mailbox on it or be a bridgehead server, which distributes the messages to other mail servers based on internal routing policies. The messages are now available for pick/delivery by the end s. For outbound messages, you can reverse the process. The s send the messages to the mail server that is configured to forward all outbound messages to the ESG appliance. All messages that the filtering policies on the ESG, flow from the VSeries appliance, through the firewall to the Internet, where they reach their final destination.
20 ESGA v7.6 201
ESG Deployment
Slide 2:3
Standard Network Deployment with DNS This network diagram is very similar to the previous one. The added steps focus on the role of the DNS lookup by a sending email server. Let’s assume that a Gmail wants to send a message to a Websense employee. The process of delivering email messages consists of the following steps: 1. The Gmail email server queries the DNS for the Websense MX record. The MX record contains the information about the first hop where to send the email bound for the websense.com domain. Tip When sending an email, the transmitting server only learns about the first relay host, which processes email for the recipient. There can be any number of relay hosts between sender and recipient. Each host only knows about the next one. None has the fully visibility of the source-todestination path. The DNS server responds with the information about the next hop where to send the messages for the Websense domain. The reply may contain one or more
ESGA v7.6 201 21
ESG Deployment
values. The sending server should always pick the one with lowest preference value. When more entries have the same preference value the sending server should process them in the order received, till find the one responding. 2. The email server connects to the next hop in the chain. This can be: - A server in the cloud - A perimeter firewall - An ESG appliance In the this example, the MX record returns the IP address of one of the routable interfaces on the firewall. 3. The messages arrive to the premise firewall. Permitted traffic is allowed through. 4. The firewall permits the messages to reach the V10000 appliance in the DMZ. 5. The ESG processes the messages and matches them against the running policies. Permitted messages flow to the email server. 6. The email server received the messages which ed the ESG filtering. This email server can have actual mailbox on it or be a bridgehead server, which distributes the messages to other mail servers based on internal routing policies. The messages are now available for pick/delivery by the end s.
22 ESGA v7.6 201
ESG Deployment
Slide 2:4
ESG Anywhere Deployment In this scenario, the customer has enabled the Hybrid filtering option. All incoming email is pre-filtered by the cloud service. This deployment scenario dramatically reduces the amount of inbound messages that the ESG on premises has to process. Considering that over 80% of all incoming messages (by number not size) is spam, using the cloud service is a very efficient and effective approach. For incoming messages, the flow is as follows: 1. The sender email server queries the DNS for the recipient MX record. The DNS server responds with the information about the next hop, in this case the cloud service. 2. The email server connects to the cloud service. 3. The Hybrid service sends the messages that ed the filtering policies, to the next hop in the chain. 4. The messages arrive to the premise firewall. Permitted traffic is allowed through. 5. The firewall permits the messages to reach the V10000 appliance in the DMZ. 6. The ESG processes the messages and matches them against the running policies. Permitted messages flow to the email server. 7. The email server received the messages which ed the ESG filtering. The messages are now available for pick/delivery by the end s.
ESGA v7.6 201 23
ESG Deployment
Slide 2:5
ESG Components This slide represents a V10000 appliance installed in dual mode; both WSG and ESG domains are running. The various components operate as follows: 1.
s can access Appliance Manager by pointing the browser to the IP address of the C interface on the appliance.
s can configure the WSG domain through the same IP address (different port).
s need to TRITON console running on an off-box server to configure the ESG domain. The ESG domain communicates with TRITON console through SSL interface.
2. TRITON Management Server
TRITON Management Server enables s to manage all of the domains on the V10000 appliance: ESG, WSG, and Appliance Manager.
TRITON Management Server reads/writes SQL Database Server to perform log and report functionalities.
3. ESG domain
24 ESGA v7.6 201
ESG Deployment
ESG domain sends and receives email messages on the E1 and E2 (if configured) interfaces. You can use, in most cases the E1 interface for both sending and receiving email. Alternatively, you can configure E1 to send and receive on the LAN side and E2 to send and receive on the DMZ side.
ESG domain stores logs and statistics into SQL Database Server in order to generate reports and dashboard charts.
ESGA v7.6 201 25
ESG Deployment
Slide 2:6
Clustering Deployment with Load Balancer This network deployment shows load balancer with three ESG appliances connected to it. The load balancer redirects mail traffic to different ESGs with certain traffic balancing algorithm. The balancing algorithm is part of the load balancer configuration and not ESG. The flow of traffic is as follows: 1. The firewall receives the email from the Internet. 2. If the messages are permitted, the firewall forwards them to the IP address of the load balancer. 3. The load balancer, using the specified algorithm, decides to which of the ESG appliance send the mail message. 4. The ESG, which receives the email from the load balancer, processes the message. formation is stored about the message. The permitted message is forwarded to the email server. 5. The email server receives the messages which ed the ESG filtering. The messages are now available for pick/delivery by the end s.
Clustering details Each ESG in a cluster has one of three roles:
26 ESGA v7.6 201
ESG Deployment
Primary
The primary node when working in cluster mode. Every cluster can have only 1 primary ESG. The primary is the only node in cluster to be configured via TRITON console.
s configure the policies on the primary node through TRITON console. The configuration is then synchronized to all the secondary node.
Secondary
The secondary node when working in cluster mode.
Every cluster can have 1 to 7 secondary ESGs.
The secondary node can not be configured via TRITON console directly.
The secondary nodes synchronize the configuration from the primary node.
The status and logs are synchronized time to time as well.
The secondary nodes and the primary process mail traffic based on the same policies.
Standalone
Any appliance that is not set as primary and not negotiated to be a secondary stays in the role of standalone.
Important One cluster has a maximum of 8 nodes. All nodes of cluster must be the same appliance type: you cannot mix V10000 with V5000 in the same cluster. All nodes of cluster should have same applications deployed on the appliance: all appliance are either single mode (ESG only) or dual mode (ESG and WSG). Tip When cluster negotiation does not succeed, the primary node will not become standalone automatically. The will need to change this manually. Important When you have more than one ESG appliances, clustering is not a requirement. You can have all ESG appliances be in standalone mode; however, you will have to configure each ESG individually. This is not recommended and leads easily to inconsistent policy settings.
ESGA v7.6 201 27
ESG Deployment
Slide 2:7
Clustering Deployment with MX Record This scenario does not have a visible load balancer deployed. It loads and balances email traffic on multiple ESG appliances using MX record. Let’s take the diagram as an example, in which you have three ESG appliances in your cluster. You can re-configure your DNS as one of the following:
Configure the MX record in DNS to return three routable IP addresses which bind to the external interface of the firewall. Each record should have the same preference value, so that the traffic is evenly distributed on the basis of number of incoming connection requests. The firewall maps the external IP addresses to the internal one of the corresponding E1 interfaces on the ESG appliances (1-to-1 NAT)
Configure the MX record in DNS to return three routable IP addresses which bind to each of the E2 interfaces of the ESG. The E2 interfaces are in the DMZ. The E1 interfaces are on the LAN side.
The flow of traffic is as follows 1. The sending email server queries the DNS MX record for the destination domain.
The DNS returns three IP addresses with the same preference value.
The order of the IP addresses are set to round robin mode.
28 ESGA v7.6 201
ESG Deployment
The email server connects to the first IP address in the list (the other addresses are used if the first one does not respond).
The DNS returns the information about the next hop as well.
2. The email server connects to the next hop in the chain. 3. The messages arrive to the premise firewall. Permitted traffic is allowed through. 4. The firewall receives the incoming SMTP request and es it to the corresponding ESG appliance (using one of the two techniques listed above). 5. The layer 2 switch receives the connection request for the address of the selected appliance. This appliance receives and processes the email message. If the policy filter allows the email through, the appliance sends the email to the email server. 6. The email server receives the email that has ed filtering, which is now available for pickup or delivery by the end use.
ESGA v7.6 201 29
ESG Deployment
Slide 2:8
Clustering Operating Mode The only operating mode for the ESG cluster is Active/Active. All machines in the cluster can process email at any time. The cluster is considered active and available if at least one appliance is functional. When the primary is down, all secondary nodes will keep working and filtering as secondaries; no one will switch to the primary automatically; the primary needs to be restored manually. Furthermore, you can NOT modify the policies because the primary is the only node through which policies are configured and it is not available at the moment. If you need to make emergency changes to the policies, you can either re-configure the cluster to pick up a secondary acting as the new primary via TRITON console, or break the cluster and turn the secondaries to standalone machines. As the configuration is synchronized among clusters nodes, the new cluster will work in the same manner. The only lost is the email in the primary message queue, which can not be recovered. We will cover clustering with more details in the later chapter.
30 ESGA v7.6 201
ESG Deployment
Slide 2:9
Firewall Configuration You need to configure your firewall as the slides on this and next page show.
Port 25: permit incoming and outgoing SMTP traffic on 25 • •
On-box message processor listens on 25 to receive messages On-box message processor sends messages to 25 Note For Hybrid Service, the route of your email may occasionally be altered within the Hosted Cluster. To enable this without requiring you to make further changes, you must allow SMTP requests from all the IP ranges listed on the firewall page to port 25.
Port 80: permit outgoing traffic on 80
ESGA v7.6 201 31
ESG Deployment
•
On-box update module and module connect to 80 listened by offbox DDS Server Tip DDS, developed by Websense, performs utilized able databases for various Websense products. The databases include URL filtering database, real time update database, real time security scanning database, etc. We’ll talk about DDS again in ESG Architecture chapter.
•
Port 443: permit outgoing traffic on 443 •
On-box log, quarantine, PEM, and configuration components connect to 1443 listened by off-box MSSQL database
Port 2525: permit incoming traffic on 2525 •
On-box authentication component connects to 389 and 636 listened by offbox LDAP Server. 636 is the secure LDAP port.
Port 1443: permit outgoing traffic on 1443 •
On-box hybrid component connects to 443 listened by Hybrid Service
Port 389 and 636: permit outgoing traffic on 389 and 636 •
On-box DLP component connects to 80 listened by off-box TRITON DSS Manager
On-box message processor listens on 2525 to receive encrypted messages released by TRITON DSS Manager
Port 6671: permit incoming and outgoing traffic on 6671 • •
32 ESGA v7.6 201
On-box SSL Proxy component listens on 6671 to communicate with offbox TRITON Unified Security Center On-box PEM component connects to 6671 listened by SSL Proxy component on another appliance
ESG Deployment
Slide 2:10
Firewall Configuration
Port 6643: permit incoming traffic on 6643 •
Port 9443: permit outgoing traffic on 9443 • •
On-box DLP component listens on these ports
Port 5819, 5821, 8443, 56992, 18404: permit outgoing traffic •
On-box PEM component listens on 9449 for end s to manage their personal messages
Port 5820, 8888, 8889, 9080: permit incoming traffic •
Off-box TRITON Unified Security Center listens on 9443 On-box DLP component connects to 9443 listened by off-box TRITON DSS Manager
Port 9449: permit incoming traffic on 9449 •
On-box PEM Load Balance component listens on 6643 to dispatch PEM traffic
On-box DLP component connects to these ports listened by off-box TRITON DSS Manager
Port 50800: permit outgoing traffic on 50800 •
Off-box Log Server listens on 50800
ESGA v7.6 201 33
ESG Deployment
Slide 2:11
Sample Deployment The slides shows two of our sample deployments: A. ESG with Hybrid service B. ESG without Hybrid service
34 ESGA v7.6 201
3
ESG Architecture
ESG is optimized to handle large volumes of email messages. It also integrates seamlessly with the DLP engine with Websense Web Filter. The TRITON Console can manage all the components. The internal architecture of ESG is very flexible to achieve this level of scalability and integration. This chapter focuses on the differences between the V10000 and V5000 appliances relative to hardware configuration and software installation.
V10000:
ESG and WSG (dual mode)
ESG only (single mode)
WSG only (single mode) Tip The hardware resources on the V10000 are allocated differently depending on which security mode you select.
V5000:
ESG only (single mode)
WSG only (single mode)
ESGA v7.6 201 35
ESG Architecture
Slide 3:1
Architecture and Message Flow The ESG hardware settings (RAM, HDD, and U) can vary depending on whether it is running on a V10000 in dual mode, a V10000 in single mode, or a V5000. Consequently, the amount of traffic it can process depends on these parameters. One of the strong points of ESG is its natural integration with the DLP module. This integration is available for all installation options. You can also integrate ESG with the Web Filter which requires dual mode (V10000 only) or an external server running the Web Filter off-box. This chapter covers the actual message processing or the lifecycle of a message within the ESG module. Understanding these details enables you to better configure and position the ESG solution.
36 ESGA v7.6 201
ESG Architecture
Slide 3:2
Appliance Modules Both V10000 and V5000 run the Appliance Manager module, which is the domain to configure hardware settings for appliances. The difference between V10000 and V5000 is that a V10000 appliance can run ESG and WSG modules at the same time, while a V5000 appliance can run either ESG or WSG. The WSG module consists of three sub-modules: 1. Websense Content Gateway (WCG): the web proxy component of WSG. 2. Web Security Suite (WSS): the content filtering component of WSG, also called Web Security Enterprise (WSE). 3. Network Agent (NA): the component used to monitor and control protocols other than HTTP, HTTPS, and FTP. Except for the Appliance Manager module runs on Dom0 in V-Series appliance, all other modules run on DomU.
ESGA v7.6 201 37
ESG Architecture
Slide 3:3
U Allocation A V10000 G2 appliance can have three different configuration options when it has the ESG module running: 1. ESG only 2. ESG plus WSG without Network Agent 3. ESG plus WSG with Network Agent running In option 1, you have 12 Us dedicated to ESG. In options 2 and 3 you are limited to four Us for ESG as the other eight are allocated to WSG. Depending on the volume of web and email traffic, dual mode may or may not be the preferred option. From a U standpoint, having Network Agent running or not has no impact on the ESG module; it only affects the WSG module.
38 ESGA v7.6 201
ESG Architecture
Slide 3:4
Memory Allocation Similar to U allocation, the amount of RAM assigned to each module depends from the specific appliance configuration. When running in dual mode, the V10000 allocates the same amount of memory as the V5000 has. The Network Agent configuration has no impact on the amount of memory allocated to the ESG; it only affects the balance of memory allocated within the specific WSG components.
ESGA v7.6 201 39
ESG Architecture
Slide 3:5
Hard Disk Allocation The difference between single mode and dual mode is the amount of hard disk space allocated to Mail Transfer Agent (MTA) A V10000 running in dual mode allocates 70GB of HDD to MTA; while both the V10000 single more and the V5000 allocate 100 GB of HDD to MTA.
40 ESGA v7.6 201
ESG Architecture
Slide 3:6
ESG Internal Architecture The ESG has many internal components. Some of the most important ones appear in this slide. The idea is to help you understand that each message that arrives to the ESG flows through a series of components. At the end of the lifecycle, one of the following will happen to the message:
Delivered to the intended recipient
Delivered to the intended recipient but modified (i.e. attachment stripped)
Quarantined
Dropped/Discarded
In addition to the messages processing components, the ESG needs to communicate with the systems s, email s, and external components. The ESG provides two management UIs for two types of s:
TRITON - Email Security, incorporated UI into TRITON architecture, running on the off-box TRITON Manager Server, is for s to manage multiple ESG appliances concurrently.
PEM, running on the ESG appliance, is for end s to manage their quarantined emails.
ESGA v7.6 201 41
ESG Architecture
TRITON - Email Security and PEM communicate directly with the off-box database to fetch formation. Appliance Interface receives and processes the requests from Appliance Manager, dom0 of the appliance. SSL Interface is a SSL proxy which provides secure communication between on-box module and off-box module. The ESG management UI and the on-box Config module connect to SSL Interface. Authentication provides four functions: maintaining different types of groups, authenticating message sender, message recipient, and end to access PEM. Config consists of a policy module and a cluster module. The policy module receives configuration requirements from the ESG management UI and simple command line from back end, and stores the configuration files on the appliance. The cluster module synchronizes configuration files from primary ESG to secondary ESG in cluster mode. PEM loads and balances traffic to multiple PEM UIs, and reduce the stress for single PEM UI. It also provides the single entry for all end s to manage quarantined messages. Quarantine isolates messages which failed in going through the antivirus engine or antispam engine. Alert provides multiple methods (dashboard, email, pop-up and SNMP) for system services to report exceptions. Log receives all kinds of logs from other on-box modules, stores the system logs and audit logs to the on-box log database, and forwards the message logs to the off-box Log Server. PostgreSQL Database, the on-box log database server, saves system logs and audit logs from the Log module. The off-box Log Server runs on windows Operation System. It receives the message logs from the on-box Log module save them to the database. Hybrid provides the s for the protected domain to to Hybrid service automatically, synchronizes the communication token and pulls hybrid logs to the log database. Policy Engine is the component installed on ESG which talks to the DSS Manager for registration and receives the policy pushed out by the DSS Manager. DLP Filter enables Policy Engine to the DSS Manager, which is integrated with TRITON Unified Security Center. Scan Engine performs five types of filtering: hybrid, antivirus, antispam, disclaimer and DLP.
42 ESGA v7.6 201
ESG Architecture
Slide 3:7
External Components As mentioned before, ESG interacts with s, end s, and various offbox components. The slide displays the key components that are responsible for communication between inside and outside ESG. 1. The Log Database and the Log Server are two different entities, which can be installed on the same physical server. 2. The Encryption Service can be on-premises or on-demand in the cloud. Currently the ESG s integration with the Voltage encryption gateway. It also s natively TLS encryption. We’ll cover the encryption with more details later in the specific chapter. 3. Messages go through a pre-filtering analysis where the email sender (IP address and domain) runs through the Real-time Blackhole List Server (RBL Server) lookup and the Websense Reputation Service Server (WRS Server). The objective is to reduce the amount of messages that need to be analyzed early on in the process. RBL Server and WRS Reputation Server can easily identify and reject a large number of spam messages. 4. The ESG needs connect to the ESG through the TRITON Management Server, which runs on an off-box server. The TRITON Management Server communicates with the SSL interface as described earlier.
ESGA v7.6 201 43
ESG Architecture
5. DDS Update Server is designed for utilized able database within Websense products. The database covers URL filtering database, real time update database, real time security scanning database, license information database, etc.
44 ESGA v7.6 201
ESG Architecture
Slide 3:8
DLP Integration The DLP Filter receives the messages for scanning in standard email format (RFC 822 compliant). The PE parses the message before scanning according to the policy. Integrating DLP into ESG: 1. ESG with the DSS Manager and transfer DLP incident logs to the DSS Manager. 2. Configure and deploy DLP policies, e.g. fingerprint, on the DSS Manager. The policies are pushed out to the DLP Filter. 3. Configure DLP policies (active/disable/block/monitor) on ESG. Your organization will leverage the TruEmail DLP capability with patented PreciseID fingerprinting and natural language processing technologies.
Note To let the DLP Filter work, you need to make sure ESG subscription for DLP is valid; otherwise the message will not be send to the DLP Filter for scanning.
ESGA v7.6 201 45
ESG Architecture
The DSS Manager itself needs a subscription for normal operations, which is an XML file. However for the DLP Filter, the DSS subscription is not a prerequisite. That is, even if no subscription is imported into the DSS Manager, or the DSS subscription is expired, the ESG ed into DSS manager could still be able to make scanning. In another word, the DLP scanning of ESG does NOT need the DSS subscription, but needs the ESG subscription for DLP.
46 ESGA v7.6 201
ESG Architecture
Slide 3:9
Hybrid Integration ESG hybrid mode allows the to integrate the on-premise Email Security Gateway with Websense Hosted Email Security (HES) service, to deliver easy deployment and high-capacity email security. The workflow sequences are shown in the slide above. HES service to each ESG box is based on hybrid . All ESG hybrid configuration will be sent to HES cluster and stored as the hybrid settings. The ID is ESG license key, and the is transparent to the customer and is maintained by ESG/HES internally. Whenever ESG talks to HES Sync Server (HSS), the and is required for authentication. If the hybrid mode is enabled, an HES will be created. HES can provide hourly refreshed statistical data in last week. The statistical data is used to show the “Value of Hybrid Mode” on the dashboard reports. The data includes:
Number of spam messages blocked in the cloud
Number of total messages processed in the cloud
Total size of blocked and processed messages
ESG will pull the data hourly from HES cluster.
ESGA v7.6 201 47
ESG Architecture
The inbound email traffic flows the sequence below: 1. Emails sending from external domains are supposed to reach the protected domains. All messages go through the Hybrid service at first. 2. Filtered messages are sent to the ESG, where to be processed again. 3. Messages detected as spams, viruses, malwares, etc are discarded or isolated. 4. Allowed messages are sent to the email server for delivery.
The outbound email traffic flows the following steps: A. Outbound messages are sent from the email server to the ESG. B.
The ESG filters out the messages that do not comply with the policies.
C.
The messages that comply with policies are delivered to the intended recipient.
Note In this scenario, Hybrid service filters the inbound traffic only.
48 ESGA v7.6 201
ESG Architecture
Slide 3:10
Policy Processing ESG listens on port 25, receives SMTP messages, and filters the message according to its configuration. If the message is not coming from a protected domain (inbound message), the ESG first checks the sender IP address, domain, and other information against the RBL, WRS and any custom Always Block List. This step already filters a majority of the messages. Next, the recipients are validated against the directory service. This is an important step to prevent Directory Harvest Attacks (DHA). The last three steps are the most typical of any email filtering product. The messages is checked against size limits, spam content, virus content, and other malicious threats (file types of the attachments, compressed files, protected archives, etc.) Outbound messages do not have to be checked against the RBL, WRS, and other block lists; by definition, they come from trusted source. However, outbound messages need be checked for possible viruses. While viruses should not be there (most company run multilayer AV in their infrastructure), it is good practice to ensure that you are not responsible for spreading an outbreak.
ESGA v7.6 201 49
ESG Architecture
More importantly, and mostly unique to the ESG solution, you implement DLP policies for all outbound messages. The cost of information leakage, both financially and in of a tarnished reputation, can be devastating.
50 ESGA v7.6 201
ESG Architecture
Slide 3:11
Message Processing Flow The incoming messages (inbound or outbound alike) are received by the SMTP Server, which listens for incoming connections in T port 25. SMTP Server is in charge of dropping as many as possible of the incoming messages that maybe threats, which is called pre-filtering analysis. During this process, the SMTP service uses multiple techniques, such as Real-time Blackhole List (RBL) lookup, Reverse DNS (RDNS) lookup, Websense Reputation Service (WRS), etc to perform the analysis fast and accurately. Moreover, it greatly reduces the number of messages that need further analysis. Filter Server receives the messages after the SMTP Server processing them, facilitates the Policy Engine and the Scan Engine to filter out the additional incompatible emails based on the policies and filters set by s. This is the content filtering, with the goal of only delivering messages that are appropriate in content and do not have any threats in them. At the end of the message processing, three outcomes are possible for the messages: 1. Dropped/Discarded: The message is deemed to be a serious threat. It cannot and should not be ever released to the recipient. This is a not recoverable action. 2. Delivered to the recipient: The message ed all the security and policy scanning. It is appropriate in content and does not have any other threats.
ESGA v7.6 201 51
ESG Architecture
3. Quarantined: The message did fail one or more of the security and policy scanning. However the content is not such that is should not ever be released to the recipient, but may need to be retrieved by s or end s. Quarantined messages are stored in the local (ESG file system) quarantine queue. Information about quarantine is stored in the log database. Tip Recipients of a quarantined message can retrieve it through PEM management UI.
52 ESGA v7.6 201
ESG Architecture
Slide 3:12
Mail Filtering Framework ESG adopts 4 queues when processing the messages:
Filter queue The messages that through pre-filtering will be placed into priority based filter queues. If quarantined messages are to be re-processed (scanned again), the messages will be put back to filter queue.
Incoming queue The messages processed by Filter Daemon are moved from filter queue to incoming queue. Besides, the messages that should not be filtered (bounce/delivery request from PEM) are put in the incoming queue.
Deferred queue If a message failed in being delivered for some transient reasons (it might succeed later), the message is placed in the deferred queue.
Active queue The messages in the active queue will be sent to SMTP Client and delivered.
The messages are processed in the following procedure within ESG:
ESGA v7.6 201 53
ESG Architecture
1. First layer of filtering is what we called pre-filtering, handled by SMTP Server. Pre-filtering rejects most zombie connections. Clean Up Filter inserts the permitted messages into priority based filter queues according to the scores and informs Filter Daemon. 2. Filter Daemon asks Policy Server about the policies of the message, call the related Scan Engines, and execute the configured actions. After filter daemon scans the message, the message is moved from filter queue to incoming queue. 3. The message that fail in filtering is dropped or quarantined. 4. The acceptable message is sent to Queue Manager. Queue Manager moves the message from incoming/deferred queue to active queue. After a message is scheduled, Queue Manager informs SMTP Client with meta info of the message. Queue Manager scans the deferred queue periodically, brings messages in deferred queue to active queue and schedules the messages according to system resource. After the messages have been tried for max times, the messages will be bounced to sender. If the first bounce failed, the message will be bounced to configured postmaster address. 5. SMTP Client gets the message from active queue and deliver the message, and replies the status of the message to Queue Manager after delivering.
54 ESGA v7.6 201
ESG Architecture
Slide 3:13
RBL and WRS The slides shows the detail processing of Real-time Blackhole List (RBL) check and Websense Reputation Service (WRS) lookup. The IP address and domain of the email sender go through the external RBL Server at first then the WRS Server and get an reputation score. Based on different scores, messages will be placed into priority based filter queues.
ESGA v7.6 201 55
ESG Architecture
56 ESGA v7.6 201
4
Personal Email Manager
Personal Email Manager (PEM) is a component of Websense® Email Security. PEM enables a to look at isolated spam messages and decide whether to delete the messages or treat them as legitimate email messages. It also identifies the IP address of the appliance on which an end can access PEM. You can also authorize an end to manage personal Always Block List and Always Permit List, configure the appearance of the PEM end display, and deploy appliances in groups for load balancing purposes. PEM monitors specially configured Websense Email Security queues and notifies s if they have blocked inbound email. It also provides end-s a facility for managing their blocked email. To achieve the desired behavior and results, PEM must be istered on an ongoing basis. These tasks include:
Configuring queues
Customizing the notification email templates
Language PEM includes adaptive language for Spanish, Portuguese, Italian, and German. When an end- or logs into the PEM facility, they can determine the language displayed in the interface.
ESGA v7.6 201 57
Personal Email Manager
Slide 4:1
PEM Key Features PEM is a built-in feature of the ESG appliance. There is no need for the to install additional software or hardware. The PEM module is fully ed in a clustered environment. In a clustered environment, PEM can be accessed through the primary ESG or the secondary ESG. If the primary ESG is not available, s can also view their isolated message list through the secondary ESG. However, isolated messages stored on the primary ESG cannot be viewed, delivered, or deleted. The PEM can be successfully implemented and used any environment regardless of size. To even greater scalability, the ESG can store the isolated messages on a remote file system. ESG can connect to a NFS share or a CIFS one (via SAMBA). In environment where more than ESG exists, even if all appliances use the same share, each ESG still creates it own separate and independent queues. One of the key benefits of the PEM is greatly reduced load; s are in charge of setting their own Always Permit List and Always Block List and release messages which may have been isolated by mistake. With tight and granular control, only messages that do not present a threat to the company can be available for release.
58 ESGA v7.6 201
Personal Email Manager
Slide 4:2
Quarantine System Overview The ESG process the messages; when it finds that an email is a spam, virus, or the message failed to be processed, it could not be encrypted or could not be decrypted the ESG informs the quarantine daemon to quarantine the email. The email can be quarantined locally or on a remote storage mounted on ESG through NFS or SAMBA (CIFS). s and end s can view and manage the quarantined emails and the quarantine daemon will record corresponding quarantine logs to ESG log database. Quarantine logs are sent directly to log database without being relayed by the ESG log server. These logs need to be sent out one by one instead of in bulk for high reliability requirement.
ESGA v7.6 201 59
Personal Email Manager
Slide 4:3
Mail Queues Quarantine messages are isolated into different quarantine queues (folders) in of their different attributes. By default, there are six built-in queues: 1. Virus: Used to store messages, which contain a virus. 2. Spam: Used to store messages, which triggered one of the spam filters. 3. Exception: Used to store exceptional emails (failed several non specific filters). 4. Encryption-fail: Used to store messages that did not encrypt correctly. 5. Decryption-fail: Used to store messages that did not decrypt successfully. 6. Archive: Used to store all incoming messages. Tip These 6 queues cannot be deleted. However, can edit their properties or add new queues. As mentioned before, messages in each queue can be stored locally (on ESG box) or on a remote storage. ESG s two remote storage types:
NFS
CIFS (through Samba)
60 ESGA v7.6 201
Personal Email Manager
On the Blocked Message page, s can see the quarantine queue list.
ESGA v7.6 201 61
Personal Email Manager
Slide 4:4
Queues Monitor Every 30 seconds, ESG will perform a checking on queue status, including remote queue availability and available disk space check.
If the remote queue is found unavailable, the queue storage status will be marked as "mounted error", and ESG will generate a corresponding alert. If the remote queue is available again, the "mounted error" status will be cleared as "ok" and the alert will be cleared too.
If the remote storage is unavailable, trying to move the queue with copying the messages will fail.
If the disk is available, ESG will check the space usage of the queue. If the percentage of free disk space is less than 10%, ESG will mark the queue status as "disk space low" and send an alert. Also, if a queue's free disk space is return to more than 10%, the status will be cleared and alert will also be cleared.
62 ESGA v7.6 201
Personal Email Manager
Slide 4:5
Queues Configuration In cluster mode, the "Queue list" page is similar to standalone mode. The "volume" is the sum of all machines' volumes, and the "size/total" is the sum of all machines' size and size limit. For example, there a 2 machines in a cluster, both of which has a 10GB size limit (Their limits are always same because the configuration is synced between cluster). Machine 1 has 10 files in it and its size is 5MB, machine 2 has 5 files in it and its size is 3MB. In this case, the "volume" and "size/total" displayed in the UI will be "15" and "8MB/20GB". Compared with standalone mode, the error status description of the queue is more complex. In standalone mode, if the hard disk space of the queue is low, there will be an icon displayed which shows "hard disk low". In cluster mode, if the storage type is locally, the content displayed will be a list of "Device X.X.X.X: hard disk low", because maybe more than 1 device has no enough free disk space; if the storage type is remote, the content displayed will be "Remote hard disk low". Regarding the "remote storage invalid" status, if a queue of at least one machine is in this status, the WebUI should show "remote storage invalid".
ESGA v7.6 201 63
Personal Email Manager
Slide 4:6
PEM Components PEM is composed by two sides: the side and the end side:
side - PEM general configuration: the PEM digest settings, the end- access control and the authentication mode.
End side - PEM management: the end can manage the messages processed by ESG and take the actions allowed he/she prefer.
64 ESGA v7.6 201
Personal Email Manager
Slide 4:7
Notification Message The ESG PEM enables end s to configure the notification message that will be sent out from the ESG when any of the emails, with the recipients or senders as themselves, are blocked. End s are able to configure the notification message in the following aspects:
Max messages The maximum number of the message digests in the notification email to the end . Such as, if set to 10, then, only 10 message digests will be in the notification message even if the end has more than 10 quarantine or spam messages. The maximum value of this setting is 100 and the default is 10.
Digest send interval This interval decides when the digest message will be sent out according to a time and day setting. Times are set by hour and minute while days can be
Every day
Every weekday
A specific day in Every Week For example, 3:00 Monday, Every week or 6:00, 12:00, 20:00, Every day.
Digest message template Message operation options define the operations which will be included in the digest message and the end can see and execute.
ESGA v7.6 201 65
Personal Email Manager
The slide show the screen when setting the notification message: 1. Set the max number of messages per notification and the actions available in the notification that the end are able to take. 2. Configure the information of the notification:
Company Define the brief info to the customer company displayed in the digest message
Description Define the brief info to the ESG which the customer prefers.
Sender Define the sender email address of the digest notification message to the end . The address format is name@domaininfo, such as “[email protected]”, meaning that the sender of the digest message will be this address. This is ONLY one email address.
Subject Define the subject of the digest message. The maximum length of this value is 100 characters.
3. Set the header The header info will be displayed in the digest message. It can provide some key words to the end before he/she open the message text. 4. Set the footer The footer info can provide information that the end can use, like, the telephone number of email address when the end gets a problem. The end- access list allows the ESG to define the list who can use the PEM to manage the quarantine or spam messages. The will only be chosen from the existed " Directory" objects. If the ESG wants to use one new directory, first, it's required to add it in the page at which the directory can be defined. This setting will list all the existed un selected directorires in the left compound box and all the checked directories in the right box.
66 ESGA v7.6 201
Personal Email Manager
Slide 4:8
Queues Actions Quarantine message logs will be stored on a remote SQL server database, together with ESG message logs. By clicking the queue name in "Queue list" page, can view all messages of the queue. can view the messages by date, and can also search messages by sender, recipients, subject and policy. can delete, , reprocess, continue reprocess, deliver (release) the message(s) or forward the message(s) to one or more email addresses. can also report a false-positive to Websense, or add the sender of a message into ESG global Always Permit/Block List. Sometimes when a queue operation is ongoing, some of the message operations will return a failure of "retry later". The following actions are ed:
Delete: Delete the message
Deliver: Deliver the message to its recipients
Reprocess: Delete the message from the queue and reprocess it, just like it has been received by ESG again. It's useful when sometimes technical team and development team want to debug false-positives of filter. They can turn on the debug, reprocess the message and find why this message is captured by filter.
ESGA v7.6 201 67
Personal Email Manager
Forward: Forward the message to one or more recipients, the forwarded message will be embedded in the forward mail as an attachment. The subject of the forward mail is "Fw: Forward Message(s)".
Add to Always Block List: Will add the message sender to Always Block List.
Add to Always Permit List: Will add the message sender to Always Permit List.
: the message in .eml format. If more than one message is ed together, they will be compressed in a zip file.
Clear queue: Clear the whole queue, all messages in the queue will be deleted.
Continue Process: Sometimes a message contains both virus and spam info. Assume it's captured by spam filter and quarantined, then finds that it's a false-positive and decides to deliver (release) it. In this case, the message will by the virus scanning. To prevent this, there is another action "Continue Process" provided to make sure the message will be scanned by other filters before it is delivered.
Not Spam: Clicking this button will cause two actions: to send a copy of this email to [email protected] for false positive research, then release the email to its original recipients.
68 ESGA v7.6 201
Personal Email Manager
Slide 4:9
PEM in Clustered ESG In a cluster environment there is always one and only one appliance which operates as the primary. For the purpose of the PEM notifications, only the primary sends out the notification. In the notification, sent to the , the IP address (or hostname) for the isolated mail link is the primary's IP (or hostname). If the primary is not available (down), the s do not receive notifications and they cannot retrieve isolated emails. The secondary will not send notification and will not participate in any election process. Once the primary returns operational, the s receive once again the notification and are able to manage the isolated messages. Similarly to the queue management, also the is only available through the primary. During normal operations, the process of retrieving isolated messages via PEM proceeds as follows: 1. ESG #1 (not the primary) receives a message, which is going to be isolated. 2. The ESG writes an entry in the log about this message. As mentioned before, a key field in the database is the ID of the ESG who isolated the message.
ESGA v7.6 201 69
Personal Email Manager
3. The recipient, which has been notified that she has received an message being isolated, connects to the PEM module. The link in the notification points to the TRITON - Email Security. 4. The primary ESG looks up the message details in the log database. Using the ID of the machine, which isolated the message, can connect to that ESG (in this case ESG #1) 5. The primary ESG communicates with the ESG #1, retrieves the message, and sends it over to the who requested it.
70 ESGA v7.6 201
5
Policy Management
Policy is the pipeline that message flows go through. ESG defines 3 types of policies based on traffic directions:
Inbound policy
Outbound policy
Internal policy
Inbound/Outbound/Internal policy have multiple sub-policies which will applies to the messages that match the policy’s matching condition. The matching condition defines the entry criteria of the policy pipeline, based on the envelope sender and recipient address of the messages. Each policy can contain a series of rules. Rules define how the message in the policy pipeline is to be scanned and how the message is processed when it triggers the filters. Each rule is combined with action and filter. Filter is individual scanning module which defines how the message in the policy pipeline is to be scanned. Action is individual module which defines how to process message when filter is matched.
How the ESG policy works could be described in the following steps: 1. Messages are divided into 3 categories: inbound, outbound, and internal. 2. In each category, messages match the condition based on the addresses of the senders and recipients respectively. 3. The specific filter, defined in the rule of the matched policy, scans the messages. 4. The messages filtered out are processed according to the action.
ESGA v7.6 201 71
Policy Management
Slide 5:1
Key Features Policy rules comprise the filters and filter actions that determine how a message that matches a policy’s sender/recipient conditions is handled. Filters provide the basis for email scanning for viruses and spam, and filter actions determine the final disposition of a message when it triggers a particular filter. Define new filters, or copy and edit an existing filter to suit your organization’s needs. Add and define new actions or customize default actions as needed. After you have created and configured filters and filter actions, they are available for inclusion in your policies.
72 ESGA v7.6 201
Policy Management
Slide 5:2
Policy Components Flow 1. Policies are directional-based and depend on the following factors:
Protected domain
Email sender address
Email recipient address
2. Policies contain conditions; when matched, rules are applied. 3. Rules contain both filters and actions:
Filters are individual scanning modules which define how messages in the policy pipeline are scanned.
Actions are individual modules which define how a message is processed. Note Policy order is important. ESG policies are similar to firewall rules. If one policy is matched, then the rest will be skipped.
In ESG, there can be up to 32 policies per mail route direction.
ESGA v7.6 201 73
Policy Management
Slide 5:3
Policy Directions An Email Security Gateway policy is applied based on defined sender/recipient conditions and the direction of the email. You can apply a different policy to different groups of senders and recipients. For example, you might apply a policy to a marketing department group in your organization and a different policy to a human resources group. After you define a set of senders and recipients in a policy, you can add the policy rules (a combination of filter and filter action) to apply when the sender/recipient conditions of the email match the policy. Email Security has 3 general types of policies, depending on the direction of the email inbound, outbound, or internal. Message direction is determined on the basis of an organization’s protected domains:
Inbound - The sender address is not from a protected domain, and the recipient address is in a protected domain
Outbound - The sender address is from a protected domain, and the recipient address is not in a protected domain
Internal - Both the sender and recipient addresses are in a protected domain.
ESG has one predefined default policy for each email direction, as well as a default Data Security policy for each direction. Data Security policies may be applied to email in any direction. These policies are configured in the Data Security module of TRITON Unified Security Center and can only be enabled or disabled in ESG. You
74 ESGA v7.6 201
Policy Management
need to Email Security with the Data Security management server and click Deploy in the Data Security module for the policies to be active.
The inbound policy logic is as follows: 1. Pre-filtering/Reputation 2. Always Permit/Block List 3. Antivirus (sender/recipient/filter-option/action) 4. Antispam (sender/recipient/filter-option/action) 5. Content Guardian (sender/recipient/filters/action) 6. DLP
The outbound policy logic is as follows: 1. Antivirus (sender/recipient/filter-option/action) 2. Antispam (sender/recipient/filter-option/action) 3. DLP
The internal policy logic is as follows: 1. Antivirus (sender/recipient/filter-option/action) 2. Antispam (sender/recipient/filter-option/action) 3. DLP or by DLP
ESGA v7.6 201 75
Policy Management
Slide 5:4
Policy Conditions Match conditions define the entry criteria of a policy. Each match condition consists of a pair of sender and recipient envelope addresses, marked as From address and To address. The address can be a simple address like [email protected], or can be a wildcard address *@somewhere.com, or can be just * for any address. When the envelope sender address of the message matches the From address of a match condition and the recipient address matches To address of a match condition, then this match condition is considered matched and the message goes into this policy. If the message has multiple envelope recipient addresses, it is possible that a message is split into different policies. The address in match condition can also be from certain Directory, which is logic group of addresses. A policy can have multiple match conditions, and messages will try to match these match conditions one by one, from top to bottom. The order of match conditions in policy is not important. When match conditions are empty, the policy will be skipped directly. There are at most 32 match conditions per policy.
76 ESGA v7.6 201
Policy Management
Slide 5:5
Rules Rules define message scanning process of policy. If the sender and the recipients of a message match any match condition of a policy, then messages will try to match rules one by one, from top to bottom. If one rule is matched, the rest will be skipped. Rules cannot be deleted/added, but can be edited. Rules contain the following:
General configuration, including Name, Enable/Disable, Order
Filters: scanning modules to use
Actions: what action to take if a filter scan is matched
There are 3 default rules
Antivirus
Antispam
Disclaimer
ESGA v7.6 201 77
Policy Management
Slide 5:6
Filters Email Security Gateway has three predefined default filter types: virus, spam, and disclaimer. The virus filter analyzes an email message and its attachments for the presence of viruses and other threats. The spam filter scans email content and compares it against a database of known spam characteristics. You can select from a variety of antispam tools, including the Websense Web Security URL scanning tool. If you want to add text at the beginning or end of a message, use the disclaimer filter. Filters are created and managed via the Main > Policy Management > Filters page. Click Add to open the Add Filter page and set the properties of your new filter. You can also copy a filter whether or not it is in use by a policy. A filter can be deleted, as long as it is not in use by any policy. However, you cannot copy or delete an Email Security default filter. Up to 32 filters can be assigned per policy.
78 ESGA v7.6 201
Policy Management
Slide 5:7
Actions A filter action determines a message’s final disposition. ESG scans messages and their attachments then performs an action based on applicable policy settings. Actions are created in the Main > Policy Management > Actions page. You can add a defined action to a policy rule when you configure your email policies. Create a new filter action by clicking Add and selecting action properties. You can remove a filter action by marking the check box to the left of the filter name to select it and clicking Delete. You can delete a filter action only if its current status is Not in use. A filter action that is in use does not have a check box for selection. Note You cannot remove an Email Security default filter action. There are at most 32 actions per policy.
ESGA v7.6 201 79
Policy Management
Slide 5:8
Summary 1. All traffics are divided into 3 directions: inbound, outbound, and internal. 2. Messages match the specific conditions. 3. Filters are triggered for the messages of the conditions. 4. Actions are performed if the filters are matched.
80 ESGA v7.6 201
6
Advanced Encryption
ESG encryption enables the secure delivery of sensitive email communications to business partners and individuals. Easy istration, no complex key management, no additional hardware or software requirements eliminate the traditional cost and complexity barriers to implementing email encryption.
The ESG provides 3 types of message encryption:
Transport Layer Security (TLS): to encrypt/decrypt emails by ESG locally
Hybrid service encryption: to encrypt/decrypt emails by Hybrid service
Third party encryption: to encrypt/decrypt emails by integrated third party encryption gateway
ESGA v7.6 201 81
Advanced Encryption
Slide 6:1
Advanced Email Encryption Email Security s the following 3 types of message encryption:
Transport Layer Security (TLS)
Hybrid service encryption
Third party encryption gateway
We’ll talk about the 3 encryptions in detail!
82 ESGA v7.6 201
Advanced Encryption
Slide 6:2
TLS introduction Transport Layer Security (TLS) and its predecessor, Secure Sockets Layer (SSL), are cryptographic protocols that provide communications security over the Internet. TLS and SSL encrypt the segments of network connections above the Transport Layer, using symmetric cryptography for privacy and a keyed message authentication code for message reliability. Several versions of the protocols are in widespread use in applications such as web browsing, electronic mail, Internet faxing, instant messaging and voice-over-IP (VoIP). TLS is an IETF standards track protocol, last updated in RFC 5246 and is based on the earlier SSL specifications developed by Netscape Corporation.
ESGA v7.6 201 83
Advanced Encryption
Slide 6:3
TLS Encryption TLS is an Internet protocol that provides security for all email transmissions— inbound, outbound, and internal. The client and server negotiate a secure connection for the transmission to occur, provided both the client and the server the same version of TLS. The ESG uses a TLS encryption level of 128 bits. The ESG uses 2 levels of TLS: Opportunistic TLS and Mandatory TLS.
Opportunistic TLS It is used during the message routing process. You can enable TLS for a specified message route on the Settings > Receive/Send > Mail Routing page. Create a new route that uses the TLS delivery option or edit an existing mail route to add the TLS option. At the bottom of the page, mark the Use Transport Layer Security (TLS) check box to use opportunistic TLS for message routing. With opportunistic TLS, the protocol must be implemented on both sides of a connection for an encrypted transfer. If a connection attempt is made using the TLS protocol, the connection recipient must provide appropriate TLS credentials for an encrypted data transfer. If the TLS “handshake” fails, the data transfer is made via plain text, rather than encrypted text. In either case, the data transfer is successfully accomplished.
84 ESGA v7.6 201
Advanced Encryption
Mandatory TLS Mandatory TLS is the default encryption method that is enabled on the Settings > Inbound/Outbound > Encryption page. For mandatory TLS, both sides of an email connection must use the protocol. An encrypted data transfer occurs when the TLS handshake process is successful. Unlike opportunistic TLS, if the handshake fails during the connection attempt, the connection is terminated and no data transfer occurs.
A. If the next hop MTA has the ability of TLS, the encrypted email will be sent out. B. Mandatory TLS: if the next hot MTA does not TLS, the email is placed in a deferred message queue for a later delivery attempt B’. Opportunistic TLS: if the client and server cannot negotiate a secure connection, the message will be sent to the recipient.
ESGA v7.6 201 85
Advanced Encryption
Slide 6:4
Prerequisite of Hybrid Service Encryption Prerequisites of Hybrid service encryption A. Hybrid service encryption is available only if your subscription includes the Hybrid service and the Hybrid service is ed and enabled. Otherwise, you can not select Hybrid service encryption from TRITON - Email Security. B. Hybrid service encryption requires a certificate from a hybrid service trusted CA. In order to obtain a certificate from one of these CAs, you need Generating a Certificate g Request (CSR) for the server on which you want to install the certificate. After you have generated a CSR, follow the third-party CA acquisition procedures for the certificate you want to purchase. Use the Settings > General > TLS Certificate page for certificate import.
Generating a Certificate g Request (CSR) You can generate a CSR using the following procedure: 1. Open the command-line interface on the V-Series appliance where Email Security resides.
86 ESGA v7.6 201
Advanced Encryption
2. Execute the following commands: openssl genrsa -out [keyfile] 1024 Where [keyfile] is used to import the certificate to the ESG. openssl req -new -out reqfile -key keyfile –subj /C=CN/ ST=Bj/L=Bj/O=Websense/CN=ESG\/ [email protected]/ Where [keyfile] is the CSR file and /C=CN/ST=Bj/L=Bj/O=Websense/ CN=ESG\/emailAddress=@websense. com/ is the subject of the certificate. The certificate subject can be modified if necessary.
List of trusted CAs for hybrid service The Email Security Gateway hybrid service can use a certificate from any one of the following trusted CAs:
Entrust.net 2048 Secure Server CA
Entrust.net Secure Personal CA
Entrust.net Secure Server CA
Equifax CA
Equifax Secure CA
GTE CyberTrust Global Root
GTE CyberTrust Root 2
GTE CyberTrust Root 3
GTE CyberTrust Root 4
GTE CyberTrust Root 5
GTE CyberTrust Root CA
GlobalSign Partners CA
GlobalSign Primary Class 1 CA
GlobalSign Primary Class 2 CA
GlobalSign Primary Class 3 CA
GlobalSign Root CA
Thawte Personal Basic CA
Thawte Personal Freemail CA
Thawte Personal CA
Thawte Server CA
Thawte Server CA
Thawte Universal CA Root
ValiCert Class 1 VA
ValiCert Class 2 VA
ValiCert Class 3 VA
ESGA v7.6 201 87
Advanced Encryption
VeriSign Class 4 Primary CA
Verisign Class 1 Public Primary Certification Authority
Verisign Class 1 Public Primary Certification Authority - G2
Verisign Class 1 Public Primary Certification Authority - G3
Verisign Class 2 Public Primary Certification Authority
Verisign Class 2 Public Primary Certification Authority - G2
Verisign Class 2 Public Primary Certification Authority - G3
Verisign Class 3 Public Primary Certification Authority
Verisign Class 3 Public Primary Certification Authority - G2
Verisign Class 3 Public Primary Certification Authority - G3
Verisign Class 4 Public Primary Certification Authority - G2
Verisign Class 4 Public Primary Certification Authority - G3
Verisign/RSA Commercial CA
Verisign/RSA Secure Server CA
88 ESGA v7.6 201
Advanced Encryption
Slide 6:5
Hybrid Service Encryption for Outbound Email Hybrid service encryption is enabled on the Settings > Inbound/Outbound > Encryption page. Note Hybrid service encryption is mandatory encryption. If the encryption is failed, the message will be placed in a deferred message queue for a later delivery attempt. For mandatory TLS, both sides of an email connection must use the protocol. An encrypted data transfer occurs when the TLS handshake process is successful. Unlike opportunistic TLS, if the handshake fails during the connection attempt, the connection is terminated and no data transfer occurs. Hybrid service does not encrypt inbound or internal messages. Even though Data Security policy identifies an inbound or internal message for encryption and Hybrid service encryption is selected, the message will not be encrypted and an alert (HESencryption for inbound/internal messages) will be sent. Therefore Data Security policy must be modified to designate only outbound messages for encryption when Hybrid service is used. Hybrid service encryption for the outbound email following the below steps:
ESGA v7.6 201 89
Advanced Encryption
1. The Email Server sends out an unencrypted email to the ESG. 2. Data Security policy labels the email for encryption. 3. The ESG sends the email to the Hybrid Filter for encryption on SMTP forced TLS. 4. The Hybrid Filter encrypts the email and delivers the email to the recipients. The Hybrid Filter address is configured during the Hybrid service setup process.
The Hybrid Filter identifies the message by
IP address of ESG
Sender domain
Encryption x-header: indicating the message should be encrypted
TLS state
If the message contains X-ESG-HYBRID header, but fails in any of the other 3 criterions, the Hybrid Filter will bounce the message. IP addresses of ESG are pushed to the Hybrid Filter during Hybrid configuration; and later changes of IP addresses or network topology are notified to the Hybrid Filter too. If encryption fails, the message is placed in a deferred message queue for a later delivery attempt. If the Hybrid Filter detects a spam or virus in an encrypted outbound message, the mail is bounced to the message sender. Bounce means the Hybrid Filter returns a 5.x.x permanent failure to ESG on SMTP conversation; or generates an NDR message (with return of content) depending on the original message and delivers it to ESG on SMTP opportunistic TLS. ESG should always STARTTLS to the Hybrid Filter and accept TLS handshake from the Hybrid Filter. Messages sent from ESG to the Hybrid Filter for encryption may be put in the Hybrid Filter queue for some time.
90 ESGA v7.6 201
Advanced Encryption
Slide 6:6
Hybrid Service Encryption for Inbound Email For inbound messages, Hybrid service encryption attempts to decrypt inbound encrypted mails. The detailed process is described on the slide: 1. An encrypted email arrives at the Hybrid Filter. 2. The Hybrid Filter decrypts the email and adds an x-header to the email to indicate whether the decryption succeeded, and scans the email and sends the permitted unencrypted email to the ESG. Note Message scanning is performed regardless of whether message decryption is successful. 3. The email may trigger DSS incidents. 4. The ESG scans and delivers the permitted unencrypted email to the recipients. The Hybrid Filter will deliver the messages to the ESG on SMTP opportunistic TLS with the following x-headers:
Decryption success/fail x-header, indicating decryption successful or failed
Virus x-header
ESGA v7.6 201 91
Advanced Encryption
Spam x-header
The ESG always STARTTLS to the Hybrid Filter and accept TLS handshake from the Hybrid Filter.
92 ESGA v7.6 201
Advanced Encryption
Slide 6:7
Third Party Encryption On-demand for Outbound Email Third party encryption is enabled on the Settings > Inbound/Outbound > Encryption page. Note Third-party encryption is opportunistic encryption. If the encryption failed, the message will be sent to the recipient in plain text. When Data Security policy identifies an outbound message for encryption and third party encryption is selected, the messages are sent to third party encryption gateway. The process is described on the slide: 1. The Email Server sends out an unencrypted email to the ESG. 2. Data Security policy labels the email for encryption. 3. The ESG sends the email to third party encryption gateway for encryption on SMTP opportunistic TLS, for third party gateway may not enable TLS to receive messages. 4. Third party gateway encrypts the message and delivers it to the recipients directly.
ESGA v7.6 201 93
Advanced Encryption
s configure x-headers to add to an outbound message via TRITON Email Security. These x-headers facilitate communications between the ESG appliance and third party gateway and they must match the corresponding settings in third party gateway.
Encryption x-header, indicating the message should be encrypted
Encryption success x-header, indicating the encryption is successful
Encryption fail x-header, indicating the encryption is failed
s can require s to present credentials to view encrypted mails: supply the desired name and in TRITON - Email Security. Authentication must be ed and configured on third party gateway as well. Besides x-header and credential configuration, s are able to set desired encryption failure options:
Isolate messages to queue
Send notification to original sender
Deliver messages
Drop messages
If third party encryption gateway is not configured, ESG will generate an alert (invalid third party encryption gateway).
94 ESGA v7.6 201
Advanced Encryption
Slide 6:8
Third Party Encryption On-premises for Outbound Email The difference between on premises and on demand is that in this case, third party encryption gateway routes the encrypted messages back to ESG instead of delivering the messages out. ESG identifies the messages from third party gateways by configured IP addresses of the gateway. The sequence is listed on the slide: 1. The Email Server sends out an unencrypted email to the ESG. 2. Data Security policy labels the email for encryption. 3. The ESG sends the email to third party encryption gateway for encryption. If encryption is successful, the third party gateway forwards the message (possible multiple messages from multiple recipients) to ESG with encryption success xheader. 4. ESG will deliver the encrypted message without further scanning. If encryption is failed, the third party gateway forwards the original message with encryption fail x-header to ESG. ESG will apply the encryption fail configuration, quarantine the message, notify senders, deliver or drop the message. An alert (third party encryption gateway encrypt fail) will be generated.
ESGA v7.6 201 95
Advanced Encryption
If there is no encryption success/fail x-header, there may be errors in configuring ESG and third party encryption gateway. An alert (third party encryption gateway does not encrypt messages) will be generated.
96 ESGA v7.6 201
Advanced Encryption
Slide 6:9
Third Party Encryption On-premises for Inbound Email In order to decrypt messages, third party encryption gateway should point their mail route back to ESG and the decryption option is enabled on ESG configuration. Decryption option can only be enabled when third party encryption gateway is selected and IP address of third party gateway is configured. The slide shows the process of third party encryption for inbound email: 1. The encrypted email arrives at the ESG. ESG checks if the inbound email are encrypted- those with content-type or xheaders are encrypted messages, which will be send to third party encryption gateway on SMTP opportunistic TLS to decrypt. 2. Third party gateway decrypts the email and routes it back to the ESG with decryption success x-header. 3. The email may trigger DSS incidents. 4. The ESG scans and delivers the permitted unencrypted email to the recipients. If decryption is failed, third party gateway will forward the original message with decryption fail x-header back to ESG. ESG will apply the decryption fail configuration: to quarantine the message, then scan the message. An alert (third party encryption gateway decrypt fail) will be generated.
ESGA v7.6 201 97
Advanced Encryption
s can configure x-headers for messages sent to third party encryption gateway via TRITON - Email Security. These x-headers facilitate communications between ESG and third party gateway and they must match the corresponding settings in third party gateway.
X-header, indicating decryption is allowed
Decryption x-header, indicating the message should be decrypted
Decryption success x-header, indicating the decryption is successful
Decryption fail x-header, indicating the decryption is failed
If there is no decrypt success/fail x-header, there may be errors in configuring ESG or third party encryption gateway. An alert (third party encryption gateway does not decrypt messages). Sandwich mode(decrypt, scan and then encrypt messages) is not ed. So decrypt success and decrypt fail messages will not be encrypted again. If Data Security returns encryption for decrypt success and decrypt fail messages, ESG will treat it as unencryption.
98 ESGA v7.6 201
7
Clustering Details
In this chapter you will learn about the details of linking multiple ESG nodes together to form a cluster to share traffic load or function as a single virtual ESG. Clustering benefits you in balancing process work among different ESG nodes when traffic is heavy, consolidating messages received in all separate ESG nodes for PEM, and easing management and configuration. If you have multiple ESG appliances running, clustering is not a requirement. You could have all of the appliances be as standalone and managed by one TRITON Management Server. In this way, the type of appliances and the running mode could mix with each other. This is not recommended, for it leads to inconsistent policy settings easily.
Each ESG has three types of roles:
Standalone: single ESG processes mail traffic. ESG works as standalone by default. Standalone ESG can be configured via TRITON console.
Primary: the primary node when working in cluster mode. Every cluster can have only 1 primary ESG. The primary is the only node in cluster to be configured via TRITON console.
Secondary: the secondary node when working in cluster mode. Every cluster can have 1 to 7 secondary ESGs. The secondary ESG can not be configured via TRITON console directly. The configuration is synchronized from the primary ESG.
After cluster negotiation succeeds, the primary and secondaries will work in cluster mode. If cluster negotiation does not succeed, all nodes in cluster will be standalone. One cluster is composed of up to 8 ESG nodes: one as the primary and others as secondaries. All the nodes within one cluster must be the same type of appliance and run in the same mode. For example, a V10000 and a V5000 cannot cluster together, a V10000 in dual mode and a V10000 in ESG mode too. All the ESG nodes within one cluster share the same configuration and same log database, accept mail traffic and deliver to destination mail server according to the same policies.
ESGA v7.6 201 99
Clustering Details
Slide 7:1
ESG Clustering Key Features In ESG cluster mode, mail traffic is distributed to different ESG appliances based on some traffic balancing algorithm. The traffic balancing algorithm is not on the ESG. We’ll talk about it later. All ESG nodes are managed centrally and can log to a central log server for consolidated logging across the enterprise. Even in a case with multiple clusters, all log events should be centrally stored, and reporting should be aggregated across all boxes. Even in a case with multiple clusters, all quarantined emails can be stored locally in each ESG appliance or centrally on an external storage device, but should be accessed from the same interface(s). ESG cluster is an Active/Active load balancing feature that provides real time synchronization between the primary and all secondary nodes for device configuration, which provides the most flexibility and maximizes system investment as requests are load-balanced across all available processing capacity. In Active/ Active deployment, each node must stay in a consistent state and must reflect the current state of the system
100 ESGA v7.6 201
Clustering Details
Slide 7:2
Clustering with Load Balancer This network deployment shows load balancer with three ESG appliances connected to it. The load balancer redirects mail traffic to different ESGs with certain traffic balancing algorithm. The balancing algorithm is part of the load balancer configuration and not ESG. The flow of traffic is as follows: 1. The firewall receives the email from the Internet. 2. If the messages are permitted, the firewall forwards them to the IP address of the load balancer. 3. The load balancer, using the specified algorithm, decides to which of the ESG appliance send the mail message. 4. The ESG, which receives the email from the load balancer, processes the message. formation is stored about the message. The permitted message is forwarded to the email server. 5. The email server receives the messages which ed the ESG filtering. The messages are now available for pick/delivery by the end s.
Clustering details Each ESG in a cluster has one of three roles:
ESGA v7.6 201 101
Clustering Details
Primary
The primary node when working in cluster mode. Every cluster can have only 1 primary ESG. The primary is the only node in cluster to be configured via TRITON console.
s configure the policies on the primary node through TRITON console. The configuration is then synchronized to all the secondary node.
Secondary
The secondary node when working in cluster mode.
Every cluster can have 1 to 7 secondary ESGs.
The secondary node can not be configured via TRITON console directly.
The secondary nodes synchronize the configuration from the primary node.
The status and logs are synchronized time to time as well.
The secondary nodes and the primary process mail traffic based on the same policies.
Standalone
Any appliance that is not set as primary and not negotiated to be a secondary stays in the role of standalone.
Important One cluster has a maximum of 8 nodes. All nodes of cluster must be the same appliance type: you cannot mix V10000 with V5000 in the same cluster. All nodes of cluster should have same applications deployed on the appliance: all appliance are either single mode (ESG only) or dual mode (ESG and WSG). Tip When cluster negotiation does not succeed, the primary node will not become standalone automatically. The will need to change this manually. Important When you have more than one ESG appliances, clustering is not a requirement. You can have all ESG appliances be in standalone mode; however, you will have to configure each ESG individually. This is not recommended and leads easily to inconsistent policy settings.
102 ESGA v7.6 201
Clustering Details
Slide 7:3
Clustering with MX Record This scenario does not have a visible load balancer deployed. It loads and balances email traffic on multiple ESG appliances using MX record. Let’s take the diagram as an example, in which you have three ESG appliances in your cluster. You can re-configure your DNS as one of the following:
Configure the MX record in DNS to return three routable IP addresses which bind to the external interface of the firewall. Each record should have the same preference value, so that the traffic is evenly distributed on the basis of number of incoming connection requests. The firewall maps the external IP addresses to the internal one of the corresponding E1 interfaces on the ESG appliances (1-to-1 NAT)
Configure the MX record in DNS to return three routable IP addresses which bind to each of the E2 interfaces of the ESG. The E2 interfaces are in the DMZ. The E1 interfaces are on the LAN side.
The flow of traffic is as follows 1. The sending email server queries the DNS MX record for the destination domain.
The DNS returns three IP addresses with the same preference value.
The order of the IP addresses are set to round robin mode.
ESGA v7.6 201 103
Clustering Details
The email server connects to the first IP address in the list (the other addresses are used if the first one does not respond).
The DNS returns the information about the next hop as well.
2. The email server connects to the next hop in the chain. 3. The messages arrive to the premise firewall. Permitted traffic is allowed through. 4. The firewall receives the incoming SMTP request and es it to the corresponding ESG appliance (using one of the two techniques listed above). 5. The layer 2 switch receives the connection request for the address of the selected appliance. This appliance receives and processes the email message. If the policy filter allows the email through, the appliance sends the email to the email server. The email server receives the email that has ed filtering, which is now available for pickup or delivery by the end use.
104 ESGA v7.6 201
Clustering Details
Slide 7:4
Centralized Management Under cluster mode, the primary and all secondary nodes are managed centrally. 1. The log on TRITON Unified Security Center to configure the primary node. 2. The configuration of the primary is synchronized to all secondaries. Note Only the primary has read/write privilege for ESG configuration. 3. The primary and all secondaries of the cluster performs process inbound and outbound emails based on the same policies. 4. All off-box logs generated from different ESG nodes will be sent to same log server and stored to same database. Note Only the primary has privilege to view ESG dashboard/ logs and generates reporting, the scope of reporting data covers all ESG nodes in cluster.
ESGA v7.6 201 105
Clustering Details
Note The secondary only has the privilege to filter emails and report everything to the primary. When the primary is down, all secondaries will keep working and filtering as secondaries; no one will switch to the primary automatically. can reconfigure the cluster to pick a secondary to act as the new the primary via TRITON Console. As the configuration is synchronized among the cluster nodes, the new cluster will work in the same manner. The only lost is the emails in primary work queue, which can not be recovered. When a secondary appliance is down, the cluster is not affected. Cluster machines continue to process mail traffic and share a configuration.
106 ESGA v7.6 201
Clustering Details
Slide 7:5
Standard Deployment A load balance device is deployed in front of ESG cluster. It redirects mail traffic to different ESG nodes with certain traffic balancing algorithm. This needs to be done by customer when ESG is deployed as cluster mode.
ESGA v7.6 201 107
Clustering Details
Slide 7:6
Configuration Synchronization Configuration synchronization is handled by Config Daemon. The following tasks are synchronized:
Configuration: All updates in configuration will be propagated from the primary to secondaries.
Log: In-box logs are synchronized from secondaries to the primary; off-box logs are saved on the same external systems, no need to synchronize.
Quarantine: All quarantine related database and files are synchronized from secondaries to the primary.
DSS registration: The primary s with DSS Manager; DSS registration is synchronized to secondaries.
Not to be synchronized (only on the primary)
Signature database update
Hot fix/patch update
Hybrid service registration
108 ESGA v7.6 201
Clustering Details
Note Only when the primary has the same version and platform, and run in the same mode with the secondary, configuration will be synchronized. Otherwise the node status would be “not synchronized” with an alert being sent.
License synchronization The same license can be imported to each node of the cluster before negotiating, and the license can be imported after negotiating too: to the primary at first then synchronized to all the secondaries automatically. Customers must assure licenses of each cluster node are either same or empty. If different licenses are imported to the nodes of a cluster, the cluster negotiation will fail.
ESGA v7.6 201 109
Clustering Details
Slide 7:7
Hybrid Service and DSS Integration Hybrid registration only happens on the primary and is not synchronized with the secondaries. Hybrid registration result is synchronized to the secondary. When the primary is down, the secondaries still process Hybrid mail traffic. DSS registration happens on the primary and is synchronized to the secondaries. ESG cluster works in below way:
The primary s with the DSS manager.
The primary and secondaries point to same DSS manager and log & reporting server
All configuration except cluster related configuration (IP, primary/secondary) are synced from primary to all secondary appliances.
Inbound traffic: traffic will be load balanced by MX record or external L7 load balancer.
Outbound traffic: DLP scanning is on secondaries locally. But if the mail quarantine is required, it will send the message to primary DSS manager for processing.
110 ESGA v7.6 201
Clustering Details
Slide 7:8
PEM Under cluster mode, the end can log on either the primary or secondaries to utilize PEM functions. ESG cluster provides 2 options to deploy PEM:
End s log on PEM of the primary only: The primary will redirect to the relative node for PEM, which is transparent for end s.
End s log on the specific ESG nodes for PEM: End s are assigned to different PEM address, which point to respective nodes in cluster.
The slide takes the first option as an example. The process of retrieving isolated messages via PEM proceeds as follows: 1. ESG #1 (not the primary) receives a message, which is going to be isolated. 2. The ESG #1 writes an entry in the log about this message. As mentioned before, a key field in the database is the ID of the ESG who isolated the message. 3. The recipient, which has been notified that he/she has received an message being isolated, connects to the PEM of the primary node. The link in the notification points to the TRITON - Email Security. 4. The primary ESG looks up the message details in the log database. Using the ID of the machine, which isolated the message, can connect to that ESG (in this case ESG #1)
ESGA v7.6 201 111
Clustering Details
5. The primary ESG communicates with the ESG #1, retrieves the message, and sends it over to the who requested it.
112 ESGA v7.6 201
8
Archive
As businesses develop, many organizations face the email issues in storage, discovery, and retention: Storage Management: Quota limits on mailboxes frustrate end s- yet unlimited mailbox storage is costly for IT to provide and will only get worse as email traffic is expected to grow to 507 billion messages per day by March 2013. Discovery: 87% of U.S. companies faced law suits and 56% of companies have been ordered to produce employee email by a court or regulatory body- yet only one in seven organizations can recover email that is older than one year. Retention and Compliance: Many laws and regulations require retention of email for extended periods of time. Only 31% of organizations are moderately prepared to comply.
Websense Email Archive is an add-on feature to ESGA. It is a full software-as-aservice (SaaS) email archiving solution that integrates with email servers to accumulate email messages and provides quick, searchable access to s. It features comprehensive e-discovery and compliance capabilities, which s can setup for select s to access within the system.
ESG has the basic archive feature: saving all incoming messages to an archive queue locally or remotely. This feature is different with the add-on archie that we are going to talk about in this chapter.
ESGA v7.6 201 113
Archive
Slide 8:1
Websense Email Archive Solution Add-Ons Websense offers three email archiving add-ons that address unique challenges:
Websense Email Archive: Provides end s unlimited mailbox size and allows them to quickly and easily access their personal archived email.
Websense Discovery Archive: Provides Legal counsel/HR/s the ability to investigate, search, tag, and export email across the entire organization to satisfy discovery requests and email retention.
Websense Email Continuity: Provides end s/IT with “standby mailboxes”, enabling them to send and receive email when their company’s email server is unavailable as if it were online.
The three add-ons are available for all Websense email security products (ESGA, ESG, HES, WES). Customers must have one of the following products to purchase email archive add-ons:
Websense Email Security Gateway Anywhere
Websense Email Security Gateway
Websense Hosted Email Security
Websense Email Security
114 ESGA v7.6 201
Archive
Slide 8:2
Email Archive Package and License Email Archive Package Each email archive add-on could be purchased individually or together in various package combinations:
Websense Email Archive Suite: Websense Email Archive + Websense Discovery Archive
Websense Email Archive with Hosted Continuity: Websense Email Archive + Websense Email Continuity
Websense Email Archive Suite with Hosted Continuity: Websense Email Archive + Discovery Archive + Email Continuity
ESGA v7.6 201 115
Archive
Websense Email Archive Solutions add-ons are available as a service in the cloud and does not require the cost and complexity of any on-premise equipment. Note Websense Email Continuity cannot be orderable as an individual add-on. It must be combined with Websense Email Archive (Websense Email Archive with Hosted Continuity or Websense Email Archive Suite with Hosted Continuity).
License For each email archive add-on, customers purchase a subscription license based on the number of mailboxes/s. Flat rate pricing for all bands. The number of license s for Websense Email Archive Solutions doesn’t need to match the number of s for the Websense email security product. Customer could purchase a subset of s. For example, customers may purchase licenses for a defined group of internal s (i.e. HR, Legal) rather than for the entire email population. The minimum number of s/mailboxes that could be purchased is 25.
116 ESGA v7.6 201
Archive
Slide 8:3
Websense Email Archive - Personal Archive The slide above shows the key features of Websense Email Archive: 1. Roles Management: Websense Personal Archive s any number of customized roles, each with a unique and customizable set of permissions associated with them.
: All new s of the archive can be granted access to their own Personal Archive which will give them the ability to search their own historical emails.
Manager: This role can be customized to set up new s, reset s, and have additional privileges.
Policy Manager: The Policy Manager can view and update policy settings (e.g., set time zones, specify storage of emails based on email direction).
Role Manager: The Role Manager can view and modify privileges of other s.
: The role has all the above/available permissions.
2. to the Personal Archive
ESGA v7.6 201 117
Archive
s can access their archives from Microsoft Outlook, IBM Lotus Notes, Outlook Web Access, Lotus Notes Web Client, BlackBerry devices and through a browser-based, secure website, which is a convenient option for employees working from home or on the go.
s only have to enter the names and s one time when opening Personal Archive through Outlook. After one successful , they will automatically be logged in on all subsequent visits to the archive.
3. Outlook Look and Feel In addition to giving s the power to restore lost or deleted emails back to their inboxes, Email Archive also allows them to compose, reply to and forward messages directly from their archives, just like they would in Outlook or Notes. 4. Quick Search With Quick Search, simply typing desired search into the Message field and clicking on the magnifying glass icon will begin the search. The search results appear in a list of messages, and the search are highlighted in yellow in the preview pane. 5. Advanced search
The Advanced Search option gives the s the ability to customize their searches based on a variety of criterion, such as message keywords, to, from, subject, date(s), attachments, etc.
Search multiple archives directly from Outlook, enabling better knowledge management and retention of archives from departed employees.
6. Search Filter Search Filters help further hone searches, allowing s to drill into specific categories, such as sender, date, tags and attachments.
Websense Email Archive benefits:
Unlimited Mailbox Websense Personal Archive gives an unlimited mailbox and allows s to quickly and easily access their archived emails directly from Microsoft Outlook or Lotus Notes. For added convenience, Folder Sync keeps s’ Outlook folders intact in the archive for easier search and retrieval. Restoring lost or deleted emails gives end s the power to restore lost emails themselves- even things they may have deleted from their desktop, laptop, BlackBerry® device or Windows Phone 7. Reducing the need for PSTs/NSFs eliminates free-range email files (PST/NSF), and minimizes the burden on IT staffs and/or help desk Improving server performance improves performance by reducing the footprint on customers’ mail servers, and shrink backup windows
118 ESGA v7.6 201
Archive
Boost Productivity Do you know how much time people spend sorting and filing their emails? Almost 9 out of 10 Internet s spend seven hours a week managing email. That’s a lot of lost productivity! Plus, 95% of the email sitting in your inbox will never be looked at again. With the Email Archive, s can delete email with confidence and stop wasting time managing the inbox.
Familiar, Easy-to-Use Interface Email Archive seamlessly integrates with Outlook or Notes, so it’s easy to complete the most common day-to-day tasks, using a variety of familiar shortcut buttons.
Comprehensive Search Features Email Archive makes retrieving old or deleted email easy. s can search for specific emails and attachments two ways: Quick Search and Advanced Search. The Advanced Search feature allows s to fine-tune their searches with key parameters. The Search Filters help further hone searches. s can also save their searches for future reference (which means they don’t have to keep recreating their most common searches).
Eliminate Local Archives Removing local archives helps reclaim space on s’ shared drives, reduces data leak protection issues when laptops are lost or stolen and eliminates the hassle involved in searching individual desktops when they are responding to a legal discovery request.
ESGA v7.6 201 119
Archive
Slide 8:4
Websense Discovery Archive Motions to discover electronic data are commonplace in today’s world of litigation; yet, more often than not, companies are unprepared to find electronically stored information, especially email and attachments. Discovery costs represent at least 50% of the cost of litigation- most of which relates to the collection and review of email. Websense Discovery Archive is a complete and affordable Security-as-a-Service (SaaS) solution designed to seamlessly manage the e-discovery needs. Our fully managed, web-based service meets and exceeds the e-discovery and mailbox management needs of businesses and public organizations of all sizes. Discovery Archive also helps significantly reduce the time it takes to respond to e-discovery requests.
Store and index every email, attachment, IM and BlackBerry® message (SMS text, PIN-to-PIN, call log) in a centralized, online, and tamper-proof repository.
Accelerate legal discovery and HR inquiries with real-time search and retrieval and multiple export capabilities. Search the entire contents of archived emails and attachments.
Enforce email preservation for lawsuits/legal holds, and protect attorney-client privileged communications.
120 ESGA v7.6 201
Archive
Websense Discovery Archive Features Store & Index
Discription
Secure and Automated Message Capture
Discovery Archive automatically captures all messages that are sent or received without any intervention and securely transmits them using TLS or VPN encryption to the LiveOffice data centers for retention.
Single-Instance Storage
Once archived, all email is de-duplicated and stored as a single instance to optimize storage space and streamline the searches.
Unlimited Storage
Unlimited storage eliminates the need for mailbox quotas and compounding storage costs.
Replicated Storage
All email messages and attachments are stored in real time and then replicated to multiple, geographically dispersed top-tier data centers.
Secure Storage
Websense enterprise-class infrastructure meets the highest industry standards for secure data storage, including SAS-70 Type II, and guarantees the authenticity of the archived data.
Tamper-Proof Storage (Optional)
For companies with stricter compliance requirements, WORM (write once, read many) storage is also available.
Search & Retrieval
Description
Lightning-Fast Search and Retrieval
Quickly search the entire contents of archived emails and attachments using a variety of search criteria, including to, from, date, subject, message body, message attachments and other message properties.
Online Review and Tagging
Reviewers can efficiently navigate through search results, identify highlighted search and tag potentially harmful emails, so they are easily retrievable for further review.
PST/NSF File Export
Once s have tagged all emails related to a specific case or matter, they can use Discovery Archive’s export feature to load the files into a case management solution or other application for further review and analysis.
EDRM XML Export
Approved IT and legal personnel can seamlessly export messages from Websense Discovery Archive to e-discovery solutions from Clearwell Systems, CT Summation, FTI Technology and iConect.
ESGA v7.6 201 121
Archive
Search & Retrieval
Description
Re-Forward or Print
Reviewers have the ability to print or re-forward suspect emails to the appropriate parties for further review and case submission.
Saved Searches
Reviewers can create and save customized email searches based on the organization’s email policies, and re-run them as necessary.
Policy Alerts
Reviewers can setup Policy Alerts to notify them when an email meets Saved Search criteria (e.g., contains specific words or phrases).
Access & Control
Description
Familiar Interface with Full Outlook and Notes Integration
Discovery Archive’s highly intuitive, Microsoft Outlook-like interface helps s get up to speed quickly. In addition, full Outlook and Notes integration allows s to centrally deploy Websense Personal Archive (optional addon service) to some or all s.
Authentication and Granular Permissions
s authenticate using SSL to , ensuring secure access to the archive. Discovery Archive has two distinct permission levels: s (full access) and Reviewers (monitoring rights for selected mailboxes).
Policy-Driven Retention and Destruction
Discovery Archive retains the emails and attachments as required to meet customer’s company’s specific retention or regulatory requirements. We also purge and destroy older email stores based on the custom criteria (e.g., any emails five or more years old).
Automated Legal Hold Enforcement
If customer’s company is in pending litigation, they can place legal holds on specific communications (based upon search criteria) to safeguard the staff or automated deletion policies from inadvertently deleting case-relevant emails.
Attorney-Client Privilege Tagging
s and reviewers can flag attorneyclient privileged communications, which are excluded from e-discovery requests.
Full Audit Trail
Discovery Archive’s search log captures the auditing activities of the reviewers, so s can ensure that they are conducting appropriate reviews and meeting compliance requirements.
122 ESGA v7.6 201
Archive
Access & Control
Description
Groups
s have the ability to group s based on functional, divisional or custom criteria. Reviewers can then search across these groups.
Mail Servers
Discovery Archive s Microsoft Exchange 2003, Microsoft Exchange 2007, Microsoft Exchange 2010, IBM Lotus Domino 6.x – 8.x, Novell GroupWise and Linux/Unix mail servers (SMTP).
ESGA v7.6 201 123
Archive
Slide 8:5
Websense Email Continuity Websense Email Continuity is an always-on backup that provides customers with “standby mailboxes”, enabling them to send and receive email when the primary server is unavailable as if it were online. Email Continuity allows the IT resources to focus on the root cause of the outage instead of wasting time implementing temporary solutions (e.g. MX repoints, etc.).
Serves as “standby” mailbox when the primary email server is down
s can easily create, reply to and forward messages during outage with Outlook Look and Feel
Email Restore s Microsoft Exchange, IBM Lotus Notes and Novell Groupwise
All emails sent or received during the outage are automatically routed back to the email server after the outage
30-days of email history
Overcome email server breakdowns, stay productive and protect customer’s business with Websense Email Continuity.
124 ESGA v7.6 201
Archive
Slide 8:6
Inbound Email Flow The slide above describes the inbound email flow with Websense Email Archive. 1. During normal operation The inbound messages are transmitted via the Internet, filtered for spam and virus by ESGA, routed to the email server and distributed to the recipients. At the same time, the inbound messages are captured in the Websense Email Archive via journaling. End s can access the these messages through Outlook, Lotus Notes, or web-based archive interface anytime. 2. During the email server outage Email messages are transmitted via the Internet, filtered for spam and virus by ESGA and automatically routed to the Email Continuity archive, bying the failed email server. End s can access the messages that are sent before and during the outage. Once the email server is restored, messages are routed from the archive back to the email server (no duplicate messages are stored).
ESGA v7.6 201 125
Archive
Slide 8:7
Outbound Email Flow The slide above describes the outbound email flow with Websense Email Archive. 1. During normal operation The outbound messages are sent from end s, routed to the email server, filtered for virus and confidential data leakage by ESGA, and transmitted over the Internet. At the same time, the outbound messages are captured in the Websense Email Archive via journaling. End s can access the these messages through Outlook, Lotus Notes, or web-based archive interface anytime. 2. During the email server outage End s compose, reply to and forward messages directly from the Email Continuity interface. The messages are then scanned for viruses and confidential data leakage by ESGA prior to being transmitted over the Internet.
126 ESGA v7.6 201
Archive
Slide 8:8
Websense Ingestion Service The diagram above illustrates the Websense ingestion service that help to imgrate the legacy data from customer premises to Websense Email Archive server and preserve chain of custody at the same time.
Legacy Data Legacy data is a generally referred to as information stored in an old or obsolete format (or computer system) that is, therefore, difficult to access or process. In the email archiving world, legacy data refers to email data sitting on:
Email storages (e.g., Microsoft Exchange, Lotus Notes) on servers
Personal archives (e.g., PST/NSF files) on desktops, laptops and servers
Legacy email archives (e.g., Autonomy, EMC, CA)
Backup tapes
By ingesting your legacy email into Websense’s archiving solutions, you have a complete, living record of all your email history, both past and present for that ingested email. But, it is critical to preserve the chain of custody when you move your email data from one store to another.
ESGA v7.6 201 127
Archive
Chain of Custody If “chain of custody” is a new or a vague concept to you, you are not alone. In the simplest , a proper chain of custody establishes the integrity of a piece of evidence, showing that it wasn’t tampered with or otherwise altered since it was first collected. Chain of custody has significant practical implications for IT managers and other professionals. Chain of custody plays a critical function in litigation, especially when opposing counsel is challenging the authenticity of evidence – particularly where digital evidence and emails are involved. We’ve found that very few organizations adequately for chain of custody or understand its importance until the authenticity is called into question.
Two Optional of Ingestion Services Websense Email Archive solutions do not require implementation/professional services. However, there are 2 optional services that are available:
Websense Data Ingestion Service: Service generally purchased up front to import all legacy email data from desktops, laptops, servers, into the Websense archive solution.
Websense Data Export Service: Service generally purchased when a customer terminates their Websense archive subscription so that they can re-capture the data stored in our cloud.
The services are one-time fee based on a per GB basis. They aren’t performed at customers’ sites. Customers merely ship all storage media desired to be archived to Websense. Websense will then import/export the data.
Websense Data Ingestion Process As the illustration above depicts, the Websense Data Ingestion Service process involves 6 key steps: 1. Extraction In the extraction phase, the customer extracts legacy email data from existing repositories. This is a critical first step and one that provides the greatest opportunity to break the chain of custody. So, it’s important to document the number of original files/PSTs/NSFs sitting in customer’s existing stores in order to compare to the total number of files/PSTs/NSFs imported into the archive.
128 ESGA v7.6 201
Archive
The following table lists the descriptions for the ed formats:
Description
PST
• • • •
EML
• Does not preserve folder structure
MSG
• Longer conversion and ingestion time • Does not preserve folder structure
NSF
• Ability to preserve folder structure upon request • Longer conversion and ingestion time • Additional cost
Files need to be 2 GB or less in size PST files cannot be protected Files can be zipped with a Ability to preserve folder structure upon request
Physical Media: • •
Format
If the volume of data is less than 10 GB, send the data on CD/DVD, Flash Drive or via S-FTP (secure FTP). If the data volume is greater than 10 GB, send the data via a USB hard drive (high speed USB 2.0 with external power recommended).
Encryption: Websense requires a basic level of encryption (i.e., protected zip file) for all email data transferred and strongly recommends using TrueCrypt Freeware (truecrypt.org) or PGP (pgp.com) to encrypt data before shipping it to us as a best practice. Zipping the data with a is also acceptable, but less desirable, since it delays the import. We do not accept -protected PSTs, since they are generally not secure and cannot be processed by our legacy system.
2. Shipment During this phase of the project, Websense outlines the key steps for the customer project. This includes having the customer complete our Legacy Order Form, which captures key chain of custody information, such as the date and time sent, number of files, number of emails, format of legacy data (e.g., PSTs), amount of data and information. Then the customer ships the data along with the Legacy Order Form to us, addressed to the attention of the Websense Data Import Center. 3. Verification Upon receipt of the data, we log, initial and date the Chain of Custody Form (part of the Legacy Order Form) under received data. The case is then assigned to one of our specialists, who reviews the details and adds the customer project to our schedule. The completion date is based on the current volume of legacy
ESGA v7.6 201 129
Archive
s in our queue. In the meantime, Websense places the data in its current format in our data vault for safe storage. After the data is extracted and ed, the specialist from the customer verifies that the data volumes match the information they provided on the Chain of Custody Form and updates the customer any deviations. If there is a material difference in the email received/extracted vs. the data on the form, we then explore the reasons for the missing data. Note If the data does not match the Chain of Custody Form or the data is corrupt, the customer will be notified to inform our findings and determine the next steps. 4. Data Import After the legacy data is prepped, the specialist from the customer imports it into the archive. Websense securely captures and indexes all messages (and attachments) imported into one of our archiving solutions – Discovery Archive or Email Archive. Once indexed, s and s can quickly and easily search the entire contents of archived emails and attachments, using a variety of search criteria, including to, from, date, subject, message body, message attachments and other message properties. As a final check, the specialist from the customer verifies the data and signs off on the Chain of Custody Form. 5. Chain of Custody Documentation Once the import is complete the customer will be notified with the results. Any deviations from the original Chain of Custody Form prior to ingestion are noted and reviewed. 6. Returned Data/Destroyed Data The final step in the process involves Websense either returning the original data to the customer or destroying it. In either case, we document the process and note the date the media was either returned or destroyed per the customer wishes and retain the final copy of the Chain of Custody Form in our locked data vault for records.
Key Points About Websense Data Ingestion Service
Use of FTP: Websense recommends using the FTP option when shipping data for only small data sets (i.e., less than 10 GB).
ed Formats: Websense s a variety of email formats, including PST, EML, MSG, NSF and MBOX. PST is the required format if preserving the folder structure for the data that is imported.
130 ESGA v7.6 201
Archive
File Sizes: Websense can process PSTs up to 20 GB, but we prefer files that are 2 GB or less. Smaller PSTs have less chance of corruption and process faster.
Media Types: Though we accept DVDs, Websense recommends physical hard drives or flash drives when the customer ships data to us.
Encryption: While Websense requires a basic level of encryption for email data shipped to us, we do not accept -protected PSTs, since they are generally not secure and cannot be processed by our legacy system.
Mapping Archiving s: When we import the PST files, we first scan them to determine how to map the customer s to individual archive s. If a sender’s email address is found in more than 80 percent of the messages in the PST file, the emails are archived under the sender’s address. This process, however, is error prone, so we recommend providing us with a CSV file for mapping purposes. If a CSV file is not provided, the system automatically determines the mapping.
ed Message Types: We currently only archive post items. Non-post items, such as calendars, s, system messages and voicemails, are not currently ed.
Legacy Unassigned Archive: If we cannot match an email address or common name to an archived , we place the emails in the Unassigned Legacy Archive. Emails in the Unassigned Legacy Archive must have a common name (unmatched) or an email address with a recognized domain in the archived .
Corrupted Data: Before sending us data, we encourage the customer to run scanpst.exe, a utility included with Microsoft Outlook, to determine whether or not the PST file is corrupted. We cannot import corrupted PST files.
ESGA v7.6 201 131
Archive
Slide 8:9
Websense Email Archive - Folder Sync Folder Sync is an add-on service to Websense Personal Archive. As end s create, delete, and rename folders, as well as move email messages from one Outlook folder to another, the scheduled synchronization service replicates the identical folders and message location in Personal Archive. After initial synchronization, Folder Sync sends only incremental mailbox folder changes, which can be scheduled hourly, daily or weekly as s see fit. Folder Sync does not require plug-ins or hardware. It is compatible with Microsoft Exchange Server 2003, 2007 and 2010. s can selectively deploy Folder Sync on a per- and/or Exchange Server basis.
132 ESGA v7.6 201
Archive
Slide 8:10
Quick Start - Configuration on Exchange Server You need to activate Microsoft Exchange Journaling and direct emails to the Websense Email Archive servers. As a result, every email that is sent to and from the exchange server will be copied and transfered to the Websense Email Archive servers. 1. Log on to the exchange server with . 2. Follow the steps outlined in journaling guides, available on our website. The instructions depend on the version of Exchange. Exchange 2003 Standard Journaling Guide
http://www.websense.com/content// library/email/shared/exchjour_2003_std/ first.aspx
Exchange 2003 Envelope Journaling Guide
http://www.websense.com/content// library/email/shared/exchjour_2003_env/ first.aspx
Exchange 2007 Journaling Guide
http://www.websense.com/content// library/email/shared/exchjour_2007/first.aspx
Exchange 2010 Journaling Guide
http://www.websense.com/content// library/email/shared/exchjour_2010/first.aspx
ESGA v7.6 201 133
Archive
3. When prompted, point the email to the Websense address shown in the table.
134 ESGA v7.6 201
EMEA servers
@WebsenseJournal.archivesuite.com
U.S. and others
@WebsenseJournal.archivecloud.net
Archive
Slide 8:11
Quick Start - Configuration on Websense Archive Server Enabling view of archive As a security measure, the default settings on the archive disallow view of end email. To view all messages in the archive, perform as follows: 1. Log on with the credentials provided by Websense. 2. Click the istration bar at bottom left of the page. 3. Click the “@...” address. 4. Under the Privilege option, check View All s.
ing the setup After completing the setup process, test the archive as the follows: 1. Log on with the credentials provided by WEbsense. 2. Click s on the left-hand side of the screen, and select the email address of your choice. 3. To explore the features of archive, click Help on the left side of the page.
ESGA v7.6 201 135
Archive
Slide 8:12
Quick Start - Configuration on Client The end s can use Personal Archive Access (PAA) to access their individual archives directly from Microsoft Outlook. Provide the setup instructions to the end s: 1. Create a new folder in Outlook (File > New > Folder). 2. Highlight the new folder, and select File > Folder > Properties. 3. Select the Home Page tab. 4. Enter the URL for Personal Archive Access: EMEA
https://websense.archivesuite.com
U.S and others
https://websense.archivecloud.net
5. Mark the box Show home page by default for this folder. 6. Click OK. 7. Click the newly created folder. After the Personal Archive is configured, end s are prompted only once for their name and . Additional details for the end s are contained here:
136 ESGA v7.6 201
Archive
http://www.websense.com/content//library/email/shared/email_archive/ WEA_end__guide.pdf Note Websense offers Websense CloudLink, an application to help to deploy Outlook Web folders to all of the end s. CloudLink is included with the Websense service.
ESGA v7.6 201 137
Archive
Slide 8:13
Archive vs Backup Backups are for disaster recovery, while archives are for retention and discovery. Backups were never really intended to meet regulatory requirements and other compliance needs. They most effectively serve as a short-term insurance policy to facilitate disaster recovery (assuming they are kept off-site). Archiving, on the other hand, is specifically designed to quickly and easily meet regulatory requirements and other compliance needs. In addition, archiving serves another important role. It can reduce the strain on customer’s in-house email servers. With growing email volumes and skyrocketing storage costs, email archiving can:
Reduce the size of email stores
Reduce backup windows
Reduce the amount of IT budget earmarked for storage
For most modern organizations, the operational, legal and compliance challenges of email necessitate going beyond simple backups.
138 ESGA v7.6 201
9
ESG Log and Reporting
ESG logs and collects data about email traffic and system activity to generate presentation reports and dashboard charts and statistics for today and history. Records of email filtering activities and system events are stored either in the on-box database, that is, local PostgreSQL, or received by the off-box log server and then transferred to the remote database, Microsoft SQL Server. From TRITON - Email Security, ESG classifies the logs into: message log, audit log, system log and console log; and catalogs the reports into: today dashboard, history dashboard, and presentation report. ESG log and reporting provide the facilities for s to configure the local and remote log database and the database maintenance options, set report preferences, and customize a report template, schedule and run a presentation report. Now let’s look at log and reporting in more details!
ESGA v7.6 201 139
ESG Log and Reporting
Slide 9:1
Primary Concepts ESG logs
Message log records information about each inbound message received by ESG. Message logs are collected from the ESG internal component: Filter Demon.
Audit log, only available for super s, shows which s have accessed the Email Security module of TRITON console, as well as any changes made to policies and settings. Audit logs are collected from the ESG internal component: Config Daemon.
System log records all ESG system level events, along with any errors or warnings produced. System logs are collected from each component inside the ESG, such as the Filter Daemon, Update Daemon, Config Daemon, etc.
Console log records any activities or changes made to the Email Security module of TRITON console. Console logs are collected from TRITON Email Security, not from the ESG internal components.
ESG logs are searchable by predefined time periods or customized conditions. For message logs, you can refine the search using conditions like email address, scanning
140 ESGA v7.6 201
ESG Log and Reporting
result, or message status and export the search results to an Excel, HTML, or XML file. For other three types of log, you can exported to an Excel or HTML file. Note The maximum number of log entries exported cannot be greater than 100,000.
ESG reporting
Today dashboard provides high level summary statistics of today to identify potential security and email flow and delivery issues with high quality and interactive graphics.
History dashboard of last 30 days, similar with today dashboard, get an overview of email scanning activity for up to the past 30 days.
Presentation reports include a set of predefined charts and tabular report templates with which you can generate graphical reports of Email Security Gateway message traffic activities.
ESGA v7.6 201 141
ESG Log and Reporting
Slide 9:2
Log and Reporting System Architecture The slide displays the main components of ESG log and reporting system. The log system is built in ESG, including log collection, on-box log database, and Log Daemon. The log system collects logs from ESG internal components then stores the logs on-box or off-box. The reporting system is composed of Log Server and MS SQL Server, which is similar with WSG reporting system. 1. A collection of raw data, in the form of logs from various ESG internal components, are fed into the Log Daemon. 2. On-box logs are transferred to PostgreSQL. 3. Off-box logs are transferred to the Log Server. 4. If the Log Server is unreachable, a Cache Log is utilized. Once the logs are in MS SQL Server, it utilizes an Extract/Transform/Load (ETL) process in every 20 seconds to convert original logs to reporting data. 5. Stored procedures are utilized to generate reports and dashboards via the WebUI. As you will see, the ESG WebUI is very similar to WSG Reporting UI.
142 ESGA v7.6 201
ESG Log and Reporting
Slide 9:3
Message Log The Message Log records information about each inbound message received by Email Security. Access the Message Log on the Main > Status > Logs page. You can configure the number of entries per log page, between 25 and 200, in the Per page drop-down list in the log table banner. Scroll through Message Log pages by clicking the back and next arrows in the banner, or enter a specific page number in the Page field and click Go. Message records are saved for 30 days. To preserve message records longer than 30 days, use the Export option to export the log on a regular basis. Exporting does not remove records from the Message Log. It transfers log data to an Excel, HTML, or XML file. When the Message Log page appears, the most recent records are shown. Use the View from/to fields to specify the date/time range for the log entries you want to see. The calendar includes the following options:
Click the time in the lower left of the calendar to set the time in hours and minutes.
Click Today to set the calendar date to today’s date.
Click Apply to activate your date/time calendar changes.
If you click Clean and then Apply, the current date/time selection is removed.
ESGA v7.6 201 143
ESG Log and Reporting
The Message cludes several search options, including date range or keyword searches. Determine the date/time range for a search by selecting dates in the View from/to field calendar controls. Default value for the from or to field is the date and time that you open the log.
144 ESGA v7.6 201
ESG Log and Reporting
Slide 9:4
Audit Log Websense Email Security provides an audit trail showing which s have accessed TRITON - Email Security, as well as any changes made to policies and settings. This information is available only to Super s. Monitoring changes through the Audit Log enables you to ensure that system is controlled responsibly and in accordance with your organization acceptable policies. Click the Audit Log tab on the Main > Status > Logs page to view the Audit Log, and to export selected portions of it to an Excel spreadsheet or an HTML file, if desired. Audit records are saved for 30 days. To preserve audit records longer than 30 days, use the Export option to export the log on a regular basis. Exporting does not remove records from the Audit Log. It transfers log data to an Excel or HTML file. When the Audit Log page opens, the most recent records are shown. Use the View drop-down list options located above the log to select the range of log entries you want to see: All, One Day, One Week, One Month, or Custom. When you select Custom, use the View from/to fields to specify the desired date/time range for the log entries you want to see. The calendar includes the following options:
Click the time in the lower left of the calendar to set the time in hours and minutes
Click Today to set the calendar date to today’s date.
Click Apply to activate your date/time calendar changes.
If you click Clean and then Apply, the current date/time selection is removed.
ESGA v7.6 201 145
ESG Log and Reporting
Slide 9:5
System Log System Log records for Email Security Gateway reflect the current state of the system, along with any errors or warnings produced. Click the System Log tab on the Main > Status > Logs page to view the System Log, and to export selected portions of it to an Excel spreadsheet or an HTML file, if desired. System Log records are saved for 30 days. To preserve System Log records longer than 30 days, use the Export option to export the log on a regular basis. Exporting does not remove records from the System Log. It transfers log data to an Excel or HTML file. When the System Log page opens, the most recent records are shown. Use the View drop-down list options located above the log to select the range of log entries you want to see: All, One Day, One Week, One Month, or Custom. When you select Custom, use the View from/to fields to specify the desired date/time range for the log entries you want to see. The calendar includes the following options:
Click the time in the lower left of the calendar to set the time in hours and minutes.
Click Today to set the calendar date to today’s date.
Click Apply to activate your date/time calendar changes.
If you click Clean and then Apply, the current date/time selection is removed.
146 ESGA v7.6 201
ESG Log and Reporting
Slide 9:6
Console Log The Console Log is a record of any activities or changes made to the Email Security module of the TRITON Unified Security Center. Click the Console Log tab on the Main > Status > Logs page to view the Console Log, and to export selected portions of it to an Excel spreadsheet or an HTML file, if desired. Console Log records are saved for 30 days. To preserve Console Log records longer than 30 days, use the Export option to export the log on a regular basis. Exporting does not remove records from the Console Log. It transfers log data to an Excel or HTML file. When the Console Log page opens, the most recent records are shown. Use the View drop-down list options located above the log to select the range of log entries you want to see: All, One Day, One Week, One Month, or Custom. When you select Custom, use the View from/to fields to specify the desired date/time range for the log entries you want to see. The calendar includes the following options:
Click the time in the lower left of the calendar to set the time in hours and minutes.
Click Today to set the calendar date to today’s date.
Click Apply to activate your date/time calendar changes.
If you click Clean and then Apply, the current date/time selection is removed.
ESGA v7.6 201 147
ESG Log and Reporting
Slide 9:7
Dashboard The Main > Status > Today: Health, Security and Value Since Midnight page appears when you first log on to TRITON - Email Security. It displays alert messages and graphical charts that show the current state of your email scanning software, focusing on email traffic activity in your network. The charts on this page cover the 24-hour period beginning at 12:01 a.m. according to the time set on the Log Database machine. At the top of the page, two summary sections provide a quick overview of current status: 1. The Health Alert Summary shows the status of your Websense software. Click an error or warning alert message to open the Alerts page, where more detailed alert information is available. Information in the Health Alert Summary is updated every 30 seconds. 2. Under Business Value, see examples of how Websense Email Security has protected your network today by blocking malicious email traffic, as well as the total number of messages handled. Below the summary information, up to four -designated Flash charts provide information about email scanning activities. These charts are available to Super s, and to other s who are granted permission to view reports on the Today page. Click Customize to select the four charts you want
148 ESGA v7.6 201
ESG Log and Reporting
displayed. Information in these charts is updated every two minutes. You may need to scroll down to see all of the charts. Up to two buttons appear at the top of the Today page: 1. Customize, available to Super s only, opens a page where you can select which charts to display on the Today page. 2. Print, available to all s, opens a secondary window with a printerfriendly version of the charts on the Today page. Use browser options to print the page.
ESGA v7.6 201 149
ESG Log and Reporting
Slide 9:8
Today Report Use the Today > Customize page to select up to 4 charts for the Status > Today page. Only Super s with unconditional policy permissions can customize the Today page.
150 ESGA v7.6 201
ESG Log and Reporting
Slide 9:9
History Report Use the Status > History: Last 30 Days page to get an overview of email scanning activity for up to the past 30 days. The four charts on the page are updated daily at 12:01 a.m. to incorporate data from the previous day, as determined by the time on the Log Database machine. You may need to scroll down to see all the charts. The exact time period covered by the charts and summary tables depends on how ESG has been processing mail. During the first month that Websense software is installed, the page shows data for the number of days since installation. After that, the reports cover the 30 days prior to today. Depending on the reporting permissions granted to the role, some s may not see the charts on the History page.
ESGA v7.6 201 151
ESG Log and Reporting
Slide 9:10
Presentation Reports Presentation reports include a set of predefined charts and tabular report templates with which you can generate graphical reports of Email Security Gateway message traffic activities. You can run a report, customize a report template, or mark a frequently used report as a Favorite. You can run any presentation report immediately, or schedule it to run at a particular time or on a repeating cycle. Not all report templates can be customized. Report templates that can be customized display a different icon from reports that cannot be customized. If the Save As button is enabled when you select a report name, then you can save and edit that report to suit your needs. The Save As button is not enabled if you select a report that cannot be customized. Use the Main > Status > Presentation Reports page to generate charts and tabular reports based on templates in the Report Catalog. The Report Catalog organizes a list of predefined report templates and custom reports into groups. Expand a group to see its corresponding templates and custom reports. Click on a template or report title to see a brief description of what it includes.
152 ESGA v7.6 201
ESG Log and Reporting
Slide 9:11
Advanced Features Log Database Option istrating the Log Database through TRITON – Email Security involves controlling many aspects of database operations, including the timing of maintenance, the conditions for creating new database partitions, and which partitions are available for reporting. Use the Settings > Reporting > Log Database page to manage Log Database operations. Database Partition Database partition is the unit to store logs. You can create up to 70 partitions, with the maximum size for each partition is 5120 Mega Bite. Keep in mind that extremely large individual partitions are not recommended. Reporting performance can slow if data is not divided into multiple, smaller partitions. When a new database partition is created, reporting is automatically enabled for the partition. Database Roll over Roll over is the process that the Log Database creates a new database partition. Use the Roll over every option to indicate whether database partitions should roll over based on size (MB) or date (weeks or months).
ESGA v7.6 201 153
ESG Log and Reporting
If the rollover begins during a busy part of the day, performance may slow during the rollover process. To avoid this possibility, some environments choose to set the automatic rollover to a long time period or large maximum size. Then, they perform regular manual rollovers to prevent the automatic rollover from occurring. Database Maintenance Use the Maintenance Configuration to control certain aspects of database processing, such as the time for running the database maintenance job, some of the maintenance tasks performed, and deletion of database partitions and error logs.
Presentation Reports Drill down With Presentation Reports, you can drill down into the underlying data of detailed events, such as identifying the top 10 external addresses for outbound messages, sorted by number of messages in a scheduled time range. Presentation Reports provides comprehensive report catalogs covering from inbound and outbound messages, hybrid and appliance, spam, virus and data security, message transfer and system capacity. And it offers the flexibilities in defining the report time schedule. You can get the expected reports through selecting the recurrence pattern, scheduling time and data range, and specifying the output format.
The following are typical presentation reports available in ESG: Overall message summary
Inbound Messages
Incoming message summary
Top Destination Domains by Message Size
Top N Internal Recipients by Volume
Outgoing message summary
Top Destination Domains by Message Size
Top Destination Domains by Volume
Data security summary
Top Data Security Policies
Spam and virus summary
Spam Quartile Chart
Top Virus Types
Message transfer summary
154 ESGA v7.6 201
ESG Log and Reporting
Delivery Failure
Encryption Gateway Routing
System capacity
Average message in work queue
ESGA v7.6 201 155
ESG Log and Reporting
156 ESGA v7.6 201
10
Troubleshooting Guide
This chapter talks about some basic troubleshooting techniques when working with your ESGA deployments. The following topics are covered: The latter sections will cover various areas of an ESGA implementation and discuss issues in a question and answer format.
General Information: establishes a baseline to ensure that certain hardware and software requirements are met.
License: explains the license expiration, counting, and reset issues.
ESG: tells about the possible errors when connecting the TRITON - Email Security to the ESG and discusses potential reasons.
Log Server & Log Database: covers the Log Server and the Log Database installation and troubleshooting methods.
Cluster: discusses the problems that may arise during the cluster implementation.
Hybrid: involves Q&A about the Hybrid connection, debug, and synchronization.
DLP: talks about the issues of DLP registration, DLP deployment, and policies.
Quarantine: includes the quarantine files location and quarantine Q&A.
System Counter: provides the detail descriptions about system counters.
Appendix: includes the references to ESG Shell, System Diagnosis, and Health Daemon.
ESGA v7.6 201 157
Troubleshooting Guide
Slide 10:1
General Information Before we start investigating the various areas of an ESGA implementation and discuss various troubleshooting techniques, it is important to discuss certain installation specifications. The table above provides minimum requirements for each module. The information below provides more detail surrounding some of the specific items such as database versions and the location of certain log files.
Database version Web (WSG) only:
SQL Server 2005, all editions except Express, all service packs, 32-bit and 64-bit, not IA64
All products:
SQL Server 2008, all editions except Web, Express and Compact, all service packs, 32-bit and 64-bit, not IA64
SQL Server 2008 R2, all editions except Web and Compact, all service packs, 32bit and 64-bit, not IA64
SQL Server Clustering, for all versions of SQL Server that we
158 ESGA v7.6 201
Troubleshooting Guide
Slide 10:2
Troubleshooting From the ESG appliance command line, you can perform the initial configuration, modify settings, check appliance status, start/stop the modules via the commands like firstboot, ip dns, module start, etc. Run help to acquire the command descriptions that the ESG appliance provids. Furthermore, you can enalbe SSH to connect with the ESG shell to check logs and turn on debug mode. This is an effective way for troubleshooting and often used by Technical .
A: Enable SSH 1. From the ESG appliance console, enter the following command to enable SSH access: ssh enable 2. Enter the following command to get a temporary code: local-access
ESGA v7.6 201 159
Troubleshooting Guide
3. From a Windows computer, open Internet Explorer and browse to http:// code.websense.com. Note This website is only accessible within the Websense intranet. 4. Input the temporary code and click Submit me! to convert it to a real SSH . 5. Back to the ESG appliance console, as websense-ts and enter the obtained in above step. 6. Run the command su and the is websense#123. 7. Run the below command to access the ESG shell: ssh esg
B: Log Files Location
Installation log: /var/log/esg_install.log
SSL Proxy log: /var/log/sslproxy.log
ESG statistic log: /var/logs
daily_counter: daily statistics since installation
counter_current: current days statistics
DLP logs: /opt/websense/PolicyEngine/Logs
Default ESG error log:
/var/log/maillog
/var/log/messages
ESG configuration processing log: /var/log/confd.log
ESG Log Server log:
To check the database issues during ESG Log Server installation: $(installpath)\ESG Log Server\SQL\CreateDbInstall.log and *.txt output files
To check the ESG Log Server issues: Windows system event view
C: Turn on Debug Mode 1. From the ESG shell, enter the following command: cd /etc/debug 2. Run the command below: ls The available modules, e.g. filter, scan_engine, mta, etc. are listed here. 3. Run the following command: esgshell
160 ESGA v7.6 201
Troubleshooting Guide
4. Run the following command to debug: debug module <module_name> event all For example, to debug the scan_engine module, enter: debug module scan_engine event all 5. Control the debug output location: a. For standard output, enter: debug output cli b. For file output, enter: debug file <path> <max_size> (wrap | nowrap) path
If the path is a relative path, the file will be placed in the “/var/log” folder.
max_size
Value: 1 - 256 Unit: MB
wrap | nowrap
To wraparound the file or not.
ESGA v7.6 201 161
Troubleshooting Guide
Slide 10:3
Troubleshooting - Example The slide is an example to troubleshooting via SSH (ESG shell). As the diagram shows, the message with confidential data shall be dropped based on the DLP policy; while it is delivered to the destination directly. The troubleshoots the issue in the following ways:
Via TRITON Unified Security Center. Check logs, policies, and configurations.
Via SSH a. to the ESG shell b. Check the log files c. Turn on Debug mode to track the filtering and delivering process.
162 ESGA v7.6 201
Troubleshooting Guide
Slide 10:4
License Issue A: What is the behavior when the ESG license is exceeded? All features in a license key have the same expiration date with the key. E.g, if a key has three features, AV/AS/DLP, and will expire by Dec 22, 2012, AV/AS/DLP will expire on Dec 22, 2012 too. Even if customer adds another feature before Dec 22, 2012, the expiration date is still Dec 22, 2012. UpdateD considers a key expired when DDS server returns the expired flag, which is the unique way to judge the key’s validation. So it’s useless to change system time. ESG sends messages to ESG alert system according to the remaining days in service period:
Message level in 7~30 days
Warning level in 0~7 days
Critical level after expired
Issue B: How are licenses counted? ESG counts the licenses in this way: the total number of all the outbound email senders in 7 days.
ESGA v7.6 201 163
Troubleshooting Guide
Issue C: How do view the current licensed count? From the ESG Shell, enter the following command: counter | grep subscriber
Issue D: Is there a way to reset the counter? From the ESG Shell, enter the following command: counter –f
164 ESGA v7.6 201
Troubleshooting Guide
Slide 10:5
License Reset You may need to perform an ESG subscription key reset after you have uninstalled a fully licensed version of TRITON - Email Security, and then reinstalled the product. An error message warns you that ESG is already licensed.
From the V-Series Appliance Manager 1. to the Appliance Manager: https://Appliance_C_IP:9447/appmng/. 2. In the > Toolbox page, click Launch Utility in the Command Line Utility section. 3. Select Websense Email Security Gateway in the Module drop-down list. 4. Select esg-license-reset in the Command drop-down list. 5. Click Run. The Console output displays "Success" when the subscription key is successfully reset.
From the ESG Shell 1. to the ESG Shell as the root .
ESGA v7.6 201 165
Troubleshooting Guide
2. Run the following command: [root@v10000-esg ~]# reset license The Console output displays "Success" when the subscription key is successfully reset.
166 ESGA v7.6 201
Troubleshooting Guide
Slide 10:6
ESG Error Issue A: Why Email Security could not be launched? The screenshot indicates that TRITON - Email Security service either hasn’t started or met a starting issue. Follow the steps to try to resolve this issue: 1. to the machine where TRITON Unified Security Center is installed. 2. List all the services. 3. Restart the Websense TRITON - Email Security service. 4. Click Retry on TRITON - Email Security interface.
Issue B: Unable to connect to the ESG appliance The connection error page will be displayed when TRITON - Email Security cannot connect to the ESG appliance. Perform as follows to determine the cause of this issue: In the Setting > Email Appliance page, check the unconnected reason by moving the mouse over the warning icon. We’ll talk about the way to solve this in the next slide.
ESGA v7.6 201 167
Troubleshooting Guide
Slide 10:7
Appliance Connection Error The slide lists the possible reasons of the appliance connection error:
Appliance is not accessible:
Ping the IP addresses of the C, E, and P interfaces. Restart the ESG modules if the console output returns with errors.
Check the statuses of all services in the appliance. If the network runs well, the SSLProxy and config-daemon service issues would cause the connection errors. Reset these services to fix the connection issue.
The TRITON - Email Security license does not match the appliance license:
The TRITON - Email Security version must be higher than the appliance:
Reset the appliance license on the appliance interface, then set license to the appliance. Update the TRITON - Email Security or the appliance to the appropriate version.
The TRITON - Email Security device ID does not match the appliance device ID:
168 ESGA v7.6 201
This error means the appliance is not the original appliance. Change the IP address of the appliance and add it to the ESG appliance. Then you can remove the original appliance from the ESG appliance.
Troubleshooting Guide
Slide 10:8
Log Server & Log Database Installation ESG Log Server s two installation modes:
Windows authentication mode In this mode, you must to the window with the existing domain and can access the SQL Server without . The windows must have the permissions to access the following system resources:
ODBC resource (create/edit ODBC data source)
Service control manager (create/delete windows service)
Database authentication mode The database must meet the following permission requirements:
ESG database owner •
• •
The database owner must have the hip in one of the following roles in the MS SQL database: “SQLAgent” role “SQLAgentReader” role “SQLAgentOperator” role The database owner must have the hip in “db_datareader” role The database owner must be a member of the “dbcreator fixed server” role
ESGA v7.6 201 169
Troubleshooting Guide
•
170 ESGA v7.6 201
The installation must have the “sys” permissions, because ESG stored procedures will access (start/stop operation) the SQL Server Agent service.
Troubleshooting Guide
Slide 10:9
Log Server & Log Database Deployment There are at least three deployment modes when you install the TRITON Management Server (EIP Server), Log Server and Log Database: 1. EIP Server + Log Server in one machine (A), Log Database in another dedicated machine (B); 2. EIP Server in one machine (A), Log server + Log Database in another machine (B); 3. All in one machine (A).
We recommend using a dedicated SQL Server to run the ESG off-box products, saying mode 1, thus you can get the best performance. Especially for mode 1, you need to install the SQL Server component Tools on machine (A) in advance. In this way, the Log Server can work in B mode and speed up the log record insertion.
ESGA v7.6 201 171
Troubleshooting Guide
Slide 10:10
Log Server Errors Issue A: What might cause the failure of the Log Server service start?
Failed to establish the connection with the Log Database.
The IP address/hostname of the Log Database may be changed.
Issue B: What might cause the Log Server failed in receiving ESG log records The ESG failed to establish the connection with the Log Server.
Check settings from the TRITON - Email Security: Settings > Report Settings > Log Server.
Check if the firewall is blocking the connection request.
172 ESGA v7.6 201
Troubleshooting Guide
Slide 10:11
Log Database Errors Issue A: What might cause the failure of the Log Database installation?
The Log Database directory must be an existing directory.
The Log Database directory must be writable on the remote SQL Server.
The Log Database must have the permission to run the scripts.
Issue B: What might cause no data in ESG report?
No data is sent to the Log Server.
No ETL job is running on the Log Database. Check job status: a. Connect to the Log Database with the SQL Server Management Studio b. Run the below command: exec usp_esg_get_job_status
Issue C: What might cause the alert “Log database extract, transform, and load process is not functioning properly” occurred?
The modified the Log Database system time after installation. ESGA v7.6 201 173
Troubleshooting Guide
Recreate the background ESG job with the following command: exec usp_create_background_jobs
How can I check job performance?
Run the following command and check the value of timeusage (ms): exec usp_esg_get_job_perf
174 ESGA v7.6 201
Troubleshooting Guide
Slide 10:12
Log Database ETL Failed
Check the error log table: select * from wse_error_logs
Check the job history: select * from msdb.dbo.sysjobhistory his, msdb.dbo.sysjobs jobs
ESGA v7.6 201 175
Troubleshooting Guide
Check the SQL Server the SQL Server management studio:
176 ESGA v7.6 201
Troubleshooting Guide
Slide 10:13
Cluster - Secondary Not Sync The slide lists the possible reasons of the second appliance not sync:
The cluster communication IP address can’t connected between the primary and the secondary nodes.
Set the correct cluster communication IP address for the primary or the secondary nodes.
Check if the primary and the secondary can reach each other on T port 6671.
The appliances within a cluster are not the same V-Series platforms (V10000 G2 or V5000 G2).
Don’t use different V-Series platforms.
Check if the V10000 platforms and ESG versions are the same for all cluster nodes:
Open the file /usr/local/etc/.xml and check the following fields to make sure the platforms are the same: ... ...
Open the file /usr/local/etc/esg.version and check the field major_verion to confirm the ESG versions are the same.
ESGA v7.6 201 177
Troubleshooting Guide
Check the file /tmp/md5sum.txt. This file lists all the synchronized files between the primary and the secondary nodes. The synchronized files are customized in /usr/local/etc/cluster_sync.conf.
178 ESGA v7.6 201
Troubleshooting Guide
Slide 10:14
Cluster - Primary or Secondary Crash Issue A: When the primary ESG has crashed and cannot be recovered, how to re-setup the cluster with the left nodes? 1. In the TRITON - Email Security > Setting > General > Email Appliances page, release all the cluster nodes to standalone mode. 2. Re-setup the cluster.
Issue B: When the secondary ESG has crashed and cannot be recovered, how to remove it from the cluster? In the TRITON - Email Security > Setting > General > Email Appliances page, remove the crashed node from the cluster.
ESGA v7.6 201 179
Troubleshooting Guide
Slide 10:15
Cluster - How to Track Issue A: How to track the cluster negotiation result?
Check the cluster status
to the ESG Shell as the root .
Execute the command show cluster to show the current cluster status Pay more attentions to the following fields: •
•
180 ESGA v7.6 201
Work Mode indicates the ESG work mode - standalone - primary - secondary Work Cur State indicates the cluster negotiation status - 0 negotiation failed - 1 negotiation processing - 2 configuration is out of sync - 3 configuration is synchronized - 4 lost network connection to peer - 5 local version is higher than peer - 6 local version is lower than peer - 7 platform is not matched Check this field for each secondary from the primary.
Troubleshooting Guide
•
Work Cur Mode indicates the cluster negotiation status. The meaning of the status code is the same as above. Check this field from the secondary.
Debug the cluster negotiation and configure the synchronized procedure
to the ESG Shell as the root .
Execute the following commands: a. debug module heartbeat_daemon event all b. debug module config_daemon event all c. debug output cli d. debug start
Check the system logs from TRITON - Email Security interface.
Check the system alerts from TRITON - Email Security interface.
Issue B: How to track the on-box logs? 1. to the ESG Shell. 2. Run the following command: [root@v10000-esg ~]# psql –U postgres 3. Run the following command to connect to the ESG on-box database: \c esgdb 4. Run the following command to list all the tables: \d 5. Run the following command to show the audit log: a. select * from audit_log b. \g 6. Run the following command to show the system log: a.
select * from system_log
b. \g
ESGA v7.6 201 181
Troubleshooting Guide
Slide 10:16
Hybrid Issue A: Check if the sync service is running 1. to the ESG Shell as the root . 2. Run the command ps aux|grep sync_daemon The Console output displays like the follows if the sync service is running: root 2905 0.0 2.6 157692 102220 ? Sl 2010 0:01 /usr/local/sbin/sync_daemon
Issue B: Export ESG sync service debug log 1. to the ESG Shell as the root . 2. Run the following commands: a. bash-3.1# esgshell b. esg1153-esg# debug module syncserver event all c. esg1153-esg# debug output cli d. esg1153-esg# debug start
182 ESGA v7.6 201
Troubleshooting Guide
Issue C: Check if the ESG can access the Hosted sync server 1. to the ESG Shell as the root . 2. Run the command: grep -i "HES_Sync_server" /usr/local/etc/hes_sync.conf 3. The Console output displays like below. This is the HES sync server name. HES_Sync_server = t8-hsync-web.odd.blackspider.com 4. Telnet the sync server on port 443 like the below example: telnet t8-hsync-web.odd.blackspider.com 443 Trying 172.16.167.146... Connected to t8-hsync-web.odd.blackspider.com (172.16.167.146). Escape character is '^]'. The Console output displays successful if the ESG can access the Hosted sync server. If not, configure the firewall settings to allow HTTPS traffic between the ESG and HES servers.
Issue D: Allow the Hosted cluster to access ESG SMTP 25 port During the hybrid configuration, the TRITON - Email Security shows the Hosted cluster IP addresses. Make sure the firewall allows the email traffic from these IP addresses to the ESG on port 25.
ESGA v7.6 201 183
Troubleshooting Guide
Slide 10:17
DLP Registration Failed If the ESG failed in registration to the DSS Manager, check as the follows: 1. The ESG appliance needs a valid and unique FQDN. Run the following command to view FQDN: hostname -f 2. The DSS Manager should be reachable on port 5819, 5821, 8443, 9443 and 80. 3. The information in the manual registration is valid and has the “Manage system module” privilege. 4. If the registration still fails, run the script manually to find the reason: /usr/local/sbin/dlp_manager.sh address name address
DSS Manager IP address
name/
information
184 ESGA v7.6 201
Troubleshooting Guide
Slide 10:18
DLP Policy Deployment Failed If the deployment from the DSS Manager fails, check the follows:
If Eth0 of the ESG is reachable on port 5820, 8888, 8889 and 9080
If the Policy Engine (PE) service is running on the ESG by the PE service status
The warning messages during deployment
The following directory in the TRITON Management Server: C:\Prograam Files(x86)\Websense\Data Security\tomcat\logs\dlp\dlp-all
ESGA v7.6 201 185
Troubleshooting Guide
Slide 10:19
DLP Policy Errors I have added policies and rules in the DSS manager. Why they are not working? If the added policies and rules aren’t working, check the following: 1. If the ESG is ed If not, an alert displays on the TRITON - Email Security dashboard, saying that the ESG is not ed. 2. If the DLP policy has been deployed from the DSS Manager If not, an alert displays on the TRITON - Email Security, saying that the DLP load configuration failed. 3. If the DLP policy on the ESG is in Enforce mode 4. If the rules action on the DSS Manager is Block All; or for the email DLP policy, the action is Quarantine 5. If the rules of the DSS Manager is configured for the right direction: inbound/ Outbound/Internal 6. If the policies are still not working, un and re- the ESG as the next section described
186 ESGA v7.6 201
Troubleshooting Guide
The process to un and re- ESG 1. Un ESG on the TRITON - Email Security. 2. Remove the ESG module on the TRITON - Date Security > System module. 3. Deploy on the DSS manager. 4. ESG on the TRITON - Email Security. 5. Deploy again on the DSS manager.
The message has been quarantined. Why the link to DSS incidence is not there in ESG message log? Mark the Audit incident option in the Action Plan of the rules.
Does the DLP scanning on ESG require DSS manager subscription? Generally, the DLP scanning of ESG requires Data Loss Prevention feature on the ESG subscription, but it does not require the DSS manager subscription. While the questionable image feature does require the specific DSS subscription. If the DSS subscription doesn’t contain the questionable image feature, the feature wouldn’t be working.
If the DSS manager is down, is DLP policy still working on ESG? The DLP policy would be working regardless of the availability of the DSS manager. But if the DSS manager or the connection to the DSS manager is down, the incidents and forensics would be cached locally on ESG.
How to debug the DLP scanning on ESG? DLP scanning can be debugged with following command: 1. to the ESG Shell as the root . 2. Run the following commands to debug the DLP scanning: a. debug module scan_engine event all b. debug output cli c. debug start The DLP scanning result and rules triggered would be shown in xml format. See the debug framework document for more details.
ESGA v7.6 201 187
Troubleshooting Guide
Slide 10:20
Quarantine Location Quarantine Folders All the quarantine relevant folders are located in the directory /var/spool/postfix/qd/. They are either used to store quarantined messages, or to store information and temporary files used by quarantine system. The following table lists details of the quarantine folders. qfolder
Refer to queue folders. All quarantined files are stored under this folder. Each queue has a separate subfolder. For example, the spam queue uses the qfolder/spam folder. If a queue is using a remote storage, the subfolder will be a symbolic link with the same name. For example, if the virus queue is using NFS storage, the qfolder/virus will be a symbolic link to the mountpoint
qfile/qlink
All quarantine messages are stored here temporarily before they are put into the destination queue folders. If there are many files under this queue, probably the quarantine daemon stops processing messages
blacklog
All quarantine requests from the filter module are stored here if the quarantine daemon stops processing messages
188 ESGA v7.6 201
Troubleshooting Guide
defer
All logs of "delayed messages" are queued here before they are processed and inserted into SQL server database
dbsync
If quarantine daemon fails to insert a quarantine to SQL server database (connection problem, execution timeout, execution failed), the log will be saved in this folder
emls
Used to temporarily store quarantined or delayed messages in plain text format (eml). The eml files are generated when you or view the messages through TRITON - Email Security interface. It will be then deleted, so supposedly there should be no files in this folder
delay
If a queue is moving, for example, from the local storage to the remote NFS storage, the incoming emails to this queue will be temporarily saved in this folder because the destination queue folder is unavailable. After the queue moving is done, these mails will be flushed into the associated queue folder
rdelay
If a remote queue is temporarily unavailable, for example, the network connection is down, the incoming emails to this queue will be temporarily saved in this folder. After the queue recovers, these mails will be flushed into the associated queue folder
failconf
Used to store the last configuration of a queue. If you see a file under this folder, the queue is either moved, or failed to move
flusher, dlcache
Not used
Configuration File The configuration file used by the quarantine daemon to connect to the Log Database is /etc/freetds.conf.
ESGA v7.6 201 189
Troubleshooting Guide
Slide 10:21
Quarantine - No Messages in Blocked Messages I’ve sent a spam email, why can’t I see quarantined logs in “Blocked Messages”? 1. to the ESG Shell with the root . 2. Check /var/log/maillog to see if the email is captured by the scan engine. The log should look like the following in normal situations: /usr/local/sbin/filter[10435]: message id 2D55FA50002, filter ASA-Filter, policy Outbound Default, rule AntiSpam, matched! 3. Make sure the quarantine action for spam and virus filters is turned on. 4. Check whether there are files under the /dbsync folder.
Yes. It is likely to be a issue with the Log Database connection. Run the below command to check whether the Log Database connects with the quarantine daemon: netstat –anp | grep quarantine_daemon a. The connection is well. The SQL command for inserting quarantine logs may be failed. Start debugging the quarantine daemon: - Sent another email.
190 ESGA v7.6 201
Troubleshooting Guide
- View the SQL command that is sent to the SQL server. - If the debug information shows that the SQL command failed, copy this SQL string and execute it in the SQL Server Management Studio manually. b. No connection. Check the Log Database configuration and the network connection.
No. to the SQL Server Management Studio to check whether the logs are inserted into esglogdb76.dbo.esg_detail_quarantine_message and esglogdb76.dbo.esg_detail_quarantine_recipient. a. If yes, it is the TRITON - Email Security interface problem. Check the tomcat ogs on TRITON - Email Security for details. b. If not, the quarantine daemon may fail to process messages. Check whether there is message file under the /qfile/qlink folder; Check whether there are core files under the /var/cores.
Why are the numbers of messages in queue list page and in message list page different? 1. to the ESG Shell as the root . 2. Taking the spam queue as an example, run the following command to calculate the total number of messages: find /var/spool/postfix/qd/spam –type f | wc -l 3. Compare the number with the queue list number and the message list number If the number is the same as the queue list, but different with the message list, some logs are probably not inserted into the Log Database successfully. Check the dbsync folder to find these logs.
When I moved a queue from the local storage to NFS, TRITON - Email Security interface shows “Data transfer ongoing” for a very long time. What happened? If the ESG works in a cluster mode, perform as the follows: 1. to the ESG Shell as the root . 2. Run the command below to see the queue moving status of each cluster node: show queuemovestatus 3. Debug the quarantine daemon to check whether the file copy is ongoing. If you don’t see any output of the file copy, cancel the data transfer from TRITON - Email Security interface and try again.
ESGA v7.6 201 191
Troubleshooting Guide
Slide 10:22
System Counters How to show counter in ESG? 1. to the ESG Shell as the root . 2. Run the following command: [root@localhost filter]# counter
What is the difference between ESG persistent counters and ESG nonpersistent counters? We can ignore the persistent counters. They have the same values as the non-persistent counters.
What does each counter mean? Many counters are for the debugging purpose (dbg_* counters).
192 ESGA v7.6 201
Troubleshooting Guide
The below table lists the counters and the descriptions as the reference. Counter Name
Description
received_msg
Messages number received externally. The messages include all directions(inbound, outbound, internal, and open relay)
processed_msg
Messages which have been processed (policy has been applied on the message)
Delivered_msg
Message which have been delivered
Dropped_msg
Message which have been dropped
Quarantined_msg
Message which have been quarantined.
In_queue_msg
Messages waiting to be processed or waiting to be delivered. A large value of in_queue_msg means that the system is under stress.
In_deferred_queue _msg
Messages in deferred queue
Queue_total/used/ freed_space
The 3 values indicate the disk space which can be used for message processing
Inb_conn
Total smtp connections
Inb_reputation_rec
Connections dropped by reputation
Inb_rbl_rec
Connections dropped by rbl
Inb_rdns_rec
Connections dropped by rdns check
Inb_spf_rec
Connections dropped by spf
Av_hit
Number of virus-infected messages
Av_clean
Number of virus-clean messages
Av_err
Number of messages which have error in av-scanning
As_hit
Number of spam messages
As_clean
Number of not-spam messages
As_err
Number of messages which have error in as-scanning
Dlp_hit
Number of messages which have matched dlp policy
Dlp_clean
Number of messages which have not matched dlp policy
Dlp_err
Number of messages which have error in dlp-scanning
Filter_exception
Number of exception messages\
Encrypt
Number of messages which need encryption
Encrypt_success
Messages which succeed in encryption
Encrypt_fail
Messages which fail encryption
Decrypt
Number of messages which need decryption
Decrypt_success
Messages which succeed in decryption
Decrypt_fail
Messages which fail decryption
ESGA v7.6 201 193
Troubleshooting Guide
Counter Name
Description
Hes_dkim_
Messages from HES which need dkim verification
Hes_dkim_
Messages which have ed dkim verification
Relations within different counters If a message with multiple recipients is applied with default policy and no exception message happens, the below formula should be true: Received_msg = processed_msg = delivered_msg + quarantined_msg + filter_exception + bounced_msg + in_deferred_queue_msg Dropped_msg = quarantined_msg
Why the above formula does not match? The possible reasons could be:
The message has multiple recipients; different recipients have different policies; e.g. some recipients are quarantined, and some recipients are delivered. In this case, the counters will not match.
Exceptions Some counters may be increased twice for a single mail.
Manual operations For example, reprocess some messages from TRITON - Email Security > Main > Message Management.
Messages are always coming into the ESG system. Counters will not match until finishing all the messages process.
194 ESGA v7.6 201
Troubleshooting Guide
Appendix
ESG Shell Q&A Debug tool do not show log. Why?
The debug daemon or modules to be debugged may not start.
Run the follow command: svstat /service/debugd/
Run the below command to start it: svc -du /service/ debugd /
Debug setting may be output to the wrong place.
Run the blow command to view the debug setting: debug show
Some services don’t function well.
Restart the ESG service to fix it: /usr/local/sbin/esg_restart.sh
Several configuration values in the share memory are none or wrong. Why? Some services may not start. 1. Check whether the core files are created in the /var/cores/ directory. 2. Run the below command to start all services: /usr/local/sbin/esg_restart.sh
The set/reset command do not function or report errors. Why? The errors should be treated case by case. The following commands may help:
Run the set/reset command again. The previous error maybe due to the related service busy.
Check the file /var/log/confd.log to find some clue.
Restart the ESG services.
System Diagnosis Q&A Failed to generate the summary file. Why? The summary file contains the configuration files and logs of all the modules, therefore any errors from any module may cause the error.
ESGA v7.6 201 195
Troubleshooting Guide
The disk space and the network issues could be the most probable reasons.
Failed to open the configuration file. Why? The file is encrypted with the decode key “f~S#-/5A”. Changes to the key may lead to fail to decode the file. If the ESG file in the whole package is damaged, check the hard disk usage of the ESG module.
Health Daemon Service didn’t start
Service restart by health daemon
Description
Service which configured in /usr/local/etc/health.conf is not started. Health daemon will start it by the configured command
Trigger condition
Service which configured in /usr/local/etc/health.conf is not started. Health daemon will start it by the configured command
Message
Service %s down, start it by health_daemon
Alert
ALERT_TYPE_SERVICE_DOWN The {%DyInfo} service is not available
Action
Start service by the configured command
Recovery
No need
Service restarted by manually
Description
Service which configured in /usr/local/etc/health.conf is not started. Health daemon will generate warning message to remind to start the service manually
Trigger condition
Service which configured in /usr/local/etc/health.conf is not started. Health daemon will start it by the configured command
Message
Service %s down. Restart by itself
Alert
ALERT_TYPE_SERVICE_DOWN The {%DyInfo} service is not available
Action
should start the service manually
Recovery
No need
196 ESGA v7.6 201
Troubleshooting Guide
Service start repeatedly Description
Service restart and terminated abnormal repeatedly and exceeds a threshold, health daemon will stop it
Trigger condition
Service restart and terminated exceeds 3 times
Message
Restart service:name=%s pid=%d,mem=%f,max=%d,max_ratio=%f,cmd=%s service memory exceed the configured limitation, restart it by the configured command
Alert
No
Action
Health daemon stop the service. should check system. The service can be restarted by command "svc -u servicename" manually
Recovery
No need
Service memory leak Description
Service memory keeps increasing and exceeds a configured threshold (/usr/locat/etc/health.conf)
Trigger condition
If the "memory increase threshold" which configured by /usr/local/ etc/health.conf is not 0, the trigger condition is: when the memory increase step is bigger the increase threshold and the memory is bigger than the max limitation value. If the "memory increase threshold" which configured by /usr/local/ etc/health.conf is 0, the trigger condition is: when the memory is bigger than the max limitation value
Message
Kill service:%s pid=%d,mem_inc_thr=%d, mem_last_val=%d,mem=%f,max=%d,max_ratio=%f cmd is [%s] service memory exceed the configured limitation, killed by health daemon
Alert
No
Action
Health daemon kill the service and generate warning messages in the /var/log/messages
Recovery
No need
Service file handler leak Description
Service generate too many pipe file handlers. It may be a file handler leak. Health daemon will kill it and generate warning messages
Trigger condition
Service generate too many pipe file handlers
Message
Process name [%s] pid [%u] may be fd leak, fd number[%u]. Please check it Service generate too many pipe file handlers. it may be a file handler leak. Killed by health daemon
Alert
No
ESGA v7.6 201 197
Troubleshooting Guide
Action
Health daemon kill the service and generate warning messages in the /var/log/messages
Recovery
No need
System disk space too small
/dev/xvda1 free space too small
Description
/dev/xvda1 free space is too little. Health daemon will set disk stress flag. Mta will not be process email
Trigger condition
/dev/xvda1 free space is little then 4G
Message
/dev/xvda1 free space is little then 4G, set disk_stress
Alert
ALERT_TYPE_DISK_STRESS Adequate amount of free disk space is not available
Action
Health daemon set the disk stress flag, mta will not process emails
Recovery
Release more disk space. Disk free space is bigger than 8G
/dev/xvda2 free space too small
Description
/dev/xvda2 free space is too little. Health daemon will set disk stress flag. Mta will not be process email
Trigger condition
/dev/xvda2 free space is little then 4G
Message
/dev/xvda2 free space is little then 4G, set disk_stress
Alert
ALERT_TYPE_DISK_STRESS Adequate amount of free disk space is not available
Action
Health daemon set the disk stress flag, mta will not process emails
Recovery
Release more disk space. Disk free space is bigger than 8G
/dev/xvda1 free inodes too small
Description
/dev/xvda1 free inodes is too small. Health daemon will set disk stress flag. Mta will not be process email
Trigger condition
/dev/xvda1 free inodes is little then 1024 * 200
Message
/dev/xvda1 free inode is little then %d, set disk_stress
Alert
ALERT_TYPE_DISK_STRESS Adequate amount of free disk space is not available
Action
Health daemon set the disk stress flag, mta will not process emails
Recovery
Release more disk space. Disk free space is bigger than 1024 * 300
198 ESGA v7.6 201
Troubleshooting Guide
/dev/xvda2 free inodes too small
Description
/dev/xvda2 free inodes is too small. Health daemon will set disk stress flag. Mta will not be process email
Trigger condition
/dev/xvda2 free inodes is little then 1024 * 200
Message
/dev/xvda2 free inode is little then %d, set disk_stress
Alert
ALERT_TYPE_DISK_STRESS Adequate amount of free disk space is not available
Action
Health daemon set the disk stress flag, mta will not process emails
Recovery
Release more disk space. Disk free space is bigger than 1024 * 300
ESGA v7.6 201 199
Troubleshooting Guide
Log stress monitor Description
Log daemon can not send data to data base successfully. /log-defer has files older than 3 days. Health daemon will set "log stress flag" and generate warning messages in /var/log/messages. Mta will not process emails
Trigger condition
/log-defer has files older than 3 days
Message
Log has files older than 3 days, set log stress
Alert
ALERT_TYPE_LOG_STRESS Email Security log queues contain cache files older than 3 days
Action
Health daemon set the log stress flag, mta will not process emails
Recovery
Check the connection to data base. Clear all files (/log-defer) which older than 3 days
Quarantine monitor Description
Quarantine daemon can not send data to data base successfully. Health daemon will set "log stress flag" and generate warning messages in /var/log/messages. Mta will not process emails
Trigger condition
/var/spool/postfix/qd/dbsync has files older than 3 days and recent successfully sending data time is older than 3 days or /var/spool/postfix/qd/dbsync has files more than 100000
Message
DBsync files and sync time is older than 3 days, or files number is bigger than %d, set quarantine stress
Alert
ALERT_TYPE_QUARANT_STRESS Email Security quarantine queue contains cache files older than 3 days or too many files
Action
Health daemon set the quarantine stress flag, mta will not process emails
Recovery
Check the connection to data base. Clear all files (/var/spool/ postfix/qd/dbsync)
Filter thread monitor Description
Filter update thread or scanning thread take too long running time. Health daemon will kill filter and generate warning message in / var/log/messages
Trigger condition
Filter update thread or scanning thread running time is more than 20 minute
Message
Filter update running time too long. Current system time is [%lu], filter update_start time is [%lu] Filter update thread running time is longer than 20 minute. filter service was killed by health daemon
Alert
No
200 ESGA v7.6 201
Troubleshooting Guide
Action
Health daemon kill filter and generate warning message in /var/log/ messages
Recovery
No need
Share memory dead lock Description
Service share memory dead lock. Health daemon kill the service
Trigger condition
Same service and same function and same line hold the same share memory lock for too long time
Message
Share memory [%s] is dead locked by function [%s] line [%u] service hold share memory for too long time, it may be a dead lock. Killed by health daemon
Alert
No
Action
Health daemon kill service and generate warning message
Recovery
No need
ESGA v7.6 201 201
Troubleshooting Guide
202 ESGA v7.6 201
Techni calEnabl ementPr ogr am
E S GAv 7. 6Cour s e201 S t udentGui de Rev . 691267
©2011 Websense, Inc. All rights reserved.
Websense and the Websense logo are ed trademarks of Websense, Inc. in the United States and other countries. TRITON is a trademark of Websense, Inc. in the United States and other countries. All other trademarks are properties of their respective owners.
Table of Contents Websense Email Security Gateway Initial Setup. . . . . . . . . . . . . . . 7 ESG Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 ESG Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Personal Email Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 Policy Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Advanced Encryption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Clustering Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 Archive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 ESG Log and Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 Troubleshooting Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
ESGA v7.6 201 5
6 ESGA v7.6 201
1
Websense Email Security Gateway Initial Setup
This chapter focuses on the initial setup and configuration of the Email Security Gateway (ESG) module in a Websense V-Series appliance. Prior to the physical and logical deployment of the appliance within the network, you need to understand the fundamental concepts behind email security. Much of the initial setup has to do with your email domains and ensuring only authorized s can send and receive email through the ESG appliance. During the initial setup of ESG, the wizard requires you to define: 1. Email server IP address: The IP address of the mailboxes server, which stores, receives and sends email to and from local s. You can add additional email server after the initial setup. 2. Protected domain address: This is the email domain where the mailboxes reside. For example, if the email is [email protected] then the protected domain is websense.com. You can add additional protected domain after the initial setup. 3. Trusted IP address: In this field you enter the IP address of the hosts which are allowed to send email (inbound connection) through ESG and not be subject to IP filtering rules. 4. Data security server IP address: In this field you enter the IP address of the DSS server so you can take advantage of the DLP integration offered by ESG. 5. Log server IP address: In order for ESG to be able to generate reports, you need to configure it to talk to an off-box log server. The log server should be installed before you commence the ESG initial setup. 6. Notification email address: A valid email address is required which will serve as the default address for all notifications from ESG. This email address should be a shared mailbox that is available to multiple level individuals. This ensures maximum ability for your environment.
ESGA v7.6 201 7
Websense Email Security Gateway Initial Setup
Slide 1:1
ESG Initial Setup Before you begin the configuration and installation of the ESG appliance you need to gather some key information needed during the initial setup process.
Email server IP address
Protected domain
Trusted IP address
IP address and information for the DSS server
IP address of the log server
Shared mailbox (or ) where ESG can send all email alerts and notifications
Steps of ESG Initial Setup 1. Install the Websense V-Series appliance, including setting up the rack, connecting network cables, connecting the input device for command line interface, etc. 2. Run the first boot wizard through command line interface. Security mode and Appliance Controller interface (C) are configured during this step. a. Security mode - the appliance performs email security protection (ESG mode), or web security protection (WSG mode), or both (ESG+WSG). We’ll talk about security mode with more details later.
8 ESGA v7.6 201
Websense Email Security Gateway Initial Setup
b. Appliance Controller interface - IP address of Appliance Manager logon portal. Note This is NOT ESG configuration wizard. 3. Access Appliance Manager via browser to configure ESG DomU IP address and other information. 4. Install TRITON Unified Security Center, also called TRITON console, including Email Security, Data Security, etc; install log database (if not pre-installed) and log server. 5. to TRITON console, switch to “Email Security” tab, then run the ESG configuration wizard. Email server, protected domain, trusted IP, IP addresses of DSS server, log server, and shared mailbox for notifications are set during this step.
ESGA v7.6 201 9
Websense Email Security Gateway Initial Setup
Slide 1:2
Protected Domains The concept of a protected domain has a slightly different meaning when you refer to inbound messages and outbound messages. An inbound message is defined as a message sent from outside your company domain to an within your company domain. The filtering of inbound messages is fairly simple. The ESG reads the recipient address and if it does not match any of the configured protected domains, the email is immediately rejected. For example, if an ESG appliance configured with websense.com and surfcontrol.com as protected domains receives an message sent from [email protected] does not match either websense.com or surfcontrol.com. In ESG, there are four directions defined:
Internal: From protected domain to protected domain
Inbound: From other domain to protected domain
Outbound: From protected domain to other domain
Open Relay: From other domain to other domain
10 ESGA v7.6 201
Websense Email Security Gateway Initial Setup
Slide 1:3
Email Relay During the ESG configuration wizard, you have entered some key data, which makes the ESG a closed relay. Only the following email messages are routed (relayed):
From any sender (coming from any IP) to the protected domain
From permitted IP (typically the internal email servers) and from a sender in the protected domain to any recipient
The idea is to ensure the ESG only routes valid email and does not become an open relay. Hackers and spammers often use open relays and misconfigured email servers to launch an attack or send out massive amounts of emails, in relative security. All traffic points back to the compromised email server.
ESGA v7.6 201 11
Websense Email Security Gateway Initial Setup
Slide 1:4
Required Components Two domains inside ESG appliance Any appliance running ESG has at least two domains configured:
ESG domain: contains an SSL interface for communicating with TRITON Management Server and the Personal Email Manager (PEM) module for end s to manage their isolated email according to the policies defined by s. Both TRITON console and the PEM module store data on Log and Report Server.
Appliance Manager: provides the facilities for s to connect and configure the appliance.
We’ll bring more details about the ESG domains in the ESG Architecture chapter.
Two types of s There are two types of s who can access the various modules inside the two domains:
can connect and configure the appliance directly
can’t access ESG domain directly, but through TRITON console running on an off-box server to set ESG configuration and policies
12 ESGA v7.6 201
Websense Email Security Gateway Initial Setup
End with a valid mailbox
can access the PEM module in ESG domain directly without TRITON console
Important There is no direct istrative access to ESG module. s have to install TRITON console off-box to be able to configure ESG domain.
Let’s assume the following scenario:
ESG domain has IP address 172.16.90.100
Appliance Manager has IP address 172.16.90.101
TRITON Management Server has IP address 172.16.90.200
The following statements are correct:
The logs to the TRITON manager, on IP address 172.16.90.200, to configure the ESG domain. You need to point your browser to the IP address 172.16.90.200
The TRITON Management Server can communicate directly with the ESG module at IP address 172.16.90.100
The can configure the appliance settings by pointing the browser the IP address 172.16.90.101
An end can access the PEM module by pointing the browser directly to the IP address of the ESG module at 172.16.90.100
Tip Any appliance running ESG needs at least two interfaces with two separate IP addresses: one bound to ESG domain, the other to Appliance Manager.
ESGA v7.6 201 13
Websense Email Security Gateway Initial Setup
Slide 1:5
Network Interfaces The V5000 appliance has 4 NIC and the V10000 has 6 NIC. The Appliance Manager domain always binds to the C interface.
On the V10000
N interface is used for the Network Agent domain.
E1 and E2 interfaces are bound to the ESG domain. You can use one or both interfaces depending on your environment. You could only use E1; and you could use E1 as internal interface and E2 as external interface too.
P1 and P2 are bound to the WSG domain. Tip On a V10000 you can run both WSG and ESG at the same time.
On the V5000 (running ESG only)
P1 and P2 interface are bound to the ESG domain.
C interface is bound to the Appliance Manager.
14 ESGA v7.6 201
Websense Email Security Gateway Initial Setup
On the V5000 (running WSG only)
N interface is bound to the Network Agent domain.
C interface is bound to the Appliance Manager.
P1 and P2 are bound to the WSG domain. Tip On a V5000 you can run either WSG or ESG.
ESGA v7.6 201 15
Websense Email Security Gateway Initial Setup
16 ESGA v7.6 201
2
ESG Deployment
ESG is typically deployed in DMZ zone. While other deployment options are possible, this is the most common and the one that Websense recommends. By design, ESG can process both incoming and outgoing messages. Some companies may decide to process only incoming messages; however it is most common to process messages in both directions. When processing inbound messages, the ESG appliance receives the email from the Internet (or possibly the Intranet, for large scale complex deployments), scans the messages, and forwards them to company internal mail server. The incoming messages can be directed to the ESG appliances in a variety of ways, depending on the customer preferences and final objectives:
MX Record re-configuration
Firewall routing policies
In order to process outbound messages from inside the company, there are two main possible configuration options. The options mostly depend on the mail client settings:
Clients not using SMTP - the mail configures the SMTP server to relay all outbound traffic to the ESG appliance. This is the most common configuration.
Clients with configured SMTP server - the mail can configure all clients to point to ESG directly. Then the ESG routes the messages as necessary. This setting, while not very common enables filtering of all internal email messages. Typically, it is easier to assign the IP address of the SMTP server to the ESG appliance and then assign a new IP address to the SMTP server.
Warning All configurations require firewall re-configuration.
ESGA v7.6 201 17
ESG Deployment
Slide 2:1
ESG Deployment The simplest deployment for an email appliance is to put in on premise and configure the firewalls or the DNS (or both) to direct incoming mail to the newly deployed appliance. You then configure the existing email server (or the client, in rare cases) to send all the outbound email through the ESG appliance. In addition to the on premises deployment, you can integrate ESG with the cloud solution and you can pre-filter all incoming email in the cloud; this reduces significantly the amount of email that the ESG has to process on premises. To scale to any size environment, the ESG s clustering. This advanced feature also ensures that email filtering is always up and running in your network.
18 ESGA v7.6 201
ESG Deployment
Slide 2:2
Standard Network Deployment This network diagram represents a very simple network configuration. Regardless of the complexity of the actual network where you plan to deploy ESG, you can always reduce it, at least conceptually, to the one shown above. Inbound email messages flow according to the following steps: 1. The email messages arrive from the Internet to the perimeter firewall. Permitted traffic is allowed through. 2. The firewall permits the messages to reach the V10000 appliance in the DMZ. Tip Typically you should install the V-Series appliances running ESG in the DMZ. On appliances configured in dual mode, you can have the E2 and P2 interface in the DMZ and the E1 and P1 interfaces in the LAN. 3. The ESG processes the messages and matches them against the running policies. Permitted messages flow to the email server.
ESGA v7.6 201 19
ESG Deployment
4. The email server receives the messages which ed the ESG filtering. This email server can have actual mailbox on it or be a bridgehead server, which distributes the messages to other mail servers based on internal routing policies. The messages are now available for pick/delivery by the end s. For outbound messages, you can reverse the process. The s send the messages to the mail server that is configured to forward all outbound messages to the ESG appliance. All messages that the filtering policies on the ESG, flow from the VSeries appliance, through the firewall to the Internet, where they reach their final destination.
20 ESGA v7.6 201
ESG Deployment
Slide 2:3
Standard Network Deployment with DNS This network diagram is very similar to the previous one. The added steps focus on the role of the DNS lookup by a sending email server. Let’s assume that a Gmail wants to send a message to a Websense employee. The process of delivering email messages consists of the following steps: 1. The Gmail email server queries the DNS for the Websense MX record. The MX record contains the information about the first hop where to send the email bound for the websense.com domain. Tip When sending an email, the transmitting server only learns about the first relay host, which processes email for the recipient. There can be any number of relay hosts between sender and recipient. Each host only knows about the next one. None has the fully visibility of the source-todestination path. The DNS server responds with the information about the next hop where to send the messages for the Websense domain. The reply may contain one or more
ESGA v7.6 201 21
ESG Deployment
values. The sending server should always pick the one with lowest preference value. When more entries have the same preference value the sending server should process them in the order received, till find the one responding. 2. The email server connects to the next hop in the chain. This can be: - A server in the cloud - A perimeter firewall - An ESG appliance In the this example, the MX record returns the IP address of one of the routable interfaces on the firewall. 3. The messages arrive to the premise firewall. Permitted traffic is allowed through. 4. The firewall permits the messages to reach the V10000 appliance in the DMZ. 5. The ESG processes the messages and matches them against the running policies. Permitted messages flow to the email server. 6. The email server received the messages which ed the ESG filtering. This email server can have actual mailbox on it or be a bridgehead server, which distributes the messages to other mail servers based on internal routing policies. The messages are now available for pick/delivery by the end s.
22 ESGA v7.6 201
ESG Deployment
Slide 2:4
ESG Anywhere Deployment In this scenario, the customer has enabled the Hybrid filtering option. All incoming email is pre-filtered by the cloud service. This deployment scenario dramatically reduces the amount of inbound messages that the ESG on premises has to process. Considering that over 80% of all incoming messages (by number not size) is spam, using the cloud service is a very efficient and effective approach. For incoming messages, the flow is as follows: 1. The sender email server queries the DNS for the recipient MX record. The DNS server responds with the information about the next hop, in this case the cloud service. 2. The email server connects to the cloud service. 3. The Hybrid service sends the messages that ed the filtering policies, to the next hop in the chain. 4. The messages arrive to the premise firewall. Permitted traffic is allowed through. 5. The firewall permits the messages to reach the V10000 appliance in the DMZ. 6. The ESG processes the messages and matches them against the running policies. Permitted messages flow to the email server. 7. The email server received the messages which ed the ESG filtering. The messages are now available for pick/delivery by the end s.
ESGA v7.6 201 23
ESG Deployment
Slide 2:5
ESG Components This slide represents a V10000 appliance installed in dual mode; both WSG and ESG domains are running. The various components operate as follows: 1.
s can access Appliance Manager by pointing the browser to the IP address of the C interface on the appliance.
s can configure the WSG domain through the same IP address (different port).
s need to TRITON console running on an off-box server to configure the ESG domain. The ESG domain communicates with TRITON console through SSL interface.
2. TRITON Management Server
TRITON Management Server enables s to manage all of the domains on the V10000 appliance: ESG, WSG, and Appliance Manager.
TRITON Management Server reads/writes SQL Database Server to perform log and report functionalities.
3. ESG domain
24 ESGA v7.6 201
ESG Deployment
ESG domain sends and receives email messages on the E1 and E2 (if configured) interfaces. You can use, in most cases the E1 interface for both sending and receiving email. Alternatively, you can configure E1 to send and receive on the LAN side and E2 to send and receive on the DMZ side.
ESG domain stores logs and statistics into SQL Database Server in order to generate reports and dashboard charts.
ESGA v7.6 201 25
ESG Deployment
Slide 2:6
Clustering Deployment with Load Balancer This network deployment shows load balancer with three ESG appliances connected to it. The load balancer redirects mail traffic to different ESGs with certain traffic balancing algorithm. The balancing algorithm is part of the load balancer configuration and not ESG. The flow of traffic is as follows: 1. The firewall receives the email from the Internet. 2. If the messages are permitted, the firewall forwards them to the IP address of the load balancer. 3. The load balancer, using the specified algorithm, decides to which of the ESG appliance send the mail message. 4. The ESG, which receives the email from the load balancer, processes the message. formation is stored about the message. The permitted message is forwarded to the email server. 5. The email server receives the messages which ed the ESG filtering. The messages are now available for pick/delivery by the end s.
Clustering details Each ESG in a cluster has one of three roles:
26 ESGA v7.6 201
ESG Deployment
Primary
The primary node when working in cluster mode. Every cluster can have only 1 primary ESG. The primary is the only node in cluster to be configured via TRITON console.
s configure the policies on the primary node through TRITON console. The configuration is then synchronized to all the secondary node.
Secondary
The secondary node when working in cluster mode.
Every cluster can have 1 to 7 secondary ESGs.
The secondary node can not be configured via TRITON console directly.
The secondary nodes synchronize the configuration from the primary node.
The status and logs are synchronized time to time as well.
The secondary nodes and the primary process mail traffic based on the same policies.
Standalone
Any appliance that is not set as primary and not negotiated to be a secondary stays in the role of standalone.
Important One cluster has a maximum of 8 nodes. All nodes of cluster must be the same appliance type: you cannot mix V10000 with V5000 in the same cluster. All nodes of cluster should have same applications deployed on the appliance: all appliance are either single mode (ESG only) or dual mode (ESG and WSG). Tip When cluster negotiation does not succeed, the primary node will not become standalone automatically. The will need to change this manually. Important When you have more than one ESG appliances, clustering is not a requirement. You can have all ESG appliances be in standalone mode; however, you will have to configure each ESG individually. This is not recommended and leads easily to inconsistent policy settings.
ESGA v7.6 201 27
ESG Deployment
Slide 2:7
Clustering Deployment with MX Record This scenario does not have a visible load balancer deployed. It loads and balances email traffic on multiple ESG appliances using MX record. Let’s take the diagram as an example, in which you have three ESG appliances in your cluster. You can re-configure your DNS as one of the following:
Configure the MX record in DNS to return three routable IP addresses which bind to the external interface of the firewall. Each record should have the same preference value, so that the traffic is evenly distributed on the basis of number of incoming connection requests. The firewall maps the external IP addresses to the internal one of the corresponding E1 interfaces on the ESG appliances (1-to-1 NAT)
Configure the MX record in DNS to return three routable IP addresses which bind to each of the E2 interfaces of the ESG. The E2 interfaces are in the DMZ. The E1 interfaces are on the LAN side.
The flow of traffic is as follows 1. The sending email server queries the DNS MX record for the destination domain.
The DNS returns three IP addresses with the same preference value.
The order of the IP addresses are set to round robin mode.
28 ESGA v7.6 201
ESG Deployment
The email server connects to the first IP address in the list (the other addresses are used if the first one does not respond).
The DNS returns the information about the next hop as well.
2. The email server connects to the next hop in the chain. 3. The messages arrive to the premise firewall. Permitted traffic is allowed through. 4. The firewall receives the incoming SMTP request and es it to the corresponding ESG appliance (using one of the two techniques listed above). 5. The layer 2 switch receives the connection request for the address of the selected appliance. This appliance receives and processes the email message. If the policy filter allows the email through, the appliance sends the email to the email server. 6. The email server receives the email that has ed filtering, which is now available for pickup or delivery by the end use.
ESGA v7.6 201 29
ESG Deployment
Slide 2:8
Clustering Operating Mode The only operating mode for the ESG cluster is Active/Active. All machines in the cluster can process email at any time. The cluster is considered active and available if at least one appliance is functional. When the primary is down, all secondary nodes will keep working and filtering as secondaries; no one will switch to the primary automatically; the primary needs to be restored manually. Furthermore, you can NOT modify the policies because the primary is the only node through which policies are configured and it is not available at the moment. If you need to make emergency changes to the policies, you can either re-configure the cluster to pick up a secondary acting as the new primary via TRITON console, or break the cluster and turn the secondaries to standalone machines. As the configuration is synchronized among clusters nodes, the new cluster will work in the same manner. The only lost is the email in the primary message queue, which can not be recovered. We will cover clustering with more details in the later chapter.
30 ESGA v7.6 201
ESG Deployment
Slide 2:9
Firewall Configuration You need to configure your firewall as the slides on this and next page show.
Port 25: permit incoming and outgoing SMTP traffic on 25 • •
On-box message processor listens on 25 to receive messages On-box message processor sends messages to 25 Note For Hybrid Service, the route of your email may occasionally be altered within the Hosted Cluster. To enable this without requiring you to make further changes, you must allow SMTP requests from all the IP ranges listed on the firewall page to port 25.
Port 80: permit outgoing traffic on 80
ESGA v7.6 201 31
ESG Deployment
•
On-box update module and module connect to 80 listened by offbox DDS Server Tip DDS, developed by Websense, performs utilized able databases for various Websense products. The databases include URL filtering database, real time update database, real time security scanning database, etc. We’ll talk about DDS again in ESG Architecture chapter.
•
Port 443: permit outgoing traffic on 443 •
On-box log, quarantine, PEM, and configuration components connect to 1443 listened by off-box MSSQL database
Port 2525: permit incoming traffic on 2525 •
On-box authentication component connects to 389 and 636 listened by offbox LDAP Server. 636 is the secure LDAP port.
Port 1443: permit outgoing traffic on 1443 •
On-box hybrid component connects to 443 listened by Hybrid Service
Port 389 and 636: permit outgoing traffic on 389 and 636 •
On-box DLP component connects to 80 listened by off-box TRITON DSS Manager
On-box message processor listens on 2525 to receive encrypted messages released by TRITON DSS Manager
Port 6671: permit incoming and outgoing traffic on 6671 • •
32 ESGA v7.6 201
On-box SSL Proxy component listens on 6671 to communicate with offbox TRITON Unified Security Center On-box PEM component connects to 6671 listened by SSL Proxy component on another appliance
ESG Deployment
Slide 2:10
Firewall Configuration
Port 6643: permit incoming traffic on 6643 •
Port 9443: permit outgoing traffic on 9443 • •
On-box DLP component listens on these ports
Port 5819, 5821, 8443, 56992, 18404: permit outgoing traffic •
On-box PEM component listens on 9449 for end s to manage their personal messages
Port 5820, 8888, 8889, 9080: permit incoming traffic •
Off-box TRITON Unified Security Center listens on 9443 On-box DLP component connects to 9443 listened by off-box TRITON DSS Manager
Port 9449: permit incoming traffic on 9449 •
On-box PEM Load Balance component listens on 6643 to dispatch PEM traffic
On-box DLP component connects to these ports listened by off-box TRITON DSS Manager
Port 50800: permit outgoing traffic on 50800 •
Off-box Log Server listens on 50800
ESGA v7.6 201 33
ESG Deployment
Slide 2:11
Sample Deployment The slides shows two of our sample deployments: A. ESG with Hybrid service B. ESG without Hybrid service
34 ESGA v7.6 201
3
ESG Architecture
ESG is optimized to handle large volumes of email messages. It also integrates seamlessly with the DLP engine with Websense Web Filter. The TRITON Console can manage all the components. The internal architecture of ESG is very flexible to achieve this level of scalability and integration. This chapter focuses on the differences between the V10000 and V5000 appliances relative to hardware configuration and software installation.
V10000:
ESG and WSG (dual mode)
ESG only (single mode)
WSG only (single mode) Tip The hardware resources on the V10000 are allocated differently depending on which security mode you select.
V5000:
ESG only (single mode)
WSG only (single mode)
ESGA v7.6 201 35
ESG Architecture
Slide 3:1
Architecture and Message Flow The ESG hardware settings (RAM, HDD, and U) can vary depending on whether it is running on a V10000 in dual mode, a V10000 in single mode, or a V5000. Consequently, the amount of traffic it can process depends on these parameters. One of the strong points of ESG is its natural integration with the DLP module. This integration is available for all installation options. You can also integrate ESG with the Web Filter which requires dual mode (V10000 only) or an external server running the Web Filter off-box. This chapter covers the actual message processing or the lifecycle of a message within the ESG module. Understanding these details enables you to better configure and position the ESG solution.
36 ESGA v7.6 201
ESG Architecture
Slide 3:2
Appliance Modules Both V10000 and V5000 run the Appliance Manager module, which is the domain to configure hardware settings for appliances. The difference between V10000 and V5000 is that a V10000 appliance can run ESG and WSG modules at the same time, while a V5000 appliance can run either ESG or WSG. The WSG module consists of three sub-modules: 1. Websense Content Gateway (WCG): the web proxy component of WSG. 2. Web Security Suite (WSS): the content filtering component of WSG, also called Web Security Enterprise (WSE). 3. Network Agent (NA): the component used to monitor and control protocols other than HTTP, HTTPS, and FTP. Except for the Appliance Manager module runs on Dom0 in V-Series appliance, all other modules run on DomU.
ESGA v7.6 201 37
ESG Architecture
Slide 3:3
U Allocation A V10000 G2 appliance can have three different configuration options when it has the ESG module running: 1. ESG only 2. ESG plus WSG without Network Agent 3. ESG plus WSG with Network Agent running In option 1, you have 12 Us dedicated to ESG. In options 2 and 3 you are limited to four Us for ESG as the other eight are allocated to WSG. Depending on the volume of web and email traffic, dual mode may or may not be the preferred option. From a U standpoint, having Network Agent running or not has no impact on the ESG module; it only affects the WSG module.
38 ESGA v7.6 201
ESG Architecture
Slide 3:4
Memory Allocation Similar to U allocation, the amount of RAM assigned to each module depends from the specific appliance configuration. When running in dual mode, the V10000 allocates the same amount of memory as the V5000 has. The Network Agent configuration has no impact on the amount of memory allocated to the ESG; it only affects the balance of memory allocated within the specific WSG components.
ESGA v7.6 201 39
ESG Architecture
Slide 3:5
Hard Disk Allocation The difference between single mode and dual mode is the amount of hard disk space allocated to Mail Transfer Agent (MTA) A V10000 running in dual mode allocates 70GB of HDD to MTA; while both the V10000 single more and the V5000 allocate 100 GB of HDD to MTA.
40 ESGA v7.6 201
ESG Architecture
Slide 3:6
ESG Internal Architecture The ESG has many internal components. Some of the most important ones appear in this slide. The idea is to help you understand that each message that arrives to the ESG flows through a series of components. At the end of the lifecycle, one of the following will happen to the message:
Delivered to the intended recipient
Delivered to the intended recipient but modified (i.e. attachment stripped)
Quarantined
Dropped/Discarded
In addition to the messages processing components, the ESG needs to communicate with the systems s, email s, and external components. The ESG provides two management UIs for two types of s:
TRITON - Email Security, incorporated UI into TRITON architecture, running on the off-box TRITON Manager Server, is for s to manage multiple ESG appliances concurrently.
PEM, running on the ESG appliance, is for end s to manage their quarantined emails.
ESGA v7.6 201 41
ESG Architecture
TRITON - Email Security and PEM communicate directly with the off-box database to fetch formation. Appliance Interface receives and processes the requests from Appliance Manager, dom0 of the appliance. SSL Interface is a SSL proxy which provides secure communication between on-box module and off-box module. The ESG management UI and the on-box Config module connect to SSL Interface. Authentication provides four functions: maintaining different types of groups, authenticating message sender, message recipient, and end to access PEM. Config consists of a policy module and a cluster module. The policy module receives configuration requirements from the ESG management UI and simple command line from back end, and stores the configuration files on the appliance. The cluster module synchronizes configuration files from primary ESG to secondary ESG in cluster mode. PEM loads and balances traffic to multiple PEM UIs, and reduce the stress for single PEM UI. It also provides the single entry for all end s to manage quarantined messages. Quarantine isolates messages which failed in going through the antivirus engine or antispam engine. Alert provides multiple methods (dashboard, email, pop-up and SNMP) for system services to report exceptions. Log receives all kinds of logs from other on-box modules, stores the system logs and audit logs to the on-box log database, and forwards the message logs to the off-box Log Server. PostgreSQL Database, the on-box log database server, saves system logs and audit logs from the Log module. The off-box Log Server runs on windows Operation System. It receives the message logs from the on-box Log module save them to the database. Hybrid provides the s for the protected domain to to Hybrid service automatically, synchronizes the communication token and pulls hybrid logs to the log database. Policy Engine is the component installed on ESG which talks to the DSS Manager for registration and receives the policy pushed out by the DSS Manager. DLP Filter enables Policy Engine to the DSS Manager, which is integrated with TRITON Unified Security Center. Scan Engine performs five types of filtering: hybrid, antivirus, antispam, disclaimer and DLP.
42 ESGA v7.6 201
ESG Architecture
Slide 3:7
External Components As mentioned before, ESG interacts with s, end s, and various offbox components. The slide displays the key components that are responsible for communication between inside and outside ESG. 1. The Log Database and the Log Server are two different entities, which can be installed on the same physical server. 2. The Encryption Service can be on-premises or on-demand in the cloud. Currently the ESG s integration with the Voltage encryption gateway. It also s natively TLS encryption. We’ll cover the encryption with more details later in the specific chapter. 3. Messages go through a pre-filtering analysis where the email sender (IP address and domain) runs through the Real-time Blackhole List Server (RBL Server) lookup and the Websense Reputation Service Server (WRS Server). The objective is to reduce the amount of messages that need to be analyzed early on in the process. RBL Server and WRS Reputation Server can easily identify and reject a large number of spam messages. 4. The ESG needs connect to the ESG through the TRITON Management Server, which runs on an off-box server. The TRITON Management Server communicates with the SSL interface as described earlier.
ESGA v7.6 201 43
ESG Architecture
5. DDS Update Server is designed for utilized able database within Websense products. The database covers URL filtering database, real time update database, real time security scanning database, license information database, etc.
44 ESGA v7.6 201
ESG Architecture
Slide 3:8
DLP Integration The DLP Filter receives the messages for scanning in standard email format (RFC 822 compliant). The PE parses the message before scanning according to the policy. Integrating DLP into ESG: 1. ESG with the DSS Manager and transfer DLP incident logs to the DSS Manager. 2. Configure and deploy DLP policies, e.g. fingerprint, on the DSS Manager. The policies are pushed out to the DLP Filter. 3. Configure DLP policies (active/disable/block/monitor) on ESG. Your organization will leverage the TruEmail DLP capability with patented PreciseID fingerprinting and natural language processing technologies.
Note To let the DLP Filter work, you need to make sure ESG subscription for DLP is valid; otherwise the message will not be send to the DLP Filter for scanning.
ESGA v7.6 201 45
ESG Architecture
The DSS Manager itself needs a subscription for normal operations, which is an XML file. However for the DLP Filter, the DSS subscription is not a prerequisite. That is, even if no subscription is imported into the DSS Manager, or the DSS subscription is expired, the ESG ed into DSS manager could still be able to make scanning. In another word, the DLP scanning of ESG does NOT need the DSS subscription, but needs the ESG subscription for DLP.
46 ESGA v7.6 201
ESG Architecture
Slide 3:9
Hybrid Integration ESG hybrid mode allows the to integrate the on-premise Email Security Gateway with Websense Hosted Email Security (HES) service, to deliver easy deployment and high-capacity email security. The workflow sequences are shown in the slide above. HES service to each ESG box is based on hybrid . All ESG hybrid configuration will be sent to HES cluster and stored as the hybrid settings. The ID is ESG license key, and the is transparent to the customer and is maintained by ESG/HES internally. Whenever ESG talks to HES Sync Server (HSS), the and is required for authentication. If the hybrid mode is enabled, an HES will be created. HES can provide hourly refreshed statistical data in last week. The statistical data is used to show the “Value of Hybrid Mode” on the dashboard reports. The data includes:
Number of spam messages blocked in the cloud
Number of total messages processed in the cloud
Total size of blocked and processed messages
ESG will pull the data hourly from HES cluster.
ESGA v7.6 201 47
ESG Architecture
The inbound email traffic flows the sequence below: 1. Emails sending from external domains are supposed to reach the protected domains. All messages go through the Hybrid service at first. 2. Filtered messages are sent to the ESG, where to be processed again. 3. Messages detected as spams, viruses, malwares, etc are discarded or isolated. 4. Allowed messages are sent to the email server for delivery.
The outbound email traffic flows the following steps: A. Outbound messages are sent from the email server to the ESG. B.
The ESG filters out the messages that do not comply with the policies.
C.
The messages that comply with policies are delivered to the intended recipient.
Note In this scenario, Hybrid service filters the inbound traffic only.
48 ESGA v7.6 201
ESG Architecture
Slide 3:10
Policy Processing ESG listens on port 25, receives SMTP messages, and filters the message according to its configuration. If the message is not coming from a protected domain (inbound message), the ESG first checks the sender IP address, domain, and other information against the RBL, WRS and any custom Always Block List. This step already filters a majority of the messages. Next, the recipients are validated against the directory service. This is an important step to prevent Directory Harvest Attacks (DHA). The last three steps are the most typical of any email filtering product. The messages is checked against size limits, spam content, virus content, and other malicious threats (file types of the attachments, compressed files, protected archives, etc.) Outbound messages do not have to be checked against the RBL, WRS, and other block lists; by definition, they come from trusted source. However, outbound messages need be checked for possible viruses. While viruses should not be there (most company run multilayer AV in their infrastructure), it is good practice to ensure that you are not responsible for spreading an outbreak.
ESGA v7.6 201 49
ESG Architecture
More importantly, and mostly unique to the ESG solution, you implement DLP policies for all outbound messages. The cost of information leakage, both financially and in of a tarnished reputation, can be devastating.
50 ESGA v7.6 201
ESG Architecture
Slide 3:11
Message Processing Flow The incoming messages (inbound or outbound alike) are received by the SMTP Server, which listens for incoming connections in T port 25. SMTP Server is in charge of dropping as many as possible of the incoming messages that maybe threats, which is called pre-filtering analysis. During this process, the SMTP service uses multiple techniques, such as Real-time Blackhole List (RBL) lookup, Reverse DNS (RDNS) lookup, Websense Reputation Service (WRS), etc to perform the analysis fast and accurately. Moreover, it greatly reduces the number of messages that need further analysis. Filter Server receives the messages after the SMTP Server processing them, facilitates the Policy Engine and the Scan Engine to filter out the additional incompatible emails based on the policies and filters set by s. This is the content filtering, with the goal of only delivering messages that are appropriate in content and do not have any threats in them. At the end of the message processing, three outcomes are possible for the messages: 1. Dropped/Discarded: The message is deemed to be a serious threat. It cannot and should not be ever released to the recipient. This is a not recoverable action. 2. Delivered to the recipient: The message ed all the security and policy scanning. It is appropriate in content and does not have any other threats.
ESGA v7.6 201 51
ESG Architecture
3. Quarantined: The message did fail one or more of the security and policy scanning. However the content is not such that is should not ever be released to the recipient, but may need to be retrieved by s or end s. Quarantined messages are stored in the local (ESG file system) quarantine queue. Information about quarantine is stored in the log database. Tip Recipients of a quarantined message can retrieve it through PEM management UI.
52 ESGA v7.6 201
ESG Architecture
Slide 3:12
Mail Filtering Framework ESG adopts 4 queues when processing the messages:
Filter queue The messages that through pre-filtering will be placed into priority based filter queues. If quarantined messages are to be re-processed (scanned again), the messages will be put back to filter queue.
Incoming queue The messages processed by Filter Daemon are moved from filter queue to incoming queue. Besides, the messages that should not be filtered (bounce/delivery request from PEM) are put in the incoming queue.
Deferred queue If a message failed in being delivered for some transient reasons (it might succeed later), the message is placed in the deferred queue.
Active queue The messages in the active queue will be sent to SMTP Client and delivered.
The messages are processed in the following procedure within ESG:
ESGA v7.6 201 53
ESG Architecture
1. First layer of filtering is what we called pre-filtering, handled by SMTP Server. Pre-filtering rejects most zombie connections. Clean Up Filter inserts the permitted messages into priority based filter queues according to the scores and informs Filter Daemon. 2. Filter Daemon asks Policy Server about the policies of the message, call the related Scan Engines, and execute the configured actions. After filter daemon scans the message, the message is moved from filter queue to incoming queue. 3. The message that fail in filtering is dropped or quarantined. 4. The acceptable message is sent to Queue Manager. Queue Manager moves the message from incoming/deferred queue to active queue. After a message is scheduled, Queue Manager informs SMTP Client with meta info of the message. Queue Manager scans the deferred queue periodically, brings messages in deferred queue to active queue and schedules the messages according to system resource. After the messages have been tried for max times, the messages will be bounced to sender. If the first bounce failed, the message will be bounced to configured postmaster address. 5. SMTP Client gets the message from active queue and deliver the message, and replies the status of the message to Queue Manager after delivering.
54 ESGA v7.6 201
ESG Architecture
Slide 3:13
RBL and WRS The slides shows the detail processing of Real-time Blackhole List (RBL) check and Websense Reputation Service (WRS) lookup. The IP address and domain of the email sender go through the external RBL Server at first then the WRS Server and get an reputation score. Based on different scores, messages will be placed into priority based filter queues.
ESGA v7.6 201 55
ESG Architecture
56 ESGA v7.6 201
4
Personal Email Manager
Personal Email Manager (PEM) is a component of Websense® Email Security. PEM enables a to look at isolated spam messages and decide whether to delete the messages or treat them as legitimate email messages. It also identifies the IP address of the appliance on which an end can access PEM. You can also authorize an end to manage personal Always Block List and Always Permit List, configure the appearance of the PEM end display, and deploy appliances in groups for load balancing purposes. PEM monitors specially configured Websense Email Security queues and notifies s if they have blocked inbound email. It also provides end-s a facility for managing their blocked email. To achieve the desired behavior and results, PEM must be istered on an ongoing basis. These tasks include:
Configuring queues
Customizing the notification email templates
Language PEM includes adaptive language for Spanish, Portuguese, Italian, and German. When an end- or logs into the PEM facility, they can determine the language displayed in the interface.
ESGA v7.6 201 57
Personal Email Manager
Slide 4:1
PEM Key Features PEM is a built-in feature of the ESG appliance. There is no need for the to install additional software or hardware. The PEM module is fully ed in a clustered environment. In a clustered environment, PEM can be accessed through the primary ESG or the secondary ESG. If the primary ESG is not available, s can also view their isolated message list through the secondary ESG. However, isolated messages stored on the primary ESG cannot be viewed, delivered, or deleted. The PEM can be successfully implemented and used any environment regardless of size. To even greater scalability, the ESG can store the isolated messages on a remote file system. ESG can connect to a NFS share or a CIFS one (via SAMBA). In environment where more than ESG exists, even if all appliances use the same share, each ESG still creates it own separate and independent queues. One of the key benefits of the PEM is greatly reduced load; s are in charge of setting their own Always Permit List and Always Block List and release messages which may have been isolated by mistake. With tight and granular control, only messages that do not present a threat to the company can be available for release.
58 ESGA v7.6 201
Personal Email Manager
Slide 4:2
Quarantine System Overview The ESG process the messages; when it finds that an email is a spam, virus, or the message failed to be processed, it could not be encrypted or could not be decrypted the ESG informs the quarantine daemon to quarantine the email. The email can be quarantined locally or on a remote storage mounted on ESG through NFS or SAMBA (CIFS). s and end s can view and manage the quarantined emails and the quarantine daemon will record corresponding quarantine logs to ESG log database. Quarantine logs are sent directly to log database without being relayed by the ESG log server. These logs need to be sent out one by one instead of in bulk for high reliability requirement.
ESGA v7.6 201 59
Personal Email Manager
Slide 4:3
Mail Queues Quarantine messages are isolated into different quarantine queues (folders) in of their different attributes. By default, there are six built-in queues: 1. Virus: Used to store messages, which contain a virus. 2. Spam: Used to store messages, which triggered one of the spam filters. 3. Exception: Used to store exceptional emails (failed several non specific filters). 4. Encryption-fail: Used to store messages that did not encrypt correctly. 5. Decryption-fail: Used to store messages that did not decrypt successfully. 6. Archive: Used to store all incoming messages. Tip These 6 queues cannot be deleted. However, can edit their properties or add new queues. As mentioned before, messages in each queue can be stored locally (on ESG box) or on a remote storage. ESG s two remote storage types:
NFS
CIFS (through Samba)
60 ESGA v7.6 201
Personal Email Manager
On the Blocked Message page, s can see the quarantine queue list.
ESGA v7.6 201 61
Personal Email Manager
Slide 4:4
Queues Monitor Every 30 seconds, ESG will perform a checking on queue status, including remote queue availability and available disk space check.
If the remote queue is found unavailable, the queue storage status will be marked as "mounted error", and ESG will generate a corresponding alert. If the remote queue is available again, the "mounted error" status will be cleared as "ok" and the alert will be cleared too.
If the remote storage is unavailable, trying to move the queue with copying the messages will fail.
If the disk is available, ESG will check the space usage of the queue. If the percentage of free disk space is less than 10%, ESG will mark the queue status as "disk space low" and send an alert. Also, if a queue's free disk space is return to more than 10%, the status will be cleared and alert will also be cleared.
62 ESGA v7.6 201
Personal Email Manager
Slide 4:5
Queues Configuration In cluster mode, the "Queue list" page is similar to standalone mode. The "volume" is the sum of all machines' volumes, and the "size/total" is the sum of all machines' size and size limit. For example, there a 2 machines in a cluster, both of which has a 10GB size limit (Their limits are always same because the configuration is synced between cluster). Machine 1 has 10 files in it and its size is 5MB, machine 2 has 5 files in it and its size is 3MB. In this case, the "volume" and "size/total" displayed in the UI will be "15" and "8MB/20GB". Compared with standalone mode, the error status description of the queue is more complex. In standalone mode, if the hard disk space of the queue is low, there will be an icon displayed which shows "hard disk low". In cluster mode, if the storage type is locally, the content displayed will be a list of "Device X.X.X.X: hard disk low", because maybe more than 1 device has no enough free disk space; if the storage type is remote, the content displayed will be "Remote hard disk low". Regarding the "remote storage invalid" status, if a queue of at least one machine is in this status, the WebUI should show "remote storage invalid".
ESGA v7.6 201 63
Personal Email Manager
Slide 4:6
PEM Components PEM is composed by two sides: the side and the end side:
side - PEM general configuration: the PEM digest settings, the end- access control and the authentication mode.
End side - PEM management: the end can manage the messages processed by ESG and take the actions allowed he/she prefer.
64 ESGA v7.6 201
Personal Email Manager
Slide 4:7
Notification Message The ESG PEM enables end s to configure the notification message that will be sent out from the ESG when any of the emails, with the recipients or senders as themselves, are blocked. End s are able to configure the notification message in the following aspects:
Max messages The maximum number of the message digests in the notification email to the end . Such as, if set to 10, then, only 10 message digests will be in the notification message even if the end has more than 10 quarantine or spam messages. The maximum value of this setting is 100 and the default is 10.
Digest send interval This interval decides when the digest message will be sent out according to a time and day setting. Times are set by hour and minute while days can be
Every day
Every weekday
A specific day in Every Week For example, 3:00 Monday, Every week or 6:00, 12:00, 20:00, Every day.
Digest message template Message operation options define the operations which will be included in the digest message and the end can see and execute.
ESGA v7.6 201 65
Personal Email Manager
The slide show the screen when setting the notification message: 1. Set the max number of messages per notification and the actions available in the notification that the end are able to take. 2. Configure the information of the notification:
Company Define the brief info to the customer company displayed in the digest message
Description Define the brief info to the ESG which the customer prefers.
Sender Define the sender email address of the digest notification message to the end . The address format is name@domaininfo, such as “[email protected]”, meaning that the sender of the digest message will be this address. This is ONLY one email address.
Subject Define the subject of the digest message. The maximum length of this value is 100 characters.
3. Set the header The header info will be displayed in the digest message. It can provide some key words to the end before he/she open the message text. 4. Set the footer The footer info can provide information that the end can use, like, the telephone number of email address when the end gets a problem. The end- access list allows the ESG to define the list who can use the PEM to manage the quarantine or spam messages. The will only be chosen from the existed " Directory" objects. If the ESG wants to use one new directory, first, it's required to add it in the page at which the directory can be defined. This setting will list all the existed un selected directorires in the left compound box and all the checked directories in the right box.
66 ESGA v7.6 201
Personal Email Manager
Slide 4:8
Queues Actions Quarantine message logs will be stored on a remote SQL server database, together with ESG message logs. By clicking the queue name in "Queue list" page, can view all messages of the queue. can view the messages by date, and can also search messages by sender, recipients, subject and policy. can delete, , reprocess, continue reprocess, deliver (release) the message(s) or forward the message(s) to one or more email addresses. can also report a false-positive to Websense, or add the sender of a message into ESG global Always Permit/Block List. Sometimes when a queue operation is ongoing, some of the message operations will return a failure of "retry later". The following actions are ed:
Delete: Delete the message
Deliver: Deliver the message to its recipients
Reprocess: Delete the message from the queue and reprocess it, just like it has been received by ESG again. It's useful when sometimes technical team and development team want to debug false-positives of filter. They can turn on the debug, reprocess the message and find why this message is captured by filter.
ESGA v7.6 201 67
Personal Email Manager
Forward: Forward the message to one or more recipients, the forwarded message will be embedded in the forward mail as an attachment. The subject of the forward mail is "Fw: Forward Message(s)".
Add to Always Block List: Will add the message sender to Always Block List.
Add to Always Permit List: Will add the message sender to Always Permit List.
: the message in .eml format. If more than one message is ed together, they will be compressed in a zip file.
Clear queue: Clear the whole queue, all messages in the queue will be deleted.
Continue Process: Sometimes a message contains both virus and spam info. Assume it's captured by spam filter and quarantined, then finds that it's a false-positive and decides to deliver (release) it. In this case, the message will by the virus scanning. To prevent this, there is another action "Continue Process" provided to make sure the message will be scanned by other filters before it is delivered.
Not Spam: Clicking this button will cause two actions: to send a copy of this email to [email protected] for false positive research, then release the email to its original recipients.
68 ESGA v7.6 201
Personal Email Manager
Slide 4:9
PEM in Clustered ESG In a cluster environment there is always one and only one appliance which operates as the primary. For the purpose of the PEM notifications, only the primary sends out the notification. In the notification, sent to the , the IP address (or hostname) for the isolated mail link is the primary's IP (or hostname). If the primary is not available (down), the s do not receive notifications and they cannot retrieve isolated emails. The secondary will not send notification and will not participate in any election process. Once the primary returns operational, the s receive once again the notification and are able to manage the isolated messages. Similarly to the queue management, also the is only available through the primary. During normal operations, the process of retrieving isolated messages via PEM proceeds as follows: 1. ESG #1 (not the primary) receives a message, which is going to be isolated. 2. The ESG writes an entry in the log about this message. As mentioned before, a key field in the database is the ID of the ESG who isolated the message.
ESGA v7.6 201 69
Personal Email Manager
3. The recipient, which has been notified that she has received an message being isolated, connects to the PEM module. The link in the notification points to the TRITON - Email Security. 4. The primary ESG looks up the message details in the log database. Using the ID of the machine, which isolated the message, can connect to that ESG (in this case ESG #1) 5. The primary ESG communicates with the ESG #1, retrieves the message, and sends it over to the who requested it.
70 ESGA v7.6 201
5
Policy Management
Policy is the pipeline that message flows go through. ESG defines 3 types of policies based on traffic directions:
Inbound policy
Outbound policy
Internal policy
Inbound/Outbound/Internal policy have multiple sub-policies which will applies to the messages that match the policy’s matching condition. The matching condition defines the entry criteria of the policy pipeline, based on the envelope sender and recipient address of the messages. Each policy can contain a series of rules. Rules define how the message in the policy pipeline is to be scanned and how the message is processed when it triggers the filters. Each rule is combined with action and filter. Filter is individual scanning module which defines how the message in the policy pipeline is to be scanned. Action is individual module which defines how to process message when filter is matched.
How the ESG policy works could be described in the following steps: 1. Messages are divided into 3 categories: inbound, outbound, and internal. 2. In each category, messages match the condition based on the addresses of the senders and recipients respectively. 3. The specific filter, defined in the rule of the matched policy, scans the messages. 4. The messages filtered out are processed according to the action.
ESGA v7.6 201 71
Policy Management
Slide 5:1
Key Features Policy rules comprise the filters and filter actions that determine how a message that matches a policy’s sender/recipient conditions is handled. Filters provide the basis for email scanning for viruses and spam, and filter actions determine the final disposition of a message when it triggers a particular filter. Define new filters, or copy and edit an existing filter to suit your organization’s needs. Add and define new actions or customize default actions as needed. After you have created and configured filters and filter actions, they are available for inclusion in your policies.
72 ESGA v7.6 201
Policy Management
Slide 5:2
Policy Components Flow 1. Policies are directional-based and depend on the following factors:
Protected domain
Email sender address
Email recipient address
2. Policies contain conditions; when matched, rules are applied. 3. Rules contain both filters and actions:
Filters are individual scanning modules which define how messages in the policy pipeline are scanned.
Actions are individual modules which define how a message is processed. Note Policy order is important. ESG policies are similar to firewall rules. If one policy is matched, then the rest will be skipped.
In ESG, there can be up to 32 policies per mail route direction.
ESGA v7.6 201 73
Policy Management
Slide 5:3
Policy Directions An Email Security Gateway policy is applied based on defined sender/recipient conditions and the direction of the email. You can apply a different policy to different groups of senders and recipients. For example, you might apply a policy to a marketing department group in your organization and a different policy to a human resources group. After you define a set of senders and recipients in a policy, you can add the policy rules (a combination of filter and filter action) to apply when the sender/recipient conditions of the email match the policy. Email Security has 3 general types of policies, depending on the direction of the email inbound, outbound, or internal. Message direction is determined on the basis of an organization’s protected domains:
Inbound - The sender address is not from a protected domain, and the recipient address is in a protected domain
Outbound - The sender address is from a protected domain, and the recipient address is not in a protected domain
Internal - Both the sender and recipient addresses are in a protected domain.
ESG has one predefined default policy for each email direction, as well as a default Data Security policy for each direction. Data Security policies may be applied to email in any direction. These policies are configured in the Data Security module of TRITON Unified Security Center and can only be enabled or disabled in ESG. You
74 ESGA v7.6 201
Policy Management
need to Email Security with the Data Security management server and click Deploy in the Data Security module for the policies to be active.
The inbound policy logic is as follows: 1. Pre-filtering/Reputation 2. Always Permit/Block List 3. Antivirus (sender/recipient/filter-option/action) 4. Antispam (sender/recipient/filter-option/action) 5. Content Guardian (sender/recipient/filters/action) 6. DLP
The outbound policy logic is as follows: 1. Antivirus (sender/recipient/filter-option/action) 2. Antispam (sender/recipient/filter-option/action) 3. DLP
The internal policy logic is as follows: 1. Antivirus (sender/recipient/filter-option/action) 2. Antispam (sender/recipient/filter-option/action) 3. DLP or by DLP
ESGA v7.6 201 75
Policy Management
Slide 5:4
Policy Conditions Match conditions define the entry criteria of a policy. Each match condition consists of a pair of sender and recipient envelope addresses, marked as From address and To address. The address can be a simple address like [email protected], or can be a wildcard address *@somewhere.com, or can be just * for any address. When the envelope sender address of the message matches the From address of a match condition and the recipient address matches To address of a match condition, then this match condition is considered matched and the message goes into this policy. If the message has multiple envelope recipient addresses, it is possible that a message is split into different policies. The address in match condition can also be from certain Directory, which is logic group of addresses. A policy can have multiple match conditions, and messages will try to match these match conditions one by one, from top to bottom. The order of match conditions in policy is not important. When match conditions are empty, the policy will be skipped directly. There are at most 32 match conditions per policy.
76 ESGA v7.6 201
Policy Management
Slide 5:5
Rules Rules define message scanning process of policy. If the sender and the recipients of a message match any match condition of a policy, then messages will try to match rules one by one, from top to bottom. If one rule is matched, the rest will be skipped. Rules cannot be deleted/added, but can be edited. Rules contain the following:
General configuration, including Name, Enable/Disable, Order
Filters: scanning modules to use
Actions: what action to take if a filter scan is matched
There are 3 default rules
Antivirus
Antispam
Disclaimer
ESGA v7.6 201 77
Policy Management
Slide 5:6
Filters Email Security Gateway has three predefined default filter types: virus, spam, and disclaimer. The virus filter analyzes an email message and its attachments for the presence of viruses and other threats. The spam filter scans email content and compares it against a database of known spam characteristics. You can select from a variety of antispam tools, including the Websense Web Security URL scanning tool. If you want to add text at the beginning or end of a message, use the disclaimer filter. Filters are created and managed via the Main > Policy Management > Filters page. Click Add to open the Add Filter page and set the properties of your new filter. You can also copy a filter whether or not it is in use by a policy. A filter can be deleted, as long as it is not in use by any policy. However, you cannot copy or delete an Email Security default filter. Up to 32 filters can be assigned per policy.
78 ESGA v7.6 201
Policy Management
Slide 5:7
Actions A filter action determines a message’s final disposition. ESG scans messages and their attachments then performs an action based on applicable policy settings. Actions are created in the Main > Policy Management > Actions page. You can add a defined action to a policy rule when you configure your email policies. Create a new filter action by clicking Add and selecting action properties. You can remove a filter action by marking the check box to the left of the filter name to select it and clicking Delete. You can delete a filter action only if its current status is Not in use. A filter action that is in use does not have a check box for selection. Note You cannot remove an Email Security default filter action. There are at most 32 actions per policy.
ESGA v7.6 201 79
Policy Management
Slide 5:8
Summary 1. All traffics are divided into 3 directions: inbound, outbound, and internal. 2. Messages match the specific conditions. 3. Filters are triggered for the messages of the conditions. 4. Actions are performed if the filters are matched.
80 ESGA v7.6 201
6
Advanced Encryption
ESG encryption enables the secure delivery of sensitive email communications to business partners and individuals. Easy istration, no complex key management, no additional hardware or software requirements eliminate the traditional cost and complexity barriers to implementing email encryption.
The ESG provides 3 types of message encryption:
Transport Layer Security (TLS): to encrypt/decrypt emails by ESG locally
Hybrid service encryption: to encrypt/decrypt emails by Hybrid service
Third party encryption: to encrypt/decrypt emails by integrated third party encryption gateway
ESGA v7.6 201 81
Advanced Encryption
Slide 6:1
Advanced Email Encryption Email Security s the following 3 types of message encryption:
Transport Layer Security (TLS)
Hybrid service encryption
Third party encryption gateway
We’ll talk about the 3 encryptions in detail!
82 ESGA v7.6 201
Advanced Encryption
Slide 6:2
TLS introduction Transport Layer Security (TLS) and its predecessor, Secure Sockets Layer (SSL), are cryptographic protocols that provide communications security over the Internet. TLS and SSL encrypt the segments of network connections above the Transport Layer, using symmetric cryptography for privacy and a keyed message authentication code for message reliability. Several versions of the protocols are in widespread use in applications such as web browsing, electronic mail, Internet faxing, instant messaging and voice-over-IP (VoIP). TLS is an IETF standards track protocol, last updated in RFC 5246 and is based on the earlier SSL specifications developed by Netscape Corporation.
ESGA v7.6 201 83
Advanced Encryption
Slide 6:3
TLS Encryption TLS is an Internet protocol that provides security for all email transmissions— inbound, outbound, and internal. The client and server negotiate a secure connection for the transmission to occur, provided both the client and the server the same version of TLS. The ESG uses a TLS encryption level of 128 bits. The ESG uses 2 levels of TLS: Opportunistic TLS and Mandatory TLS.
Opportunistic TLS It is used during the message routing process. You can enable TLS for a specified message route on the Settings > Receive/Send > Mail Routing page. Create a new route that uses the TLS delivery option or edit an existing mail route to add the TLS option. At the bottom of the page, mark the Use Transport Layer Security (TLS) check box to use opportunistic TLS for message routing. With opportunistic TLS, the protocol must be implemented on both sides of a connection for an encrypted transfer. If a connection attempt is made using the TLS protocol, the connection recipient must provide appropriate TLS credentials for an encrypted data transfer. If the TLS “handshake” fails, the data transfer is made via plain text, rather than encrypted text. In either case, the data transfer is successfully accomplished.
84 ESGA v7.6 201
Advanced Encryption
Mandatory TLS Mandatory TLS is the default encryption method that is enabled on the Settings > Inbound/Outbound > Encryption page. For mandatory TLS, both sides of an email connection must use the protocol. An encrypted data transfer occurs when the TLS handshake process is successful. Unlike opportunistic TLS, if the handshake fails during the connection attempt, the connection is terminated and no data transfer occurs.
A. If the next hop MTA has the ability of TLS, the encrypted email will be sent out. B. Mandatory TLS: if the next hot MTA does not TLS, the email is placed in a deferred message queue for a later delivery attempt B’. Opportunistic TLS: if the client and server cannot negotiate a secure connection, the message will be sent to the recipient.
ESGA v7.6 201 85
Advanced Encryption
Slide 6:4
Prerequisite of Hybrid Service Encryption Prerequisites of Hybrid service encryption A. Hybrid service encryption is available only if your subscription includes the Hybrid service and the Hybrid service is ed and enabled. Otherwise, you can not select Hybrid service encryption from TRITON - Email Security. B. Hybrid service encryption requires a certificate from a hybrid service trusted CA. In order to obtain a certificate from one of these CAs, you need Generating a Certificate g Request (CSR) for the server on which you want to install the certificate. After you have generated a CSR, follow the third-party CA acquisition procedures for the certificate you want to purchase. Use the Settings > General > TLS Certificate page for certificate import.
Generating a Certificate g Request (CSR) You can generate a CSR using the following procedure: 1. Open the command-line interface on the V-Series appliance where Email Security resides.
86 ESGA v7.6 201
Advanced Encryption
2. Execute the following commands: openssl genrsa -out [keyfile] 1024 Where [keyfile] is used to import the certificate to the ESG. openssl req -new -out reqfile -key keyfile –subj /C=CN/ ST=Bj/L=Bj/O=Websense/CN=ESG\/ [email protected]/ Where [keyfile] is the CSR file and /C=CN/ST=Bj/L=Bj/O=Websense/ CN=ESG\/emailAddress=@websense. com/ is the subject of the certificate. The certificate subject can be modified if necessary.
List of trusted CAs for hybrid service The Email Security Gateway hybrid service can use a certificate from any one of the following trusted CAs:
Entrust.net 2048 Secure Server CA
Entrust.net Secure Personal CA
Entrust.net Secure Server CA
Equifax CA
Equifax Secure CA
GTE CyberTrust Global Root
GTE CyberTrust Root 2
GTE CyberTrust Root 3
GTE CyberTrust Root 4
GTE CyberTrust Root 5
GTE CyberTrust Root CA
GlobalSign Partners CA
GlobalSign Primary Class 1 CA
GlobalSign Primary Class 2 CA
GlobalSign Primary Class 3 CA
GlobalSign Root CA
Thawte Personal Basic CA
Thawte Personal Freemail CA
Thawte Personal CA
Thawte Server CA
Thawte Server CA
Thawte Universal CA Root
ValiCert Class 1 VA
ValiCert Class 2 VA
ValiCert Class 3 VA
ESGA v7.6 201 87
Advanced Encryption
VeriSign Class 4 Primary CA
Verisign Class 1 Public Primary Certification Authority
Verisign Class 1 Public Primary Certification Authority - G2
Verisign Class 1 Public Primary Certification Authority - G3
Verisign Class 2 Public Primary Certification Authority
Verisign Class 2 Public Primary Certification Authority - G2
Verisign Class 2 Public Primary Certification Authority - G3
Verisign Class 3 Public Primary Certification Authority
Verisign Class 3 Public Primary Certification Authority - G2
Verisign Class 3 Public Primary Certification Authority - G3
Verisign Class 4 Public Primary Certification Authority - G2
Verisign Class 4 Public Primary Certification Authority - G3
Verisign/RSA Commercial CA
Verisign/RSA Secure Server CA
88 ESGA v7.6 201
Advanced Encryption
Slide 6:5
Hybrid Service Encryption for Outbound Email Hybrid service encryption is enabled on the Settings > Inbound/Outbound > Encryption page. Note Hybrid service encryption is mandatory encryption. If the encryption is failed, the message will be placed in a deferred message queue for a later delivery attempt. For mandatory TLS, both sides of an email connection must use the protocol. An encrypted data transfer occurs when the TLS handshake process is successful. Unlike opportunistic TLS, if the handshake fails during the connection attempt, the connection is terminated and no data transfer occurs. Hybrid service does not encrypt inbound or internal messages. Even though Data Security policy identifies an inbound or internal message for encryption and Hybrid service encryption is selected, the message will not be encrypted and an alert (HESencryption for inbound/internal messages) will be sent. Therefore Data Security policy must be modified to designate only outbound messages for encryption when Hybrid service is used. Hybrid service encryption for the outbound email following the below steps:
ESGA v7.6 201 89
Advanced Encryption
1. The Email Server sends out an unencrypted email to the ESG. 2. Data Security policy labels the email for encryption. 3. The ESG sends the email to the Hybrid Filter for encryption on SMTP forced TLS. 4. The Hybrid Filter encrypts the email and delivers the email to the recipients. The Hybrid Filter address is configured during the Hybrid service setup process.
The Hybrid Filter identifies the message by
IP address of ESG
Sender domain
Encryption x-header: indicating the message should be encrypted
TLS state
If the message contains X-ESG-HYBRID header, but fails in any of the other 3 criterions, the Hybrid Filter will bounce the message. IP addresses of ESG are pushed to the Hybrid Filter during Hybrid configuration; and later changes of IP addresses or network topology are notified to the Hybrid Filter too. If encryption fails, the message is placed in a deferred message queue for a later delivery attempt. If the Hybrid Filter detects a spam or virus in an encrypted outbound message, the mail is bounced to the message sender. Bounce means the Hybrid Filter returns a 5.x.x permanent failure to ESG on SMTP conversation; or generates an NDR message (with return of content) depending on the original message and delivers it to ESG on SMTP opportunistic TLS. ESG should always STARTTLS to the Hybrid Filter and accept TLS handshake from the Hybrid Filter. Messages sent from ESG to the Hybrid Filter for encryption may be put in the Hybrid Filter queue for some time.
90 ESGA v7.6 201
Advanced Encryption
Slide 6:6
Hybrid Service Encryption for Inbound Email For inbound messages, Hybrid service encryption attempts to decrypt inbound encrypted mails. The detailed process is described on the slide: 1. An encrypted email arrives at the Hybrid Filter. 2. The Hybrid Filter decrypts the email and adds an x-header to the email to indicate whether the decryption succeeded, and scans the email and sends the permitted unencrypted email to the ESG. Note Message scanning is performed regardless of whether message decryption is successful. 3. The email may trigger DSS incidents. 4. The ESG scans and delivers the permitted unencrypted email to the recipients. The Hybrid Filter will deliver the messages to the ESG on SMTP opportunistic TLS with the following x-headers:
Decryption success/fail x-header, indicating decryption successful or failed
Virus x-header
ESGA v7.6 201 91
Advanced Encryption
Spam x-header
The ESG always STARTTLS to the Hybrid Filter and accept TLS handshake from the Hybrid Filter.
92 ESGA v7.6 201
Advanced Encryption
Slide 6:7
Third Party Encryption On-demand for Outbound Email Third party encryption is enabled on the Settings > Inbound/Outbound > Encryption page. Note Third-party encryption is opportunistic encryption. If the encryption failed, the message will be sent to the recipient in plain text. When Data Security policy identifies an outbound message for encryption and third party encryption is selected, the messages are sent to third party encryption gateway. The process is described on the slide: 1. The Email Server sends out an unencrypted email to the ESG. 2. Data Security policy labels the email for encryption. 3. The ESG sends the email to third party encryption gateway for encryption on SMTP opportunistic TLS, for third party gateway may not enable TLS to receive messages. 4. Third party gateway encrypts the message and delivers it to the recipients directly.
ESGA v7.6 201 93
Advanced Encryption
s configure x-headers to add to an outbound message via TRITON Email Security. These x-headers facilitate communications between the ESG appliance and third party gateway and they must match the corresponding settings in third party gateway.
Encryption x-header, indicating the message should be encrypted
Encryption success x-header, indicating the encryption is successful
Encryption fail x-header, indicating the encryption is failed
s can require s to present credentials to view encrypted mails: supply the desired name and in TRITON - Email Security. Authentication must be ed and configured on third party gateway as well. Besides x-header and credential configuration, s are able to set desired encryption failure options:
Isolate messages to queue
Send notification to original sender
Deliver messages
Drop messages
If third party encryption gateway is not configured, ESG will generate an alert (invalid third party encryption gateway).
94 ESGA v7.6 201
Advanced Encryption
Slide 6:8
Third Party Encryption On-premises for Outbound Email The difference between on premises and on demand is that in this case, third party encryption gateway routes the encrypted messages back to ESG instead of delivering the messages out. ESG identifies the messages from third party gateways by configured IP addresses of the gateway. The sequence is listed on the slide: 1. The Email Server sends out an unencrypted email to the ESG. 2. Data Security policy labels the email for encryption. 3. The ESG sends the email to third party encryption gateway for encryption. If encryption is successful, the third party gateway forwards the message (possible multiple messages from multiple recipients) to ESG with encryption success xheader. 4. ESG will deliver the encrypted message without further scanning. If encryption is failed, the third party gateway forwards the original message with encryption fail x-header to ESG. ESG will apply the encryption fail configuration, quarantine the message, notify senders, deliver or drop the message. An alert (third party encryption gateway encrypt fail) will be generated.
ESGA v7.6 201 95
Advanced Encryption
If there is no encryption success/fail x-header, there may be errors in configuring ESG and third party encryption gateway. An alert (third party encryption gateway does not encrypt messages) will be generated.
96 ESGA v7.6 201
Advanced Encryption
Slide 6:9
Third Party Encryption On-premises for Inbound Email In order to decrypt messages, third party encryption gateway should point their mail route back to ESG and the decryption option is enabled on ESG configuration. Decryption option can only be enabled when third party encryption gateway is selected and IP address of third party gateway is configured. The slide shows the process of third party encryption for inbound email: 1. The encrypted email arrives at the ESG. ESG checks if the inbound email are encrypted- those with content-type or xheaders are encrypted messages, which will be send to third party encryption gateway on SMTP opportunistic TLS to decrypt. 2. Third party gateway decrypts the email and routes it back to the ESG with decryption success x-header. 3. The email may trigger DSS incidents. 4. The ESG scans and delivers the permitted unencrypted email to the recipients. If decryption is failed, third party gateway will forward the original message with decryption fail x-header back to ESG. ESG will apply the decryption fail configuration: to quarantine the message, then scan the message. An alert (third party encryption gateway decrypt fail) will be generated.
ESGA v7.6 201 97
Advanced Encryption
s can configure x-headers for messages sent to third party encryption gateway via TRITON - Email Security. These x-headers facilitate communications between ESG and third party gateway and they must match the corresponding settings in third party gateway.
X-header, indicating decryption is allowed
Decryption x-header, indicating the message should be decrypted
Decryption success x-header, indicating the decryption is successful
Decryption fail x-header, indicating the decryption is failed
If there is no decrypt success/fail x-header, there may be errors in configuring ESG or third party encryption gateway. An alert (third party encryption gateway does not decrypt messages). Sandwich mode(decrypt, scan and then encrypt messages) is not ed. So decrypt success and decrypt fail messages will not be encrypted again. If Data Security returns encryption for decrypt success and decrypt fail messages, ESG will treat it as unencryption.
98 ESGA v7.6 201
7
Clustering Details
In this chapter you will learn about the details of linking multiple ESG nodes together to form a cluster to share traffic load or function as a single virtual ESG. Clustering benefits you in balancing process work among different ESG nodes when traffic is heavy, consolidating messages received in all separate ESG nodes for PEM, and easing management and configuration. If you have multiple ESG appliances running, clustering is not a requirement. You could have all of the appliances be as standalone and managed by one TRITON Management Server. In this way, the type of appliances and the running mode could mix with each other. This is not recommended, for it leads to inconsistent policy settings easily.
Each ESG has three types of roles:
Standalone: single ESG processes mail traffic. ESG works as standalone by default. Standalone ESG can be configured via TRITON console.
Primary: the primary node when working in cluster mode. Every cluster can have only 1 primary ESG. The primary is the only node in cluster to be configured via TRITON console.
Secondary: the secondary node when working in cluster mode. Every cluster can have 1 to 7 secondary ESGs. The secondary ESG can not be configured via TRITON console directly. The configuration is synchronized from the primary ESG.
After cluster negotiation succeeds, the primary and secondaries will work in cluster mode. If cluster negotiation does not succeed, all nodes in cluster will be standalone. One cluster is composed of up to 8 ESG nodes: one as the primary and others as secondaries. All the nodes within one cluster must be the same type of appliance and run in the same mode. For example, a V10000 and a V5000 cannot cluster together, a V10000 in dual mode and a V10000 in ESG mode too. All the ESG nodes within one cluster share the same configuration and same log database, accept mail traffic and deliver to destination mail server according to the same policies.
ESGA v7.6 201 99
Clustering Details
Slide 7:1
ESG Clustering Key Features In ESG cluster mode, mail traffic is distributed to different ESG appliances based on some traffic balancing algorithm. The traffic balancing algorithm is not on the ESG. We’ll talk about it later. All ESG nodes are managed centrally and can log to a central log server for consolidated logging across the enterprise. Even in a case with multiple clusters, all log events should be centrally stored, and reporting should be aggregated across all boxes. Even in a case with multiple clusters, all quarantined emails can be stored locally in each ESG appliance or centrally on an external storage device, but should be accessed from the same interface(s). ESG cluster is an Active/Active load balancing feature that provides real time synchronization between the primary and all secondary nodes for device configuration, which provides the most flexibility and maximizes system investment as requests are load-balanced across all available processing capacity. In Active/ Active deployment, each node must stay in a consistent state and must reflect the current state of the system
100 ESGA v7.6 201
Clustering Details
Slide 7:2
Clustering with Load Balancer This network deployment shows load balancer with three ESG appliances connected to it. The load balancer redirects mail traffic to different ESGs with certain traffic balancing algorithm. The balancing algorithm is part of the load balancer configuration and not ESG. The flow of traffic is as follows: 1. The firewall receives the email from the Internet. 2. If the messages are permitted, the firewall forwards them to the IP address of the load balancer. 3. The load balancer, using the specified algorithm, decides to which of the ESG appliance send the mail message. 4. The ESG, which receives the email from the load balancer, processes the message. formation is stored about the message. The permitted message is forwarded to the email server. 5. The email server receives the messages which ed the ESG filtering. The messages are now available for pick/delivery by the end s.
Clustering details Each ESG in a cluster has one of three roles:
ESGA v7.6 201 101
Clustering Details
Primary
The primary node when working in cluster mode. Every cluster can have only 1 primary ESG. The primary is the only node in cluster to be configured via TRITON console.
s configure the policies on the primary node through TRITON console. The configuration is then synchronized to all the secondary node.
Secondary
The secondary node when working in cluster mode.
Every cluster can have 1 to 7 secondary ESGs.
The secondary node can not be configured via TRITON console directly.
The secondary nodes synchronize the configuration from the primary node.
The status and logs are synchronized time to time as well.
The secondary nodes and the primary process mail traffic based on the same policies.
Standalone
Any appliance that is not set as primary and not negotiated to be a secondary stays in the role of standalone.
Important One cluster has a maximum of 8 nodes. All nodes of cluster must be the same appliance type: you cannot mix V10000 with V5000 in the same cluster. All nodes of cluster should have same applications deployed on the appliance: all appliance are either single mode (ESG only) or dual mode (ESG and WSG). Tip When cluster negotiation does not succeed, the primary node will not become standalone automatically. The will need to change this manually. Important When you have more than one ESG appliances, clustering is not a requirement. You can have all ESG appliances be in standalone mode; however, you will have to configure each ESG individually. This is not recommended and leads easily to inconsistent policy settings.
102 ESGA v7.6 201
Clustering Details
Slide 7:3
Clustering with MX Record This scenario does not have a visible load balancer deployed. It loads and balances email traffic on multiple ESG appliances using MX record. Let’s take the diagram as an example, in which you have three ESG appliances in your cluster. You can re-configure your DNS as one of the following:
Configure the MX record in DNS to return three routable IP addresses which bind to the external interface of the firewall. Each record should have the same preference value, so that the traffic is evenly distributed on the basis of number of incoming connection requests. The firewall maps the external IP addresses to the internal one of the corresponding E1 interfaces on the ESG appliances (1-to-1 NAT)
Configure the MX record in DNS to return three routable IP addresses which bind to each of the E2 interfaces of the ESG. The E2 interfaces are in the DMZ. The E1 interfaces are on the LAN side.
The flow of traffic is as follows 1. The sending email server queries the DNS MX record for the destination domain.
The DNS returns three IP addresses with the same preference value.
The order of the IP addresses are set to round robin mode.
ESGA v7.6 201 103
Clustering Details
The email server connects to the first IP address in the list (the other addresses are used if the first one does not respond).
The DNS returns the information about the next hop as well.
2. The email server connects to the next hop in the chain. 3. The messages arrive to the premise firewall. Permitted traffic is allowed through. 4. The firewall receives the incoming SMTP request and es it to the corresponding ESG appliance (using one of the two techniques listed above). 5. The layer 2 switch receives the connection request for the address of the selected appliance. This appliance receives and processes the email message. If the policy filter allows the email through, the appliance sends the email to the email server. The email server receives the email that has ed filtering, which is now available for pickup or delivery by the end use.
104 ESGA v7.6 201
Clustering Details
Slide 7:4
Centralized Management Under cluster mode, the primary and all secondary nodes are managed centrally. 1. The log on TRITON Unified Security Center to configure the primary node. 2. The configuration of the primary is synchronized to all secondaries. Note Only the primary has read/write privilege for ESG configuration. 3. The primary and all secondaries of the cluster performs process inbound and outbound emails based on the same policies. 4. All off-box logs generated from different ESG nodes will be sent to same log server and stored to same database. Note Only the primary has privilege to view ESG dashboard/ logs and generates reporting, the scope of reporting data covers all ESG nodes in cluster.
ESGA v7.6 201 105
Clustering Details
Note The secondary only has the privilege to filter emails and report everything to the primary. When the primary is down, all secondaries will keep working and filtering as secondaries; no one will switch to the primary automatically. can reconfigure the cluster to pick a secondary to act as the new the primary via TRITON Console. As the configuration is synchronized among the cluster nodes, the new cluster will work in the same manner. The only lost is the emails in primary work queue, which can not be recovered. When a secondary appliance is down, the cluster is not affected. Cluster machines continue to process mail traffic and share a configuration.
106 ESGA v7.6 201
Clustering Details
Slide 7:5
Standard Deployment A load balance device is deployed in front of ESG cluster. It redirects mail traffic to different ESG nodes with certain traffic balancing algorithm. This needs to be done by customer when ESG is deployed as cluster mode.
ESGA v7.6 201 107
Clustering Details
Slide 7:6
Configuration Synchronization Configuration synchronization is handled by Config Daemon. The following tasks are synchronized:
Configuration: All updates in configuration will be propagated from the primary to secondaries.
Log: In-box logs are synchronized from secondaries to the primary; off-box logs are saved on the same external systems, no need to synchronize.
Quarantine: All quarantine related database and files are synchronized from secondaries to the primary.
DSS registration: The primary s with DSS Manager; DSS registration is synchronized to secondaries.
Not to be synchronized (only on the primary)
Signature database update
Hot fix/patch update
Hybrid service registration
108 ESGA v7.6 201
Clustering Details
Note Only when the primary has the same version and platform, and run in the same mode with the secondary, configuration will be synchronized. Otherwise the node status would be “not synchronized” with an alert being sent.
License synchronization The same license can be imported to each node of the cluster before negotiating, and the license can be imported after negotiating too: to the primary at first then synchronized to all the secondaries automatically. Customers must assure licenses of each cluster node are either same or empty. If different licenses are imported to the nodes of a cluster, the cluster negotiation will fail.
ESGA v7.6 201 109
Clustering Details
Slide 7:7
Hybrid Service and DSS Integration Hybrid registration only happens on the primary and is not synchronized with the secondaries. Hybrid registration result is synchronized to the secondary. When the primary is down, the secondaries still process Hybrid mail traffic. DSS registration happens on the primary and is synchronized to the secondaries. ESG cluster works in below way:
The primary s with the DSS manager.
The primary and secondaries point to same DSS manager and log & reporting server
All configuration except cluster related configuration (IP, primary/secondary) are synced from primary to all secondary appliances.
Inbound traffic: traffic will be load balanced by MX record or external L7 load balancer.
Outbound traffic: DLP scanning is on secondaries locally. But if the mail quarantine is required, it will send the message to primary DSS manager for processing.
110 ESGA v7.6 201
Clustering Details
Slide 7:8
PEM Under cluster mode, the end can log on either the primary or secondaries to utilize PEM functions. ESG cluster provides 2 options to deploy PEM:
End s log on PEM of the primary only: The primary will redirect to the relative node for PEM, which is transparent for end s.
End s log on the specific ESG nodes for PEM: End s are assigned to different PEM address, which point to respective nodes in cluster.
The slide takes the first option as an example. The process of retrieving isolated messages via PEM proceeds as follows: 1. ESG #1 (not the primary) receives a message, which is going to be isolated. 2. The ESG #1 writes an entry in the log about this message. As mentioned before, a key field in the database is the ID of the ESG who isolated the message. 3. The recipient, which has been notified that he/she has received an message being isolated, connects to the PEM of the primary node. The link in the notification points to the TRITON - Email Security. 4. The primary ESG looks up the message details in the log database. Using the ID of the machine, which isolated the message, can connect to that ESG (in this case ESG #1)
ESGA v7.6 201 111
Clustering Details
5. The primary ESG communicates with the ESG #1, retrieves the message, and sends it over to the who requested it.
112 ESGA v7.6 201
8
Archive
As businesses develop, many organizations face the email issues in storage, discovery, and retention: Storage Management: Quota limits on mailboxes frustrate end s- yet unlimited mailbox storage is costly for IT to provide and will only get worse as email traffic is expected to grow to 507 billion messages per day by March 2013. Discovery: 87% of U.S. companies faced law suits and 56% of companies have been ordered to produce employee email by a court or regulatory body- yet only one in seven organizations can recover email that is older than one year. Retention and Compliance: Many laws and regulations require retention of email for extended periods of time. Only 31% of organizations are moderately prepared to comply.
Websense Email Archive is an add-on feature to ESGA. It is a full software-as-aservice (SaaS) email archiving solution that integrates with email servers to accumulate email messages and provides quick, searchable access to s. It features comprehensive e-discovery and compliance capabilities, which s can setup for select s to access within the system.
ESG has the basic archive feature: saving all incoming messages to an archive queue locally or remotely. This feature is different with the add-on archie that we are going to talk about in this chapter.
ESGA v7.6 201 113
Archive
Slide 8:1
Websense Email Archive Solution Add-Ons Websense offers three email archiving add-ons that address unique challenges:
Websense Email Archive: Provides end s unlimited mailbox size and allows them to quickly and easily access their personal archived email.
Websense Discovery Archive: Provides Legal counsel/HR/s the ability to investigate, search, tag, and export email across the entire organization to satisfy discovery requests and email retention.
Websense Email Continuity: Provides end s/IT with “standby mailboxes”, enabling them to send and receive email when their company’s email server is unavailable as if it were online.
The three add-ons are available for all Websense email security products (ESGA, ESG, HES, WES). Customers must have one of the following products to purchase email archive add-ons:
Websense Email Security Gateway Anywhere
Websense Email Security Gateway
Websense Hosted Email Security
Websense Email Security
114 ESGA v7.6 201
Archive
Slide 8:2
Email Archive Package and License Email Archive Package Each email archive add-on could be purchased individually or together in various package combinations:
Websense Email Archive Suite: Websense Email Archive + Websense Discovery Archive
Websense Email Archive with Hosted Continuity: Websense Email Archive + Websense Email Continuity
Websense Email Archive Suite with Hosted Continuity: Websense Email Archive + Discovery Archive + Email Continuity
ESGA v7.6 201 115
Archive
Websense Email Archive Solutions add-ons are available as a service in the cloud and does not require the cost and complexity of any on-premise equipment. Note Websense Email Continuity cannot be orderable as an individual add-on. It must be combined with Websense Email Archive (Websense Email Archive with Hosted Continuity or Websense Email Archive Suite with Hosted Continuity).
License For each email archive add-on, customers purchase a subscription license based on the number of mailboxes/s. Flat rate pricing for all bands. The number of license s for Websense Email Archive Solutions doesn’t need to match the number of s for the Websense email security product. Customer could purchase a subset of s. For example, customers may purchase licenses for a defined group of internal s (i.e. HR, Legal) rather than for the entire email population. The minimum number of s/mailboxes that could be purchased is 25.
116 ESGA v7.6 201
Archive
Slide 8:3
Websense Email Archive - Personal Archive The slide above shows the key features of Websense Email Archive: 1. Roles Management: Websense Personal Archive s any number of customized roles, each with a unique and customizable set of permissions associated with them.
: All new s of the archive can be granted access to their own Personal Archive which will give them the ability to search their own historical emails.
Manager: This role can be customized to set up new s, reset s, and have additional privileges.
Policy Manager: The Policy Manager can view and update policy settings (e.g., set time zones, specify storage of emails based on email direction).
Role Manager: The Role Manager can view and modify privileges of other s.
: The role has all the above/available permissions.
2. to the Personal Archive
ESGA v7.6 201 117
Archive
s can access their archives from Microsoft Outlook, IBM Lotus Notes, Outlook Web Access, Lotus Notes Web Client, BlackBerry devices and through a browser-based, secure website, which is a convenient option for employees working from home or on the go.
s only have to enter the names and s one time when opening Personal Archive through Outlook. After one successful , they will automatically be logged in on all subsequent visits to the archive.
3. Outlook Look and Feel In addition to giving s the power to restore lost or deleted emails back to their inboxes, Email Archive also allows them to compose, reply to and forward messages directly from their archives, just like they would in Outlook or Notes. 4. Quick Search With Quick Search, simply typing desired search into the Message field and clicking on the magnifying glass icon will begin the search. The search results appear in a list of messages, and the search are highlighted in yellow in the preview pane. 5. Advanced search
The Advanced Search option gives the s the ability to customize their searches based on a variety of criterion, such as message keywords, to, from, subject, date(s), attachments, etc.
Search multiple archives directly from Outlook, enabling better knowledge management and retention of archives from departed employees.
6. Search Filter Search Filters help further hone searches, allowing s to drill into specific categories, such as sender, date, tags and attachments.
Websense Email Archive benefits:
Unlimited Mailbox Websense Personal Archive gives an unlimited mailbox and allows s to quickly and easily access their archived emails directly from Microsoft Outlook or Lotus Notes. For added convenience, Folder Sync keeps s’ Outlook folders intact in the archive for easier search and retrieval. Restoring lost or deleted emails gives end s the power to restore lost emails themselves- even things they may have deleted from their desktop, laptop, BlackBerry® device or Windows Phone 7. Reducing the need for PSTs/NSFs eliminates free-range email files (PST/NSF), and minimizes the burden on IT staffs and/or help desk Improving server performance improves performance by reducing the footprint on customers’ mail servers, and shrink backup windows
118 ESGA v7.6 201
Archive
Boost Productivity Do you know how much time people spend sorting and filing their emails? Almost 9 out of 10 Internet s spend seven hours a week managing email. That’s a lot of lost productivity! Plus, 95% of the email sitting in your inbox will never be looked at again. With the Email Archive, s can delete email with confidence and stop wasting time managing the inbox.
Familiar, Easy-to-Use Interface Email Archive seamlessly integrates with Outlook or Notes, so it’s easy to complete the most common day-to-day tasks, using a variety of familiar shortcut buttons.
Comprehensive Search Features Email Archive makes retrieving old or deleted email easy. s can search for specific emails and attachments two ways: Quick Search and Advanced Search. The Advanced Search feature allows s to fine-tune their searches with key parameters. The Search Filters help further hone searches. s can also save their searches for future reference (which means they don’t have to keep recreating their most common searches).
Eliminate Local Archives Removing local archives helps reclaim space on s’ shared drives, reduces data leak protection issues when laptops are lost or stolen and eliminates the hassle involved in searching individual desktops when they are responding to a legal discovery request.
ESGA v7.6 201 119
Archive
Slide 8:4
Websense Discovery Archive Motions to discover electronic data are commonplace in today’s world of litigation; yet, more often than not, companies are unprepared to find electronically stored information, especially email and attachments. Discovery costs represent at least 50% of the cost of litigation- most of which relates to the collection and review of email. Websense Discovery Archive is a complete and affordable Security-as-a-Service (SaaS) solution designed to seamlessly manage the e-discovery needs. Our fully managed, web-based service meets and exceeds the e-discovery and mailbox management needs of businesses and public organizations of all sizes. Discovery Archive also helps significantly reduce the time it takes to respond to e-discovery requests.
Store and index every email, attachment, IM and BlackBerry® message (SMS text, PIN-to-PIN, call log) in a centralized, online, and tamper-proof repository.
Accelerate legal discovery and HR inquiries with real-time search and retrieval and multiple export capabilities. Search the entire contents of archived emails and attachments.
Enforce email preservation for lawsuits/legal holds, and protect attorney-client privileged communications.
120 ESGA v7.6 201
Archive
Websense Discovery Archive Features Store & Index
Discription
Secure and Automated Message Capture
Discovery Archive automatically captures all messages that are sent or received without any intervention and securely transmits them using TLS or VPN encryption to the LiveOffice data centers for retention.
Single-Instance Storage
Once archived, all email is de-duplicated and stored as a single instance to optimize storage space and streamline the searches.
Unlimited Storage
Unlimited storage eliminates the need for mailbox quotas and compounding storage costs.
Replicated Storage
All email messages and attachments are stored in real time and then replicated to multiple, geographically dispersed top-tier data centers.
Secure Storage
Websense enterprise-class infrastructure meets the highest industry standards for secure data storage, including SAS-70 Type II, and guarantees the authenticity of the archived data.
Tamper-Proof Storage (Optional)
For companies with stricter compliance requirements, WORM (write once, read many) storage is also available.
Search & Retrieval
Description
Lightning-Fast Search and Retrieval
Quickly search the entire contents of archived emails and attachments using a variety of search criteria, including to, from, date, subject, message body, message attachments and other message properties.
Online Review and Tagging
Reviewers can efficiently navigate through search results, identify highlighted search and tag potentially harmful emails, so they are easily retrievable for further review.
PST/NSF File Export
Once s have tagged all emails related to a specific case or matter, they can use Discovery Archive’s export feature to load the files into a case management solution or other application for further review and analysis.
EDRM XML Export
Approved IT and legal personnel can seamlessly export messages from Websense Discovery Archive to e-discovery solutions from Clearwell Systems, CT Summation, FTI Technology and iConect.
ESGA v7.6 201 121
Archive
Search & Retrieval
Description
Re-Forward or Print
Reviewers have the ability to print or re-forward suspect emails to the appropriate parties for further review and case submission.
Saved Searches
Reviewers can create and save customized email searches based on the organization’s email policies, and re-run them as necessary.
Policy Alerts
Reviewers can setup Policy Alerts to notify them when an email meets Saved Search criteria (e.g., contains specific words or phrases).
Access & Control
Description
Familiar Interface with Full Outlook and Notes Integration
Discovery Archive’s highly intuitive, Microsoft Outlook-like interface helps s get up to speed quickly. In addition, full Outlook and Notes integration allows s to centrally deploy Websense Personal Archive (optional addon service) to some or all s.
Authentication and Granular Permissions
s authenticate using SSL to , ensuring secure access to the archive. Discovery Archive has two distinct permission levels: s (full access) and Reviewers (monitoring rights for selected mailboxes).
Policy-Driven Retention and Destruction
Discovery Archive retains the emails and attachments as required to meet customer’s company’s specific retention or regulatory requirements. We also purge and destroy older email stores based on the custom criteria (e.g., any emails five or more years old).
Automated Legal Hold Enforcement
If customer’s company is in pending litigation, they can place legal holds on specific communications (based upon search criteria) to safeguard the staff or automated deletion policies from inadvertently deleting case-relevant emails.
Attorney-Client Privilege Tagging
s and reviewers can flag attorneyclient privileged communications, which are excluded from e-discovery requests.
Full Audit Trail
Discovery Archive’s search log captures the auditing activities of the reviewers, so s can ensure that they are conducting appropriate reviews and meeting compliance requirements.
122 ESGA v7.6 201
Archive
Access & Control
Description
Groups
s have the ability to group s based on functional, divisional or custom criteria. Reviewers can then search across these groups.
Mail Servers
Discovery Archive s Microsoft Exchange 2003, Microsoft Exchange 2007, Microsoft Exchange 2010, IBM Lotus Domino 6.x – 8.x, Novell GroupWise and Linux/Unix mail servers (SMTP).
ESGA v7.6 201 123
Archive
Slide 8:5
Websense Email Continuity Websense Email Continuity is an always-on backup that provides customers with “standby mailboxes”, enabling them to send and receive email when the primary server is unavailable as if it were online. Email Continuity allows the IT resources to focus on the root cause of the outage instead of wasting time implementing temporary solutions (e.g. MX repoints, etc.).
Serves as “standby” mailbox when the primary email server is down
s can easily create, reply to and forward messages during outage with Outlook Look and Feel
Email Restore s Microsoft Exchange, IBM Lotus Notes and Novell Groupwise
All emails sent or received during the outage are automatically routed back to the email server after the outage
30-days of email history
Overcome email server breakdowns, stay productive and protect customer’s business with Websense Email Continuity.
124 ESGA v7.6 201
Archive
Slide 8:6
Inbound Email Flow The slide above describes the inbound email flow with Websense Email Archive. 1. During normal operation The inbound messages are transmitted via the Internet, filtered for spam and virus by ESGA, routed to the email server and distributed to the recipients. At the same time, the inbound messages are captured in the Websense Email Archive via journaling. End s can access the these messages through Outlook, Lotus Notes, or web-based archive interface anytime. 2. During the email server outage Email messages are transmitted via the Internet, filtered for spam and virus by ESGA and automatically routed to the Email Continuity archive, bying the failed email server. End s can access the messages that are sent before and during the outage. Once the email server is restored, messages are routed from the archive back to the email server (no duplicate messages are stored).
ESGA v7.6 201 125
Archive
Slide 8:7
Outbound Email Flow The slide above describes the outbound email flow with Websense Email Archive. 1. During normal operation The outbound messages are sent from end s, routed to the email server, filtered for virus and confidential data leakage by ESGA, and transmitted over the Internet. At the same time, the outbound messages are captured in the Websense Email Archive via journaling. End s can access the these messages through Outlook, Lotus Notes, or web-based archive interface anytime. 2. During the email server outage End s compose, reply to and forward messages directly from the Email Continuity interface. The messages are then scanned for viruses and confidential data leakage by ESGA prior to being transmitted over the Internet.
126 ESGA v7.6 201
Archive
Slide 8:8
Websense Ingestion Service The diagram above illustrates the Websense ingestion service that help to imgrate the legacy data from customer premises to Websense Email Archive server and preserve chain of custody at the same time.
Legacy Data Legacy data is a generally referred to as information stored in an old or obsolete format (or computer system) that is, therefore, difficult to access or process. In the email archiving world, legacy data refers to email data sitting on:
Email storages (e.g., Microsoft Exchange, Lotus Notes) on servers
Personal archives (e.g., PST/NSF files) on desktops, laptops and servers
Legacy email archives (e.g., Autonomy, EMC, CA)
Backup tapes
By ingesting your legacy email into Websense’s archiving solutions, you have a complete, living record of all your email history, both past and present for that ingested email. But, it is critical to preserve the chain of custody when you move your email data from one store to another.
ESGA v7.6 201 127
Archive
Chain of Custody If “chain of custody” is a new or a vague concept to you, you are not alone. In the simplest , a proper chain of custody establishes the integrity of a piece of evidence, showing that it wasn’t tampered with or otherwise altered since it was first collected. Chain of custody has significant practical implications for IT managers and other professionals. Chain of custody plays a critical function in litigation, especially when opposing counsel is challenging the authenticity of evidence – particularly where digital evidence and emails are involved. We’ve found that very few organizations adequately for chain of custody or understand its importance until the authenticity is called into question.
Two Optional of Ingestion Services Websense Email Archive solutions do not require implementation/professional services. However, there are 2 optional services that are available:
Websense Data Ingestion Service: Service generally purchased up front to import all legacy email data from desktops, laptops, servers, into the Websense archive solution.
Websense Data Export Service: Service generally purchased when a customer terminates their Websense archive subscription so that they can re-capture the data stored in our cloud.
The services are one-time fee based on a per GB basis. They aren’t performed at customers’ sites. Customers merely ship all storage media desired to be archived to Websense. Websense will then import/export the data.
Websense Data Ingestion Process As the illustration above depicts, the Websense Data Ingestion Service process involves 6 key steps: 1. Extraction In the extraction phase, the customer extracts legacy email data from existing repositories. This is a critical first step and one that provides the greatest opportunity to break the chain of custody. So, it’s important to document the number of original files/PSTs/NSFs sitting in customer’s existing stores in order to compare to the total number of files/PSTs/NSFs imported into the archive.
128 ESGA v7.6 201
Archive
The following table lists the descriptions for the ed formats:
Description
PST
• • • •
EML
• Does not preserve folder structure
MSG
• Longer conversion and ingestion time • Does not preserve folder structure
NSF
• Ability to preserve folder structure upon request • Longer conversion and ingestion time • Additional cost
Files need to be 2 GB or less in size PST files cannot be protected Files can be zipped with a Ability to preserve folder structure upon request
Physical Media: • •
Format
If the volume of data is less than 10 GB, send the data on CD/DVD, Flash Drive or via S-FTP (secure FTP). If the data volume is greater than 10 GB, send the data via a USB hard drive (high speed USB 2.0 with external power recommended).
Encryption: Websense requires a basic level of encryption (i.e., protected zip file) for all email data transferred and strongly recommends using TrueCrypt Freeware (truecrypt.org) or PGP (pgp.com) to encrypt data before shipping it to us as a best practice. Zipping the data with a is also acceptable, but less desirable, since it delays the import. We do not accept -protected PSTs, since they are generally not secure and cannot be processed by our legacy system.
2. Shipment During this phase of the project, Websense outlines the key steps for the customer project. This includes having the customer complete our Legacy Order Form, which captures key chain of custody information, such as the date and time sent, number of files, number of emails, format of legacy data (e.g., PSTs), amount of data and information. Then the customer ships the data along with the Legacy Order Form to us, addressed to the attention of the Websense Data Import Center. 3. Verification Upon receipt of the data, we log, initial and date the Chain of Custody Form (part of the Legacy Order Form) under received data. The case is then assigned to one of our specialists, who reviews the details and adds the customer project to our schedule. The completion date is based on the current volume of legacy
ESGA v7.6 201 129
Archive
s in our queue. In the meantime, Websense places the data in its current format in our data vault for safe storage. After the data is extracted and ed, the specialist from the customer verifies that the data volumes match the information they provided on the Chain of Custody Form and updates the customer any deviations. If there is a material difference in the email received/extracted vs. the data on the form, we then explore the reasons for the missing data. Note If the data does not match the Chain of Custody Form or the data is corrupt, the customer will be notified to inform our findings and determine the next steps. 4. Data Import After the legacy data is prepped, the specialist from the customer imports it into the archive. Websense securely captures and indexes all messages (and attachments) imported into one of our archiving solutions – Discovery Archive or Email Archive. Once indexed, s and s can quickly and easily search the entire contents of archived emails and attachments, using a variety of search criteria, including to, from, date, subject, message body, message attachments and other message properties. As a final check, the specialist from the customer verifies the data and signs off on the Chain of Custody Form. 5. Chain of Custody Documentation Once the import is complete the customer will be notified with the results. Any deviations from the original Chain of Custody Form prior to ingestion are noted and reviewed. 6. Returned Data/Destroyed Data The final step in the process involves Websense either returning the original data to the customer or destroying it. In either case, we document the process and note the date the media was either returned or destroyed per the customer wishes and retain the final copy of the Chain of Custody Form in our locked data vault for records.
Key Points About Websense Data Ingestion Service
Use of FTP: Websense recommends using the FTP option when shipping data for only small data sets (i.e., less than 10 GB).
ed Formats: Websense s a variety of email formats, including PST, EML, MSG, NSF and MBOX. PST is the required format if preserving the folder structure for the data that is imported.
130 ESGA v7.6 201
Archive
File Sizes: Websense can process PSTs up to 20 GB, but we prefer files that are 2 GB or less. Smaller PSTs have less chance of corruption and process faster.
Media Types: Though we accept DVDs, Websense recommends physical hard drives or flash drives when the customer ships data to us.
Encryption: While Websense requires a basic level of encryption for email data shipped to us, we do not accept -protected PSTs, since they are generally not secure and cannot be processed by our legacy system.
Mapping Archiving s: When we import the PST files, we first scan them to determine how to map the customer s to individual archive s. If a sender’s email address is found in more than 80 percent of the messages in the PST file, the emails are archived under the sender’s address. This process, however, is error prone, so we recommend providing us with a CSV file for mapping purposes. If a CSV file is not provided, the system automatically determines the mapping.
ed Message Types: We currently only archive post items. Non-post items, such as calendars, s, system messages and voicemails, are not currently ed.
Legacy Unassigned Archive: If we cannot match an email address or common name to an archived , we place the emails in the Unassigned Legacy Archive. Emails in the Unassigned Legacy Archive must have a common name (unmatched) or an email address with a recognized domain in the archived .
Corrupted Data: Before sending us data, we encourage the customer to run scanpst.exe, a utility included with Microsoft Outlook, to determine whether or not the PST file is corrupted. We cannot import corrupted PST files.
ESGA v7.6 201 131
Archive
Slide 8:9
Websense Email Archive - Folder Sync Folder Sync is an add-on service to Websense Personal Archive. As end s create, delete, and rename folders, as well as move email messages from one Outlook folder to another, the scheduled synchronization service replicates the identical folders and message location in Personal Archive. After initial synchronization, Folder Sync sends only incremental mailbox folder changes, which can be scheduled hourly, daily or weekly as s see fit. Folder Sync does not require plug-ins or hardware. It is compatible with Microsoft Exchange Server 2003, 2007 and 2010. s can selectively deploy Folder Sync on a per- and/or Exchange Server basis.
132 ESGA v7.6 201
Archive
Slide 8:10
Quick Start - Configuration on Exchange Server You need to activate Microsoft Exchange Journaling and direct emails to the Websense Email Archive servers. As a result, every email that is sent to and from the exchange server will be copied and transfered to the Websense Email Archive servers. 1. Log on to the exchange server with . 2. Follow the steps outlined in journaling guides, available on our website. The instructions depend on the version of Exchange. Exchange 2003 Standard Journaling Guide
http://www.websense.com/content// library/email/shared/exchjour_2003_std/ first.aspx
Exchange 2003 Envelope Journaling Guide
http://www.websense.com/content// library/email/shared/exchjour_2003_env/ first.aspx
Exchange 2007 Journaling Guide
http://www.websense.com/content// library/email/shared/exchjour_2007/first.aspx
Exchange 2010 Journaling Guide
http://www.websense.com/content// library/email/shared/exchjour_2010/first.aspx
ESGA v7.6 201 133
Archive
3. When prompted, point the email to the Websense address shown in the table.
134 ESGA v7.6 201
EMEA servers
U.S. and others
Archive
Slide 8:11
Quick Start - Configuration on Websense Archive Server Enabling view of archive As a security measure, the default settings on the archive disallow view of end email. To view all messages in the archive, perform as follows: 1. Log on with the credentials provided by Websense. 2. Click the istration bar at bottom left of the page. 3. Click the “@...” address. 4. Under the Privilege option, check View All s.
ing the setup After completing the setup process, test the archive as the follows: 1. Log on with the credentials provided by WEbsense. 2. Click s on the left-hand side of the screen, and select the email address of your choice. 3. To explore the features of archive, click Help on the left side of the page.
ESGA v7.6 201 135
Archive
Slide 8:12
Quick Start - Configuration on Client The end s can use Personal Archive Access (PAA) to access their individual archives directly from Microsoft Outlook. Provide the setup instructions to the end s: 1. Create a new folder in Outlook (File > New > Folder). 2. Highlight the new folder, and select File > Folder > Properties. 3. Select the Home Page tab. 4. Enter the URL for Personal Archive Access: EMEA
https://websense.archivesuite.com
U.S and others
https://websense.archivecloud.net
5. Mark the box Show home page by default for this folder. 6. Click OK. 7. Click the newly created folder. After the Personal Archive is configured, end s are prompted only once for their name and . Additional details for the end s are contained here:
136 ESGA v7.6 201
Archive
http://www.websense.com/content//library/email/shared/email_archive/ WEA_end__guide.pdf Note Websense offers Websense CloudLink, an application to help to deploy Outlook Web folders to all of the end s. CloudLink is included with the Websense service.
ESGA v7.6 201 137
Archive
Slide 8:13
Archive vs Backup Backups are for disaster recovery, while archives are for retention and discovery. Backups were never really intended to meet regulatory requirements and other compliance needs. They most effectively serve as a short-term insurance policy to facilitate disaster recovery (assuming they are kept off-site). Archiving, on the other hand, is specifically designed to quickly and easily meet regulatory requirements and other compliance needs. In addition, archiving serves another important role. It can reduce the strain on customer’s in-house email servers. With growing email volumes and skyrocketing storage costs, email archiving can:
Reduce the size of email stores
Reduce backup windows
Reduce the amount of IT budget earmarked for storage
For most modern organizations, the operational, legal and compliance challenges of email necessitate going beyond simple backups.
138 ESGA v7.6 201
9
ESG Log and Reporting
ESG logs and collects data about email traffic and system activity to generate presentation reports and dashboard charts and statistics for today and history. Records of email filtering activities and system events are stored either in the on-box database, that is, local PostgreSQL, or received by the off-box log server and then transferred to the remote database, Microsoft SQL Server. From TRITON - Email Security, ESG classifies the logs into: message log, audit log, system log and console log; and catalogs the reports into: today dashboard, history dashboard, and presentation report. ESG log and reporting provide the facilities for s to configure the local and remote log database and the database maintenance options, set report preferences, and customize a report template, schedule and run a presentation report. Now let’s look at log and reporting in more details!
ESGA v7.6 201 139
ESG Log and Reporting
Slide 9:1
Primary Concepts ESG logs
Message log records information about each inbound message received by ESG. Message logs are collected from the ESG internal component: Filter Demon.
Audit log, only available for super s, shows which s have accessed the Email Security module of TRITON console, as well as any changes made to policies and settings. Audit logs are collected from the ESG internal component: Config Daemon.
System log records all ESG system level events, along with any errors or warnings produced. System logs are collected from each component inside the ESG, such as the Filter Daemon, Update Daemon, Config Daemon, etc.
Console log records any activities or changes made to the Email Security module of TRITON console. Console logs are collected from TRITON Email Security, not from the ESG internal components.
ESG logs are searchable by predefined time periods or customized conditions. For message logs, you can refine the search using conditions like email address, scanning
140 ESGA v7.6 201
ESG Log and Reporting
result, or message status and export the search results to an Excel, HTML, or XML file. For other three types of log, you can exported to an Excel or HTML file. Note The maximum number of log entries exported cannot be greater than 100,000.
ESG reporting
Today dashboard provides high level summary statistics of today to identify potential security and email flow and delivery issues with high quality and interactive graphics.
History dashboard of last 30 days, similar with today dashboard, get an overview of email scanning activity for up to the past 30 days.
Presentation reports include a set of predefined charts and tabular report templates with which you can generate graphical reports of Email Security Gateway message traffic activities.
ESGA v7.6 201 141
ESG Log and Reporting
Slide 9:2
Log and Reporting System Architecture The slide displays the main components of ESG log and reporting system. The log system is built in ESG, including log collection, on-box log database, and Log Daemon. The log system collects logs from ESG internal components then stores the logs on-box or off-box. The reporting system is composed of Log Server and MS SQL Server, which is similar with WSG reporting system. 1. A collection of raw data, in the form of logs from various ESG internal components, are fed into the Log Daemon. 2. On-box logs are transferred to PostgreSQL. 3. Off-box logs are transferred to the Log Server. 4. If the Log Server is unreachable, a Cache Log is utilized. Once the logs are in MS SQL Server, it utilizes an Extract/Transform/Load (ETL) process in every 20 seconds to convert original logs to reporting data. 5. Stored procedures are utilized to generate reports and dashboards via the WebUI. As you will see, the ESG WebUI is very similar to WSG Reporting UI.
142 ESGA v7.6 201
ESG Log and Reporting
Slide 9:3
Message Log The Message Log records information about each inbound message received by Email Security. Access the Message Log on the Main > Status > Logs page. You can configure the number of entries per log page, between 25 and 200, in the Per page drop-down list in the log table banner. Scroll through Message Log pages by clicking the back and next arrows in the banner, or enter a specific page number in the Page field and click Go. Message records are saved for 30 days. To preserve message records longer than 30 days, use the Export option to export the log on a regular basis. Exporting does not remove records from the Message Log. It transfers log data to an Excel, HTML, or XML file. When the Message Log page appears, the most recent records are shown. Use the View from/to fields to specify the date/time range for the log entries you want to see. The calendar includes the following options:
Click the time in the lower left of the calendar to set the time in hours and minutes.
Click Today to set the calendar date to today’s date.
Click Apply to activate your date/time calendar changes.
If you click Clean and then Apply, the current date/time selection is removed.
ESGA v7.6 201 143
ESG Log and Reporting
The Message cludes several search options, including date range or keyword searches. Determine the date/time range for a search by selecting dates in the View from/to field calendar controls. Default value for the from or to field is the date and time that you open the log.
144 ESGA v7.6 201
ESG Log and Reporting
Slide 9:4
Audit Log Websense Email Security provides an audit trail showing which s have accessed TRITON - Email Security, as well as any changes made to policies and settings. This information is available only to Super s. Monitoring changes through the Audit Log enables you to ensure that system is controlled responsibly and in accordance with your organization acceptable policies. Click the Audit Log tab on the Main > Status > Logs page to view the Audit Log, and to export selected portions of it to an Excel spreadsheet or an HTML file, if desired. Audit records are saved for 30 days. To preserve audit records longer than 30 days, use the Export option to export the log on a regular basis. Exporting does not remove records from the Audit Log. It transfers log data to an Excel or HTML file. When the Audit Log page opens, the most recent records are shown. Use the View drop-down list options located above the log to select the range of log entries you want to see: All, One Day, One Week, One Month, or Custom. When you select Custom, use the View from/to fields to specify the desired date/time range for the log entries you want to see. The calendar includes the following options:
Click the time in the lower left of the calendar to set the time in hours and minutes
Click Today to set the calendar date to today’s date.
Click Apply to activate your date/time calendar changes.
If you click Clean and then Apply, the current date/time selection is removed.
ESGA v7.6 201 145
ESG Log and Reporting
Slide 9:5
System Log System Log records for Email Security Gateway reflect the current state of the system, along with any errors or warnings produced. Click the System Log tab on the Main > Status > Logs page to view the System Log, and to export selected portions of it to an Excel spreadsheet or an HTML file, if desired. System Log records are saved for 30 days. To preserve System Log records longer than 30 days, use the Export option to export the log on a regular basis. Exporting does not remove records from the System Log. It transfers log data to an Excel or HTML file. When the System Log page opens, the most recent records are shown. Use the View drop-down list options located above the log to select the range of log entries you want to see: All, One Day, One Week, One Month, or Custom. When you select Custom, use the View from/to fields to specify the desired date/time range for the log entries you want to see. The calendar includes the following options:
Click the time in the lower left of the calendar to set the time in hours and minutes.
Click Today to set the calendar date to today’s date.
Click Apply to activate your date/time calendar changes.
If you click Clean and then Apply, the current date/time selection is removed.
146 ESGA v7.6 201
ESG Log and Reporting
Slide 9:6
Console Log The Console Log is a record of any activities or changes made to the Email Security module of the TRITON Unified Security Center. Click the Console Log tab on the Main > Status > Logs page to view the Console Log, and to export selected portions of it to an Excel spreadsheet or an HTML file, if desired. Console Log records are saved for 30 days. To preserve Console Log records longer than 30 days, use the Export option to export the log on a regular basis. Exporting does not remove records from the Console Log. It transfers log data to an Excel or HTML file. When the Console Log page opens, the most recent records are shown. Use the View drop-down list options located above the log to select the range of log entries you want to see: All, One Day, One Week, One Month, or Custom. When you select Custom, use the View from/to fields to specify the desired date/time range for the log entries you want to see. The calendar includes the following options:
Click the time in the lower left of the calendar to set the time in hours and minutes.
Click Today to set the calendar date to today’s date.
Click Apply to activate your date/time calendar changes.
If you click Clean and then Apply, the current date/time selection is removed.
ESGA v7.6 201 147
ESG Log and Reporting
Slide 9:7
Dashboard The Main > Status > Today: Health, Security and Value Since Midnight page appears when you first log on to TRITON - Email Security. It displays alert messages and graphical charts that show the current state of your email scanning software, focusing on email traffic activity in your network. The charts on this page cover the 24-hour period beginning at 12:01 a.m. according to the time set on the Log Database machine. At the top of the page, two summary sections provide a quick overview of current status: 1. The Health Alert Summary shows the status of your Websense software. Click an error or warning alert message to open the Alerts page, where more detailed alert information is available. Information in the Health Alert Summary is updated every 30 seconds. 2. Under Business Value, see examples of how Websense Email Security has protected your network today by blocking malicious email traffic, as well as the total number of messages handled. Below the summary information, up to four -designated Flash charts provide information about email scanning activities. These charts are available to Super s, and to other s who are granted permission to view reports on the Today page. Click Customize to select the four charts you want
148 ESGA v7.6 201
ESG Log and Reporting
displayed. Information in these charts is updated every two minutes. You may need to scroll down to see all of the charts. Up to two buttons appear at the top of the Today page: 1. Customize, available to Super s only, opens a page where you can select which charts to display on the Today page. 2. Print, available to all s, opens a secondary window with a printerfriendly version of the charts on the Today page. Use browser options to print the page.
ESGA v7.6 201 149
ESG Log and Reporting
Slide 9:8
Today Report Use the Today > Customize page to select up to 4 charts for the Status > Today page. Only Super s with unconditional policy permissions can customize the Today page.
150 ESGA v7.6 201
ESG Log and Reporting
Slide 9:9
History Report Use the Status > History: Last 30 Days page to get an overview of email scanning activity for up to the past 30 days. The four charts on the page are updated daily at 12:01 a.m. to incorporate data from the previous day, as determined by the time on the Log Database machine. You may need to scroll down to see all the charts. The exact time period covered by the charts and summary tables depends on how ESG has been processing mail. During the first month that Websense software is installed, the page shows data for the number of days since installation. After that, the reports cover the 30 days prior to today. Depending on the reporting permissions granted to the role, some s may not see the charts on the History page.
ESGA v7.6 201 151
ESG Log and Reporting
Slide 9:10
Presentation Reports Presentation reports include a set of predefined charts and tabular report templates with which you can generate graphical reports of Email Security Gateway message traffic activities. You can run a report, customize a report template, or mark a frequently used report as a Favorite. You can run any presentation report immediately, or schedule it to run at a particular time or on a repeating cycle. Not all report templates can be customized. Report templates that can be customized display a different icon from reports that cannot be customized. If the Save As button is enabled when you select a report name, then you can save and edit that report to suit your needs. The Save As button is not enabled if you select a report that cannot be customized. Use the Main > Status > Presentation Reports page to generate charts and tabular reports based on templates in the Report Catalog. The Report Catalog organizes a list of predefined report templates and custom reports into groups. Expand a group to see its corresponding templates and custom reports. Click on a template or report title to see a brief description of what it includes.
152 ESGA v7.6 201
ESG Log and Reporting
Slide 9:11
Advanced Features Log Database Option istrating the Log Database through TRITON – Email Security involves controlling many aspects of database operations, including the timing of maintenance, the conditions for creating new database partitions, and which partitions are available for reporting. Use the Settings > Reporting > Log Database page to manage Log Database operations. Database Partition Database partition is the unit to store logs. You can create up to 70 partitions, with the maximum size for each partition is 5120 Mega Bite. Keep in mind that extremely large individual partitions are not recommended. Reporting performance can slow if data is not divided into multiple, smaller partitions. When a new database partition is created, reporting is automatically enabled for the partition. Database Roll over Roll over is the process that the Log Database creates a new database partition. Use the Roll over every option to indicate whether database partitions should roll over based on size (MB) or date (weeks or months).
ESGA v7.6 201 153
ESG Log and Reporting
If the rollover begins during a busy part of the day, performance may slow during the rollover process. To avoid this possibility, some environments choose to set the automatic rollover to a long time period or large maximum size. Then, they perform regular manual rollovers to prevent the automatic rollover from occurring. Database Maintenance Use the Maintenance Configuration to control certain aspects of database processing, such as the time for running the database maintenance job, some of the maintenance tasks performed, and deletion of database partitions and error logs.
Presentation Reports Drill down With Presentation Reports, you can drill down into the underlying data of detailed events, such as identifying the top 10 external addresses for outbound messages, sorted by number of messages in a scheduled time range. Presentation Reports provides comprehensive report catalogs covering from inbound and outbound messages, hybrid and appliance, spam, virus and data security, message transfer and system capacity. And it offers the flexibilities in defining the report time schedule. You can get the expected reports through selecting the recurrence pattern, scheduling time and data range, and specifying the output format.
The following are typical presentation reports available in ESG: Overall message summary
Inbound Messages
Incoming message summary
Top Destination Domains by Message Size
Top N Internal Recipients by Volume
Outgoing message summary
Top Destination Domains by Message Size
Top Destination Domains by Volume
Data security summary
Top Data Security Policies
Spam and virus summary
Spam Quartile Chart
Top Virus Types
Message transfer summary
154 ESGA v7.6 201
ESG Log and Reporting
Delivery Failure
Encryption Gateway Routing
System capacity
Average message in work queue
ESGA v7.6 201 155
ESG Log and Reporting
156 ESGA v7.6 201
10
Troubleshooting Guide
This chapter talks about some basic troubleshooting techniques when working with your ESGA deployments. The following topics are covered: The latter sections will cover various areas of an ESGA implementation and discuss issues in a question and answer format.
General Information: establishes a baseline to ensure that certain hardware and software requirements are met.
License: explains the license expiration, counting, and reset issues.
ESG: tells about the possible errors when connecting the TRITON - Email Security to the ESG and discusses potential reasons.
Log Server & Log Database: covers the Log Server and the Log Database installation and troubleshooting methods.
Cluster: discusses the problems that may arise during the cluster implementation.
Hybrid: involves Q&A about the Hybrid connection, debug, and synchronization.
DLP: talks about the issues of DLP registration, DLP deployment, and policies.
Quarantine: includes the quarantine files location and quarantine Q&A.
System Counter: provides the detail descriptions about system counters.
Appendix: includes the references to ESG Shell, System Diagnosis, and Health Daemon.
ESGA v7.6 201 157
Troubleshooting Guide
Slide 10:1
General Information Before we start investigating the various areas of an ESGA implementation and discuss various troubleshooting techniques, it is important to discuss certain installation specifications. The table above provides minimum requirements for each module. The information below provides more detail surrounding some of the specific items such as database versions and the location of certain log files.
Database version Web (WSG) only:
SQL Server 2005, all editions except Express, all service packs, 32-bit and 64-bit, not IA64
All products:
SQL Server 2008, all editions except Web, Express and Compact, all service packs, 32-bit and 64-bit, not IA64
SQL Server 2008 R2, all editions except Web and Compact, all service packs, 32bit and 64-bit, not IA64
SQL Server Clustering, for all versions of SQL Server that we
158 ESGA v7.6 201
Troubleshooting Guide
Slide 10:2
Troubleshooting From the ESG appliance command line, you can perform the initial configuration, modify settings, check appliance status, start/stop the modules via the commands like firstboot, ip dns, module start, etc. Run help to acquire the command descriptions that the ESG appliance provids. Furthermore, you can enalbe SSH to connect with the ESG shell to check logs and turn on debug mode. This is an effective way for troubleshooting and often used by Technical .
A: Enable SSH 1. From the ESG appliance console, enter the following command to enable SSH access: ssh enable 2. Enter the following command to get a temporary code: local-access
ESGA v7.6 201 159
Troubleshooting Guide
3. From a Windows computer, open Internet Explorer and browse to http:// code.websense.com. Note This website is only accessible within the Websense intranet. 4. Input the temporary code and click Submit me! to convert it to a real SSH . 5. Back to the ESG appliance console, as websense-ts and enter the obtained in above step. 6. Run the command su and the is websense#123. 7. Run the below command to access the ESG shell: ssh esg
B: Log Files Location
Installation log: /var/log/esg_install.log
SSL Proxy log: /var/log/sslproxy.log
ESG statistic log: /var/logs
daily_counter: daily statistics since installation
counter_current: current days statistics
DLP logs: /opt/websense/PolicyEngine/Logs
Default ESG error log:
/var/log/maillog
/var/log/messages
ESG configuration processing log: /var/log/confd.log
ESG Log Server log:
To check the database issues during ESG Log Server installation: $(installpath)\ESG Log Server\SQL\CreateDbInstall.log and *.txt output files
To check the ESG Log Server issues: Windows system event view
C: Turn on Debug Mode 1. From the ESG shell, enter the following command: cd /etc/debug 2. Run the command below: ls The available modules, e.g. filter, scan_engine, mta, etc. are listed here. 3. Run the following command: esgshell
160 ESGA v7.6 201
Troubleshooting Guide
4. Run the following command to debug: debug module <module_name> event all For example, to debug the scan_engine module, enter: debug module scan_engine event all 5. Control the debug output location: a. For standard output, enter: debug output cli b. For file output, enter: debug file <path> <max_size> (wrap | nowrap) path
If the path is a relative path, the file will be placed in the “/var/log” folder.
max_size
Value: 1 - 256 Unit: MB
wrap | nowrap
To wraparound the file or not.
ESGA v7.6 201 161
Troubleshooting Guide
Slide 10:3
Troubleshooting - Example The slide is an example to troubleshooting via SSH (ESG shell). As the diagram shows, the message with confidential data shall be dropped based on the DLP policy; while it is delivered to the destination directly. The troubleshoots the issue in the following ways:
Via TRITON Unified Security Center. Check logs, policies, and configurations.
Via SSH a. to the ESG shell b. Check the log files c. Turn on Debug mode to track the filtering and delivering process.
162 ESGA v7.6 201
Troubleshooting Guide
Slide 10:4
License Issue A: What is the behavior when the ESG license is exceeded? All features in a license key have the same expiration date with the key. E.g, if a key has three features, AV/AS/DLP, and will expire by Dec 22, 2012, AV/AS/DLP will expire on Dec 22, 2012 too. Even if customer adds another feature before Dec 22, 2012, the expiration date is still Dec 22, 2012. UpdateD considers a key expired when DDS server returns the expired flag, which is the unique way to judge the key’s validation. So it’s useless to change system time. ESG sends messages to ESG alert system according to the remaining days in service period:
Message level in 7~30 days
Warning level in 0~7 days
Critical level after expired
Issue B: How are licenses counted? ESG counts the licenses in this way: the total number of all the outbound email senders in 7 days.
ESGA v7.6 201 163
Troubleshooting Guide
Issue C: How do view the current licensed count? From the ESG Shell, enter the following command: counter | grep subscriber
Issue D: Is there a way to reset the counter? From the ESG Shell, enter the following command: counter –f
164 ESGA v7.6 201
Troubleshooting Guide
Slide 10:5
License Reset You may need to perform an ESG subscription key reset after you have uninstalled a fully licensed version of TRITON - Email Security, and then reinstalled the product. An error message warns you that ESG is already licensed.
From the V-Series Appliance Manager 1. to the Appliance Manager: https://Appliance_C_IP:9447/appmng/. 2. In the > Toolbox page, click Launch Utility in the Command Line Utility section. 3. Select Websense Email Security Gateway in the Module drop-down list. 4. Select esg-license-reset in the Command drop-down list. 5. Click Run. The Console output displays "Success" when the subscription key is successfully reset.
From the ESG Shell 1. to the ESG Shell as the root .
ESGA v7.6 201 165
Troubleshooting Guide
2. Run the following command: [root@v10000-esg ~]# reset license The Console output displays "Success" when the subscription key is successfully reset.
166 ESGA v7.6 201
Troubleshooting Guide
Slide 10:6
ESG Error Issue A: Why Email Security could not be launched? The screenshot indicates that TRITON - Email Security service either hasn’t started or met a starting issue. Follow the steps to try to resolve this issue: 1. to the machine where TRITON Unified Security Center is installed. 2. List all the services. 3. Restart the Websense TRITON - Email Security service. 4. Click Retry on TRITON - Email Security interface.
Issue B: Unable to connect to the ESG appliance The connection error page will be displayed when TRITON - Email Security cannot connect to the ESG appliance. Perform as follows to determine the cause of this issue: In the Setting > Email Appliance page, check the unconnected reason by moving the mouse over the warning icon. We’ll talk about the way to solve this in the next slide.
ESGA v7.6 201 167
Troubleshooting Guide
Slide 10:7
Appliance Connection Error The slide lists the possible reasons of the appliance connection error:
Appliance is not accessible:
Ping the IP addresses of the C, E, and P interfaces. Restart the ESG modules if the console output returns with errors.
Check the statuses of all services in the appliance. If the network runs well, the SSLProxy and config-daemon service issues would cause the connection errors. Reset these services to fix the connection issue.
The TRITON - Email Security license does not match the appliance license:
The TRITON - Email Security version must be higher than the appliance:
Reset the appliance license on the appliance interface, then set license to the appliance. Update the TRITON - Email Security or the appliance to the appropriate version.
The TRITON - Email Security device ID does not match the appliance device ID:
168 ESGA v7.6 201
This error means the appliance is not the original appliance. Change the IP address of the appliance and add it to the ESG appliance. Then you can remove the original appliance from the ESG appliance.
Troubleshooting Guide
Slide 10:8
Log Server & Log Database Installation ESG Log Server s two installation modes:
Windows authentication mode In this mode, you must to the window with the existing domain and can access the SQL Server without . The windows must have the permissions to access the following system resources:
ODBC resource (create/edit ODBC data source)
Service control manager (create/delete windows service)
Database authentication mode The database must meet the following permission requirements:
ESG database owner •
• •
The database owner must have the hip in one of the following roles in the MS SQL database: “SQLAgent” role “SQLAgentReader” role “SQLAgentOperator” role The database owner must have the hip in “db_datareader” role The database owner must be a member of the “dbcreator fixed server” role
ESGA v7.6 201 169
Troubleshooting Guide
•
170 ESGA v7.6 201
The installation must have the “sys” permissions, because ESG stored procedures will access (start/stop operation) the SQL Server Agent service.
Troubleshooting Guide
Slide 10:9
Log Server & Log Database Deployment There are at least three deployment modes when you install the TRITON Management Server (EIP Server), Log Server and Log Database: 1. EIP Server + Log Server in one machine (A), Log Database in another dedicated machine (B); 2. EIP Server in one machine (A), Log server + Log Database in another machine (B); 3. All in one machine (A).
We recommend using a dedicated SQL Server to run the ESG off-box products, saying mode 1, thus you can get the best performance. Especially for mode 1, you need to install the SQL Server component Tools on machine (A) in advance. In this way, the Log Server can work in B mode and speed up the log record insertion.
ESGA v7.6 201 171
Troubleshooting Guide
Slide 10:10
Log Server Errors Issue A: What might cause the failure of the Log Server service start?
Failed to establish the connection with the Log Database.
The IP address/hostname of the Log Database may be changed.
Issue B: What might cause the Log Server failed in receiving ESG log records The ESG failed to establish the connection with the Log Server.
Check settings from the TRITON - Email Security: Settings > Report Settings > Log Server.
Check if the firewall is blocking the connection request.
172 ESGA v7.6 201
Troubleshooting Guide
Slide 10:11
Log Database Errors Issue A: What might cause the failure of the Log Database installation?
The Log Database directory must be an existing directory.
The Log Database directory must be writable on the remote SQL Server.
The Log Database must have the permission to run the scripts.
Issue B: What might cause no data in ESG report?
No data is sent to the Log Server.
No ETL job is running on the Log Database. Check job status: a. Connect to the Log Database with the SQL Server Management Studio b. Run the below command: exec usp_esg_get_job_status
Issue C: What might cause the alert “Log database extract, transform, and load process is not functioning properly” occurred?
The modified the Log Database system time after installation. ESGA v7.6 201 173
Troubleshooting Guide
Recreate the background ESG job with the following command: exec usp_create_background_jobs
How can I check job performance?
Run the following command and check the value of timeusage (ms): exec usp_esg_get_job_perf
174 ESGA v7.6 201
Troubleshooting Guide
Slide 10:12
Log Database ETL Failed
Check the error log table: select * from wse_error_logs
Check the job history: select * from msdb.dbo.sysjobhistory his, msdb.dbo.sysjobs jobs
ESGA v7.6 201 175
Troubleshooting Guide
Check the SQL Server the SQL Server management studio:
176 ESGA v7.6 201
Troubleshooting Guide
Slide 10:13
Cluster - Secondary Not Sync The slide lists the possible reasons of the second appliance not sync:
The cluster communication IP address can’t connected between the primary and the secondary nodes.
Set the correct cluster communication IP address for the primary or the secondary nodes.
Check if the primary and the secondary can reach each other on T port 6671.
The appliances within a cluster are not the same V-Series platforms (V10000 G2 or V5000 G2).
Don’t use different V-Series platforms.
Check if the V10000 platforms and ESG versions are the same for all cluster nodes:
Open the file /usr/local/etc/.xml and check the following fields to make sure the platforms are the same: ... ...
Open the file /usr/local/etc/esg.version and check the field major_verion to confirm the ESG versions are the same.
ESGA v7.6 201 177
Troubleshooting Guide
Check the file /tmp/md5sum.txt. This file lists all the synchronized files between the primary and the secondary nodes. The synchronized files are customized in /usr/local/etc/cluster_sync.conf.
178 ESGA v7.6 201
Troubleshooting Guide
Slide 10:14
Cluster - Primary or Secondary Crash Issue A: When the primary ESG has crashed and cannot be recovered, how to re-setup the cluster with the left nodes? 1. In the TRITON - Email Security > Setting > General > Email Appliances page, release all the cluster nodes to standalone mode. 2. Re-setup the cluster.
Issue B: When the secondary ESG has crashed and cannot be recovered, how to remove it from the cluster? In the TRITON - Email Security > Setting > General > Email Appliances page, remove the crashed node from the cluster.
ESGA v7.6 201 179
Troubleshooting Guide
Slide 10:15
Cluster - How to Track Issue A: How to track the cluster negotiation result?
Check the cluster status
to the ESG Shell as the root .
Execute the command show cluster to show the current cluster status Pay more attentions to the following fields: •
•
180 ESGA v7.6 201
Work Mode indicates the ESG work mode - standalone - primary - secondary Work Cur State indicates the cluster negotiation status - 0 negotiation failed - 1 negotiation processing - 2 configuration is out of sync - 3 configuration is synchronized - 4 lost network connection to peer - 5 local version is higher than peer - 6 local version is lower than peer - 7 platform is not matched Check this field for each secondary from the primary.
Troubleshooting Guide
•
Work Cur Mode indicates the cluster negotiation status. The meaning of the status code is the same as above. Check this field from the secondary.
Debug the cluster negotiation and configure the synchronized procedure
to the ESG Shell as the root .
Execute the following commands: a. debug module heartbeat_daemon event all b. debug module config_daemon event all c. debug output cli d. debug start
Check the system logs from TRITON - Email Security interface.
Check the system alerts from TRITON - Email Security interface.
Issue B: How to track the on-box logs? 1. to the ESG Shell. 2. Run the following command: [root@v10000-esg ~]# psql –U postgres 3. Run the following command to connect to the ESG on-box database: \c esgdb 4. Run the following command to list all the tables: \d 5. Run the following command to show the audit log: a. select * from audit_log b. \g 6. Run the following command to show the system log: a.
select * from system_log
b. \g
ESGA v7.6 201 181
Troubleshooting Guide
Slide 10:16
Hybrid Issue A: Check if the sync service is running 1. to the ESG Shell as the root . 2. Run the command ps aux|grep sync_daemon The Console output displays like the follows if the sync service is running: root 2905 0.0 2.6 157692 102220 ? Sl 2010 0:01 /usr/local/sbin/sync_daemon
Issue B: Export ESG sync service debug log 1. to the ESG Shell as the root . 2. Run the following commands: a. bash-3.1# esgshell b. esg1153-esg# debug module syncserver event all c. esg1153-esg# debug output cli d. esg1153-esg# debug start
182 ESGA v7.6 201
Troubleshooting Guide
Issue C: Check if the ESG can access the Hosted sync server 1. to the ESG Shell as the root . 2. Run the command: grep -i "HES_Sync_server" /usr/local/etc/hes_sync.conf 3. The Console output displays like below. This is the HES sync server name. HES_Sync_server = t8-hsync-web.odd.blackspider.com 4. Telnet the sync server on port 443 like the below example: telnet t8-hsync-web.odd.blackspider.com 443 Trying 172.16.167.146... Connected to t8-hsync-web.odd.blackspider.com (172.16.167.146). Escape character is '^]'. The Console output displays successful if the ESG can access the Hosted sync server. If not, configure the firewall settings to allow HTTPS traffic between the ESG and HES servers.
Issue D: Allow the Hosted cluster to access ESG SMTP 25 port During the hybrid configuration, the TRITON - Email Security shows the Hosted cluster IP addresses. Make sure the firewall allows the email traffic from these IP addresses to the ESG on port 25.
ESGA v7.6 201 183
Troubleshooting Guide
Slide 10:17
DLP Registration Failed If the ESG failed in registration to the DSS Manager, check as the follows: 1. The ESG appliance needs a valid and unique FQDN. Run the following command to view FQDN: hostname -f 2. The DSS Manager should be reachable on port 5819, 5821, 8443, 9443 and 80. 3. The information in the manual registration is valid and has the “Manage system module” privilege. 4. If the registration still fails, run the script manually to find the reason: /usr/local/sbin/dlp_manager.sh address name address
DSS Manager IP address
name/
information
184 ESGA v7.6 201
Troubleshooting Guide
Slide 10:18
DLP Policy Deployment Failed If the deployment from the DSS Manager fails, check the follows:
If Eth0 of the ESG is reachable on port 5820, 8888, 8889 and 9080
If the Policy Engine (PE) service is running on the ESG by the PE service status
The warning messages during deployment
The following directory in the TRITON Management Server: C:\Prograam Files(x86)\Websense\Data Security\tomcat\logs\dlp\dlp-all
ESGA v7.6 201 185
Troubleshooting Guide
Slide 10:19
DLP Policy Errors I have added policies and rules in the DSS manager. Why they are not working? If the added policies and rules aren’t working, check the following: 1. If the ESG is ed If not, an alert displays on the TRITON - Email Security dashboard, saying that the ESG is not ed. 2. If the DLP policy has been deployed from the DSS Manager If not, an alert displays on the TRITON - Email Security, saying that the DLP load configuration failed. 3. If the DLP policy on the ESG is in Enforce mode 4. If the rules action on the DSS Manager is Block All; or for the email DLP policy, the action is Quarantine 5. If the rules of the DSS Manager is configured for the right direction: inbound/ Outbound/Internal 6. If the policies are still not working, un and re- the ESG as the next section described
186 ESGA v7.6 201
Troubleshooting Guide
The process to un and re- ESG 1. Un ESG on the TRITON - Email Security. 2. Remove the ESG module on the TRITON - Date Security > System module. 3. Deploy on the DSS manager. 4. ESG on the TRITON - Email Security. 5. Deploy again on the DSS manager.
The message has been quarantined. Why the link to DSS incidence is not there in ESG message log? Mark the Audit incident option in the Action Plan of the rules.
Does the DLP scanning on ESG require DSS manager subscription? Generally, the DLP scanning of ESG requires Data Loss Prevention feature on the ESG subscription, but it does not require the DSS manager subscription. While the questionable image feature does require the specific DSS subscription. If the DSS subscription doesn’t contain the questionable image feature, the feature wouldn’t be working.
If the DSS manager is down, is DLP policy still working on ESG? The DLP policy would be working regardless of the availability of the DSS manager. But if the DSS manager or the connection to the DSS manager is down, the incidents and forensics would be cached locally on ESG.
How to debug the DLP scanning on ESG? DLP scanning can be debugged with following command: 1. to the ESG Shell as the root . 2. Run the following commands to debug the DLP scanning: a. debug module scan_engine event all b. debug output cli c. debug start The DLP scanning result and rules triggered would be shown in xml format. See the debug framework document for more details.
ESGA v7.6 201 187
Troubleshooting Guide
Slide 10:20
Quarantine Location Quarantine Folders All the quarantine relevant folders are located in the directory /var/spool/postfix/qd/. They are either used to store quarantined messages, or to store information and temporary files used by quarantine system. The following table lists details of the quarantine folders. qfolder
Refer to queue folders. All quarantined files are stored under this folder. Each queue has a separate subfolder. For example, the spam queue uses the qfolder/spam folder. If a queue is using a remote storage, the subfolder will be a symbolic link with the same name. For example, if the virus queue is using NFS storage, the qfolder/virus will be a symbolic link to the mountpoint
qfile/qlink
All quarantine messages are stored here temporarily before they are put into the destination queue folders. If there are many files under this queue, probably the quarantine daemon stops processing messages
blacklog
All quarantine requests from the filter module are stored here if the quarantine daemon stops processing messages
188 ESGA v7.6 201
Troubleshooting Guide
defer
All logs of "delayed messages" are queued here before they are processed and inserted into SQL server database
dbsync
If quarantine daemon fails to insert a quarantine to SQL server database (connection problem, execution timeout, execution failed), the log will be saved in this folder
emls
Used to temporarily store quarantined or delayed messages in plain text format (eml). The eml files are generated when you or view the messages through TRITON - Email Security interface. It will be then deleted, so supposedly there should be no files in this folder
delay
If a queue is moving, for example, from the local storage to the remote NFS storage, the incoming emails to this queue will be temporarily saved in this folder because the destination queue folder is unavailable. After the queue moving is done, these mails will be flushed into the associated queue folder
rdelay
If a remote queue is temporarily unavailable, for example, the network connection is down, the incoming emails to this queue will be temporarily saved in this folder. After the queue recovers, these mails will be flushed into the associated queue folder
failconf
Used to store the last configuration of a queue. If you see a file under this folder, the queue is either moved, or failed to move
flusher, dlcache
Not used
Configuration File The configuration file used by the quarantine daemon to connect to the Log Database is /etc/freetds.conf.
ESGA v7.6 201 189
Troubleshooting Guide
Slide 10:21
Quarantine - No Messages in Blocked Messages I’ve sent a spam email, why can’t I see quarantined logs in “Blocked Messages”? 1. to the ESG Shell with the root . 2. Check /var/log/maillog to see if the email is captured by the scan engine. The log should look like the following in normal situations: /usr/local/sbin/filter[10435]: message id 2D55FA50002, filter ASA-Filter, policy Outbound Default, rule AntiSpam, matched! 3. Make sure the quarantine action for spam and virus filters is turned on. 4. Check whether there are files under the /dbsync folder.
Yes. It is likely to be a issue with the Log Database connection. Run the below command to check whether the Log Database connects with the quarantine daemon: netstat –anp | grep quarantine_daemon a. The connection is well. The SQL command for inserting quarantine logs may be failed. Start debugging the quarantine daemon: - Sent another email.
190 ESGA v7.6 201
Troubleshooting Guide
- View the SQL command that is sent to the SQL server. - If the debug information shows that the SQL command failed, copy this SQL string and execute it in the SQL Server Management Studio manually. b. No connection. Check the Log Database configuration and the network connection.
No. to the SQL Server Management Studio to check whether the logs are inserted into esglogdb76.dbo.esg_detail_quarantine_message and esglogdb76.dbo.esg_detail_quarantine_recipient. a. If yes, it is the TRITON - Email Security interface problem. Check the tomcat ogs on TRITON - Email Security for details. b. If not, the quarantine daemon may fail to process messages. Check whether there is message file under the /qfile/qlink folder; Check whether there are core files under the /var/cores.
Why are the numbers of messages in queue list page and in message list page different? 1. to the ESG Shell as the root . 2. Taking the spam queue as an example, run the following command to calculate the total number of messages: find /var/spool/postfix/qd/spam –type f | wc -l 3. Compare the number with the queue list number and the message list number If the number is the same as the queue list, but different with the message list, some logs are probably not inserted into the Log Database successfully. Check the dbsync folder to find these logs.
When I moved a queue from the local storage to NFS, TRITON - Email Security interface shows “Data transfer ongoing” for a very long time. What happened? If the ESG works in a cluster mode, perform as the follows: 1. to the ESG Shell as the root . 2. Run the command below to see the queue moving status of each cluster node: show queuemovestatus 3. Debug the quarantine daemon to check whether the file copy is ongoing. If you don’t see any output of the file copy, cancel the data transfer from TRITON - Email Security interface and try again.
ESGA v7.6 201 191
Troubleshooting Guide
Slide 10:22
System Counters How to show counter in ESG? 1. to the ESG Shell as the root . 2. Run the following command: [root@localhost filter]# counter
What is the difference between ESG persistent counters and ESG nonpersistent counters? We can ignore the persistent counters. They have the same values as the non-persistent counters.
What does each counter mean? Many counters are for the debugging purpose (dbg_* counters).
192 ESGA v7.6 201
Troubleshooting Guide
The below table lists the counters and the descriptions as the reference. Counter Name
Description
received_msg
Messages number received externally. The messages include all directions(inbound, outbound, internal, and open relay)
processed_msg
Messages which have been processed (policy has been applied on the message)
Delivered_msg
Message which have been delivered
Dropped_msg
Message which have been dropped
Quarantined_msg
Message which have been quarantined.
In_queue_msg
Messages waiting to be processed or waiting to be delivered. A large value of in_queue_msg means that the system is under stress.
In_deferred_queue _msg
Messages in deferred queue
Queue_total/used/ freed_space
The 3 values indicate the disk space which can be used for message processing
Inb_conn
Total smtp connections
Inb_reputation_rec
Connections dropped by reputation
Inb_rbl_rec
Connections dropped by rbl
Inb_rdns_rec
Connections dropped by rdns check
Inb_spf_rec
Connections dropped by spf
Av_hit
Number of virus-infected messages
Av_clean
Number of virus-clean messages
Av_err
Number of messages which have error in av-scanning
As_hit
Number of spam messages
As_clean
Number of not-spam messages
As_err
Number of messages which have error in as-scanning
Dlp_hit
Number of messages which have matched dlp policy
Dlp_clean
Number of messages which have not matched dlp policy
Dlp_err
Number of messages which have error in dlp-scanning
Filter_exception
Number of exception messages\
Encrypt
Number of messages which need encryption
Encrypt_success
Messages which succeed in encryption
Encrypt_fail
Messages which fail encryption
Decrypt
Number of messages which need decryption
Decrypt_success
Messages which succeed in decryption
Decrypt_fail
Messages which fail decryption
ESGA v7.6 201 193
Troubleshooting Guide
Counter Name
Description
Hes_dkim_
Messages from HES which need dkim verification
Hes_dkim_
Messages which have ed dkim verification
Relations within different counters If a message with multiple recipients is applied with default policy and no exception message happens, the below formula should be true: Received_msg = processed_msg = delivered_msg + quarantined_msg + filter_exception + bounced_msg + in_deferred_queue_msg Dropped_msg = quarantined_msg
Why the above formula does not match? The possible reasons could be:
The message has multiple recipients; different recipients have different policies; e.g. some recipients are quarantined, and some recipients are delivered. In this case, the counters will not match.
Exceptions Some counters may be increased twice for a single mail.
Manual operations For example, reprocess some messages from TRITON - Email Security > Main > Message Management.
Messages are always coming into the ESG system. Counters will not match until finishing all the messages process.
194 ESGA v7.6 201
Troubleshooting Guide
Appendix
ESG Shell Q&A Debug tool do not show log. Why?
The debug daemon or modules to be debugged may not start.
Run the follow command: svstat /service/debugd/
Run the below command to start it: svc -du /service/ debugd /
Debug setting may be output to the wrong place.
Run the blow command to view the debug setting: debug show
Some services don’t function well.
Restart the ESG service to fix it: /usr/local/sbin/esg_restart.sh
Several configuration values in the share memory are none or wrong. Why? Some services may not start. 1. Check whether the core files are created in the /var/cores/ directory. 2. Run the below command to start all services: /usr/local/sbin/esg_restart.sh
The set/reset command do not function or report errors. Why? The errors should be treated case by case. The following commands may help:
Run the set/reset command again. The previous error maybe due to the related service busy.
Check the file /var/log/confd.log to find some clue.
Restart the ESG services.
System Diagnosis Q&A Failed to generate the summary file. Why? The summary file contains the configuration files and logs of all the modules, therefore any errors from any module may cause the error.
ESGA v7.6 201 195
Troubleshooting Guide
The disk space and the network issues could be the most probable reasons.
Failed to open the configuration file. Why? The file is encrypted with the decode key “f~S#-/5A”. Changes to the key may lead to fail to decode the file. If the ESG file in the whole package is damaged, check the hard disk usage of the ESG module.
Health Daemon Service didn’t start
Service restart by health daemon
Description
Service which configured in /usr/local/etc/health.conf is not started. Health daemon will start it by the configured command
Trigger condition
Service which configured in /usr/local/etc/health.conf is not started. Health daemon will start it by the configured command
Message
Service %s down, start it by health_daemon
Alert
ALERT_TYPE_SERVICE_DOWN The {%DyInfo} service is not available
Action
Start service by the configured command
Recovery
No need
Service restarted by manually
Description
Service which configured in /usr/local/etc/health.conf is not started. Health daemon will generate warning message to remind to start the service manually
Trigger condition
Service which configured in /usr/local/etc/health.conf is not started. Health daemon will start it by the configured command
Message
Service %s down. Restart by itself
Alert
ALERT_TYPE_SERVICE_DOWN The {%DyInfo} service is not available
Action
should start the service manually
Recovery
No need
196 ESGA v7.6 201
Troubleshooting Guide
Service start repeatedly Description
Service restart and terminated abnormal repeatedly and exceeds a threshold, health daemon will stop it
Trigger condition
Service restart and terminated exceeds 3 times
Message
Restart service:name=%s pid=%d,mem=%f,max=%d,max_ratio=%f,cmd=%s service memory exceed the configured limitation, restart it by the configured command
Alert
No
Action
Health daemon stop the service. should check system. The service can be restarted by command "svc -u servicename" manually
Recovery
No need
Service memory leak Description
Service memory keeps increasing and exceeds a configured threshold (/usr/locat/etc/health.conf)
Trigger condition
If the "memory increase threshold" which configured by /usr/local/ etc/health.conf is not 0, the trigger condition is: when the memory increase step is bigger the increase threshold and the memory is bigger than the max limitation value. If the "memory increase threshold" which configured by /usr/local/ etc/health.conf is 0, the trigger condition is: when the memory is bigger than the max limitation value
Message
Kill service:%s pid=%d,mem_inc_thr=%d, mem_last_val=%d,mem=%f,max=%d,max_ratio=%f cmd is [%s] service memory exceed the configured limitation, killed by health daemon
Alert
No
Action
Health daemon kill the service and generate warning messages in the /var/log/messages
Recovery
No need
Service file handler leak Description
Service generate too many pipe file handlers. It may be a file handler leak. Health daemon will kill it and generate warning messages
Trigger condition
Service generate too many pipe file handlers
Message
Process name [%s] pid [%u] may be fd leak, fd number[%u]. Please check it Service generate too many pipe file handlers. it may be a file handler leak. Killed by health daemon
Alert
No
ESGA v7.6 201 197
Troubleshooting Guide
Action
Health daemon kill the service and generate warning messages in the /var/log/messages
Recovery
No need
System disk space too small
/dev/xvda1 free space too small
Description
/dev/xvda1 free space is too little. Health daemon will set disk stress flag. Mta will not be process email
Trigger condition
/dev/xvda1 free space is little then 4G
Message
/dev/xvda1 free space is little then 4G, set disk_stress
Alert
ALERT_TYPE_DISK_STRESS Adequate amount of free disk space is not available
Action
Health daemon set the disk stress flag, mta will not process emails
Recovery
Release more disk space. Disk free space is bigger than 8G
/dev/xvda2 free space too small
Description
/dev/xvda2 free space is too little. Health daemon will set disk stress flag. Mta will not be process email
Trigger condition
/dev/xvda2 free space is little then 4G
Message
/dev/xvda2 free space is little then 4G, set disk_stress
Alert
ALERT_TYPE_DISK_STRESS Adequate amount of free disk space is not available
Action
Health daemon set the disk stress flag, mta will not process emails
Recovery
Release more disk space. Disk free space is bigger than 8G
/dev/xvda1 free inodes too small
Description
/dev/xvda1 free inodes is too small. Health daemon will set disk stress flag. Mta will not be process email
Trigger condition
/dev/xvda1 free inodes is little then 1024 * 200
Message
/dev/xvda1 free inode is little then %d, set disk_stress
Alert
ALERT_TYPE_DISK_STRESS Adequate amount of free disk space is not available
Action
Health daemon set the disk stress flag, mta will not process emails
Recovery
Release more disk space. Disk free space is bigger than 1024 * 300
198 ESGA v7.6 201
Troubleshooting Guide
/dev/xvda2 free inodes too small
Description
/dev/xvda2 free inodes is too small. Health daemon will set disk stress flag. Mta will not be process email
Trigger condition
/dev/xvda2 free inodes is little then 1024 * 200
Message
/dev/xvda2 free inode is little then %d, set disk_stress
Alert
ALERT_TYPE_DISK_STRESS Adequate amount of free disk space is not available
Action
Health daemon set the disk stress flag, mta will not process emails
Recovery
Release more disk space. Disk free space is bigger than 1024 * 300
ESGA v7.6 201 199
Troubleshooting Guide
Log stress monitor Description
Log daemon can not send data to data base successfully. /log-defer has files older than 3 days. Health daemon will set "log stress flag" and generate warning messages in /var/log/messages. Mta will not process emails
Trigger condition
/log-defer has files older than 3 days
Message
Log has files older than 3 days, set log stress
Alert
ALERT_TYPE_LOG_STRESS Email Security log queues contain cache files older than 3 days
Action
Health daemon set the log stress flag, mta will not process emails
Recovery
Check the connection to data base. Clear all files (/log-defer) which older than 3 days
Quarantine monitor Description
Quarantine daemon can not send data to data base successfully. Health daemon will set "log stress flag" and generate warning messages in /var/log/messages. Mta will not process emails
Trigger condition
/var/spool/postfix/qd/dbsync has files older than 3 days and recent successfully sending data time is older than 3 days or /var/spool/postfix/qd/dbsync has files more than 100000
Message
DBsync files and sync time is older than 3 days, or files number is bigger than %d, set quarantine stress
Alert
ALERT_TYPE_QUARANT_STRESS Email Security quarantine queue contains cache files older than 3 days or too many files
Action
Health daemon set the quarantine stress flag, mta will not process emails
Recovery
Check the connection to data base. Clear all files (/var/spool/ postfix/qd/dbsync)
Filter thread monitor Description
Filter update thread or scanning thread take too long running time. Health daemon will kill filter and generate warning message in / var/log/messages
Trigger condition
Filter update thread or scanning thread running time is more than 20 minute
Message
Filter update running time too long. Current system time is [%lu], filter update_start time is [%lu] Filter update thread running time is longer than 20 minute. filter service was killed by health daemon
Alert
No
200 ESGA v7.6 201
Troubleshooting Guide
Action
Health daemon kill filter and generate warning message in /var/log/ messages
Recovery
No need
Share memory dead lock Description
Service share memory dead lock. Health daemon kill the service
Trigger condition
Same service and same function and same line hold the same share memory lock for too long time
Message
Share memory [%s] is dead locked by function [%s] line [%u] service hold share memory for too long time, it may be a dead lock. Killed by health daemon
Alert
No
Action
Health daemon kill service and generate warning message
Recovery
No need
ESGA v7.6 201 201
Troubleshooting Guide
202 ESGA v7.6 201
Related Documents c2h70
Websense Esga V76 201 Student Guide 18205o
November 2022 0
Tutorial Websense 1b3212
August 2022 0
Student Guide 4n2dt
November 2019 52
Student Guide 4n2dt
December 2019 55
Siem Websense 4g2p60
December 2019 46
Student Guide 4n2dt
November 2019 71More Documents from "Thiago Santos" 1j5e50
6g3p1u
December 2019 35
Deus Me Ama - Thalles Roberto 1w2s22
November 2019 50
6g3p1u
April 2020 26
Provaobjetiva Ca Cfg 2011 2012 94t5p
October 2022 0
Websense Esga V76 201 Student Guide 18205o
November 2022 0