Encrypted traffic has come to dominate network flows, which makes it difficult for traditional flow monitoring tools to maintain visibility. This is particularly true when the process to enable encryption occurs after an initial data exchange, causing the encryption attributes to be missed.
In this blog post we take a closer look at a new feature added to CERT’s Yet Another Flowmeter tool (YAF) to capture the attributes of encryption when it occurs after the start of the session. We call this mid-encryption. We explore what mid-encryption means, why it matters, how it works within YAF, and what benefits this brings to traffic analysis and network security teams.
From 2014 to 2024, we saw a steady increase in the percentage of traffic that is encrypted with more than 80 percent of pages loaded by Firefox and 96 percent of traffic across Google being encrypted.
CERT researchers developed Yet Another Flowmeter (YAF) 20 years ago to read network packets and create Internet Protocol Flow Information Export (IPFIX) network flow records—where each record summarizes a connection between two hosts (a network session. The infrequent use of encryption at that time meant YAF had full visibility into many of these records: YAF was able to capture the metadata of various connections, including: HTTP for web sites, Simple Mail Transport Protocol (SMTP), Internet Message Access Protocol (IMAP), and Post Office Protocol v3 (POP3).
For connections that started with an encryption request, YAF could capture attributes of the encrypted session (the Transport Layer Security (TLS) ClientHello and ServerHello) and the certificates used for encryption. Although the encrypted session itself was opaque, the captured attributes allowed network analysts to verify that certificates were legitimate, and the connection was properly encrypted.
What is Mid-Encryption?
Mid-encryption refers to a network session beginning in an unencrypted (usually text-based) state and transitioning to an encrypted state during the same session. This action is triggered using mechanisms such as STARTTLS, a command used in application-layered protocols (e.g., Simple Mail Transfer Protocol, Internet Message Access Protocol, Extensible Messaging and Presence Protocol) that begins encryption using TLS.
Typically flow sensors label the session as encrypted or unencrypted by examining the beginning of the session. While this process usually helps with labeling the correct protocol and capturing the metadata, commands such as STARTTLS may lead to potential loss of visibility and metadata because they launch the encryption process during the session.
Why Mid-Encryption Support Matters
Today’s HTTP traffic is largely encrypted, but older protocols often use an opportunistic encryption model that is easier to implement and allows servers and clients to communicate when both parties do not support encryption. With opportunistic encryption, a session starts in plain text before negotiations for encryption occur via a STARTTLS or HTTPS upgrade. Early session metadata is available to the sensor, while the rest may be nontransparent.
Without mid-encryption support, YAF may miss the indicators of when encryption occurs and fail to label the session correctly. This scenario could lead to partial loss of visibility—we don’t know if encryption was successful—and incorrectly labeled flow records, which may lead to analysts needlessly investigating benign traffic.
With mid-encryption support, YAF can capture early metadata during the clear-text phase, detect and capture the encryption indicators (e.g., STARTTLS string), annotate the flow accurately, provide TLS handshake metadata, and compute JA3 fingerprints from the metadata. The fingerprints provide a quick why to distinguish legitimate traffic from malicious traffic and to detect the use of weak or revoked certificates.
Mid Encryption Capabilities
With the new feature, YAF can now track protocol negotiations in real time and identify encryption flags (like the STARTTLS command or TLS ClientHello). The Internet Protocol Flow Information Export (IPFIX) records it generates are enriched with encryption information: when the encryption began, what protocol was negotiated, and which portions of the flow are encrypted or clear text. The record also includes TLS ClientHello metadata: TLS version, cipher suites offered and selected, and server certificate details.
Mid encryption is useful with protocols that still allow clear text preludes before upgrading, such as SMTP, POP3, IMAP, Network News Transport Protocol (NNTP), Lightweight Directory Access Protocol (LDAP), XMPP, and IRC.
Example Use Case: STARTTLS in SMTP
A mail client connects to a mail server listening on port 25. The server replies with a greeting and a list of extensions that includes STARTTLS if supported. The client may issue SMTP commands, such as EHLO, MAIL FROM, and RCPT TO, that are transmitted in clear text. At this point the session is still unencrypted. The client at some point sends a STARTTLS command to which the server, if supported, replies with a message saying it is ready to start TLS communication (e.g., 220 Ready to start TLS). The client sends TLS ClientHello messages and TLS negotiation and encryption begins.
With the mid-encryption support, YAF is able to
- parse clear text for SMTP commands
- identify the STARTTLS command and replies
- identify the TLS ClientHello message
- identify when encryption starts and ends
- provide TLS deep packet inspection (DPI) data
- detect protocol nesting and record accurately
Figure 1: With mid encryption support, YAF captures plain text commands and encryption negotiation of a SMTP connection
YAF has the ability to label the flows correctly because it keeps track of the original protocol where the plain-text session started—SMTP for this use case. YAF would also maintain a sub-record labeling the TLS DPI data that provides network analysts a more complete picture of the protocols t upgrading to an encrypted session.
Figure 2: A YAF record containing DPI for SMTP text commands and TLS metadata
What Can an Analyst Do with Mid-Encryption?
Let’s take the SMTP use case as an example. Before adding mid-encryption, a record generated by YAF summarizing an SMTP connection using STARTTLS would not contain information regarding the quality of the encryption or the certificates used. It would only contain the server’s welcome banner, the client’s EHLO command, and a Boolean noting that STARTTLS was used.
With the support of mid-encryption, the records generated by YAF are augmented with service-specific TLS attributes and certificate information as seen in the diagram (Figure 2), which illustrates the IPFIX or JSON records. Within the original record for the SMTP protocol, a TLS DPI section (using the historical name SSL) will appear that would inform the analyst that the session was encrypted, the version of TLS, the encryption cipher, and certificate attributes such as the issuer, subject, key length and validity dates. A security analyst could identify the use of weak or revoked certificates or certificates issued by suspicious parties. The analyst would then be able to expand on their fingerprinting capabilities (e.g., JA3 or JA4+) and pivot from that information. This could be used to identify misconfigured machines or insider threats within an organization, or identify sources of unwelcome email that should be blocked.
Understanding How and Why Encryption Started
As network encryption becomes the norm, visibility at the protocol layer is harder to maintain. This visibility, however, is more important than ever since it provides one of the few opportunities to examine the traffic traversing your network. The addition of mid- encryption support in YAF is a forward-thinking enhancement that helps bridge the gap between plain-text and encrypted traffic awareness.
Mid-encryption in YAF helps analysts see what happens before encryption starts and gain a better understanding of when and how encryption started. Knowing this information helps maintain context around nested protocols and improve detection of stealthy or evasive behavior.
This new capability is not just a technical upgrade; it is a shift towards smarter flow analytics in an increasingly encrypted world. When paired with certificate fingerprinting, it provides network defenders a powerful tool to find uses of revoked or weak certificates within their network and identify malicious traffic entering the network.
