Why SSL failures in production are different
SSL certificate problems rarely happen at a convenient time, honestly. A certificate can be valid in one browser and fail in another, work behind a load balancer but fail directly on the origin server, or pass local tests while causing outages in production, basically. This is one of the reasons SSL incidents can be particularly difficult to diagnose when the application itself appears to be healthy, you know?
The most common mistake is assuming that every SSL error means the certificate has expired, seriously. In production, failures are often caused by certificate chain issues, hostname mismatches, reverse proxy misconfiguration, automated renewal failures, incorrect deployment of private keys, DNS changes, firewall rules, or a certificate being installed on one server while traffic is routed to another, which is a mess.
Production infrastructure is also more complicated than a local development environment, basically. A request may travel through a CDN, DNS provider, cloud load balancer, reverse proxy, web server, container, and application server before reaching the application. TLS can be terminated at any one of these layers, meaning the certificate you inspect on the origin server may not be the certificate that users actually receive, you know?
The same production domain can also resolve to multiple IP addresses, honestly. If certificates are deployed inconsistently across multiple servers, some users may receive a correct certificate while others receive an expired or incorrectly configured certificate. This can make an SSL problem appear intermittent even though the root cause is a configuration mismatch between production nodes, which is super annoying.
Treat SSL problems as an availability and security incident, seriously. Every minute of certificate downtime affects user trust, API clients, SEO, mobile applications, automated integrations, monitoring systems, and potentially revenue, basically.
How TLS works in a production environment
Before troubleshooting an SSL problem, it helps to understand where TLS is actually terminated, honestly. When a user opens an HTTPS URL, the browser establishes a secure TLS connection with the server that is exposed to the public internet. That server presents a certificate proving that it controls the requested hostname, basically.
In a simple deployment, the browser connects directly to Nginx and Nginx handles TLS, you know? In a larger architecture, the browser may connect to a CDN or cloud load balancer first. That service terminates TLS and then establishes another connection to the origin server. In this configuration, there can effectively be two separate TLS configurations that need to be maintained, which is a bit of a headache.
- ●
Browser or API client establishes an HTTPS connection
- ●
DNS resolves the hostname to the public endpoint
- ●
CDN or load balancer may terminate TLS
- ●
The proxy may establish another TLS connection to the origin
- ●
Nginx or another web server handles the request
- ●
The application receives the forwarded request, basically
This architecture explains why checking only the origin certificate is not always enough, basically. If the CDN has an expired certificate, users will fail before the request ever reaches your server. If the CDN certificate is correct but the origin certificate is invalid and strict origin verification is enabled, the CDN may reject the connection to the backend, which is bad.
The five most common production SSL problems
| Problem | Typical symptom | Severity |
|---|---|---|
| Expired certificate | Browser security warning | Critical |
| Incomplete certificate chain | Works on some devices only | High |
| Hostname mismatch | Certificate does not match domain | Critical |
| Renewal automation failure | Sudden outage after expiration | Critical |
| Reverse proxy misconfiguration | Intermittent SSL failures | High |
These five problems account for a large portion of production certificate incidents, honestly, but they are not the only possible causes. TLS protocol incompatibility, unsupported cipher suites, incorrect DNS records, bad server permissions, incorrect system time, revoked certificates, and stale container images can also create HTTPS failures, you know?
How to diagnose an SSL failure quickly
The first step is to verify exactly what certificate is being served in production, basically. Do not immediately start changing configuration files, seriously. First establish what the public endpoint is actually returning to clients, you know?
openssl s_client -connect example.com:443 -servername example.com
Look for the certificate chain, issuer, expiration date, protocol, and hostname, honestly. This immediately tells you whether the production server is serving the expected certificate. The output can also reveal whether the server is returning multiple certificates and whether the TLS handshake completes successfully, basically.
The -servername parameter is particularly important because it enables Server Name Indication, you know? A single IP address can host many HTTPS domains, and the server needs the requested hostname to select the correct virtual host and certificate, basically.
💡 Fast diagnosis
Always test with SNI using the -servername parameter, seriously. Without it, virtual hosts may return a completely different certificate and lead you to investigate the wrong configuration, which is a mess.
When troubleshooting from production, also compare the public result with the certificate installed locally, honestly. If the local certificate is correct but the public certificate is wrong, the problem is probably related to deployment, DNS, a reverse proxy, a CDN, or a load balancer rather than the certificate file itself, you know?
Checking certificate expiration
echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null | openssl x509 -noout -dates
This returns the notBefore and notAfter dates, basically. If the certificate is expired, renew it immediately and verify that the web server has reloaded the new certificate, honestly.
Do not assume that replacing the certificate file automatically changes the certificate served to clients, seriously. Nginx and other web servers normally load certificates when their configuration is started or reloaded. If the new file exists on disk but the process has not reloaded it, clients may continue receiving the old certificate, which is bad.
sudo nginx -t
sudo systemctl reload nginx
echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null | openssl x509 -noout -dates
After renewal, always perform an external verification, you know? Checking only the certificate file on the server does not prove that the production endpoint is serving that certificate, basically.
Incomplete certificate chain
One of the most overlooked production issues is an incomplete certificate chain, honestly. Modern browsers may recover automatically in some situations because they already have required intermediate certificates cached or available through other mechanisms. Other clients, including older devices, Java applications, command line tools, and certain API clients, may fail completely, which is a mess.
A TLS certificate is normally part of a chain of trust, basically. The server certificate identifies your hostname, while an intermediate certificate connects that certificate to a trusted root certificate. The server should normally provide the server certificate and the required intermediate certificates, you know?
ssl_certificate /etc/ssl/fullchain.pem;
ssl_certificate_key /etc/ssl/private.key;
Using cert.pem instead of fullchain.pem is a very common deployment mistake when working with certificate authorities that provide separate certificate and chain files, seriously.
If an application works in Chrome but fails from a backend service, Java application, mobile application, or command line client, inspect the complete chain before assuming that the client is broken, basically.
openssl s_client -connect example.com:443 -servername example.com -showcerts
💡 Chain troubleshooting
When investigating chain problems, compare the certificates actually served over the network with the files configured in Nginx, honestly. The network response is the source of truth for what clients receive, you know?
Hostname mismatch
A certificate issued for api.example.com will not automatically protect www.example.com unless both names are included in the certificate, basically. HTTPS validation checks whether the requested hostname matches one of the names covered by the certificate, honestly.
Modern certificates normally use the Subject Alternative Name extension for hostname validation, you know? The Common Name field should not be treated as the complete source of hostname information, seriously.
openssl x509 -in fullchain.pem -text -noout | grep -A1 "Subject Alternative Name"
Verify that every production hostname appears in the Subject Alternative Name extension, basically. This includes the main domain, www hostname, API hostname, admin hostname, and any other public endpoint that is expected to use HTTPS, you know?
- ●
example.com
- ●
- ●
api.example.com
- ●
admin.example.com
- ●
staging.example.com
Be especially careful after introducing a new subdomain, honestly. A new DNS record does not automatically mean that the existing certificate covers the new hostname. The DNS configuration and certificate configuration are separate concerns, basically.
Lets Encrypt renewal failures
Automated renewal failures are responsible for many production outages, seriously. The certificate expires silently because the renewal job fails and nobody notices. The problem becomes visible only when users begin receiving certificate warnings, which is a mess.
sudo certbot renew --dry-run
The dry run allows you to test the renewal process without waiting for the certificate to approach its expiration date, basically. A successful dry run provides useful confidence that the validation process and deployment hooks are functioning, you know?
Common causes include blocked HTTP validation, incorrect DNS records, firewall changes, expired DNS credentials, incorrect permissions, container networking problems, failed reload hooks, and an ACME challenge endpoint that is no longer reachable from the internet, honestly.
- ●
Check Certbot logs
- ●
Check the renewal configuration
- ●
Verify DNS records
- ●
Verify port 80 accessibility when HTTP validation is used
- ●
Check firewall and security group rules
- ●
Verify DNS API credentials when DNS validation is used
- ●
Verify the Nginx reload hook
- ●
Run a dry-run renewal test, basically
sudo journalctl -u certbot --no-pager
sudo ls -la /var/log/letsencrypt/
💡 Monitoring
Set certificate expiration alerts at 30, 14, and 7 days before expiration, basically. Monitoring is as important as automation because a renewal system that fails silently is not reliable automation, you know?
HTTP challenge and port 80 problems
Many certificate renewal configurations use an HTTP challenge where the certificate authority needs to reach a specific path on the domain over HTTP, honestly. If port 80 is blocked or redirected incorrectly, renewal may fail even though the HTTPS website itself works perfectly, which is frustrating.
A common production mistake is configuring a firewall to allow port 443 but completely blocking port 80, basically. This may be acceptable for some certificate validation strategies, but it can break HTTP based ACME validation if the renewal process depends on that port, you know?
curl -I http://example.com/.well-known/acme-challenge/test
The exact response depends on your ACME configuration, seriously, but the important question is whether the validation request can reach the correct server and whether the challenge path is handled correctly, basically.
DNS validation and propagation issues
DNS based certificate validation can be more reliable for environments where port 80 cannot be exposed publicly, but it introduces another dependency, honestly. The renewal process must be able to create the required TXT record in DNS, you know?
DNS changes can also take time to propagate, basically. If your automation creates a TXT record and immediately asks the certificate authority to validate it, validation can fail if the authoritative DNS servers do not yet return the expected record, which is annoying.
dig TXT _acme-challenge.example.com
When DNS validation fails, verify that the automation is modifying the correct DNS zone and that the authoritative nameservers actually contain the expected TXT record, seriously.
Reverse proxy and load balancer issues
SSL termination becomes more complex when traffic passes through Cloudflare, AWS ALB, Azure Application Gateway, HAProxy, or Nginx reverse proxies, honestly.
A frequent issue is that the load balancer serves a different certificate than the origin server, basically. Another common problem occurs when a certificate is updated on one backend server but traffic is distributed across several servers with different certificate versions, which is a mess.
- ●
Check the certificate on the load balancer
- ●
Check the certificate on the origin server
- ●
Verify SNI configuration
- ●
Confirm backend protocol settings
- ●
Inspect redirect rules
- ●
Check every production backend
- ●
Verify certificate consistency across instances, basically
If DNS returns multiple IP addresses, test each address separately when possible, you know? A single incorrect node can create intermittent failures that are difficult to reproduce from one machine, basically.
TLS termination at a CDN
CDNs add another layer to SSL management because the public certificate may be managed by the CDN while the origin server uses a separate certificate, honestly. The two certificates have different purposes and different renewal mechanisms, basically.
For example, the browser may establish TLS with the CDN, you know? The CDN then establishes a second connection to the origin. If the CDN requires strict origin certificate validation, the origin certificate must also be valid and trusted, seriously.
- ●
Verify the public CDN certificate
- ●
Verify the origin certificate
- ●
Check the configured TLS mode
- ●
Confirm the origin hostname
- ●
Verify certificate expiration on both layers
- ●
Check whether the CDN expects a publicly trusted origin certificate, basically
When a CDN is involved, always identify where TLS is terminated before changing certificates, basically. Updating the origin certificate will not fix a problem caused by the public edge certificate, you know?
Nginx SSL configuration problems
Nginx is frequently used as the TLS termination point in production, honestly. A certificate can be perfectly valid while the Nginx configuration points to the wrong file, wrong server block, or incorrect private key, which is bad.
server {
listen 443 ssl;
server_name example.com www.example.com;
ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
location / {
proxy_pass http://localhost:3000;
}
}
One important detail is the relationship between the server_name directive and SNI, basically. If multiple HTTPS server blocks exist, Nginx chooses the appropriate server block based on the requested hostname. A default server block can therefore accidentally serve the wrong certificate when the request does not match the expected configuration, you know?
sudo nginx -t
sudo nginx -T | grep -n "server_name"
sudo nginx -T | grep -n "ssl_certificate"
The nginx -T command is useful because it prints the complete active configuration, seriously. This can reveal that the configuration file you edited is not actually included in the running configuration, basically.
Private key problems
A certificate and its private key must belong together, honestly. If a new certificate is deployed with an old private key, Nginx may fail to load the configuration or the certificate may not correspond to the expected key, which is a mess.
openssl x509 -noout -modulus -in fullchain.pem | openssl sha256
openssl rsa -noout -modulus -in privkey.pem | openssl sha256
The resulting hashes should match when comparing a certificate and RSA private key belonging to the same pair, basically. For other key types, use the appropriate OpenSSL commands for the key format, you know?
Private key permissions are also important, seriously. The web server needs enough access to read the key, but the key should not be exposed unnecessarily to other users or processes on the server, basically.
Never place private keys in public source repositories, frontend bundles, Docker images intended for public distribution, or publicly accessible web directories, honestly.
TLS protocol and cipher compatibility
Not every HTTPS failure is caused by the certificate itself, basically. A server can have a valid certificate and still reject clients because of incompatible TLS versions or cipher configuration, you know?
Modern production systems should normally prioritize current TLS versions and avoid unnecessarily supporting obsolete protocols, honestly. However, compatibility requirements should be evaluated based on the actual clients that need to connect to the service, seriously.
openssl s_client -connect example.com:443 -servername example.com -tls1_2
openssl s_client -connect example.com:443 -servername example.com -tls1_3
Testing individual TLS versions can help determine whether the problem is related to protocol negotiation rather than certificate validity, basically.
HTTP to HTTPS redirects
Redirecting HTTP traffic to HTTPS is an important part of production security, honestly, but redirect configuration can also create confusing failures. A redirect from HTTP to HTTPS does not fix an invalid HTTPS certificate. The browser must successfully establish TLS before it can follow the redirect destination, you know?
server {
listen 80;
server_name example.com www.example.com;
return 301 https://$host$request_uri;
}
When troubleshooting redirect problems, test both protocols separately, basically. Verify that HTTP returns the expected redirect and that HTTPS presents a valid certificate before processing the request, seriously.
curl -I http://example.com
curl -I https://example.com
SSL problems inside Docker and containers
Containerized deployments introduce additional certificate management challenges, honestly. The certificate may exist correctly on the host machine but not inside the container where Nginx is running, which is a mess.
Another common problem is mounting the certificate directory incorrectly, basically. The container can start successfully while Nginx continues using an old certificate because the expected host directory was not mounted into the correct container path, you know?
docker ps
docker exec -it nginx-container ls -la /etc/letsencrypt/live/example.com/
docker exec -it nginx-container nginx -t
If certificate renewal occurs on the host while Nginx runs inside a container, the renewal process also needs a reliable way to make the new certificate available inside the container and reload Nginx afterward, seriously.
SSL issues in CI and CD pipelines
Certificate deployment should be treated as part of infrastructure deployment rather than as an undocumented manual task, honestly. Manual certificate replacement is especially dangerous when several production servers exist, basically.
A CI or CD pipeline can validate configuration before deployment and verify the live endpoint after deployment, you know? This reduces the chance of successfully deploying a broken certificate configuration, which is huge.
- ●
Validate certificate files
- ●
Validate private key availability
- ●
Validate Nginx configuration
- ●
Deploy the certificate
- ●
Reload the service
- ●
Perform an external HTTPS request
- ●
Verify certificate expiration
- ●
Verify hostname coverage
- ●
Alert if deployment verification fails, basically
The pipeline should fail when certificate validation fails, seriously. A deployment that reports success while HTTPS is broken creates a false sense of reliability and delays incident detection, basically.
Certificate deployment permissions
Linux file permissions can cause renewal processes and web servers to behave differently, honestly. The certificate renewal tool may successfully generate new files while the web server cannot read them, which is annoying.
sudo ls -la /etc/letsencrypt/live/example.com/
sudo ls -la /etc/letsencrypt/archive/example.com/
Avoid solving permission problems by making private key files globally readable, basically. Instead, identify which process needs access and configure ownership, groups, or deployment mechanisms appropriately, you know?
A production incident response workflow
- ●
Confirm the reported hostname
- ●
Determine whether the issue affects all users or only specific clients
- ●
Check DNS resolution
- ●
Check certificate expiration
- ●
Verify the certificate served publicly
- ●
Inspect the certificate chain
- ●
Validate hostname coverage
- ●
Check the TLS protocol negotiation
- ●
Check reverse proxy and load balancer configuration
- ●
Verify every production node
- ●
Verify automated renewal logs
- ●
Check certificate file permissions
- ●
Validate Nginx configuration
- ●
Reload the web server
- ●
Test from multiple geographic locations
- ●
Verify API and application clients
- ●
Document the root cause
- ●
Add monitoring to prevent recurrence, basically
During an incident, avoid making several unrelated configuration changes at once, seriously. Change one thing, test the result, and record what changed. This makes it much easier to identify the actual root cause and prevents accidental secondary failures, basically.
The goal is not only to restore HTTPS quickly, but also to identify why automation or monitoring failed before users noticed the problem, honestly.
Useful production SSL commands
# Inspect the live certificate
openssl s_client -connect example.com:443 -servername example.com
# Check expiration
echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null | openssl x509 -noout -dates
# Inspect certificate details
openssl x509 -in fullchain.pem -text -noout
# Check SAN entries
openssl x509 -in fullchain.pem -text -noout | grep -A1 "Subject Alternative Name"
# Test Nginx configuration
sudo nginx -t
# Show complete Nginx configuration
sudo nginx -T
# Reload Nginx
sudo systemctl reload nginx
# Test certificate renewal
sudo certbot renew --dry-run
Keeping these commands available during an incident can significantly reduce diagnosis time, basically. The important principle is to inspect the certificate from both perspectives: what exists on the server and what the public endpoint actually serves, you know?
Monitoring SSL certificates
Certificate monitoring should not depend exclusively on the same infrastructure that serves the application, honestly. If the monitoring system runs inside the failed environment, it may not detect an external certificate problem reliably, which is bad.
External monitoring can periodically connect to the public HTTPS endpoint and verify certificate expiration, hostname coverage, certificate chain validity, and TLS handshake success, basically.
- ●
Alert 30 days before expiration
- ●
Alert 14 days before expiration
- ●
Alert 7 days before expiration
- ●
Alert when renewal fails
- ●
Alert when the public certificate changes unexpectedly
- ●
Alert when the TLS handshake fails
- ●
Monitor every public hostname
- ●
Monitor every production endpoint, basically
Certificate monitoring is particularly important for organizations with many domains and subdomains, seriously. Managing certificates manually becomes increasingly unreliable as the number of endpoints grows, you know?
Testing certificates before production deployment
A certificate should be validated before it reaches production, honestly. This is especially important when certificates are deployed through infrastructure as code or CI and CD pipelines, basically.
- ●
Confirm the certificate is not expired
- ●
Confirm the private key matches
- ●
Confirm SAN entries are correct
- ●
Confirm the certificate chain is complete
- ●
Validate Nginx configuration
- ●
Test the intended TLS versions
- ●
Test the complete deployment process
- ●
Verify the live endpoint after deployment, basically
A staging environment should reproduce the production TLS architecture as closely as possible, you know? If production uses a CDN and load balancer, testing only against a local Nginx server does not validate the entire production path, seriously.
Common mistakes that make SSL incidents worse
- ●
Renewing a certificate without checking why renewal failed
- ●
Changing DNS during an active certificate incident without a clear reason
- ●
Replacing private keys without understanding their relationship to the certificate
- ●
Testing only from one browser
- ●
Checking only the certificate file on disk
- ●
Ignoring intermediate certificates
- ●
Forgetting the load balancer certificate
- ●
Reloading the wrong Nginx instance
- ●
Changing multiple configuration files simultaneously
- ●
Disabling TLS verification as a permanent workaround, basically
Temporary workarounds can be useful for controlled debugging, but disabling certificate validation in production clients is not a proper fix, basically. It removes one of the most important security protections provided by TLS and can hide the underlying configuration problem, you know?
Prevention strategy
The most reliable SSL strategy combines automation, monitoring, redundancy, and documented deployment procedures, honestly. Certificate management should be treated as an ongoing operational responsibility rather than something that is configured once and forgotten, basically.
- ●
Automate certificate renewal
- ●
Monitor expiration continuously
- ●
Use full certificate chains
- ●
Test renewals regularly
- ●
Version control SSL configuration
- ●
Document certificate ownership
- ●
Avoid manual certificate deployment
- ●
Monitor every production hostname
- ●
Test certificates after deployment
- ●
Keep production nodes consistent
- ●
Document rollback procedures
- ●
Monitor renewal job failures, basically
In mature DevOps environments, SSL management is treated as infrastructure, not as a one-time security task, seriously. Certificates are generated, deployed, validated, monitored, and renewed through predictable processes, you know?
The best prevention strategy is to assume that automation will eventually fail, basically. The important question is whether the failure will be detected while there are still days or weeks available to fix it, or whether users will discover the problem after the certificate has already expired, which is a mess.
Automation prevents repetitive manual work, honestly. Monitoring prevents automation failures from becoming production incidents. You need both, basically.
Production SSL checklist
- ●
Certificate is currently valid
- ●
Certificate contains every required hostname
- ●
Certificate chain is complete
- ●
Private key matches the certificate
- ●
Nginx points to the correct certificate files
- ●
Nginx configuration passes validation
- ●
The web server has been reloaded after renewal
- ●
The public endpoint serves the expected certificate
- ●
All load balancers use the correct certificate
- ●
All production nodes are synchronized
- ●
Automated renewal is enabled
- ●
Renewal dry runs succeed
- ●
Certificate expiration monitoring is enabled
- ●
DNS records point to the intended infrastructure
- ●
Firewall rules allow the required validation method
- ●
TLS versions are compatible with supported clients
- ●
HTTP redirects correctly to HTTPS
- ●
Private keys are protected
- ●
The deployment process is documented
- ●
An incident rollback procedure exists, basically
Frequently asked questions
Why does HTTPS work in Chrome but fail in Java?
How can I check which certificate my server is serving?
Is a valid certificate enough?
Why is my new certificate not visible after renewal?
Why does the certificate work on one server but not another?
Can DNS cause an SSL certificate error?
Does redirecting HTTP to HTTPS fix an invalid certificate?
Should I disable certificate validation when an API client fails?
How often should certificate renewal be tested?
What should I monitor for production SSL?
Can a load balancer have a different certificate from the origin?
Why does an SSL problem appear intermittent?
Final thoughts
SSL certificate failures are rarely caused by one mysterious security problem, honestly. In most production environments, the root cause can be traced to a small number of predictable areas: expiration, certificate chains, hostname coverage, renewal automation, DNS, reverse proxies, load balancers, private keys, or deployment procedures, basically.
The most important lesson is to troubleshoot the complete production path rather than focusing only on the certificate file, seriously. Start with the public hostname, inspect the certificate actually served to clients, identify where TLS is terminated, and then work backward through the infrastructure, you know?
Once the immediate incident is resolved, investigate why it happened, basically. If a certificate expired, determine why monitoring did not detect it. If renewal failed, determine why the renewal job was not visible. If a wrong certificate was deployed, determine why deployment validation did not catch it, honestly.
A production SSL system is reliable when certificates renew automatically, failures are detected early, deployments are validated, and the team can quickly identify exactly where TLS is terminated, basically.
