Mastering Application Logging Best Practices

Source: Dev.to

As a DevOps engineer or developer, you’ve likely encountered a situation where an application fails in production and the only clue is a cryptic error message. You spend hours poring over code, trying to reproduce the issue, only to realize that the problem lies in a subtle mis‑configuration or unexpected user input.

This scenario highlights the importance of application logging in production environments. Effective logging enables you to debug issues efficiently, ensuring minimal downtime and an optimal user experience.

Why Logging Is Often an Afterthought

Application logging is frequently added as an after‑thought rather than being integrated into the development process. This can lead to:

• Insufficient log data – making it difficult to reproduce and debug problems.
• Overly verbose or overly sparse messages – providing little useful information.
• Improper formatting – causing difficulties in parsing and analysis.

Real‑World Example

A web application experiences intermittent 500 errors. Without proper logging, it’s hard to determine whether the root cause is:

• A database connection problem
• Server overload
• A coding error

Inadequate logging forces you to rely on trial‑and‑error, increasing time‑to‑resolution and potentially causing further issues.

Prerequisites for Effective Logging

1. Basic understanding of logging concepts – log levels, formats, and destinations.
2. Familiarity with your language’s logging library (e.g., Log4j for Java, Python’s built‑in logging).
3. Container platform (optional) – Docker, Kubernetes, etc.
4. Log aggregation tool – ELK (Elasticsearch, Logstash, Kibana), Splunk, Loki, etc.

Diagnosing Logging Issues

Start by reviewing the current logging configuration:

• Check log levels (DEBUG, INFO, WARN, ERROR).
• Verify log formats (plain text, JSON, etc.).
• Confirm output destinations (stdout, files, external services).

Example: Inspecting Pods in Kubernetes

# List all pods that are NOT in the Running state
kubectl get pods -A | grep -v Running

You can adapt this command to suit your specific logging setup.

Example: Streaming Logs from All Non‑Running Pods

kubectl get pods -A | grep -v Running \
  | awk '{print $1}' \
  | xargs -I {} kubectl logs -f {}

Note: The above command does not set the log level; it merely streams logs. Adjust the command according to your logging library and configuration.

Verifying Your Logging Implementation

When you run your application, the log output should contain:

• Timestamps for each entry
• Log level (DEBUG, INFO, WARN, ERROR)
• Relevant messages (including error details)
• Consistent formatting (e.g., JSON or a defined pattern)

Use tools like kubectl logs, your aggregation platform’s UI, or CLI utilities (curl, jq, etc.) to inspect the logs.

Sample Logging Configurations

1. Kubernetes Manifest with Logging Environment Variables

apiVersion: v1
kind: Pod
metadata:
  name: example-pod
spec:
  containers:
    - name: example-container
      image: example-image
      env:
        - name: LOG_LEVEL
          value: DEBUG
        - name: LOG_FORMAT
          value: json

2. Python Logging (built‑in logging module)

import logging

logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s %(levelname)s: %(message)s',
)
logger = logging.getLogger(__name__)

def main():
    logger.debug('This is a debug message')
    logger.info('This is an info message')
    logger.warning('This is a warning message')
    logger.error('This is an error message')

if __name__ == '__main__':
    main()

3. Java Logging with Log4j

import org.apache.log4j.Logger;

public class Example {
    private static final Logger logger = Logger.getLogger(Example.class);

    public static void main(String[] args) {
        logger.debug("This is a debug message");
        logger.info("This is an info message");
        logger.warn("This is a warning message");
        logger.error("This is an error message");
    }
}

Common Pitfalls & How to Avoid Them

Key Takeaways

• Log relevant information – user input, DB queries, error details.
• Standardize log formats – JSON or a consistent pattern simplifies parsing.
• Implement rotation & retention – prevents disks from filling up.
• Monitor & analyze logs – leverage aggregation tools for real‑time insights.
• Adjust log levels per environment – DEBUG for development, INFO/WARN/ERROR for production.

Conclusion

Application logging is a critical component of both development and debugging. By following the best practices outlined above, you can:

• Quickly pinpoint issues in production.
• Reduce downtime and improve user experience.
• Maintain clean, searchable, and manageable log data.

Remember: log the right data, in the right format, and manage it responsibly. 🚀

Application Logging

• Monitor and analyze logs
• Adjust log levels according to your application’s needs

Want to learn more about logging?

• Logging aggregation tools – Explore popular platforms such as ELK, Splunk, and Loggly.
• Log analysis & parsing – Learn techniques for parsing and analyzing log data, including regular expressions and dedicated log‑parsing libraries.
• Observability & monitoring – Understand how observability ties together logging, monitoring, and debugging in modern applications.

Kubernetes Troubleshooting Resources

• Lens – The Kubernetes IDE that makes debugging 10× faster.
• k9s – Terminal‑based Kubernetes dashboard.
• Stern – Multi‑pod log tailing for Kubernetes.
• Kubernetes Troubleshooting in 7 Days – Step‑by‑step email course ( $7 ).
• “Kubernetes in Action” – The definitive guide (Amazon).
• “Cloud Native DevOps with Kubernetes” – Production best practices.

Subscribe to the DevOps Daily Newsletter

• 3 curated articles per week
• Production incident case studies
• Exclusive troubleshooting tips

Found this helpful? Share it with your team!