A comprehensive guide to debugging and monitoring in Drupal like a pro

Observing colleagues work with Drush, it’s clear they have the basics down pat. Recently, I conducted a crash course in upskilling like a pro. Moving beyond the frequent commands many have already mastered, such as:

drush cr
drush updb
drush status

These commands are fantastic for quick tasks like clearing caches, running database updates, or checking site status. Yet, to truly take control of your environment, it’s worth expanding your knowledge of Drush and logs.

When working with Drupal, especially in enterprise or large-scale deployments, monitoring and debugging logs is critical for maintaining a stable and high-performing application. Drupal’s command-line tool, Drush, provides powerful logging capabilities that can be used to pinpoint and resolve issues effectively.

Understanding Drupal logs and their role

Drupal logs, also known as the watchdog logs, are managed through the Database Logging module (dblog) or syslog. These logs capture critical information about errors, warnings, notices, and system events that occur within a Drupal site. This becomes particularly critical when the site encounters the infamous "white screen of death." Understanding how to leverage these command-line tools can significantly minimise the time required to resolve issues. Drush provides a command-line interface to access, filter, and analyse these logs without requiring access to the Drupal admin UI.

Getting started with drush watchdog commands

Drush offers the watchdog namespace for interacting with logs. The primary command for viewing logs is:

drush watchdog:show

This command outputs the most recent log entries, displaying relevant details such as severity, type, and message. For example:

 --------- -------------- -------------------- ---------- -----------------------------------------------------------------------------------------------
  ID        Date           Type                 Severity   Message
 --------- -------------- -------------------- ---------- -----------------------------------------------------------------------------------------------
  3062412   05/Jan 12:03   database_logger      Info       5 log entries removed for 5 jobs
  3062411   05/Jan 12:02   ultimate_cron_lock   Info       Cleaned up 7 expired locks
  3062410   05/Jan 12:01   serial_launcher      Warning    No free threads available for launching jobs
  3062409   05/Jan 12:01   serial_launcher      Warning    No free threads available for launching jobs
  3062408   05/Jan 12:01   serial_launcher      Warning    No free threads available for launching jobs
  3062407   05/Jan 12:01   serial_launcher      Warning    No free threads available for launching jobs
  3062406   05/Jan 11:46   webform              Notice     Contact webform sent Email notification email.
  3062405   05/Jan 11:46   webform_submission   Error      Contact: Email not sent for Email confirmation handler because a To, CC, or BCC email was not
                                                           provided
  3062404   05/Jan 11:46   webform              Notice     Contact: Submission #0000 created.
 --------- -------------- -------------------- ---------- -----------------------------------------------------------------------------------------------

Filtering logs for specific issues

Filtering by type

Logs can be filtered based on type, which is particularly useful for debugging specific modules or features. Such as to view logs related to a custom module that uses the serial_launcher type:

drush watchdog:show --type=serial_launcher

This command isolates entries associated with the serial_launcher module, making it easier to identify patterns and recurring issues.

Common types include:

• php: Logs related to PHP errors and warnings
• cron: Logs for scheduled tasks and cron runs
• user: Logs related to user activity, such as logins and failed attempts
• database: Logs related to database queries and errors
• system: Logs for system-level events like cache clears or module installs
• content: Logs related to content creation, updates, and deletions
• config: Logs related to configuration changes
• security: Logs related to security events, such as permissions changes or access violations
• views: Logs for issues related to views and query execution
• block: Logs for block rendering and placement errors
• menu: Logs related to menu management and routing issues
• search: Logs related to search indexing and queries
• watchdog: Logs for the logging system itself

To view logs specific to database errors:

drush watchdog:show --type=database

Filtering by message

Logs can also be filtered based on specific keywords within their messages. This is particularly useful for narrowing down logs related to specific errors, warnings, or actions. To search for logs containing a keyword, use the --message option

drush watchdog:show --message="keyword"

For example, to find logs containing the keyword "permission"

drush watchdog:show --message="permission"

This command will return all logs that contain the word "permission" in their messages, enabling you to quickly isolate relevant logs.

You can also search for phrases within messages by enclosing them in quotes

drush watchdog:show --message="access denied"

This command returns all logs containing the exact phrase "access denied" in their messages, enabling you to quickly isolate relevant logs.

Combining multiple filters

Drush provides flexible filtering options to refine log searches using the watchdog:show command. For example, to display only errors related to the serial_launcher type, you can use the following command

drush watchdog:show --type=serial_launcher --severity=error

Additional filtering options include:

• --limit=10: Displays the latest 10 log entries
• --reverse: Displays logs in reverse order (newest first)
• --format=json: Outputs logs in JSON format for structured data processing
• --fields=id,type,severity,message: Customises the fields displayed in the output

Combining Multiple Options

You can combine multiple filters and output formats to refine results further. For example, the following command retrieves the latest 5 error logs for cron jobs and formats them in JSON

drush watchdog:show --type=cron --severity=error --limit=5 --format=json

This command retrieves the latest 5 error logs for cron jobs and formats them in JSON for better readability.

Benefits of Using JSON Format

Using --format=json provides several advantages:

• Improved readability: Logs are displayed in a structured format, making it easier to identify key details
• Ease of parsing: JSON is widely supported in programming languages, allowing logs to be parsed programmatically for further analysis
• Detailed information: JSON format includes all log fields without truncation, ensuring no critical information is lost
• Integration with tools: JSON logs can be imported into third-party tools like log analyzers, dashboards, or monitoring systems

Log output in JSON vs regular format

JSON format

{
   "3062427": {
       "wid": "3062427",
       "type": "client error",
       "message": "Symfony\\Component\\HttpKernel\\Exception\\BadRequestHttpException: Input value \"name\" contains a non-scalar value. in Symfony\\Component\\HttpKernel\\HttpKernel->handle() (line 83 of /var/www/vh",
       "severity": "Warning",
       "location": "https://www.codebales.com/user/password",
       "hostname": "xx.xxx.xx.xxx",
       "date": "05/Jan 13:06",
       "username": "Anonymous",
       "uid": "0"
   }
}

Regular watchdog response

3062427   05/Jan 13:06   client error         Warning    Symfony\Component\HttpKernel\Exception\BadRequestHttpException: Input value "name" contains a
                                                    non-scalar value. in Symfony\Component\HttpKernel\HttpKernel->handle() (line 83 of /var/www/vh

Key differences

• Truncation: The regular format may truncate long messages, while JSON provides the complete details
• Structure: JSON separates data into key-value pairs, making it easier to parse and analyse
• Scalability: JSON logs can be exported and processed in bulk, making them suitable for enterprise-level monitoring systems

Filtering by severity

Logs are categorised by severity levels, which help administrators quickly assess and respond to system events. These severity levels provide insight into system performance, errors, and potential issues

• Emergency (0) - System is unusable and requires immediate attention
• Alert (1) - Immediate action is required to prevent service failure
• Critical (2) - Critical conditions that may cause system instability
• Error (3) - Errors that need immediate investigation and resolution
• Warning (4) - Potential issues that should be monitored but do not require immediate action
• Notice (5) - Informational messages about system events
• Info (6) - General operational logs, useful for tracking system activities
• Debug (7) - Detailed debugging information, often used during development

You can filter logs based on severity to identify specific issues or monitor critical system events. The Drush command-line tool provides an easy way to query logs directly from the database.

Viewing Logs for a Specific Severity Level

To filter logs and display only those with a specified severity level, use the following command:

drush watchdog:show --severity=error

Explanation of the command:

• watchdog:show - Displays the log messages stored in the watchdog (database logging) table
• --severity=error - Filters logs to include only entries with the severity level set to Error (level 3)

Filtering Multiple Severity Levels

To filter logs for multiple severity levels, use their numerical codes in a comma-separated list. For example, to filter logs with severity levels Critical (2) and Error (3):

drush watchdog:show --severity=2,3

Filtering by date

To review logs within a specific timeframe, you can filter them by specifying the date range. The date format must strictly follow the ISO 8601 format (YYYY-MM-DD), including the year, month, and day components. This ensures precise filtering of logs based on specific dates and times.

drush watchdog:show --start-date="2024-01-01" --end-date="2024-01-05"

Date format:

• The dates in this example follow the ISO 8601 format (YYYY-MM-DD), where:
  • YYYY represents the four-digit year (e.g., 2024)
  • MM represents the two-digit month (e.g., 01 for January)
  • DD represents the two-digit day (e.g., 01 for the 1st day of the month)

Breakdown with example:

• --start-date="2024-01-01" specifies the starting point for the log search, January 1, 2024
• --end-date="2024-01-05" specifies the ending point for the log search, January 5, 2024

This date range filters logs recorded from January 1, 2024 (inclusive) to January 5, 2024 (inclusive).

If only a start date is provided, the end date defaults to the current day. For instance:

drush watchdog:show --start-date="2024-01-01"

This command retrieves logs from January 1, 2024, up until today.

These approaches are valuable for identifying issues that occurred during a deployment or upgrade.

Exporting Logs for Analysis

To export filtered logs into a file for further analysis, you can redirect the output as follows

drush watchdog:show --severity=error > error_logs.txt

The wrap

Exploring and debugging Drupal logs with Drush is a powerful method for maintaining site stability, diagnosing errors, and improving performance. By leveraging advanced filtering options, real-time monitoring, and log exports, developers can efficiently manage and analyse logs. Incorporating these practices into your workflow ensures better insight into site behaviour and faster issue resolution.