How to create and use a Python virtual environment in your web project

Managing dependencies in a web project can be challenging, especially when different projects require different versions of the same packages. Virtual environments offer a solution by isolating project dependencies. This article will guide you through creating and using a Python virtual environment in a web project, specifically when using PHP to execute Python scripts.

Why use a virtual environment?

Isolation: Keeps dependencies required by different projects in separate places.
Dependency Management: Avoids conflicts between package versions.
Security: Limits the scope of installed packages to the project directory, reducing potential security risks.

Steps to create and use a virtual environment

1. Install required system packages

Before creating a virtual environment, ensure that the required system packages are installed. For Debian-based systems (like Ubuntu), you can use the following commands:

sudo apt-get update
sudo apt-get install libffi-dev libssl-dev python3.12 python3.12-venv

2. Navigate to your project directory

Navigate to the directory where you want to create the virtual environment. For example, if your project directory is /var/www/html/fprog, use:

cd /var/www/html/{project}

3. Create the virtual environment

Use Python 3.12 to create a virtual environment named aisearch:

python3.12 -m venv aisearch

4. Activate the virtual environment

Activate the virtual environment. This step is necessary each time you want to use the environment.

On Windows

aisearch\Scripts\activate

On macOS/Linux

source aisearch/bin/activate

5. Install required packages

Once the virtual environment is activated, install the necessary packages. For example, to install torch and python-dotenv:

pip install \
aiohttp==3.9.0 \
aiosignal==1.3.1 \
annotated-types==0.6.0 \
anyio==4.2.0 \
asttokens==2.4.1 \
asyncio==3.4.3 \
attrs==23.1.0 \
fastapi==0.110.0 \
filetype==1.2.0 \
httpcore==1.0.2 \
httpx==0.25.1 \
httpx-sse==0.4.0 \
huggingface-hub==0.21.4 \
icecream==2.1.3 \
langchain==0.1.13 \
langchain-cli==0.0.21 \
langchain-community==0.0.29 \
langchain-core==0.1.33 \
langchain-openai==0.1.0 \
langchain-pinecone==0.0.3 \
langchain-text-splitters==0.0.1 \
langchainhub==0.1.14 \
loguru==0.7.2 \
opencv-python==4.8.1.78 \
pillow==10.2.0 \
python-dateutil==2.8.2 \
python-decouple==3.8 \
python-dotenv==1.0.0 \
python-iso639==2023.6.15 \
python-magic==0.4.27 \
scipy==1.12.0 \
torch==2.2.2 \
transformers==4.38.2 \
wrapt==1.16.0

6. Modify your PHP code to use the virtual environment

Update your PHP code to activate the virtual environment before running the Python script.  Following is an example in the PHP code

<?php

namespace Drupal\summaries\Form;

use Drupal\Core\Form\FormBase;
use Drupal\Core\Form\FormStateInterface;

class SummariesForm extends FormBase {

  /**
   * {@inheritdoc}
   */
  public function getFormId() {
    return 'summaries_form';
  }

  /**
   * {@inheritdoc}
   */
  public function buildForm(array $form, FormStateInterface $form_state) {
    // Form building code here...

    return $form;
  }

  /**
   * {@inheritdoc}
   */
  public function submitForm(array &$form, FormStateInterface $form_state) {
    // Path to the virtual environment activation script
    $venv_path = '/var/www/html/{project}/{venv}/bin/activate';
    $python_path = '/var/www/html/{project}/{venv}/bin/python';
    $open_ai_path = '/var/www/html/{project}/';
    $script_name = 'main.py';
    $question = $form_state->getValue('question'); // Assuming there's a form field named 'question'
    $request = "rag:" . $question;

    if (!$this->isValidInput($question)) {
      return;
    }

    // Construct the Python command with virtual environment activation
    $python_command = escapeshellcmd("source $venv_path && $python_path $open_ai_path$script_name $request");

    // Execute the command
    $json_response = shell_exec($python_command);

    // As a string if using shell_exec()
    // Use regular expression to match content within braces
    preg_match('/\{.*\}/s', $json_response, $matches);

    $this->log_message('JSON response', $json_response);

    $data = [];

    // Check if we found a match
    if (!empty($matches)) {
      $jsonString = $matches[0];
      
      // Decode the JSON string
      $decodedData = json_decode($jsonString, true);

      // Check if the JSON decoding was successful
      if (json_last_error() === JSON_ERROR_NONE) {
        $data['question'] = $decodedData['question'] ?? null;
        $data['response'] = $decodedData['response'] ?? null;
        $data['metadata'] = $decodedData['metadata'] ?? null;

      } else {
        $this->log_message('Failed to decode JSON', json_last_error_msg() . PHP_EOL);
      }
    }

    return $data;
  }
}

7. Set Permissions

Ensure the www-data user has the necessary permissions to execute the virtual environment and the Python script:

chown -R www-data:www-data /var/www/html/{project}
chmod -R 755 /var/www/html/{project}
chown -R www-data:www-data /var/www/html/{project}
chmod -R 755 /var/www/html/{project}

The wrap

By following these steps, you can create and use a Python virtual environment in your web project, ensuring better dependency management and security.