Python error: in putheader if _is_illegal_header_value(values[i])

Snapshot of the error

File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1271, in putheader
    if _is_illegal_header_value(values[i]):

The above error points to the putheader definition in client.py file located 

python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/

The putheader definition in the client.py file:

1250     def putheader(self, header, *values):
1251         """Send a request header line to the server.
1252
1253         For example: h.putheader('Accept', 'text/html')
1254         """
1255         if self.__state != _CS_REQ_STARTED:
1256             raise CannotSendHeader()
1257
1258         if hasattr(header, 'encode'):
1259             header = header.encode('ascii')
1260
1261         if not _is_legal_header_name(header):
1262             raise ValueError('Invalid header name %r' % (header,))
1263
1264         values = list(values)
1265         for i, one_value in enumerate(values):
1266             if hasattr(one_value, 'encode'):
1267                 values[i] = one_value.encode('latin-1')
1268             elif isinstance(one_value, int):
1269                 values[i] = str(one_value).encode('ascii')
1270
1271             if _is_illegal_header_value(values[i]):
1272                 raise ValueError('Invalid header value %r' % (values[i],))
1273
1274         value = b'\r\n\t'.join(values)
1275         header = header + b': ' + value
1276         self._output(header)

This error occurred whether run locally and noted in the errors on this page or on the server (in this case Ubuntu).

Working through the issue

Environment file (.env)

I read many suggestions that the error was a result of the variables not being set in the environment file (.env).  Given I was able to upsert the embeddings, this didn't make sense.  I cross checked by creating an .env file and added the 

PINECONE_API_KEY=/... YOUR KEY .../
PINECONE_API_ENV=/... YOUR ENV .../

As thought, no impact on the error.

Pinecone API key has a response

Reading the following post - https://stackoverflow.com/questions/77463987/tryin-to-connect-to-pinecone-but-the-code-only-work-in-jupyter-notebook-not-as, I thought about the os.getenv("PINECONE_API_KEY").  I knew this was correct and outputting.

Initialised?

Continuing from the previous steps, a realisation prompted a validation check to ensure Pinecone had been properly initialised. Despite expectations, testing revealed an oversight.

To rectify this, I connected the initialisation process in the relevant script. Specifically, within the search_from_existing_index() function, I incorporated pinecone_processor.initialise_pinecone() to address the initialisation discrepancy.

# Get Pinecone instance from an existing index.
def search_from_existing_index():
	# Initialise Pinecone
	pinecone_processor.initialise_pinecone()

	embeddings = openai_manager.embeddings

	# already have an index
	# This method is used when you already have a Pinecone index and want to instantiate a Pinecone client object to interact with that existing index.
	search_index = Pinecone.from_existing_index(index_name=common_variables.pinecone_index_name, embedding=embeddings)

	return search_index


# Run the search in the document(s0.)
def search_pinecone(search_index, prompt):

	llm = ChatOpenAI(temperature=common_variables.temperature, openai_api_key=OPENAI_API_KEY, model_name=common_variables.model)
	chain = load_qa_chain(llm, chain_type=common_variables.chain_type)

	docs = search_index.similarity_search(prompt, include_metadata=True)

	response = chain.run(input_documents=docs, question=prompt)

	return response


# Document query
# Current query question is - "What are examples of good data science teams?"
def query_pinecone_documents(prompt):
	# Use the global keyword to indicate that you're modifying the new documents
	# global common_variables.new_documents

	search_index = search_from_existing_index()

	# llm = OpenAI(temperature=0, openai_api_key=llm_key)
	# chain = load_qa_chain(llm, chain_type="stuff")
	# prompt the document register.
	response = search_pinecone(search_index, prompt)
	# output = chain.run(input_documents=docs, question=prompt)

	return response

In case you need further details, the initialise definition is:

def initialise_pinecone():
  pinecone.init(
    api_key=PINECONE_API_KEY,
    environment=PINECONE_API_ENV
  )

Appendix

def from_existing_index

I delved into the definition and requirements of from_existing_index to gain a better understanding of the functionality. This exploration was part of the process of comprehending the underlying issue.

@classmethod
def from_existing_index(
        cls,
        index_name: str,
        embedding: Embeddings,
        text_key: str = "text",
        namespace: Optional[str] = None,
    ) -> Pinecone:
        """Load pinecone vectorstore from index name."""
        try:
            import pinecone
        except ImportError:
            raise ValueError(
                "Could not import pinecone python package. "
                "Please install it with `pip install pinecone-client`."
            )

        return cls(
            pinecone.Index(index_name), embedding.embed_query, text_key, namespace
        )

The full error

Traceback (most recent call last):
  File "/Users/{user}/Sites/AI%20Search/main.py", line 73, in <module>
    outcome = process_request(file)
              ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/{user}/Sites/AI%20Search/main.py", line 42, in process_request
    outcome = query_pinecone.query_pinecone_documents(request)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/{user}/Sites/AI%20Search/processes/query_pinecone.py", line 152, in query_pinecone_documents
    search_index = search_from_existing_index()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/{user}/Sites/AI%20Search/processes/query_pinecone.py", line 121, in search_from_existing_index
    search_index = Pinecone.from_existing_index(index_name=index, embedding=embeddings)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/langchain/vectorstores/pinecone.py", line 442, in from_existing_index
    pinecone_index = cls.get_pinecone_index(index_name, pool_threads)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/langchain/vectorstores/pinecone.py", line 359, in get_pinecone_index
    indexes = pinecone.list_indexes()  # checks if provided index exists
              ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/pinecone/manage.py", line 185, in list_indexes
    response = api_instance.list_indexes()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/pinecone/core/client/api_client.py", line 776, in __call__
    return self.callable(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/pinecone/core/client/api/index_operations_api.py", line 1130, in __list_indexes
    return self.call_with_http_info(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/pinecone/core/client/api_client.py", line 838, in call_with_http_info
    return self.api_client.call_api(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/pinecone/core/client/api_client.py", line 413, in call_api
    return self.__call_api(resource_path, method,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/pinecone/core/client/api_client.py", line 200, in __call_api
    response_data = self.request(
                    ^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/pinecone/core/client/api_client.py", line 439, in request
    return self.rest_client.GET(url,
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/pinecone/core/client/rest.py", line 236, in GET
    return self.request("GET", url,
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/pinecone/core/client/rest.py", line 202, in request
    r = self.pool_manager.request(method, url,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/urllib3/_request_methods.py", line 110, in request
    return self.request_encode_url(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/urllib3/_request_methods.py", line 143, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/urllib3/poolmanager.py", line 444, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/urllib3/connectionpool.py", line 790, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/urllib3/connectionpool.py", line 496, in _make_request
    conn.request(
  File "/opt/homebrew/lib/python3.11/site-packages/urllib3/connection.py", line 394, in request
    self.putheader(header, value)
  File "/opt/homebrew/lib/python3.11/site-packages/urllib3/connection.py", line 308, in putheader
    super().putheader(header, *values)
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1271, in putheader
    if _is_illegal_header_value(values[i]):