regex

Introduction

Data extraction and filtering are crucial for developers working with large research datasets. Whether you're working on government archives, industry reports, or academic research projects, extracting meaningful insights efficiently can be challenging.  

Working on a project where the JSON dataset contains over 460,000 named records, we are preparing to upsert these records into Pinecone. However, for validation and testing purposes, it's essential to cross-check how many times a specific term appears across the dataset. To ensure data integrity, we only want to include one record per occurrence, regardless of whether the term appears once or multiple times within a record.

If you're working with a large codebase in Visual Studio Code (VS Code) and need to find specific occurrences of a term, but only when it's a standalone word possibly surrounded by spaces or parentheses, regular expressions (regex) are your best friend. This guide will walk you through the steps to efficiently search for these instances, I'll be searching for the term 'SBT' and ensuring you don't pick up unwanted matches like `'ADSBTCR'` or `'SBT123'`.