Automated company intelligence pipeline and prospect research agent built for the AI & Automation Developer Hackathon. Traverses sitemaps with Cloudflare anti-bot bypass, extracts verified contact info using regex pre-processing, and structures business profiles with Gemini 2.5 Flash.
Sales, business development, and market intelligence teams spend hundreds of hours manually browsing target company websites, hunting through sitemaps for verified contact info, and copy-pasting unstructured blurbs into CRMs. Standard scrapers frequently break on Cloudflare anti-bot challenges, while feeding raw, noisy HTML directly into LLMs wastes context tokens and produces hallucinated business summaries.
CompanyLens automates company prospect research end-to-end. It uses fuzzy string matching to prioritize high-value sitemap pages (About, Solutions, Contact), deploys cloudscraper to cleanly bypass anti-bot protections, extracts verified emails and phones via deterministic regex pre-processing, and passes clean markdown to Gemini 2.5 Flash for structured JSON synthesis and competitive positioning insights.
Intelligently parses sitemap.xml and homepage links with weighted string matching to auto-prioritize target pages like About, Contact, and Solutions.
Rotates user-agents dynamically and falls back to browser-spoofing cloudscraper to bypass endpoint protection layers seamlessly.
Pre-harvests email addresses and phone numbers directly from raw HTML, passing candidate lists to the LLM for strict verification rather than generation.
Configured with reasoning token budgets on gemini-2.5-flash with automated fallbacks to gemini-2.0-flash and gemini-2.0-flash-lite.
Real-time frontend status updates showing step-by-step progress (Scraping ➔ Analyzing ➔ Generating) with responsive visual indicators.
Modern dark-mode web dashboard featuring instant toggle between detailed card grid and expandable table views with slide-out detail drawers.
scraper.py — Hardened sitemap crawler with Brotli stream decoding resilienceenricher.py — Prompt engineering, token optimizer, and Gemini model fallback managermain.py — FastAPI production web server powering REST APIs and static servingdatabase.py — SQLite storage engine maintaining historical company profile recordsstatic/index.html — HTML5 single-page application structurestatic/app.js — Dynamic API handler, multi-stage progress indicators, and modal drawersstatic/styles.css — CSS variables, glassmorphism card components, and CSS animationsrender.yaml — Production Render blueprint for 1-click cloud deploymentReasoning LLM with structured output generation for corporate analysis and outreach personalization.
High-performance asynchronous Python REST server with auto-generated Swagger OpenAPI documentation.
Resilient web page fetching, anti-bot bypass, and DOM text content extraction.
Embedded relational store for enriched company data and custom dark-theme presentation UI.