Commit Graph

3 Commits

Author SHA1 Message Date
Patrick Plate 2ab847f51d feat(webscraper): add Brave Search hint tool and User-Agent header
- Add webscraper_search_hint() tool using Brave Search as backend
  (no CAPTCHA/GDPR consent wall, works with plain httpx)
- Add User-Agent header to _fetch_page() — fixes 403 on Wikipedia,
  Feynman Lectures, and other sites that block headless requests
- Add 5 new tests for search hint (23 total, 90% coverage)

Brave Search URL: https://search.brave.com/search?q={query}&source=web
Use sparingly — once per research task as orientation, not in loops
2026-04-05 09:37:30 +02:00
pplate 5a96359bb1 fix(mcp-webscraper): use certifi SSL context + bundled Comodo root cert
- _build_ssl_context() loads certifi bundle + all *.pem from certs/ dir
- _SSL_CTX singleton built at module load, passed to httpx.get(verify=...)
- Fixes SSLCertVerificationError on Cloudflare-served sites on Fedora 43
  (Comodo AAA root cert missing from system trust store)
- test_server.py: fix HTTPStatusError mock to include request= param
2026-04-04 09:52:26 +02:00
Patrick Plate 155d56e8e8 chore: reorganize into polyglot monorepo (workshop)
- Move bigmind/ -> mcp/bigmind/
- Move webscraper/ -> mcp/webscraper/
- Move mss-failsafe/ -> java/mss-failsafe/
- Move Wellmann-Shop/ -> java/wellmann-shop/ (normalize to kebab-case)
- Add .roo/ IDE config files to tracking
- Add plans/REPO_STRATEGY.md (monorepo strategy document)
- Expand .gitignore: Java/Maven, Node/TS, coverage, uv.lock
- Rewrite README.md as navigation index
- Update .roo/mcp.json webscraper path to mcp/webscraper/
2026-04-04 08:51:15 +02:00