No description
  • Python 95.4%
  • Just 2.9%
  • Shell 1.7%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Gurbakhshish Singh d3fb3f7826
All checks were successful
CI / SCA (Trivy) (push) Successful in 8s
CI / Tests (push) Successful in 21s
CI / SAST (Bandit) (push) Successful in 3s
CI: switch check container to gitea/runner-images:ubuntu-latest
2026-08-27 13:29:13 -04:00
.forgejo CI: switch check container to gitea/runner-images:ubuntu-latest 2026-08-27 13:29:13 -04:00
src/jobseach Add Forgejo CI: tests, Bandit SAST, Trivy SCA (adapted from suredata_client) 2026-08-27 13:09:21 -04:00
tests Phase 3: thin interfaces over core.service — typer CLI, FastAPI REST, fastmcp MCP server; interface tests 2026-08-27 12:11:39 -04:00
.gitignore Add Forgejo CI: tests, Bandit SAST, Trivy SCA (adapted from suredata_client) 2026-08-27 13:09:21 -04:00
.python-version getting started 2026-08-27 11:08:04 -04:00
AGENTS.md Add Forgejo CI: tests, Bandit SAST, Trivy SCA (adapted from suredata_client) 2026-08-27 13:09:21 -04:00
justfile Add Forgejo CI: tests, Bandit SAST, Trivy SCA (adapted from suredata_client) 2026-08-27 13:09:21 -04:00
PLAN.md Phase 3: thin interfaces over core.service — typer CLI, FastAPI REST, fastmcp MCP server; interface tests 2026-08-27 12:11:39 -04:00
pyproject.toml Phase 3: thin interfaces over core.service — typer CLI, FastAPI REST, fastmcp MCP server; interface tests 2026-08-27 12:11:39 -04:00
README.md Add docs and task runner: README, AGENTS.md, justfile 2026-08-27 12:35:35 -04:00
SPECS.md getting started 2026-08-27 11:08:04 -04:00
uv.lock Phase 3: thin interfaces over core.service — typer CLI, FastAPI REST, fastmcp MCP server; interface tests 2026-08-27 12:11:39 -04:00

jobseach

Search LinkedIn job postings through the public "guest" endpoints, normalize them into canonical records, cache them in SQLite, and serve them through a CLI, a REST API, and an MCP server.

uv run jobseach search --keywords "python developer" --location "United States" --limit 10
┃ Title            ┃ Company              ┃ Location     ┃ Posted     ┃ Salary ┃ Job ID
│ Python Developer │ Open Systems Techno… │ New York, NY │ 2026-08-26 │        │ linkedin:39…
  • No LinkedIn account or auth — uses the public guest API (/jobs-guest/...)
  • Canonical data model — every source normalizes into the same pydantic models
  • SQLite persistence — jobs deduped on (source, job_id), searches cached with per-source TTL
  • Three interfaces, one service — CLI (typer), REST (FastAPI), MCP (fastmcp), zero business logic in any of them
  • Graceful degradation — source failures serve stale cache with an error status; the service never raises

Requirements

  • Python 3.12+
  • uv
  • A residential IP. LinkedIn rate-limits guest traffic: roughly 10 pages per IP before HTTP 429, and datacenter IPs get HTTP 999 outright. jobseach throttles (23 s between requests) and retries with exponential backoff; a stealth-browser fallback (Camoufox) is available if plain requests get blocked.

Setup

uv sync                      # install deps
uv run pytest                # verify: 46 tests, all offline

Optional, only if you hit repeated 429/999 blocks:

uv run camoufox fetch        # download the Camoufox stealth browser (one time)

CLI

uv run jobseach search [OPTIONS]     # search job postings
uv run jobseach get SOURCE JOB_ID    # one job, with full description

search options (all optional except --keywords being useful):

Option Values / format Notes
--keywords free text e.g. "python developer"
--location free text e.g. "United States", "Berlin"
--posted-within past_day past_week past_month pushed to LinkedIn f_TPR
--remote flag pushed to LinkedIn f_WT=2
--seniority internship entry associate mid_senior director executive pushed to LinkedIn f_E
--salary-min annualized integer client-side post-filter; postings without stated pay are kept
--limit / --offset ints paging, pushed via LinkedIn start
--refresh flag bypass cache
--source name restrict to one source (default: all; currently linkedin)
--json flag machine-readable output instead of the rich table
# Table (default)
uv run jobseach search --keywords "data engineer" --remote --posted-within past_week

# Scripting
uv run jobseach search --keywords "data engineer" --salary-min 120000 --json | jq '.jobs[].title'

# One job with description (backfills from the detail endpoint if needed)
uv run jobseach get linkedin 3952273769

REST API

uv run fastapi dev src/jobseach/interfaces/api.py
  • GET /jobs — query params mirror the CLI options above exactly (keywords, location, posted_within, remote, seniority, salary_min, limit, offset, refresh, source). Returns SearchResponse: { "jobs": [...], "status": { "linkedin": "ok" | "cached" | "error: ..." } }
  • GET /jobs/{source}/{job_id} — one canonical Job (fetches the detail page if the stored copy has no description); 404 if unknown.
curl -s "http://127.0.0.1:8000/jobs?keywords=python&remote=true&limit=5" | jq '.status'
curl -s "http://127.0.0.1:8000/jobs/linkedin/3952273769" | jq '.description' | head

MCP server

uv run python -m jobseach.interfaces.mcp_server   # stdio transport
# or: uv run fastmcp run src/jobseach/interfaces/mcp_server.py

Tools (registered in any MCP client, e.g. Claude Desktop):

  • search_jobs(keywords, location, posted_within, remote, seniority, salary_min, limit, offset, refresh, source) — same filter surface as CLI/REST
  • get_job_detail(source, job_id) — one job with full description
# quick client check
import asyncio
from fastmcp import Client
from jobseach.interfaces.mcp_server import mcp

async def main():
    async with Client(mcp) as c:
        r = await c.call_tool("search_jobs", {"keywords": "python developer", "limit": 5})
        print(r.data["status"], len(r.data["jobs"]))

asyncio.run(main())

Configuration

Setting Default Override
Database path ~/.local/state/jobseach/jobs.sqlite3 JOBSEACH_DB=/path/to.db
Cache TTL 24 h (all sources) PER_SOURCE_TTL in src/jobseach/config.py
Request delay 23 s random between calls REQUEST_DELAY_RANGE in config
Retry/backoff 3 attempts, 2 s → 4 s → 8 s on 429/999 MAX_RETRIES in config

How it works

CLI / REST / MCP            (thin interfaces, no business logic)
        │
        ▼
core.service.SearchService  orchestration
        │  1. hash the canonical query per source
        │  2. cache hit & fresh & not refresh → serve from SQLite, no source call
        │  3. else fetch via the source's adapter, upsert jobs, rewrite cache entry
        │  4. post-filter what the source can't (salary_min) on canonical fields
        │  5. source error → serve stale cache with an error status, never raise
        ▼
sources/<name>/             pluggable adapters behind the SourceAdapter protocol
  linkedin/adapter.py       CanonicalQuery → guest-API params (keywords, location,
                            f_E, f_TPR, f_WT, start paging in steps of 25)
  linkedin/transport.py     curl_cffi with Chrome TLS impersonation + throttle +
                            backoff (primary); Camoufox stealth browser (fallback)
  linkedin/parser.py        HTML → model dicts; ALL selectors are module constants
        ▼
storage/sqlite.py           jobs PK (source, job_id) → upsert = dedup,
                            search_cache (query_hash, source) → TTL cache

Canonical Job fields: title, company.name, location.raw, workplace (onsite|remote|hybrid), description, salary (min, max, currency, period, raw — only when the employer stated pay, never inferred), posted_at (exact date from the search card), seniority, source.{source, job_id, url}, fetched_at.

Notes on data fidelity:

  • Salary appears only when LinkedIn shows a stated pay range; otherwise salary: null.
  • Exact posted_at comes from the search cards' <time datetime>; detail pages only show relative dates ("1 week ago").
  • workplace is set when the location carries a marker like (Remote); otherwise null.
  • LinkedIn changes guest HTML every few weeks. When parsing breaks, only sources/linkedin/parser.py needs updating; the fixture tests fail loudly on drift.

Adding a new source (Greenhouse, Lever, Ashby, ...)

Sources plug in behind the SourceAdapter protocol without touching core or interfaces:

  1. Create src/jobseach/sources/<name>/ with an adapter implementing name, search(CanonicalQuery) -> list[Job], and get_job(job_id) -> Job | None. Push down whatever filters the source's API supports; leave the rest to the service post-filter.
  2. register(YourAdapter()) at module import.
  3. Add the module path to _DEFAULT_SOURCES in sources/registry.py.

Cache, dedup, TTL, error handling, and all three interfaces work with the new source automatically (--source <name> / ?source=<name>).

Development

uv run pytest                  # full suite; LinkedIn tests run offline against tests/fixtures/
uv run pytest -q tests/test_linkedin_parser.py
  • tests/fixtures/ holds real HTML captured from the guest endpoints. To refresh: curl -s "https://www.linkedin.com/jobs-guest/jobs/api/seeMoreJobPostings/search?keywords=...&start=0", then update the fixture and any affected assertions.
  • Be polite when testing against live LinkedIn: the transport throttles itself, don't loop --refresh searches in scripts.

Limitations

  • Guest API exposes only what anonymous search shows: no easy-apply, employment-type, or sort filters; no exact posting counts; salary only when the employer states it.
  • Blocking is IP-based; heavy use will get throttled (backoff + stale cache absorb it).
  • Single-process SQLite; fine at personal scale (thousands of postings).