Skills › Marketing Skills › prospecting › evals.json
evals.json
191 lines · skills/prospecting/evals/evals.json at 88c79f6
1{
2 "skill_name": "prospecting",
3 "evals": [
4 {
5 "id": 1,
6 "prompt": "We're a B2B SaaS selling RevOps tooling at $30K ACV. Build me a list of 25 prospects.",
7 "expected_output": "Should check for product-marketing.md first. Should identify this as the SaaS branch. Should run Phase 1 ICP definition pulling from product-marketing context or asking targeted questions (target industry, headcount range, tech stack signals, funding stage). Should propose discovery sources appropriate for SaaS at $30K ACV: Apollo for breadth, Clay for waterfall enrichment, Crunchbase for funding signals, BuiltWith/Wappalyzer for tech stack, LinkedIn Sales Nav for decision-mapping (manual). Should ask about user's tool access before assuming. Should source 50-75 candidates (2-3x target) before qualifying. Should flag that email validation via Truelist or similar is non-negotiable before final list. Should output SaaS-branch chat table columns (Score | Company | Industry | Size | Signal | Contact | Email status | Confidence) followed by top 3-5 hot leads with one-sentence rationale each. Should reference references/saas-prospecting.md.",
8 "assertions": [
9 "Checks for product-marketing.md",
10 "Identifies SaaS branch",
11 "Runs Phase 1 ICP definition",
12 "Recommends multi-source discovery (Apollo, Clay, Crunchbase, BuiltWith)",
13 "Asks about user's tool access",
14 "Sources 2-3x candidates before qualifying",
15 "Requires email validation before final list",
16 "Outputs SaaS-branch chat table columns",
17 "Includes top 3-5 outreach targets with rationale",
18 "References saas-prospecting.md"
19 ],
20 "files": []
21 },
22 {
23 "id": 2,
24 "prompt": "Find me 25 SaaS companies that just raised a Series B in the last 60 days and use HubSpot.",
25 "expected_output": "Should recognize this as a SaaS branch prospecting task with very specific signals. Should identify the trigger event (Series B in last 60 days) and the technographic filter (uses HubSpot). Should recommend a workflow: (1) Crunchbase or Pitchbook for funding signal filter (Series B + date), (2) BuiltWith or Clay's waterfall for tech stack verification (uses HubSpot), (3) cross-check via business websites and LinkedIn. Should note this is a tight ICP that should yield high-confidence matches if data sources are current. Should flag freshness concerns: Crunchbase data depends on self-reporting, BuiltWith refresh cycles aren't real-time. Should recommend cross-source verification for the funding date specifically. Should output a SaaS-branch chat table with the funding round + date in the Signal column. Should include verified email validation before delivering.",
26 "assertions": [
27 "Identifies as SaaS branch",
28 "Identifies funding signal + tech stack filter",
29 "Recommends Crunchbase or Pitchbook for funding",
30 "Recommends BuiltWith or Clay for HubSpot verification",
31 "Notes data freshness concerns",
32 "Recommends cross-source verification",
33 "Outputs signal column showing round + date",
34 "Requires email validation"
35 ],
36 "files": []
37 },
38 {
39 "id": 3,
40 "prompt": "I run a marketing agency. Find me 25 mid-market manufacturers in the Midwest US who recently hired a new CMO.",
41 "expected_output": "Should identify this as the B2B branch (manufacturers, not SaaS). Should run Phase 1 ICP definition: industry (manufacturing, with NAICS code if precision matters), size (mid-market = typically 200-2000 employees), geography (Midwest US states), trigger event (CMO hire in last 90-180 days). Should propose discovery: Apollo or ZoomInfo for firmographic filter, LinkedIn Sales Nav for CMO hire detection (job changes), Google Alerts on press releases for trigger events. Should warn that CMO hires aren't always in public databases — LinkedIn Sales Nav alerts on job changes is the most reliable source. Should output B2B-branch chat table with the CMO trigger as the signal. Should reference references/b2b-prospecting.md. Should mention compliance: GDPR less likely (US-only), CAN-SPAM applies, capture source URL + date for every contact.",
42 "assertions": [
43 "Identifies B2B branch (not SaaS)",
44 "Runs Phase 1 ICP definition with NAICS or industry classification",
45 "Specifies mid-market size band",
46 "Specifies Midwest US geography",
47 "Identifies trigger event (CMO hire)",
48 "Recommends Apollo/ZoomInfo + LinkedIn Sales Nav",
49 "Notes CMO hires often only on LinkedIn",
50 "Outputs B2B-branch chat table",
51 "Mentions CAN-SPAM and source URL capture",
52 "References b2b-prospecting.md"
53 ],
54 "files": []
55 },
56 {
57 "id": 4,
58 "prompt": "We sell to industrial distributors. Build a list of 25 prospects.",
59 "expected_output": "Should identify this as the B2B branch. Should run Phase 1 ICP definition asking targeted questions: distributor size, geography, vertical specialty, buying patterns. Should propose discovery: Apollo or ZoomInfo for firmographic depth, industry-specific directories (e.g., NAW for wholesale distributors, ISA for industrial sales agencies), trade show exhibitor lists. Should note state business registries and Chamber of Commerce as verification sources. Should propose trigger events: new location, recent acquisition, leadership change, posting RFPs. Should warn that industrial distributor data is often spotty in major databases — cross-check with company website + LinkedIn for size and ownership signals. Should output B2B-branch chat table. Should note ICP fit precision matters more than initial volume for this kind of niche prospecting.",
60 "assertions": [
61 "Identifies B2B branch",
62 "Runs Phase 1 ICP definition asking targeted questions",
63 "Recommends industry-specific directories beyond Apollo/ZoomInfo",
64 "Mentions trade show exhibitor lists",
65 "Identifies relevant trigger events",
66 "Warns about data spottiness for industrial",
67 "Recommends cross-verification with business websites + LinkedIn",
68 "Notes ICP fit precision over volume"
69 ],
70 "files": []
71 },
72 {
73 "id": 5,
74 "prompt": "I build websites for local businesses. Find me 15 prospects near Austin, TX who don't have a website.",
75 "expected_output": "Should identify as Local SMB branch. Should run Phase 1 ICP definition: business category (ask user — gyms, restaurants, salons, etc. matter), radius (default 20 km from Austin), target count (15). Should run the browser research workflow: search Google Maps for category + Austin, build candidate list from visible results, cross-check via business name + city web search to verify website status. Should apply the 4-tier website status classification (No site found / Social only / Weak site / Has site) — prioritize No site + Social only as Hot. Should score: Hot (no site + active + phone + within radius), Warm (weak site), Cold (has site), Skip (closed/duplicate/out of scope). Should output Local SMB chat table (Score | Business | Category | Area | Distance | Website status | Website/Social | Phone | Why prospect | Confidence). Should add 'Best first outreach targets' top 3 with reasoning. Should reference references/local-prospecting.md. Should warn against bulk-scraping Google Maps (ToS violation) — browser-assisted research only.",
76 "assertions": [
77 "Identifies Local SMB branch",
78 "Asks about business category if not specified",
79 "Defaults radius to 20km",
80 "Runs browser research workflow",
81 "Applies 4-tier website status classification",
82 "Uses Hot/Warm/Cold/Skip scoring",
83 "Outputs Local SMB chat table columns",
84 "Adds top 3 outreach targets",
85 "References local-prospecting.md",
86 "Warns against bulk-scraping Google Maps"
87 ],
88 "files": []
89 },
90 {
91 "id": 6,
92 "prompt": "I have a list of 200 prospect emails from Apollo. How do I know which ones are deliverable before I start outreach?",
93 "expected_output": "Should explain the deliverability validation step in Phase 3. Should recommend Truelist (the integration in this pack) for bulk validation. Should explain the email_state classification output: ok (deliverable), email_invalid (bounces, exclude), risky (deliverable with risk like role or disposable, include cautiously), unknown (couldn't determine, skip or re-verify), accept_all (catch-all domain, include cautiously). Should warn that Apollo data accuracy is typically 60-80% — sending without validation will tank sender reputation (bounce rate >2% triggers ISP throttling and reputation damage). Should recommend the workflow: bulk POST to /api/v1/verify or CSV upload → keep ok, include risky/accept_all cautiously, exclude email_invalid, re-verify unknown → hand off to outreach. Should note Truelist also has an official MCP server for agent-driven validation. Should note cold email reputation is hard to recover once damaged — validation is non-negotiable, not optional. Should mention Hunter and Snov as alternatives with built-in verification. Should reference truelist.md integration guide.",
94 "assertions": [
95 "Recommends Truelist for bulk validation",
96 "Explains email_state values (ok, email_invalid, risky, unknown, accept_all)",
97 "Warns Apollo accuracy is 60-80%",
98 "Cites 2% bounce rate threshold for reputation damage",
99 "Recommends workflow: validate, keep ok, exclude email_invalid",
100 "Mentions Truelist MCP server for agent workflows",
101 "Mentions cold email reputation is hard to recover",
102 "References truelist.md or data-sources.md"
103 ],
104 "files": []
105 },
106 {
107 "id": 7,
108 "prompt": "I just built a tool that automates failed-payment follow-up for gym owners. I have no customers yet. Help me find my first ten — the people who are actually dealing with this problem right now.",
109 "expected_output": "Should select the Demand-signal branch (early-stage, first customers, evidence-of-demand) and load references/demand-signals.md — NOT the SMB/B2B list-building branches. Should start with a product brief, then mine the five signal buckets (explicit demand / pain / workaround / switching / timing) across public discourse (forums, communities, reviews, GitHub issues, job posts) — using last30days for recency, social-fetch/scraping to read original pages, not qualifying from snippets. Should score prospects on demand-fit (pain 25 / product fit 25 / timing 20 / reachability 15 / evidence quality 15, 0-100 with bands) rather than ICP-fit Hot/Warm/Cold, and require a cited public signal for every primary-shortlist prospect. Should draft source-based openers but never auto-send. Should produce an evidence report (verdict → ICP → top prospect → shortlist with sources+scores → repeated patterns → 7-day manual outreach plan → limits) and label prospects as 'potential customers based on public signals,' not confirmed buyers. Should honor the compliance guardrails including no data brokers/leaked data and no sensitive-trait targeting.",
110 "assertions": [
111 "Selects the Demand-signal branch, not the SMB/B2B/SaaS list-building branches",
112 "Mines the five signal buckets from public discourse rather than contact databases",
113 "Uses recency/original-source tooling (last30days, social-fetch, scraping) and does not qualify from snippets",
114 "Scores on the demand-fit rubric (0-100 weighted), not ICP-fit Hot/Warm/Cold",
115 "Requires a cited public signal for every primary-shortlist prospect",
116 "Drafts openers but never auto-sends; labels prospects as potential-based-on-public-signals",
117 "Produces the evidence report structure with a 7-day manual outreach plan and limits"
118 ],
119 "files": []
120 },
121 {
122 "id": 8,
123 "prompt": "I sell CRO services to B2B SaaS companies. How do I find companies that are likely to need us right now instead of just pulling a big list from Apollo?",
124 "expected_output": "Should load the signal plays reference. Should rank signals by closeness and pick ones that fit a CRO agency: recent funding (30-60 day window), a new VP Marketing or growth leader (first 90 days), open growth or performance marketing roles, a redesigned site or new pricing page, launched paid campaigns. Should stack signals with ICP fit rather than act on one weak signal, record signal dates, map signals to plays (value-first teardown), and assign account tiers (Tier 1 1:1 top 25-50 with account briefs, Tier 2 segments, Tier 3). Should name ways to detect each signal (job data, funding news, visitor ID) and warn against opening with the signal itself or mentioning tracked visits.",
125 "assertions": [
126 "Uses buying signals as a timing filter on top of ICP fit",
127 "Names signals relevant to a CRO agency",
128 "Gives freshness windows for signals",
129 "Maps signals to a play, including a value-first teardown",
130 "Assigns account tiers that set research depth",
131 "Says not to open with 'congrats on the raise' or mention tracked visits"
132 ],
133 "files": []
134 },
135 {
136 "id": 9,
137 "prompt": "About a fifth of the emails in my list come back as catch-all from our verifier. Should I send to them or drop them?",
138 "expected_output": "Should load the sourcing and enrichment reference and explain that catch-all domains accept any address, so basic SMTP verification can't confirm the person. Should recommend a policy: resolve catch-alls with a verifier that goes beyond SMTP checks; send resolved-valid ones from separate lower-volume mailboxes; send unresolved ones only to Tier 1 accounts while watching bounces; keep campaign bounces under 2% with automatic pause; re-verify within 7 days of sending. Should not recommend sending all of them blind or dropping all of them without considering account value.",
139 "assertions": [
140 "Explains why catch-all addresses can't be confirmed by basic verification",
141 "Recommends resolving catch-alls with a specialist verifier",
142 "Recommends separate lower-volume mailboxes for resolved catch-alls",
143 "Limits unresolved catch-alls to Tier 1",
144 "Keeps bounce rate under 2% with a pause rule",
145 "Neither sends all blindly nor drops all by default"
146 ],
147 "files": []
148 },
149 {
150 "id": 10,
151 "prompt": "We have 200 B2B prospects and several connected enrichment providers. Some records are fresh in our cache, some caches are expired, one API says429 and another finds a same-name company in the wrong country. Design a cost-bounded waterfall without overwriting verified CRM contacts.",
152 "expected_output": "Defines identity and accepted-field criteria, exact account/provider/input cache matching, differentiated expiry/no-match/error handling, and conditional ordered lookups. Preserves stronger CRM evidence, contact verification, and unresolved reasons.",
153 "assertions": [
154 "Reuses only applicable fresh cache entries",
155 "Does not treat429 or timeout as no-match",
156 "Rejects a wrong-entity result instead of counting any nonempty response as success",
157 "Stops when an accepted field is found instead of running every provider",
158 "Preserves stronger verified CRM fields and separate contact verification"
159 ],
160 "files": []
161 },
162 {
163 "id": 11,
164 "prompt": "I'm a founder selling payroll software to 20-200 person companies. Cold email reply rates have cratered. I have about 4,000 LinkedIn connections and a busy calendar. Can I turn my own network into pipeline without spamming people?",
165 "expected_output": "Should load references/signal-plays.md (warm paths) and lay out: export LinkedIn connections, calendar attendees and inbox contacts (noting LinkedIn's export only includes emails for connections who allow it); enrich and score against the ICP; match against a target account list to find first- and second-degree paths; start with recent calendar meetings to find buying moments; recruit a connector per vertical and pay them per qualified meeting with the arrangement disclosed; use double opt-in intros only, with a human approving every intro and no contact asked to take intros for two vendors in the same week.",
166 "assertions": [
167 "Starts by exporting the network (LinkedIn, calendar, inbox)",
168 "Notes LinkedIn's export lacks emails for most connections",
169 "Matches contacts against target accounts for first- and second-degree paths",
170 "Suggests starting with recent calendar meetings",
171 "Recommends connectors with disclosed compensation",
172 "Requires double opt-in intros with a human approving each one"
173 ],
174 "files": []
175 },
176 {
177 "id": 12,
178 "prompt": "I run a landing page and tracking agency. Build me a list of B2B SaaS companies that are running Meta ads but have no pixel installed or are sending ad traffic to their homepage, and draft the opener.",
179 "expected_output": "Should load references/signal-plays.md (gap plays) and pair the ad-activity signal with gaps verified from outside: ad destinations from the Meta Ad Library, tags from page source, the network tab, or a tag checker. Should require evidence (date, URL or screenshot) for every gap and treat 'no gap found' as a valid result. Should caveat that server-side tracking (Conversions API, server-side GTM) is invisible from outside, so a missing browser pixel isn't proof, and that homepage destinations can be deliberate. Should re-verify the gap within 7 days of sending. The opener should state the observation and ask a question rather than accuse, and offer the teardown only after a yes.",
180 "assertions": [
181 "Uses the Meta Ad Library to confirm active ads and their destinations",
182 "Checks tags from outside (page source, network tab, or tag checker)",
183 "Requires dated evidence for each gap and allows 'no gap found'",
184 "Warns that server-side tracking can't be seen from outside",
185 "Re-verifies gaps within 7 days of sending",
186 "Opener is an observation plus a question, not an accusation"
187 ],
188 "files": []
189 }
190 ]
191}