14 Questions to Ask Before Buying an ATS
Questions to ask an ATS vendor are the specific, probing inquiries that reveal what demos hide and contracts bury — covering AI methodology, pricing structure, data portability, implementation timelines, and bias handling — so your hiring team can distinguish genuine capability from polished sales theater. RecruitBPM found that most vendor demos are scripted, and Recruitee’s evaluation framework estimates 15–25% of total contract value is lost on poor vendor choices. These 14 questions protect that investment.
Demos Are Scripted
Why You Need a Question List
Every ATS vendor has a polished demo. They walk you through the happy path — the one scenario where everything works perfectly. The sales rep knows exactly which buttons to click and which screens to skip. Your team does not live on the happy path.
The Recruitee evaluation framework found that 15–25% of total contract value is lost on poor vendor choices. That figure includes subscription costs, implementation time, training hours, data migration pain, and the opportunity cost of a tool your team abandons after three months.
A prepared question list flips the dynamic. Instead of reacting to a rehearsed presentation, you control the conversation. Below are 14 questions that surface the information demos are designed to obscure. Use them alongside our ATS evaluation checklist for a complete vendor assessment.
1. What AI Powers Your Screening?
Most ATS platforms claim “AI-powered screening” without explaining the methodology. Keyword matching counts term frequency and penalizes candidates who use different vocabulary for the same skills. Signal-based scoring evaluates measurable outcomes — revenue generated, teams scaled, projects delivered — regardless of the specific words on the resume.
The vendor can explain exactly what their AI evaluates, how scores are calculated, and how the model was trained. They can show you the scoring breakdown for a real candidate, not just a summary number. CurriculoATS uses impact scoring that extracts quantified achievements and weighs them against role requirements.
2. Per-Seat or Flat Pricing?
Per-seat pricing punishes team growth. Adding a hiring manager to review candidates should not increase your monthly bill by $50–$100. Over 12 months, a 10-person hiring team on per-seat pricing can cost $6,000–$12,000 more than a flat-rate alternative. Check transparent pricing models to understand what flat-rate actually means.
Flat monthly pricing that covers unlimited users, or per-job pricing that scales with hiring volume rather than team size. Published pricing on the website — no “contact sales for a quote” gates. CurriculoATS charges $50/month flat with unlimited team members.
3. Can I Export My Data If I Leave?
Your candidate data, pipeline history, notes, scorecards, and communication logs are your intellectual property. If you cannot export them in standard formats (CSV, JSON) at any time, you are locked in — not by the product’s quality, but by the switching cost they engineered into the contract.
Self-service export of all data types in standard formats, available on every plan, with no additional fees. The vendor should be able to show you the export interface during the demo, not just promise it exists.
4. How Configurable Are Pipelines?
Your engineering hiring process has different stages than your sales hiring process. A rigid, one-size-fits-all pipeline forces your team to work around the tool instead of with it. You need different pipelines for different role types, with custom stages that match your actual workflow.
Unlimited custom pipelines per role type, drag-and-drop stage reordering, custom stage names, and the ability for hiring managers to modify pipelines without admin intervention. Bonus: pipeline templates you can clone for similar roles.
5. How Long to Get Up and Running?
Implementation timelines reveal product complexity. If a vendor needs six weeks and a dedicated project manager to get you live, the product was built for enterprises with dedicated HR ops teams — not for growing startups where the founder is also the recruiter. Every week of implementation is a week you are still using spreadsheets or your old broken tool.
Self-service setup that gets your first job posted within hours. No mandatory onboarding calls (though optional ones should be available). Data import from your previous tool in under a day. CurriculoATS is live in minutes — sign up, post a job, start screening.
6. What Happens When Hiring Workflows Change?
Your hiring process will change. You will add interview rounds, remove stages, change scorecards, restructure teams. The question is whether the ATS adapts to those changes in real time or requires a support ticket and a two-week turnaround for every modification.
Admin-level users can modify pipelines, scorecards, automations, and permissions without contacting support. Changes take effect immediately, not after a sync cycle. Historical data is preserved even when pipeline structures change.
7. Do You Support SSO?
Single sign-on is not just a convenience feature — it is a security requirement. Without SSO, every team member has a separate username and password for the ATS, which means orphaned accounts when people leave, password reuse across systems, and no centralized access control.
SAML 2.0 or OpenID Connect SSO included on the plan your team actually uses, not locked behind an enterprise tier that costs 5x more. Integration with common identity providers like Google Workspace, Okta, and Azure AD.
8. What’s Your Uptime and SLA?
When your ATS goes down during a hiring sprint, candidates cannot apply, hiring managers cannot review, and your team loses momentum. Ask for the actual uptime number from the last 12 months, not the SLA target. A 99.9% SLA allows 8.7 hours of downtime per year — ask when those hours happened and whether they were during business hours.
Published status page with historical uptime data. A real SLA with service credits if uptime falls below the target. Planned maintenance windows communicated at least 48 hours in advance and scheduled outside peak hours.
9. Can I Try Before I Buy?
Demos are performances. A free trial is reality. Your actual team testing real workflows with real candidate data is the only reliable way to evaluate whether an ATS fits. RecruitBPM found that scripted demos consistently hide usability problems, edge cases, and performance issues that only surface during hands-on use.
A genuine free tier or extended free trial (not 7 days) that includes core features like AI screening, pipeline management, and team collaboration. No credit card required to start. CurriculoATS offers a free Starter plan with AI screening and 1 active job — permanently free, not a trial.
10. How Do You Handle Bias?
AI screening can amplify existing bias if the training data reflects historical hiring patterns. If the system learned from a decade of biased human decisions, it will replicate those biases at scale — faster and more consistently than any human recruiter. This is not a theoretical risk; it is a documented pattern across the industry.
Documented bias detection and mitigation methodology. The ability to audit AI decisions and understand why a candidate received a specific score. Regular bias audits with published results. Explainability for every screening decision — not just a number, but the reasoning behind it.
11. What Integrations Are Native?
There is a meaningful difference between native integrations and “we connect via Zapier.” Native integrations are built and maintained by the vendor, sync in real time, and handle edge cases. Zapier connections add latency, break when APIs change, and cost extra. Ask specifically about the tools your team uses daily: calendar, email, Slack, HRIS, and job boards.
Native integrations for your core workflow tools (Google Workspace or Outlook, Slack or Teams, major job boards). A documented API for custom integrations. Clear documentation on what each integration does and does not sync.
12. What Reporting Do I Get?
Hiring decisions improve when you measure outcomes. Time-to-hire, source effectiveness, pipeline conversion rates, and interviewer calibration data tell you where your process works and where it breaks down. Without reporting, you are flying blind — making process changes based on gut feeling instead of data.
Built-in dashboards for key recruiting metrics (time-to-hire, source quality, pipeline velocity, offer acceptance rate). The ability to filter by role, team, and time period. Data export for custom analysis. Bonus: scheduled report delivery to stakeholders.
13. Can Hiring Managers Use It Without Training?
The most expensive ATS failure is adoption failure. If hiring managers need a 2-hour training session and a PDF manual to review candidates, they will revert to email and spreadsheets within a month. The interface should be obvious enough that a hiring manager can review candidates, leave feedback, and advance or reject in their first session.
During the trial, hand the tool to a hiring manager with zero context and watch what happens. They should be able to view their candidates, read AI screening summaries, leave notes, and move candidates through the pipeline without asking for help. If they need a tutorial, the UX is not ready.
14. What’s Your Data Retention Policy?
Candidate data has legal implications. GDPR requires you to have a lawful basis for storing personal data and to delete it when that basis expires. Your ATS needs to support configurable retention periods, automated deletion, and candidate consent management — not just store everything forever and hope nobody asks.
Configurable data retention periods per data type. Automated deletion workflows. Candidate self-service data requests (access, correction, deletion). Clear documentation on where data is stored geographically and which subprocessors have access. Review our security approach for an example.
Running Your Vendor Calls
Step 1: Send questions in advance. Email the 14 questions to the vendor before the demo. Serious vendors will appreciate the structure. Vendors who resist transparency will push back — which is itself an answer.
Step 2: Score responses live. Use our ATS evaluation checklist to score each vendor’s answers on a 1–5 scale during the call. Do not rely on memory — record the scores immediately.
Step 3: Trial the top 3. No amount of questioning replaces hands-on use. Run real candidate data through each finalist. Have hiring managers use the tool for an actual open role. The trial reveals what answers could not.
Step 4: Compare total cost. Factor in subscription fees, per-seat charges, implementation costs, and training time. A $149/month tool with $5,000 in setup fees costs more in year one than a $50/month flat-rate alternative with self-service setup.
If you want to start evaluating now, CurriculoATS has a free Starter plan — no credit card, no sales call, live in minutes.
What is the single most important question to ask an ATS vendor?
Ask what methodology powers their AI screening. Keyword matching counts term frequency and penalizes different vocabulary. Signal-based scoring evaluates measurable outcomes like revenue generated, teams scaled, and projects delivered. The methodology determines whether AI improves hiring quality or automates the same flawed process.
How can I tell if an ATS demo is hiding problems?
Ask to see edge cases during the demo — bulk candidate uploads, permission changes mid-pipeline, data exports, and what happens when a hiring manager makes a mistake. If the sales rep deflects or promises a follow-up, that feature probably does not work well. RecruitBPM found that most vendor demos are scripted to show only the happy path.
Should I always require a free trial before purchasing an ATS?
Yes. Any vendor that does not offer a free trial or sandbox is hiding something. Demos show the best-case scenario. A trial lets your actual team test real workflows with real data. CurriculoATS offers a free Starter plan with AI screening and 1 active job — no credit card required.
What ATS contract red flags should I watch for?
Watch for multi-year lock-ins with auto-renewal, per-seat pricing that punishes team growth, data deletion upon cancellation, implementation fees exceeding $1,000, and mandatory paid training. The Recruitee evaluation framework estimates 15–25% of total contract value is lost on poor vendor choices — much of that from unfavorable terms.
How much should a growing startup budget for an ATS?
Startups under 200 employees should budget between $0 and $150 per month. Platforms like CurriculoATS offer a free tier with AI screening at $0 and a full-featured plan at $50/month flat — no per-seat fees. Enterprise platforms like Greenhouse start at $6,000/year and scale past $125,000 for large organizations.