Why Security Questionnaires Must Evolve
Enterprise questionnaires were designed for classic SaaS: predictable releases, fixed deployments, deterministic outputs. Voice AI is different. Models update, providers change, and behavior can drift. If procurement keeps asking SaaS‑era questions, risk gets obscured, timelines stretch, and teams stall.
Today, we’re partnering with Delve to publish a questionnaire framework that centers the controls that actually determine whether an AI system is safe, reliable, and enterprise‑ready.
The AI First Security Questionnaire (Co‑Authored with Delve): What We’re Announcing
Below is the structure we’re moving toward in reviews, with acceptance criteria you can copy into your RFP or vendor due diligence. It’s designed to evaluate any AI vendor (including us) against the realities of modern model behavior, data flows, and operations.
1) Data sovereignty & residency:
- Can you attest that all voice-data processing stays within our designated geographic region, with no external API calls?
- Do you keep comprehensive audit logs for every system that touches our customer voice data, including timestamps and processing locations?
2) Hallucination risk management:
- What risk-profiling methods do you use to measure and track hallucination rates across conversation types?
- Can you provide quantitative metrics on hallucination frequency by domain (e.g., medical, financial, legal)?
- How do you detect and prevent hallucinations that aren’t captured by the model’s own confidence scoring?
- What is your baseline hallucination rate, and how do you ensure it does not degrade with model updates?
3) Non-LLM verification systems:
- What deterministic, rule-based checks verify regulatory compliance prior to delivery (e.g., schema/structured validation)?
- Do you employ non-AI guardrails (regex patterns, keyword filters) to intercept problematic outputs?
- How do you validate numerical accuracy and factual claims without relying solely on the language model?
4) Performance guarantees & resilience:
- Can you contractually commit to sub-second response times regardless of OpenAI’s or other providers’ traffic levels?
- During a third-party service outage (e.g., 2024 incidents), how do you maintain service continuity?
5) Model behavior & control:
- Can you adjust the model’s behavior within 24 hours of an issue being identified, without relying on external providers?
- What is the impact on our custom voice models and conversation data if a third-party provider changes its terms of service or pricing?
- How do you guard against bias being introduced when third-party providers update their models without your knowledge or consent?
- Can you immediately roll back to a prior model version if an update introduces unacceptable bias or behavior changes?
6) Security & compliance verification:
- Can you provide evidence that our voice data is never used to train models, including by third-party providers?
- How do you ensure HIPAA compliance when patient voice data may include PHI?
7) Cost predictability & transparency:
- Can you offer a fixed-cost model that does not fluctuate with third-party API pricing changes?
8) Third-party dependency risks:
- What is your disaster-recovery plan if a critical AI provider permanently shuts down?
- How do you handle situations where a provider’s values or actions conflict with our corporate policies?
9) Infrastructure control:
- How quickly can you implement the custom security controls or encryption methods we require?
10) Technical model optimization & hyperparameters:
- If you used LoRA for fine-tuning, what are the Alpha and R values, how many parameters were unfrozen, and which optimizer did you use?
- Do you support quantization-aware training, and at what bit precision (e.g., INT8, INT4) can models run while maintaining accuracy?
- What mixture-of-experts routing mechanisms do you use, and can the gating network be adjusted for our use case?
- How do you handle gradient checkpointing and memory optimization for large-scale deployments?
How We’ll Support Your Review
For regulated or high‑risk workflows, we’ll work with your security team to validate evidence, capture decisions, and document exceptions in your preferred format. The outcome: shorter cycles, clearer acceptance criteria, and a shared picture of risk tied to real operational controls.
We’re excited to advance this standard alongside Delve, and even more excited to help your team evaluate voice AI with the rigor it deserves.
Ready to run this questionnaire against your environment?
Talk to sales. We’ll bring the evidence, you bring the questions.