Security testing vendor Intruder sells AI-augmented penetration testing, so its recent comparison of automated agents against human testers — covered by BleepingComputer on 15 October — should be read with that commercial interest in mind. Even so, the analysis lands on a verdict most buyers will find unsurprising and useful: AI-driven testing is a real force multiplier for coverage and frequency, but it falls short of replacing human judgement on risk and business context.
Per BleepingComputer's write-up of Intruder's analysis, three weaknesses recur in AI-only testing:
Consistency. AI agents produce uneven results across runs, sometimes missing issues they surfaced previously or reporting them at varying severity. For teams that need deterministic, repeatable evidence in audit deliverables, that variability matters.
Risk assessment. Automated tooling can catalogue vulnerabilities, but ranking them against actual exposure — what is internet-facing, what holds sensitive data, what an attacker would realistically chain — remains a human competence. AI is stronger at enumeration than at judgement.
Business context. Determining which findings are exploitable in practice, which carry reputational or regulatory consequence, and which are theoretical is, at present, outside what autonomous agents reliably deliver.
Intruder positions its own platform as a middle path — automated breadth plus human review — and touts continuous, frequently refreshed scanning. Buyers should treat such performance claims as vendor assertions needing trial validation or independent references, not as benchmarked findings.
The budget question for Hong Kong IT and security managers
For local organisations weighing AI pentest tooling against traditional engagements, the more consequential question is not capability but acceptability. Banks and financial institutions operating under HKMA or SFC expectations, and any organisation handling personal data under the PDPO, typically treat penetration testing as evidence of due diligence. The operative issue: can a methodology regulators and auditors understand as human-assessed be satisfied by AI-only output, or would automated results need to be reviewed, validated, and signed off by qualified testers before carrying weight in a compliance submission? That is a question to put to assurance providers, not to resolve on vendor say-so.
The framing points toward a layered model rather than substitution. AI raises the baseline — breadth, speed, and the ability to run frequently between scheduled engagements, catching new assets and misconfigurations early. Human testers remain the control point for deep exploitation chains, subtle logic flaws, and the contextual risk sign-off that appears in audit reports and board briefings.
The practical takeaway: treat AI pentesting as a way to raise that baseline and free human testers for higher-value work, not as grounds to shrink engagement scope or eliminate expert review. Organisations preparing for audits should confirm with their assurance providers that their testing mix produces evidence meeting human-assessment expectations — and should validate vendor AI capability claims through trials or independent references before committing budget.
BleepingComputer's coverage of Intruder's analysis is available at the source link above.
安全測試廠商 Intruder 有售 AI 增強的滲透測試服務，因此其近期將自動化 agent 與人類測試員作出比較的分析——由 BleepingComputer 於 10 月 15 日報道——讀者宜考慮其商業利益背景。儘管如此，該分析得出的結論，大多數買家應會覺得並不意外而且實用：AI 驅動的測試確實能大幅提升覆蓋範圍和測試頻率，但在風險判斷和業務情境方面，仍未能取代人類的專業判斷。
根據 BleepingComputer 對 Intruder 分析的報道，純 AI 測試反覆出現三大弱點：
一致性。 AI agent 在不同次執行之間結果參差，有時會遺漏先前曾發現的問題，或就同一問題回報不同程度的嚴重性。對於需要在審計交付文件中提供確定性、可重複證據的團隊而言，這種變異性影響重大。
風險評估。 自動化工具可以編目漏洞，但要根據實際暴露情況為其排序——哪些是面向互聯網、哪些儲存敏感數據、攻擊者實際上會如何將漏洞串連利用——仍然是人類的專長。AI 擅長枚舉（enumeration），但不擅長判斷。
業務情境。 判斷哪些發現實際上可被利用、哪些會帶來聲譽或監管後果、哪些僅屬理論性質，目前仍然超出自主 agent 可可靠提供的範圍。
Intruder 將自家平台定位為中間路線——自動化覆蓋加上人類審核——並大力推廣持續、頻繁更新的掃描功能。買家應將此類性能宣傳視為廠商主張，需要透過試用驗證或獨立參考加以確認，而非視為已基準測試的結果。
香港 IT 及保安管理層的預算考量
對於本地機構而言，權衡 AI 滲透測試工具與傳統滲透測試項目時，更關鍵的問題並非能力，而是可接受性。受金管局（HKMA）或證監會（SFC）規管期望約束的銀行和金融機構，以及任何根據《私隱條例》（PDPO）處理個人資料的機構，通常都將滲透測試視為盡職審責的證據。核心問題是：監管機構和審計師所理解的「經人類評估」的方法論，能否以純 AI 產出的結果來滿足？抑或自動化結果須先由合資格測試員審閱、驗證及簽署確認，才能在合規提交中具備效力？這個問題應向保證服務提供者查詢，不應單憑廠商一面之詞來決定。
這個框架指向一種分層模式，而非簡單取代。AI 能提升基準水平——擴闊覆蓋、加快速度、並可在定期項目之間頻繁執行，及早發現新資產和錯誤配置。人類測試員仍然是深度漏洞利用鏈、細微邏輯缺陷，以及審計報告和董事會簡報中所需的情境風險簽核的關鍵把關者。
實踐上的要點是：應將 AI 滲透測試視為提升基準水平、釋放人類測試員投入更高價值工作的方法，而非縮減項目範圍或取消專家審核的理由。準備接受審計的機構，應先向保證服務提供者確認其測試組合所產生的證據符合人類評估的期望——並應在批出預算之前，透過試用或獨立參考來驗證廠商對 AI 能力的宣稱。
BleepingComputer 對 Intruder 分析的報道可見於上方原始連結。