Ask most people what a penetration test looks like, and they picture an ethical hacker probing a network or web application from the outside, with no inside knowledge, trying to break in exactly as a real attacker would. That approach — called black-box testing — is a legitimate and common method in general IT security.
For medical devices, both the FDA and the leading consensus standards in the space explicitly favor a different approach, and understanding why reveals a lot about what makes this category of testing distinct.
Black-Box, Gray-Box, White-Box: The Real Difference
The three terms describe how much internal knowledge a tester starts with. Black-box testing gives the tester zero prior information — no source code, no architecture diagrams, no design documentation.
Gray-box testing provides partial access, often credentials and some architectural context. White-box testing gives the tester full access to source code, firmware images, threat models, and design documentation before testing even begins.
For a typical corporate web application, black-box testing has real value — it simulates what an anonymous internet attacker actually sees. But medical devices are held to a different standard specifically because of what black-box testing tends to miss: deep logic flaws, business-rule violations, and vulnerabilities buried in firmware or hardware interfaces that no amount of external probing would ever surface within a realistic testing window.
The FDA’s premarket cybersecurity guidance calls for testing that demonstrates device resilience using design documentation, threat models, and source-level analysis — which is white-box testing by definition — and the AAMI TIR57 and TIR97 standards frame security testing as something that flows directly out of threat modeling and architecture analysis, not blind external probing.
The Attack Surface Is Bigger Than Most People Expect
A meaningful medical device penetration test has to account for a full stack that looks nothing like a typical corporate environment: the clinician or hospital-facing portal, cloud APIs and the data plane behind them, cellular or Wi-Fi backhaul connections, the mobile companion app patients and clinicians actually use, BLE, RF, or other wireless telemetry, service and maintenance interfaces used by field technicians, the device’s own firmware and embedded operating system, internal hardware buses like I2C, SPI, or CAN, and, at the deepest layer, the microcontroller’s boot chain.
Each layer requires genuinely different testing skills. Firmware analysis means extracting and reverse-engineering embedded binaries, and often physically accessing hardware debug interfaces like JTAG or UART.
Wireless testing means fuzzing BLE pairing and GATT implementations, or, for devices using medical-specific protocols, testing DICOM, HL7, or FHIR implementations directly rather than treating them as generic network traffic.
Cloud and mobile testing looks more like conventional application security work — REST, MQTT, or gRPC API testing, authentication review, and mobile binary analysis — but still has to be evaluated as part of the same integrated system as the device itself, not in isolation.
What Real Findings Actually Look Like
The value of testing this comprehensively shows up in the kinds of issues it surfaces — issues that a generic scan or a web-only assessment would never catch.
Documented patterns from real medical device testing include hardcoded credentials baked into a BLE pairing implementation, which would let any sufficiently close attacker pair with the device and pull data without the patient’s consent; unauthenticated firmware update endpoints on a hospital network, which could in principle let an attacker push unsigned firmware and alter dosing logic on an infusion pump; plaintext storage of protected health information in a mobile companion app’s local cache, recoverable from a lost or stolen phone with no special tools required; predictable session tokens on a cloud API, allowing session prediction that could grant access across different clinicians’ or patients’ accounts; debug interfaces like JTAG or UART left enabled in production firmware, allowing local code extraction; and outdated TLS configurations permitting downgrade attacks on a device-to-cloud telemetry channel.
None of these are exotic vulnerabilities requiring nation-state-level sophistication. Most are the kind of finding that a properly scoped, sufficiently deep test catches reliably — which is exactly the point. They’re also, notably, the same categories of finding documented in real CISA advisories over the past several years, covering devices from cardiac implants to infusion pumps.
The Letter of Attestation: A Detail That Trips Up Generic Vendors
One artifact that distinguishes a reviewer-ready penetration test report from a generic IT security deliverable is the Letter of Attestation — a signed document identifying who conducted the test, confirming their independence from the device’s development team, describing the scope of what was actually covered, and confirming that every finding was either remediated or formally risk-accepted.
The FDA’s premarket guidance specifically expects testing results to be documented in a format reviewers can audit, and the Letter of Attestation has become the standard way that expectation gets satisfied.
A generic IT security firm producing a report in its own house format, without this specific artifact, frequently triggers a deficiency letter simply asking for one — an avoidable delay for something that should have been part of the original deliverable.
Finding Issues Is Only Half the Engagement
A penetration test report full of critical findings and no path forward isn’t actually useful to an engineering team under deadline pressure. The more valuable version of this work includes engineer-to-engineer support during remediation — someone who understands both the vulnerability and the device’s constraints working alongside the manufacturer’s own developers to fix it correctly, rather than handing over a PDF and disappearing until the next billable engagement.
Re-testing matters just as much as the original assessment. A fix that looks correct on paper can introduce a new issue, fail to fully close the original vulnerability, or work as intended in one configuration but not another.
Confirming that a finding is actually resolved, not just addressed in principle, is what turns a list of open items into a clean report a submission team can actually rely on.
Manufacturers scoping an engagement should treat unlimited or clearly bounded re-testing as a baseline expectation, not a premium add-on billed separately — because a testing relationship that charges extra every time a fix needs verification creates a perverse incentive to under-test in the first place.
What This Means for Manufacturers Scoping a Test
The practical takeaway for a MedTech team preparing for a submission is to ask pointed questions before selecting a testing vendor: Does the engagement include source code, firmware, and threat model access, or is it black-box only?
Does it cover hardware interfaces and wireless protocols specific to the device, or just web APIs and the mobile app? Does the deliverable include a signed Letter of Attestation formatted the way FDA reviewers are trained to evaluate it?
A properly scoped medical device penetration testing engagement should be able to answer all three questions clearly before testing even begins — because a report that can’t cover that ground, no matter how thorough it looks on the surface, is unlikely to hold up under the kind of scrutiny an FDA reviewer, and ultimately a real attacker, will apply.
