CMS Notice of Proposed Sanctions to Theranos, Inc.
|
|
|
Refer to: WDSC-GKY
IMPORTANT NOTICE – PLEASE READ CAREFULLY
March 18, 2016
|
CLIA Number: 05D2025714 |
|
|
RE: |
PROPOSED SANCTIONS – CONDITIONS NOT MET IMMEDIATE JEOPARDY. IMPOSITION NOTICE TO FOLLOW IF PROPOSED SANCTIONS ARE IMPOSED. |
|
Dear Dr. Dhawan[1], Ms. Holmes, and Mr. Balwani:
This letter provides notice of sanctions the Centers for Medicare & Medicaid Services (CMS) is proposing to impose against the laboratory's Clinical Laboratory Improvement Amendments of 1988 (CLIA) certificate and of the laboratory's opportunity to submit in writing any evidence or information as to why the proposed sanctions should not be imposed. If the sanctions are imposed, we will provide the laboratory with a separate notice setting forth hearing rights and explaining the administrative appeals process.
CMS conducted a CLIA recertification and complaint survey at Theranos, Inc. ("Theranos" or "laboratory"). The onsite portion of the survey was completed on November 20, 2015; however, the survey concluded with the receipt of critical information received from the laboratory on December 23, 2015. Based on this survey, Theranos was found to be out of compliance with the following five CLIA Condition-level requirements:
| D5024: 42 C.F.R. § 493.1215 | Condition: Hematology; |
| D5400: 42 C.F.R. § 493.1250 | Condition: Analytic systems; |
| D6076: 42 C.F.R. § 493.1441 | Condition: Laboratories performing high complexity testing; laboratory director; |
| D6108: 42 C.F.R. 493.1447 | Condition: Laboratories performing high complexity testing; technical supervisor; and, |
| D6168: 42 C.F.R. § 493.1487 | Condition: Laboratories performing high complexity testing; testing personnel. |
In addition, CMS determined that various CLIA Standard-level requirements were not met.
By letter dated January 25, 2016, CMS provided Theranos with a listing of all deficiencies identified during the survey on Form CMS-2567, Statement of Deficiencies. The January 25, 2016 letter also notified the laboratory that the seriousness of the deficiencies cited under 42 C.F.R. § 493.1215 resulted in the finding of immediate jeopardy to patient health and safety, and requested that the laboratory take immediate action to remove the jeopardy and bring any unmet Condition-level requirements into compliance. CMS gave the laboratory 10 calendar days from the date of receipt of the January 25, 2016 letter to submit a credible allegation of compliance and acceptable evidence of correction for the cited deficiencies. On February 4, 2016, a Theranos representative requested an extension until February 12, 2016 to provide a submission, which CMS granted. CMS received a submission from the laboratory in response to the January 25, 2016 letter on February 12, 2016.
After careful review, we have determined that the laboratory's submission does not constitute a credible allegation of compliance and acceptable evidence of correction for the deficiencies cited during the CLIA recertification and complaint survey completed on December 23, 2016, and does not demonstrate that the laboratory has come into Condition-level compliance and abated immediate jeopardy. In general, we find that the statements made in the allegation of compliance and evidence of correction: 1) failed to adequately address the deficient practice cited; 2) are incomplete and failed to meet the criteria of acceptable evidence of correction; 3) do not ensure sustained compliance; and 4) show a lack of understanding of the CLIA requirements.
As the laboratory was advised in our January 25, 2016 letter, a credible allegation of compliance, as defined by regulation (42 C.F.R. § 493.2), is a statement or document that is:
1) Made by a representative of a laboratory with a history of having maintained a commitment to compliance and taking corrective action when required;
2) Realistic in terms of the possibility of the corrective action being accomplished between the date of the survey and the date of the allegation; and
3) Indicates resolution of the problem.
It is important to note that for it to be credible, the allegation of compliance must be complete and address each of the deficiencies cited in the Statement of Deficiencies. For each deficiency, the allegation of compliance must include a corrective action date that is realistic in terms of the action being accomplished between the date of the survey and the planned date of completion.
As Theranos was also advised in our January 25, 2016 letter, the laboratory's allegation of compliance must be substantiated by acceptable evidence of correction which must include:
1) Documentation showing what corrective action(s) have been taken for patients found to have been affected by the deficient practice;
2) How the laboratory has identified other patients having the potential to be affected by the same deficient practice and what corrective action(s) have been taken;
3) What measure has been put into place or what systemic changes the laboratory has made to ensure that the deficient practice does not recur; and
4) How the corrective action(s) is being monitored to ensure the deficient practice does not recur.
The following explanation details why the laboratory's February 12, 2016 submission does not constitute a credible allegation of compliance and acceptable evidence of correction:
D2094
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
Some documents pertaining to this deficiency referenced in the submission were not included. Specifically, the submission references "Ex. A, Tab 4, § 7.1.2.2.d.2." We found no such reference contained in the materials provided to CMS. In addition, Exhibit (Ex.) O, Tab 2 states: "The QC data are presented here: <Exhibit D Tab 7, Tab 8>." We found no "Tab 7" or "Tab 8" in Ex. D.
Although the laboratory's submitted protocol indicates that ungraded proficiency testing (PT) results will be evaluated, the submitted protocol does not explain how an investigation is performed and who must sign an investigation of ungraded PT samples.
In the submission, the laboratory concludes:
- "Not enough patient data available for meaningful analysis"
- "No evidence of systemic errors"
- "No patient impact is expected"
However, no information as to how the laboratory came to these conclusions related to patient outcomes was submitted. Documentation contained in Ex. O, Tab 2 also compares the "Range of Means" with no explanation as to what this refers to or how it correlates to the laboratory's ungraded PT results. The submission merely indicates that "the lab has investigated this ungraded PT event for ALP [alkaline phosphatase] and has documented its investigation and conclusion."
Furthermore, a Quality Monitoring and Process Improvement (QMPI) Program Meeting Agenda, CL FRM-00045-F1, was submitted as part of Ex A, Tab 13 and only includes PT for the Alternative Proficiency Assessment (APA). Based on information included in the submitted agenda, all PT issues were not addressed as required in CL QOP-00045, Revision A (Ex. A, Tab 12) in Sections 7.2.2.6 and 7.2.2.7.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
D2128
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
The submission references "Ex. A, Tab 4, § 7.1.2.2.d.2." We found no such reference contained in the materials provided to CMS.
Although the laboratory's submitted protocol indicates that unsatisfactory PT results would be investigated using CL FRM-00006-F1, the laboratory provided no documentation showing that the required form was used to investigate the unsatisfactory PT result. The laboratory also states that the "lab has investigated the one unacceptable challenge and documented its investigation and conclusions;" however, CL FRM-00006-Fl documenting the investigation was not submitted per the laboratory's protocol.
In the submission, the laboratory concludes:
- "Not enough patient data available for meaningful analysis"
- "No evidence of systemic errors"
- "No patient impact is expected"
However, no information as to how the laboratory came to these conclusions related to patient outcomes was submitted. Documentation contained in Ex O, Tab 3 shows that the laboratory received a passing score of 80% for blood cell identification. While this is true, the CLIA regulations require that all unacceptable results have remedial action taken and that such remedial action be documented. The submission does not contain evidence of remedial action.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program. The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
D5024
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
See our reviews of D5403, D5437, D5447, D5469, D5481, D5779, and D5801.
D5217
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
Some documents pertaining to this deficiency referenced in the submission were not included. Specifically, the submission references "Ex. A, Tab 4, § 7.1.2.2.d.2." We found no such reference contained in the materials provided to CMS. In addition, Ex. O, Tab 2 includes documentation stating: "The QC data are presented here: <Exhibit D Tab 7, Tab 8>." We found no "Tab 7" or "Tab 8" in "Exhibit D."
In the submission, the laboratory concludes:
- "Not enough patient data available for meaningful analysis"
- "No evidence of systemic errors"
- "No patient impact is expected"
However, no information as to how the laboratory came to these conclusions related to patient outcomes was submitted.
The laboratory states in Ex. O, Tab 1 that no peer data was available for the Siemens Immulite 2000. Based on lack of a peer group, the laboratory should compare its results to the "All Participants" values for troponin. Review of the laboratory's values versus the "All Participants" values shows that four of five of the laboratory's troponin values for the second proficiency testing event of 2014 were unacceptable when compared to the "All Participants" acceptable ranges. It appears the laboratory reviewed all of the mean values for all peer, instrument and method groups in order to determine an inappropriate acceptable range for its samples based on data not related to the "All Participants" values.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
D5311
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
The submission references "Ex. A, Tab 4, § 7.1.1.1.d.2." We found no such reference contained in the materials provided to CMS.
The laboratory concluded that "this issue is not likely to affect patients" because its mislabel error rate is comparable to published industry mislabel error rates. No other information related to possible patient outcomes was provided. Even though the laboratory's error rate may be comparable, the laboratory must still pursue any corrective action(s) for patients that may have been affected by this deficient practice.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
D5391
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
Finding #1
The submission references "Ex. A, Tab 4, § 7.1.1.1.d.2." We found no such reference contained in the materials provided to CMS.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to address and provide acceptable evidence of correction consisting of: what measure has been put in place or what systemic changes have been made to ensure the deficient practice does not recur; and, how the corrective action(s) is being monitored to ensure the deficient practice does not recur.
Finding #2
The submission references "Ex. A, Tab 18, § 3.7.1," and "Ex. A, Tab 23, §§ 7.14, 7.15, 8.6.1,
8.6.4, 8.6.5." We found no such references contained in the materials provided to CMS.
At the time of the onsite survey, the laboratory failed to establish a written protocol for the daily review of patient specimens referred to other laboratories for testing to ensure timely receipt and reporting of test results performed by other laboratories. The laboratory did not provide such a protocol with its submission.
In addition, it is unclear as to how the laboratory's QMPI Program would ensure the establishment of written protocols for all the laboratory's processes.
The laboratory failed to address and provide acceptable evidence of correction consisting of: what measure has been put in place or what systemic changes have been made to ensure the deficient practice does not recur; and, how the corrective action(s) is being monitored to ensure the deficient practice does not recur.
D5393
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
The submission references "Ex. A, Tab 23, §§ 7.14, 7.15, 8.6.1, 8.6.4, and 8.6.5." We found no such references contained in the materials provided to CMS.
At the time of the onsite survey, the laboratory failed to document the daily review of patient specimens referred to other laboratories for testing to ensure the timely receipt and reporting of test results performed by other laboratories.
In the submission, we found no written protocol as to how the daily review would be documented and no evidence this quality assessment activity had been documented since the survey.
In addition, without daily written documentation, it is unclear as to how the laboratory's QMPI Program would ensure this quality assessment activity has been completed.
The laboratory failed to address and provide acceptable evidence of correction consisting of: what measure has been put in place or what systemic changes have been made to ensure the deficient practice does not recur; and, how the corrective action(s) is being monitored to ensure the deficient practice does not recur.
D5400
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
See our reviews of D5403, D5407, D5413, D5421, D5423, D5429, D5437, D5447, D5449, D5469, D5477, D5481, D5775, D5779, D5787, D5791, and D5793.
D5403
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
Finding #1
Although the laboratory addressed the inclusion in its procedure manual of corrective actions to
be taken when calibration or quality control (QC) results failed to meet the laboratory's criteria for
acceptability when using the Siemens Advia 2120i instrument, in its submission the laboratory
failed to address how it will ensure its procedure manuals include applicable components as
required by 42 C.F.R. § 493.1251(b).
We also note that to ensure appropriate and required calibration or QC corrective actions were taken, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to address and provide acceptable evidence of correction consisting of: what measure has been put in place or what systemic changes have been made to ensure the deficient practice does not recur; and, how the corrective action(s) is being monitored to ensure the deficient practice does not recur.
Finding #2
The laboratory was cited for not having a QC procedure for the "Edison 3.5 Theranos System"
prior to 5/15/14. The laboratory's submission states that it performed QC before and after 5/15/14;
however, it did not address the citation. While the data submitted did show that QC was run prior
to 5/15/14, the laboratory provided no information regarding any investigation as to what
procedure the laboratory was using to determine number, type, frequency and acceptability criteria
for QC. The laboratory indicated that systemic errors using QC and patient test distribution over
time as well as random errors were used to evaluate patient impact. Specifically, the laboratory
indicated that "QC data was reviewed to identify >2SD failure and periods of elevated CV.
Namely, batches of 10 consecutive QC data points were analyzed and the CV was calculated for
each QC level during this period. If the CV was >30%, then all patient results run during that time
period will be voided." Ex E, Tab 1. The laboratory did not indicate how "periods of elevated
CV" were identified or what time periods were reviewed. The summary also included a specific
number of patient "reports [that] were identified" for each analyte that may have been affected by
the laboratory's QC errors, but did not include any documentation to indicate whether corrected
reports were generated and issued.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a “tracer audit” would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
D5407
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
Although the laboratory's submitted protocol (provided at Ex. A, Tab 1 § 4.13) indicates that the laboratory director must sign all procedures prior to use, the response does not include an explanation as to why the cited procedures were not signed by the laboratory director prior to use as required by the laboratory's protocol available at the time of the onsite survey.
In Ex. L, Tab 26 the submission states: "The lab's overarching review of its systems and primary instruments has identified the patients affected or having the potential to be affected by this issue." The laboratory did not define the criteria used to make this determination or provide documentation as to how they came to this conclusion.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
D5413
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
Finding #1
Although the laboratory's submitted protocol indicates that the allowable temperature should be posted on the unit (see Ex. A, Tab 29), the response does not address the incorrect labelling for acceptable temperatures on the freezer doors. Documentation contained in Ex. A. Tab 29, § 4.5 indicated that the laboratory supervisor or designee was responsible for monitoring and recording temperatures. However, we could not determine if the new procedure had been effectuated as no documentation was submitted to show that the freezer units had been labelled appropriately. In addition, the laboratory states that training had occurred subsequent to the survey, but it is not clear who should be trained as "laboratory supervisor or designee" was not defined. Training records provided in Ex. A, Tab 30 included only training documents for one general supervisor. Several freezers were identified as "not used in patient testing" (see Ex. N, Tab A); however, based on interviews during the onsite visit, these freezers were identified as being used for CLIA activities. The submission did not include a response or data related to the freezers identified as "not used in patient testing."
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
Finding #2
The submission references "Ex. I, Tabs 2-6." We located these tabs, but found no documentation in Tabs 2, 5, and 6.
Although the laboratory's submitted protocol indicates that the laboratory was required to check manufacturer package inserts prior to use (see Ex. A, Tab 31, §8.1.2), which was also stated in the laboratory's written procedure available prior to the onsite survey, the laboratory provided no documentation showing that this protocol has been effectuated and no indication that package inserts for new lot numbers have been checked. In addition, the submission states that "the lab has conducted training on those procedures;" however, the training documents submitted in Ex. A, Tab 32 do not include any training specific to the cited deficiency. We were unable to verify that training occurred as stated.
In Ex. I, Tab 1, the laboratory provided lists of patient specimen accession numbers for which the laboratory issued corrected reports; however, the laboratory did not provide copies of the corrected reports.
The laboratory failed to address and provide acceptable evidence of correction consisting of: what measure has been put in place or what systemic changes have been made to ensure the deficient practice does not recur; and, how the corrective action(s) is being monitored to ensure the deficient practice does not recur.
D5421
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
Finding #1 The submission references "Ex. A, Tab 9, §7.2.4.4 and Ex. B, Tabs 46-50, 52-56, 57-61." We found no such references contained in the materials provided to CMS.
The laboratory's submitted protocol (Ex. A, Tab 9, § 4.2) states: "If the verification is performed by the vendor, the laboratory is involved in all aspects of the verification and the final verification documentation is reviewed and signed off by the Laboratory Director." We were unable to determine what specific aspects of the method verification the laboratory would be performing or if vendors would continue to perform method verifications. The submitted protocol included form CL FRM-00022-Fl, Verification Results (Ex. A, Tab 9), which appears to be pre-signed by the laboratory director. We were unable to determine if the laboratory director will "review and sign off" on each method verification.
We note that in the laboratory's submission related to test performance specifications for the Advia XPT instrument, it was the laboratory's policy to accept %CV values that were up to 1.5 times the %CV values stated by the instrument manufacturer for any given chemistry analyte. In the submission, we found no explanation as to why 1.5 times the manufacturer's %CV was acceptable to the laboratory. We also note that the submitted re-verification of test performance specifications did not follow the submitted protocol for test verification; therefore, we could not determine if the re-verifications were adequate or how the laboratory determined if patients were affected or potentially affected.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
Finding #2
Although the laboratory's submitted protocol (Ex. A, Tab 9) included a method verification procedure which addressed verification of accuracy, precision, and reportable range, the laboratory provided no documentation showing that this protocol had been followed to re-verify performance specifications. The submission also stated: "Before the lab resumes any test on the Advia XPT, the lab will ensure that the test has been re-verified pursuant to the lab's improved method verification procedures that have been approved by the laboratory director." The completion date for the correction of this deficiency was indicated on the submitted Allegation of Compliance as February 12, 2016, which, in order to determine correction, should include the re-verification documentation for the Advia XPT. However, the laboratory provided no documentation that the re-verification had been performed.
We note that in the laboratory's submission related to test performance specifications for the Advia XPT instrument it was the laboratory's policy to accept %CV values that were up to 1.5 times the %CV values stated by the instrument manufacturer for any given chemistry analyte. In the submission, we found no explanation as to why 1.5 times the manufacturer's %CV was acceptable to the laboratory. We also found no explanation as how "normal patient distribution" related to determining if patients were affected or potentially affected.
In addition, the laboratory's submission (Ex. B, various analyte tabs) states: "Matrix comparisons done by the manufacturer between serum and plasma showed correlation." No documentation was submitted supporting this statement.
The submission also states that "the lab has conducted training on these procedures to ensure that the practice is consistent with them," but the laboratory did not include complete training documentation to support this assertion.
Because the laboratory has not shown whether it can follow its own validation protocols, it was uncertain whether the laboratory's quality assessment mechanisms could monitor the laboratory's corrective actions and ensure this deficient practice does not recur.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
D5423
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
Although the laboratory's submitted protocol (Ex. A, Tab 9) included a method verification procedure which addressed verification of accuracy, precision, and reportable range, the laboratory provided no documentation showing that this protocol had been followed to re-verify performance specifications. The submission also states: "Before the lab resumes any test on the Advia XPT, the lab will ensure that the test has been re-verified pursuant to the lab's improved method verification procedures that have been approved by the laboratory director." The completion date for the correction of this deficiency was indicated on the submitted Allegation of Compliance as February 12, 2016 which, in order to determine correction, should include the re-verification documentation for the Advia XPT. However, the laboratory provided no documentation that the re-verification had been performed.
We note that in the laboratory's submission related to test performance specifications for the Advia XPT instrument, it was the laboratory's policy to accept %CV values that were up to 1.5 times the %CV values stated by the instrument manufacturer for any given chemistry analyte. In the submission, we found no explanation as to why 1.5 times the manufacturer's %CV was acceptable to the laboratory. We also found no explanation as how "normal patient distribution" relates to determining if patients were affected or potentially affected.
The laboratory's procedure as well as the method verification for alkaline phosphatase was reviewed at the onsite survey and the reportable range was documented as 0 - 1100 IU/L thus modifying the test. At the time of survey, the laboratory stated that the reportable range was 10 1100 IU/L per the manufacturer, and that the reportable range of 0 - 1100 IU/L was an error. The laboratory confirmed that the lowest reportable value established by the manufacturer was 10 IU/L. The Patient Impact Analysis submitted by the laboratory at Ex. B, Tab 6 now indicates that the reportable range should be 5 - 1100 IU/L which is still lower than the range established by the manufacturer; however, the laboratory did not submit any documentation to support this new reportable range.
The Patient Impact Analysis also indicates that review of the PT showed a negative bias for PT samples CAP C-A 2015 and CAP-B 2015, but the laboratory determined that no corrected reports were needed. The submission also states: "There were no significant trends or bias in patient population data." The laboratory provided no explanations for these assertions.
In addition, the laboratory's submission states at Ex. B, Tab 6: "Matrix comparisons done by the manufacturer between serum and plasma showed correlation." The laboratory submitted no documentation supporting this statement.
The submission also states that "the lab has conducted training on these procedures to ensure that the practice is consistent with them," but did not include complete training documentation.
Because the laboratory has not shown whether it can follow its own validation protocols, it is uncertain whether the laboratory's quality assessment mechanisms could monitor the laboratory's corrective actions and ensure this deficient practice does not recur.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
D5429
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to address and provide acceptable evidence of correction consisting of: what measure has been put in place or what systemic changes have been made to ensure the deficient practice does not recur; and, how the corrective action(s) is being monitored to ensure the deficient practice does not recur.
D5437
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
Finding #1 & #2
At the time of the onsite survey, the laboratory failed to maintain calibration documentation for the complete blood count (CBC) instruments. In the submission, we found no evidence the laboratory has re-established its calibration documentation for the CBC instruments.
In Ex. F, the laboratory provided lists of patient specimen accession numbers for which the laboratory intended to issue corrected test reports; however, the laboratory provided no documentation to indicate corrected reports were generated and issued.
To ensure the maintenance of CBC instrument calibration documentation, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
In addition, we note that in the laboratory's re-verification of test performance specifications for the CBC instruments, it was the laboratory's policy to accept %CV values that were up to 1.5 times the %CV values stated by the instrument manufacturer for any given CBC analyte. In the submission, we find no explanation as to why 1.5 times the manufacturer's %CV is acceptable to the laboratory.
The laboratory failed to address and provide acceptable evidence of correction consisting of: what measure has been put in place or what systemic changes have been made to ensure the deficient practice does not recur; and, how the corrective action(s) is being monitored to ensure the deficient practice does not recur.
D5447
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
Although the laboratory's submitted protocol indicates that a two-level QC slide review is now required every day of patient testing when using the Cellavision, the laboratory provided no documentation of or information about the QC materials used, how the statistical parameters of the QC materials would be determined, how QC test results will be documented, and whether laboratory staff has been trained on this new protocol.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to address and provide acceptable evidence of correction consisting of: what measure has been put in place or what systemic changes have been made to ensure the deficient practice does not recur; and, how the corrective action(s) is being monitored to ensure the deficient practice does not recur.
D5449
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
Although the laboratory's submitted protocol indicates that an external positive QC material would be included in each run of patient Chlamydia trachomatis/Neisseria gonorrhoeae (CT/NG) testing, the laboratory provided no documentation of the positive QC material used, how the statistical parameters of the positive QC material would be determined, how the positive QC test result will be documented, and whether laboratory staff has been trained on this new protocol.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to address and provide acceptable evidence of correction consisting of: what measure has been put in place or what systemic changes have been made to ensure the deficient practice does not recur; and, how the corrective action(s) is being monitored to ensure the deficient practice does not recur.
D5469
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
Finding #1
Although the laboratory's submitted protocol indicates that the stated values of new commercially assayed CBC QC materials were to be verified through parallel testing against QC materials in use, the laboratory provided no documentation indicating that this protocol had been effectuated, no information as to how the results of the parallel testing will be documented, and no information as to whether laboratory staff has been trained on this new protocol.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to address and provide acceptable evidence of correction consisting of: what measure has been put in place or what systemic changes have been made to ensure the deficient practice does not recur; and, how the corrective action(s) is being monitored to ensure the deficient practice does not recur.
Finding #2
Although the laboratory's submitted protocol indicates that the stated values of new commercially assayed chemistry QC materials were to be verified through parallel testing against QC materials in use, the laboratory provided no documentation indicating that this protocol has been effectuated, no information as to how the results of the parallel testing will be documented, and no information as to whether laboratory staff has been trained on this new protocol.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to address and provide acceptable evidence of correction consisting of: what measure has been put in place or what systemic changes have been made to ensure the deficient practice does not recur; and, how the corrective action(s) is being monitored to ensure the deficient practice does not recur.
D5477
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
The laboratory submitted no written procedure for the bacteriology media QC protocol, and no information as to whether laboratory staff has been trained on this new protocol.
In Ex. L, Tab 8, the submission included a completed form titled "Blood Agar 5% Quality Control Logsheet." We note that the entry for the media received on 1/18/16 indicates that QC testing of this bacteriology media was completed on 1/14/16, which is before the date the media was received by the laboratory. Yet, as documented on the form, the completed form was reviewed on 2/11/16 without any indication this entry had been reviewed. We question the accuracy of this entry as well as the effectiveness of the laboratory's oversight mechanism.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to address and provide acceptable evidence of correction consisting of: what measure has been put in place or what systemic changes have been made to ensure the deficient practice does not recur; and, how the corrective action(s) is being monitored to ensure the deficient practice does not recur.
D5481
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
Finding #1
The submission references "Ex. I, Tabs 2-6." We located these tabs, but found no documentation in Tabs 2, 5 and 6.
Although the laboratory's submitted protocol requires that QC values be acceptable prior to reporting patient results, the submission states: "Theranos reviewed all quality control (QC) data for PT/INR [Prothombin Time/International Normalized Ratio] for the time period that this lot of Dade Innovin was in use." The laboratory provided no documentation of this review other than stating it was performed. We also found no documentation to indicate that the revised standard operating procedures (SOPS) have been effectuated. That is, we found no documentation of PT/INR QC failure investigations and corrective actions taken based on the revised SOPS.
In Ex. I, the laboratory provided lists of patient specimen accession numbers for which the laboratory intended to issue corrected test reports. The laboratory provided no documentation to indicate corrected reports were issued. We also note that the Patient Impact Assessment (Ex. I, Tab 1) states that "remedial action was taken on 9/25/15," but "corrected reports were issued beginning on 11/10/15 and completed on 11/12/15." There was no explanation as to why there was such a long period of time between the remedial action and issuing corrected reports.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to address and provide acceptable evidence of correction consisting of: what measure has been put in place or what systemic changes have been made to ensure the deficient practice does not recur; and, how the corrective action(s) is being monitored to ensure the deficient practice does not recur.
Finding #2
Although the laboratory submitted a QC protocol (Ex. A, Tab 9), the laboratory's submission did not include an updated protocol for QC specific to the Theranos Proprietary Devices. The reference protocol states at §§ 8.1.9-8.1.10:
No instrument or test method can be used for the purpose of performing a test to report a result to a patient prior to having passed all required QC procedures at least once for the day on which the test is to be performed. It is the responsibility of all testing personnel to ensure this does not occur. Testing personnel must immediately take the appropriate action for all failed QC runs, refer to section 8.4.
In addition, the laboratory's submission states:
To perform this analysis, systematic error was investigated using quality control (QC) data and patient distribution over time. The QC data was reviewed in Levey-Jennings plots and evaluated for bias trends. The patient data was summarized by plotting monthly means, monthly medians and monthly means ("average of normal" or AON) calculated from central 95% of values. In addition, random errors were evaluated by analysis of QC data. Namely, QC data was reviewed to identify >2SD failures and period of elevated CV. Namely, batches of 10 consecutive QC data points were analyzed and the CV was calculated for each QC level during this period. If the CV was >30%, then all patient results run during that time period will be voided. (Ex. E, Tab 1)
The laboratory's response does not include an evaluation of unacceptable QC results or patients affected or potentially affected by the cited QC failures. It is not clear how evaluating aggregate patient (i.e., monthly means and medians) and QC (i.e., bias trends, >2SD, CV) data addressed specific days that the QC was unacceptable and patient results were reported.
The summary (Ex. E, Tab 1) included analytes tested on the Theranos Proprietary Device. We note that general statements such as, but not limited to, were included:
- "Monthly patient distribution variance were noted, but they could not be further substantiated by QC trends"
- "Patient results following a QC failure and during periods of high CV are being voided"
- "Patient distribution and QC trends did not reveal significant trends."
- "Potential biases observed in patient distributions were found to be insignificant relative to the normal reference intervals"
The laboratory's submission did not provide any explanation or documentation to support these statements.
In Ex. E, the laboratory provided a list of patient specimen accession numbers for which the laboratory "identified reports," but did not specify or submit documentation to indicate whether corrected reports were generated and issued.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
D5775
The laboratory's allegation of compliance is not credible and evidence of correction not acceptable.
Finding #1
In the submitted protocol, the laboratory states: "To ensure all manual differentials performed are comparable, . . .manual differentials will be compared annually amongst the competent staff. . ." The laboratory provided no written procedure as to how "manual differentials will be compared annually." In addition, the regulation requires the laboratory to conduct such evaluations twice a year, not annually.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to address and provide acceptable evidence of correction consisting of: what measure has been put in place or what systemic changes have been made to ensure the deficient practice does not recur; and, how the corrective action(s) is being monitored to ensure the deficient practice does not recur.
Findings #2
Although the laboratory submitted a method/instrument comparison protocol (Ex. A, Tab 28), we were unable to determine if the protocol was effectuated. In its submission, the laboratory states: "The lab's technical supervisors and the quality system director will be responsible for ensuring that these procedures are followed." However, the laboratory's submission did not include any documentation to indicate that the technical supervisors or the quality systems director had been trained on the method/instrumentation comparison protocols.
The laboratory's submission also includes the following statement:
Because TPS's [Theranos Proprietary Devices] were factory calibrated by Theranos, the laboratory did not perform additional method correlation studies. As part of this investigation, Theranos examined the correlation of different TPS to evaluate their concordance. Namely, for SHGB, TT3, Vitamin B12 and Vitamin D, patient distributions were compared across representative TPS's in use during the same period (Exhibit E Tab 12-A; Tab 24-A, Tab 33-A. These data showed similar patient distributions across the different devices. In addition, comparison of assay results across different assay levels and different TPS's demonstrates that the instruments were well correlated to each other (Exhibit E, Tab 12-A; Tab 24-A, Tab 33-A). Namely, the median result for each TPS was within the total allowable error thereby demonstrating that the same assay reference ranges were appropriate for TPS.
The laboratory's response indicates that not all TPS would be included in the correlation studies rather "across representative TPS in use during the same period." The laboratory submitted correlation studies for Sex Hormone Binding Globulin (SHBG) and Vitamin D (VitD) only (Ex E, Tabs 12-A, 24-A and 33-A, respectively). However, these studies did not include a date on either the summary or raw data, so we were unable to determine when the correlation studies occurred. The submission for SHBG also included information on one document with box plots that included three devices and another document that included box plots of six devices, so we were unable to determine what devices were actually used for SHBG testing.
The laboratory's submission offered no response to correlation studies for Vitamin B12 and no explanation for the disparity between the devices used for QC and the devices used for patient testing. We also noted that there was no explanation as to where the total allowable error (TAE) of 24% on the raw data documentation came from or if any TAE was calculated for the submitted data. The laboratory used the median result for each proprietary device to determine if TAE was acceptable, but did not provide an explanation why the median results were used.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to address and provide acceptable evidence of correction consisting of: what measure has been put in place or what systemic changes have been made to ensure the deficient practice does not recur; and, how the corrective action(s) is being monitored to ensure the deficient practice does not recur.
D5779
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
In Ex. F, the laboratory provided lists of patient specimen accession numbers for which the laboratory intended to issue corrected test reports. The laboratory provided no documentation to indicate corrected reports were issued.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
D5787
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
The laboratory submitted no written procedure for the documentation of the lot of bacteriology media used to test any given patient specimen, and no information as to whether laboratory staff has been trained on this new protocol.
In its submission, the laboratory references Ex. L, Tab 6 which states: "When [media quality control] testing is completed the results are logged into an exel sheet with lot number and expiration date for each patient (see attached report)." We find no "attached report" in the submission and, therefore, no documentation indicating the laboratory's corrective action has been effectuated.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to address and provide acceptable evidence of correction consisting of: what measure has been put in place or what systemic changes have been made to ensure the deficient practice does not recur; and, how the corrective action(s) is being monitored to ensure the deficient practice does not recur.
D5791
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
Finding #1
The laboratory submitted a protocol for the SensoScientific Monitoring in Ex. A, Tab 24. The response does not specifically address the days that the freezers did not meet acceptable temperatures as documented in the SensoScientific Monitoring Audit Node Log. We could not determine if the new procedure has been effectuated as no documentation was submitted to show that the unacceptable freezer temperatures were addressed. In addition, the laboratory states that training has occurred, but it was not clear who should be trained as "laboratory supervisor or designee" was not defined. Training records in Ex. A, Tab 30, included only training documents for one general supervisor. Several freezers were identified as "not used in patient testing" (see Ex. N, Tab A); however, based on interviews during the onsite visit, these freezers were identified as being used for CLIA activities. The submission does not include a response or data to support the laboratory's assertion that certain freezers are "not used in patient testing."
The laboratory submitted an article for the use of Mean Kinetic Temperatures (MKT) (Ex. N, Tab 1). This article relates to the handling, storage, and distribution of temperature sensitive pharmaceuticals, not clinical laboratory specimens, reagents, reference materials, etc. We are unclear as to how this article applies to this deficient practice, and note that the written procedure submitted by the laboratory at Ex. A, Tab 24 does not incorporate monitoring temperature by MKT.
The laboratory also submitted an article related to the storage of bacterial samples for optimal viability (Ex. N, Tab 22). We note that the submitted written procedure relates to determining acceptable temperature viability of bacterial samples does not include any references to this article. We are unclear as to how this article applied to this deficient practice.
The laboratory submitted no evidence to support the following conclusion for the Sanyo JP Lab 7059 -20 freezer: "[The laboratory] determined that higher storage temperature had no effect on the calibrators or controls stored in this freezer. There is no patient impact from the change in temperature during the time period."
In addition, the laboratory stated for another freezer (1 BUGS 7120 -80 freezer) that the "MKT remained between -70 and -86 C for the entire date range." However, the laboratory indicated that "a small number of bacterial cultures. . . labeled to be stored at -80C or colder" were stored in this freezer. It was unclear as to why it was appropriate to store "bacterial cultures. . . labeled to be stored at -80C or colder" to be stored at temperatures of -70C to -79C.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
Finding #2
The laboratory submitted a protocol for a Master Validation Plan (Ex. E, Tab 30B) which required the %CV of replicates to be not more than 20% (25% at the lower and upper limits of detection). The submitted protocol was different than the protocol presented to the surveyor at the time of the onsite survey which required the %CV of replicates to be not more than 15% (20% at the lower and upper limits of detection). The %CV in the protocol presented to the surveyor at the time of the onsite survey matched the %CV of the predicate devices. It is unclear as to why the TPS, which is covered by the Master Validation Plan, can now have %CV's for replicates greater than the predicate devices; no explanation was submitted. In addition, we note that the %CV's (18.7% -63.8%) included in the submission does not appear to have been evaluated using this new protocol nor was it addressed in the laboratory's response provided at Ex. E, Tab 1."
The submission (Ex. E) includes a list of patient specimen accession numbers for which the laboratory "identified reports," but does not specify or include documentation to indicate whether corrected reports were generated and issued.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
Finding #3
The laboratory's review of the QC found in Ex. E, Tab 1 did not specify what period of QC the laboratory reviewed. The quality assessment (QA) documentation related to QC data from the survey in July 2014, October 2014, and February through June 2015. The laboratory's investigation did not indicate that the review had identified the cited issues related to QC.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
Finding #4
Although the laboratory's submitted protocol included information about quarterly audits and agenda items for the laboratory's QMPI Program, the submission does not provide specific actions to take when problems are identified, specifically related to clotted specimens. The laboratory states in Ex. L, Tab 35 that "[p]atient specimens that did not meet the lab's acceptance criteria were rejected. . ." However, the issue was not that the clotted specimens were rejected, rather that the laboratory's QA program did not identify the high number of clotted specimens over a period of three quarters. The submission does not include any documentation showing that the issue was identified and corrected. We note that Ex. A, Tab 12, §7.2.1.6 indicates that the QMPI Program agenda included specimen rejection rate partitioned by rejection criteria, but does not require that any action be taken based on this agenda item. In addition, we note that in Ex. A, Tab 12, §7.1, the laboratory provided the frequency of the QMPI Program meetings, but does not indicate any action should be taken in the event it is determined that the quality metrics were not met.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit,” the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to address and provide acceptable evidence of correction consisting of: what measure has been put in place or what systemic changes have been made to ensure the deficient practice does not recur; and, how the corrective action(s) is being monitored to ensure the deficient practice does not recur.
D5793
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
Finding #1
See our review of D5403, D5437, D5469, and D5779.
Finding #2
In the submission, the laboratory states:
"Based on comprehensive review, multiple examples of failed QC without the appropriate documentation with no investigation or corrective action was identified which is now addressed with revised SOPS and training. Additional competency is on-going to ensure CLS demonstrate the required proficiency with quality control and the appropriate investigation, corrective action and notification."
We find no documentation to indicate that the "revised SOPS" have been effectuated. That is, we find no documentation of cytometry QC failure investigations and corrective actions taken based on the "revised SOPS" as discovered while conducting the "comprehensive review."
In Ex. J, the laboratory provided lists of patient specimen accession numbers for which the laboratory intended to issue corrected test reports. The laboratory provided no documentation to indicate corrected reports were generated and issued.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to address and provide acceptable evidence of correction consisting of: what measure has been put in place or what systemic changes have been made to ensure the deficient practice does not recur; and, how the corrective action(s) is being monitored to ensure the deficient practice does not recur.
Finding #3
In the submission, the laboratory states:
- "07/16/2015: Level 3 QC outside manufacturer's recommended range, and not repeated, therefore level 3 QC was out of control for the day"
- "09/01/2015 to 09/13/2015: all level 3 control results are outside the manufacturer's recommended range (negative bias)"
- "10/01/2015 - 10/31/2015: all level 3 control results are outside the manufacturer's recommended range (negative bias)"
- "All 3 levels of QC were not run on 11/04/2015, 11/05/2015, 11/07/2015, 11/08/2015, 11/09/2015, 11/10/2015, 11/12/2015, 11/14/2015, and 11/15/2015" [The laboratory provided no information as to whether patient HCG specimens were tested on the November dates listed.]
Yet, the laboratory concluded: "Results for urine HCG are quantitative with a cut off of 30 mUI/ml. Because of the nature of this assay no correct reports are required."
Since QC failures may indicate possible test system problems and it is unknown as to how the HCG (human chronic gonadotropin) test results were used and interpreted, we question the laboratory's conclusion and the accuracy and reliability of any patient HCG test results reported on the July, September, October, and November dates listed.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
Finding #4
In the submission, the laboratory states:
- "Anti-HBs is a qualitative assay with a positive cutoff of 11 so no corrected reports are required."
- "Qualitative assay will not require corrected reports to be issued"
We are unclear as to how the laboratory came to these conclusions and the submission does not contain any documentation to support these statements.
Nevertheless, to ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
Findings #5
In the submission, the laboratory states: "Corrected [LH] reports to be sent for patient samples analyzed for the following time period: 7/1/2015-7/8/2015; all patient samples analyzed in 8/2015 and 9/2015." However, later in the submission, the laboratory concluded that "[b]ased on this review, 0 corrected reports are being issued." From these conflicting statements, it is unclear whether the laboratory addressed patient outcomes. The laboratory provided no documentation to indicate that corrected LH (luteinizing hormone) patient reports were issued.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
Finding #6
In the submission, the laboratory references "Ex. D, 1" and "Ex. D, 2-8." Ex. D, 1 and Ex. D, 2-8 relate to the laboratory's HCG testing, and not CA-125 testing, the analyte in the cited deficiency. However, the laboratory's CA-125 testing was found in Ex. C, Tabs 1 - 4. We considered those results in reaching the following conclusions.
In the submission, the laboratory states:
- "All patient results reported > 472 U/ml will need corrected reports."
- "Based on this review, 122 corrected [CA-125] reports are being issued to reflect the verified reportable range.”
The laboratory provided no documentation to indicate corrected reports were generated and issued. To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
Finding #7
The submission references "Ex. H, Tabs 1, 2 - 5, 16, 17 - 20, 21, 22 - 25, 26, 27 - 30, 36, 37 - 40, 46, 47 - 50, 56, and 57-60." We found no such references in the materials provided to CMS. Throughout Ex. H, the submission includes lists of patient specimen accession numbers for which the laboratory intended to issue corrected test reports. The laboratory provided no documentation to indicate corrected reports were generated and issued.
We found no information addressing how the laboratory will ensure the timely review of the effectiveness of any actions taken. Although the laboratory indicated that quarterly audits will be performed to ensure compliance with this deficient practice, it is unclear how quarterly audits will timely address the issues cited under this deficiency.
Nevertheless, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
Finding #8
The submission references "Ex. B, Tabs 51 and 53-55, and Ex. H, Tabs 1, 2 -5, 17 - 20, 26, 27 30,36,37-40, 46, 47 - 50, 56, and 57 - 60." We found no such references in the materials provided to CMS.
The laboratory submitted a new QC procedure (Ex. A, Tab 1) which requires QC Pass/Fail Criteria. Specifically, Section 8.2.1.1.a - c. provides that QC must meet certain criteria as well as ". . .the required Westgard rule pass criteria." Westgard rules included a "10x" rule for rejecting QC when 10 consecutive control measurements fall on one side of the mean. We note that on page 17 of 19 (Ex. A, Tab 6) in the updated protocol that the "10x" rule was included as part of the Westgard rules which the lab must follow. The lab provided no documentation indicating that the "10x" rule was evaluated as part of its review.
In Ex. B and Ex. H, the submission includes lists of patient specimen accession numbers for which the laboratory intended to issue corrected test reports. The laboratory provided no documentation to indicate corrected reports were generated and issued.
Because the laboratory has not shown whether it can follow its own QC protocols, it is unclear whether the laboratory's quality assessment mechanisms can monitor the laboratory's corrective actions and ensure this deficient practice does not recur.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
Finding #9
The laboratory did not submit any documentation in the referenced exhibits related to an updated protocol for Alternative Assessment Procedures (AAP); therefore, we have concluded that the current protocol applies. The laboratory's submission did not include any documentation to explain why the laboratory did not follow its AAP protocol for the Theranos Proprietary System.
Because the laboratory has not shown whether it can follow its own AAP protocols, it is unclear whether the laboratory's quality assessment mechanisms can monitor the laboratory's corrective actions and ensure this deficient practice does not recur.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
Finding #10
Ex. A, Tab 9 of the submission contains the protocol "Method Validation." Section 8.4 of this protocol, "Reference Intervals," states:
The reference intervals are established from a selected reference population. Typically, when both decreased and increased levels are clinically significant, it is defined as the interval between and including the lower and upper reference limits of the studies population. This is calculated as the central 95 percentile of the population distribution where the lower and upper reference limits are demarcated as the 2.5th and 97.5th percentiles of the underlying distribution of values. . . The reference intervals are used to establish expected level in health "normal" individuals. Values outside the reference range are considered "abnormal" or "high" or "low.
In Ex. E, Tab 1, the laboratory states:
[The] laboratory director decided to use medical decision limits based on national consensus. Accordingly, the following decision limits were displayed on all lab reports for Vitamin D. As noted in CLSI EP28-A3, "when decision limits determined by national or worldwide consensus exist, these limits, rather than reference intervals should be reported.
In Ex. E of the submission, the laboratory provided the following "medical decision limits:"
| Deficiency: | <20.0 ng/mL |
| Insufficiency: | 20.0 - 29.0 ng/mL |
| Sufficiency: | 30.0 - 100.0 ng/mL |
| Possible Toxicity: | >100.0 ng/mL |
We were unclear as to the laboratory's definition of "medical decision limits." No definition of this phrase was found in Section 3—Definitions of the "Method Validation" protocol. In addition, no reference to CLSI EP28-A3 was found in the submitted protocol. We also note that the submitted protocol does not include a reference to "medical decision limits" as a criterion for determining reference range.
Since the laboratory provided no definition of "critical medical decision level," no information regarding the laboratory's disparity between the reference range (9.3 47.9 ng/mL) in the validation documentation and raw data reports, and no explanation as to how a Vitamin D level could be both normal and deficient or insufficient, it is unclear as to whether the laboratory has followed its own "Method Validation" protocol set forth at Section 8.4.
Throughout Ex. E, the laboratory provided a list of patient specimen accession numbers for which the laboratory "identified reports," but did not specify or submit documentation to indicate whether corrected reports were generated and issued.
Because the laboratory has not shown whether it could follow its own validation protocols, it is unclear whether the laboratory's quality assessment mechanisms can monitor the laboratory's corrective actions and ensure this deficient practice does not recur.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
D5801
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
The submission references "Ex. I, Tabs 2-6." We located these tabs, but found no documentation in Tabs 2, 5 and 6.
In the submission, the laboratory states:
Before PT/INR testing resumes, the lab will also prepare a revised assay-specific procedure for PT/INR to reinforce that the International Normalized Ratio (INR) must be calculated accurately prior to reporting patient test results. The relevant testing personnel will be required to demonstrate competency to ensure the practice is consistent with these procedures.
We note that the laboratory provided no documentation to indicate that the inaccurate calculations were reviewed or that INR calculation using different lot numbers of Dade Innovin were reviewed for accuracy. In addition, we found that the laboratory did not provide documentation to ensure that PT/INR results reported from calculations were, and would continue to be, accurate.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
D5805
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
The submission references "Ex. I, Tabs 2-6." We located these tabs, but found no documentation in Tabs 2, 5 and 6.
In its submission, the laboratory states: "The lab revised its patient reports during the survey so that the interpretive note appears only under the heading for patients with therapy." However, no updated forms were shown to the surveyor during the survey and the laboratory's submission does not include documentation of the updated reports.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
D5821
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
The submission references "Ex. I, Tabs 2-6." We located these tabs, but found no documentation in Tabs 2, 5 and 6.
In Ex. I, the laboratory provided lists of patient specimen accession numbers for which the laboratory intended to issue corrected test reports. The laboratory provided no documentation to indicate corrected reports were issued. We also note that the Patient Impact Assessment (Ex. I, Tab 1) states that "remedial action was taken on 9/25/15," but "corrected reports were issued beginning on 11/10/15 and completed on 11/12/15." There was no explanation as to why there was such a long period of time between the remedial action and issuing corrected reports.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to address and provide acceptable evidence of correction consisting of: what measure has been put in place or what systemic changes have been made to ensure the deficient practice does not recur; and, how the corrective action(s) is being monitored to ensure the deficient practice does not recur.
D6076
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
See our review of D6083, D6085, D6086, D6093, D6094, D6098, and D6102.
D6079
Based on the laboratory's submission, this requirement is met.
D6083
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
See our review of D5413 and D5791.
D6085
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
See our review of D6115.
D6086
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
Finding #1
The submission references "Ex. A, Tab 9, F-1" and "Ex. A, Tab 10, F-1." We found no such references in the materials provided to CMS.
Throughout Ex. E, Tab 1, the laboratory provided lists of patient specimen accession numbers for which the laboratory intended to issue corrected test reports. The laboratory provided no documentation to indicate corrected reports were generated and issued.
We found no information addressing how the laboratory will ensure the accurate review of test method validation documentation, the issue cited in the deficiency.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
Finding #2
The submission references "Ex. H, Tabs 1, 2-5, 16, 17 - 20, 21, 22 - 25, 26, 27 - 30, 36, 37-40, 46, 47 - 50, 56, and 57 - 60." We found no such references in the materials provided to CMS.
Ex. H contains lists of patient specimen accession numbers for which the laboratory intended to issue corrected test reports. The laboratory provided no documentation to indicate corrected reports were generated and issued.
At the time of the onsite survey, the laboratory's protocol "Master Validation Plan for Routine Chemistry Assays on Theranos Devices," stated: ". . .for establishing the trueness or comparability to two procedures. . .at least 50% of samples should be outside the reference interval."
For five validation documents (ALT, BUN, calcium, glucose, and sodium testing using the b 4 b 4, a review of the test results used by the laboratory to establish "the trueness or comparability of two procedures" showed that the laboratory did not follow its established protocol and use "at least 50% of samples. . .outside the reference interval."
The submission includes the protocol "Method Validation," at Ex. A, Tab 9, Section 8.1.1.1 ("A method comparison and bias estimation design"), which states: "If the analyte has a critical medical decision level, at least 50% of the specimens must have analyte levels below this value and the remaining 50% above."
We are unclear as to the laboratory's definition of "critical medical decision level." No definition of this phrase was found in Section 3—Definitions of the "Method Validation" protocol or in Ex. H.
In Ex. H of the submission, the laboratory provided the following "assay validation" summary:
| Analyte | Number of Specimens Used | Specimen Value Range |
| ALT | 128 | 12 – 62 |
| BUN | 128 | 9 – 21 |
| Calcium | 128 | 9.2 – 10.2 |
| Glucose | 128 | 65 – 142 |
| Sodium | 128 | 134 – 142 |
Since the laboratory provided no definition of "critical medical decision level" and no information. regarding the laboratory's panic or alert values for ALT, BUN, calcium, glucose, and sodium testing, it is unclear as to whether the laboratory has followed its own "Method Validation" protocol at Section 8.1.1.1.
Because the laboratory has not shown whether it can follow its own validation protocols, it is unclear whether the laboratory's quality assessment mechanisms can monitor the laboratory's corrective actions and ensure this deficient practice does not recur.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
Finding #3
Ex. H contains lists of patient specimen accession numbers for which the laboratory intended to issue corrected test reports. The laboratory provided no documentation to indicate corrected reports were generated and issued.
At the time of the onsite survey, for five validation documents reviewed (ALT, BUN, calcium, glucose, and sodium testing using the b 4 laboratory summary information indicated that the laboratory obtained coefficient of variation (CV) results greater than CV's established by the instrument manufacturer. That is, ALT, BUN, calcium, glucose, and sodium test results obtained by the laboratory were statistically less precise than what would be expected using the b 4, The laboratory provided no explanation for and no information pertaining to any investigation as to why the laboratory's test results were less precise. These validation documents were approved by the laboratory director as evidenced by his signature.
In Ex. H of the submission, the laboratory provided summary data for ALT, BUN, calcium, glucose, and sodium similar to the CV data reviewed at the time of the onsite survey. Again, the laboratory provided no explanation for and/or information pertaining to any investigation as to why the laboratory's test results were less precise.
Because the laboratory has not shown whether its assay verification procedures are adequate to determine assay precision characteristics, we question whether the laboratory's quality assessment mechanisms can monitor the laboratory's corrective actions and ensure this deficient practice does not recur.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
Finding #4
Ex. H contains lists of patient specimen accession numbers for which the laboratory intended to issue corrected test reports. The laboratory provided no documentation to indicate corrected reports were generated and issued.
At the time of the onsite survey, for four validation documents reviewed (ALT, BUN, calcium, and glucose testing using the b 4 laboratory summary information indicated that the reference range determined by the laboratory's testing differed from the reference range on the laboratory's test reports. The following was noted:
| Analyte | Validation Document Reference Range | Test Report Reference Range |
| ALT | 0 – 52 U/L | 8 – 41 U/L |
| BUN | 5.3 – 22.5 mg/dL | 6 – 24 mg/dL |
| Calcium | 8.18 – 10.3 mg/dL | 8.3 – 10.6 mg/dL |
| Glucose | 64.0 – 112.3 mg/dL | 73 – 99 mg/dL |
In Ex. H of the submission, the laboratory provided the following reference ranges, some of which were still different than what appeared on the laboratory's test reports:
| Analyte | Reference Range |
| ALT | 8 – 41 U/L |
| BUN | 6.2 – 22 mg/dL |
| Calcium | 8.2 – 10.3 mg/dL |
| Glucose | 64 – 112 mg/dL |
Since the laboratory did not provide copies of any updated test reports in its submission, we are unable to determine whether the laboratory has corrected this deficient practice.
Because the laboratory has not shown whether it had corrected this deficient practice, we are also unable to determine whether the laboratory's quality assessment mechanisms can monitor the laboratory's corrective actions and ensure this deficient practice does not recur.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
Finding #5
The submission references "Ex. G, Tabs 31, 34, 36, 39, 41, 44, 46, 49, 51, 54, 56, 61, 64, 66, and 69." We found no documents within these tabs.
At the time of the onsite survey, the laboratory maintained no documentation showing that the validation documents for two Siemens Advia 2120i instruments had been reviewed and approved by the laboratory director.
In Ex. G of the submission, we again found no information to indicate the Siemens Advia 2120i validation documents have been reviewed and approved by the director.
Because the laboratory has not shown whether it has corrected this deficient practice, we are also unable to determine whether the laboratory's QA mechanisms can monitor the laboratory's corrective actions and ensure this deficient practice does not recur.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
Finding #6
The submission references "Ex. I, Tabs 2-6." We located these tabs, but found no documentation in Tabs 2, 5 and 6.
See our review of 05413.
Although the submitted protocol (Ex. A, Tab 31, § 8.1.2) included the requirement to review all new manufacturer package inserts and vendor notifications, the laboratory did not provide a response to the citation other than to state: "Theranos did not have the documentation for the MNPT calculation done for lot #539280 in 3/2015 before it was put in use for patient testing." It is unclear what investigation was performed to determine how the Mean Normal Prothrombin Time (MNPT) for Dade Innovin lot number 539280 was defined and entered into the Siemens BCS-XP.
In its submission the laboratory states "training has occurred," however, training documentation provided in Ex. A Tab 32 did not include all testing personnel listed on the CMS-209 (Ex. L, Tab 1).
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to address and provide acceptable evidence of correction consisting of: what measure has been put in place or what systemic changes have been made to ensure the deficient practice does not recur; and, how the corrective action(s) is being monitored to ensure the deficient practice does not recur.
D6093
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
Finding #1
See our review ofD5447, D5449, D5469, D5477, D5481, and D5779.
Finding #2
See our review of D5481.
The submission references "Ex. I, Tabs 2-6." We located these tabs, but found no documentation in Tabs 2, 5 and 6.
The laboratory's submission did not include an explanation or documentation of an investigation regarding the adjustment of QC ranges.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
D6094
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
Finding #1
See our review of D5391, D5393, D5791, and D5793.
Finding #2
See our review of D5413, D5481, D5801, D5805, and D5821.
D6098
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
See our review of D5805.
D6102
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
The laboratory submitted protocols relating to testing personnel training (Ex. A, Tab 16, §§ 4.5.3 and 6.4) which state the following:
- § 4.5.3 "Testing personnel/trainee are responsible to ensure that training is properly documented using CL-FRM-03016-F4, Training Attendance Form, approved by the designated trainer and stored in the relevant binder."
- § 6.4 " . . .It is the responsibility of the trainee and trainer to ensure the SOP is accurate."
However, the regulations require that the laboratory director, not the testing personnel, is responsible for ensuring that prior to testing patient specimens, all personnel have the appropriate training and have demonstrated that they can perform all testing operations to provide and report accurate results.
Training documentation submitted in Ex. A, Tab 11 only includes training on protocols related to testing personnel qualification requirements and delegation of responsibilities, as well as job descriptions. The laboratory provided no documentation or plan to train or retrain testing personnel on the Theranos Proprietary Device. Training documents submitted in Ex. A, Tab 17 only addressed review of the document control and training and competency protocols, but did not include training or retraining documentation for personnel performing testing in the venipuncture lab nor did it include an investigation if patient were affected or potentially affected when untrained or partially trained testing personnel were performing patent testing.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
D6108
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
See our review of 06115.
D6111
Based on the laboratory's submission, this requirement is met.
D6115
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
The laboratory submitted a protocol for a Master Validation Plan at Ex. E, Tab 30B which was a different protocol than presented to the surveyor. In Ex. E, Tab 1, under the Accuracy section, the submission indicates that "some of the immunoassays on the TSP [Theranos Proprietary System] were 'calibrated' or 'corrected' to remove systemic bias between the two methods." It is not clear what specific biases were used for each analyte which used a corrected results or if the corrected or uncorrected results was reported. In addition, the Accuracy section described that the "samples spanned the medical decision levels (MDL)". We are unclear as to the laboratory's definition of "medical decision limits." No definition of this phrase was found in Section 3—Definitions of the "Method Validation" protocol or in Ex. E. We also note that the submitted protocol does not include a reference to "medical decision limits" as a criterion for determining accuracy.
We also note that the laboratory's submission provided no indication why the validation report for SHBG had an effective date of 7/14/14, but was not approved by the laboratory director until 9/19/15, even though patient testing began 7/28/14.
The laboratory's submission in Ex. E, Tab 1 states:
The procedure and acceptance criteria outlined in the "Guidance for Industry: Bioanalytical Method Validation, 2001" document from FDA was followed to establish the analytical measurement range (AMR) (reportable range) for these immunoassays run on the TPS. Theranos did not follow the validation plan for immunoassays (ELISAs) as outlined in Master Validation Plan for ELISA Assays on Theranos Devices.
The laboratory's response does not include a copy of the FDA protocol, and does not state in the submitted master validation protocol, CL PLN-14002, Rev. A, that the FDA protocol should be used to determine AMR.
The laboratory submitted a protocol for a Master Validation Plan at Ex. E, Tab 30B which requires the %CV of replicates to be not more than 20% (25% at the lower and upper limits of detection). The submitted protocol was different than the protocol presented to the surveyor at the time of the onsite survey which required the %CV of replicates to be not more than 15% (20% at the lower and upper limits of detection). The %CV in the protocol presented to the surveyor at the time of the onsite survey matched the %CV of the predicate devices. Based on the protocol submitted in the submission, it is unclear as to why the TPS, which is covered by the Master Validation Plan, can now have %CV's for replicates greater than the predicate devices. No explanation for the change in %CV was submitted. In addition, we noted that the information submitted in Ex. E, Tab 1 did not include a 20-day precision study as required by the submitted protocol.
The laboratory's submitted protocol requires that 120 samples be used to determine Reference Intervals (see Ex. E, Tab 30B), Ex. E, Tab 1 states:
However, the validation plan noted that the reference range would be established with 120 samples per partition. However, the study followed CLSI procedure for transferring a reference range—the process by which one adapts a previously established reference range to a new analytical method. Furthermore, the transferred reference range was verified following CLSI guidelines—namely "the process by which one ensures, with reasonable confidence, using a relative small number of reference individuals (eg, n=20), that a reference interval established elsewhere, or transferred from another study, can be used locally.
We question the laboratory's procedure for "transferring a reference range" since the laboratory provided no reference document to support "transferring a reference range." We note that the submitted protocol does not state in Section 18, Reference Range, that the laboratory can "transfer" a reference range. It is unclear as to how the laboratory determined that the predicate method's reference range could be adapted or "transferred" to the Theranos Proprietary Devices, which used different methodology than the predicate method.
The laboratory's submission does not indicate that the % Recovery and Allowable Bias citations had been addressed.
Ex. E contains a list of patient specimen accession numbers for which the laboratory "identified reports", but did not specify, or submit documentation to show that corrected reports were generated and issued.
Because the laboratory has not shown that it can follow its own validation protocols, it is unclear whether the laboratory's quality assessment mechanisms could monitor the laboratory's corrective actions and ensure this deficient practice does not recur.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
D6124
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
Ex. A, Tab 16, § 7.3 of the submission contains a competency assessment protocol, Annual Competency. However, the protocol does not address the cited deficiency that direct observation was required for instrument maintenance and function checks. The protocol at § 1.1.7 (page 12) allows for a choice of validation method of which direct observation is one choice. The submitted laboratory protocol also requires the use of CL FRM 03016-F3 for annual competency. This form also allows the assessor to choose the validation method for competency assessment. Neither the submitted protocol nor the form to document competency assessment indicate that the six required procedures for CLIA competency assessment are required.
Ex. L, Tab 4 of the submission states: "All testing personnel currently performing tests have completed competency testing with direct observation of, among other things, maintenance and function checks for the tests they are performing." We are unable to determine if the documentation is complete as there is no information submitted about which testing personnel were performing testing on which instrument(s).
Because the laboratory has not shown that it has corrected this deficient practice, we are also unable to determine whether the laboratory's quality assessment mechanisms can monitor the laboratory's corrective actions and ensure this deficient practice does not recur.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
D6168
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
See our review of 06170 and 06171.
D6170
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
Ex. J, Tab 46 contains lists of patient specimen accession numbers for which the laboratory intended to issue corrected test reports. The laboratory provided no documentation to indicate corrected reports were generated and issued. The submission further states:
- "Based on comprehensive review, multiple examples of failed QC [quality control] without the appropriate documentation with no investigation or corrective action was identified which is now addressed with revised SOPS and training."
- " . . .the review identified multiple examples where technical service provided PM and that service was not documented in addition to failed QC run as part of the service which is now addressed with revised SOPs and training."
The laboratory provided no documentation of investigation or corrective action for these QC failures and lack of maintenance documentation.
Ex. A, Tab 10, § 5.10 states: "Unlicensed laboratory personnel are responsible for performing the activities listed below, under direct and constant supervision by licensed test personnel, with strict adherence to regulatory requirements. . ."
One of the activities listed at§ 5.10.2 is "operation of. . .moderately complex testing instruments. . ."
CMS confirmed with the California Department of Public Health, Laboratory Field Services that unlicensed laboratory personnel cannot operate moderately complex testing instruments in California. The laboratory relied on its written policies and procedures to ensure that appropriately licensed testing personnel were performing and reporting patient test results. However, if the laboratory's policies and procedures are inconsistent with state requirements, we question whether this mechanism would prevent the deficient practice from recurring, and question the laboratory's document control system to ensure accurately written policies and procedures.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
D6171
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
Ex. L, Tab 3 of the submission states: "TP14 has been retrained to ensure that TP14 only performs activities within the scope of TP14's job description as a clinical laboratory associate, under the supervision of testing personnel. Because TP14 is not testing personnel, the high complexity personnel educational requirements do not apply." The CLIA job responsibilities in the job description provided at Ex. L, Tab 3 does include CLIA regulatory responsibilities for testing personnel, and as such TP14 was required to meet the educational requirements. The laboratory's submission does not provide any additional documentation that establishes that TP14 was qualified to perform high complexity testing. TP14 remains unqualified to perform high complexity testing.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
D6178
The laboratory's allegation of compliance is not credible and evidence of correction is not acceptable.
Ex. F contains lists of patient specimen accession numbers for which the laboratory intended to issue corrected test reports. The laboratory provided no documentation to indicate corrected reports were generated and issued.
To ensure the deficient practice does not recur, the laboratory indicated that quarterly audits will be performed and suggested that the audits results would be reviewed within the laboratory's QMPI Program. However, the laboratory did not establish the procedure by which these quarterly audits are to be conducted. In its submission, the laboratory indicates that a "tracer audit may [emphasis added] be used," but did not provided a protocol for a "tracer audit," the means by which a "tracer audit" would be documented, and whether the results of a "tracer audit" would be the information reviewed by the QMPI Program.
The laboratory failed to adequately address this deficiency and provide acceptable evidence of correction consisting of the required documentation and information set forth above and in our January 25, 2016 letter.
Proposed Sanctions
Accordingly, pursuant to 42 C.F.R. §§ 493.1806, 493.1814, and 493.1840(a)(3), based on the finding of immediate jeopardy and the laboratory's failure to meet all CLIA Condition-level requirements, and based on the failure by the owners and director of the laboratory to comply with certificate requirements and performance standards as evidenced by the deficiencies cited during the CLIA recertification and complaint survey completed on December 23, 2015, CMS is proposing the following sanctions against the CLIA certificate of Theranos, Inc.:
- 42 U.S.C. § 263a(i), and 42 C.F.R. §§ 493.1806, 493.1840(a)(3), and 493.1840(e)—Principal Sanction: Revocation of the laboratory's CLIA certificate effective 60 calendar days from the notice of imposition. If imposed the laboratory has 60 calendar days to appeal the determination to revoke the laboratory's CLIA certificate. If a timely hearing request is received, revocation of the laboratory's CLIA certificate will become effective following the administrative hearing decision, if our determination of non-compliance is upheld.
- 42 C.F.R. §§ 493.1806, 493.1812, 493.1840(a)(3), and 493.1840(d)(2)(i)—Principal Sanction: Limitation of the laboratory's CLIA certificate for the specialty of hematology effective eight calendar days from the notice of imposition based on the finding of immediate jeopardy. The limitation will take effect regardless of whether a hearing request is filed and will remain in effect until the laboratory's CLIA certificate is revoked.
- 42 C.R.R. §§ 493.1806(c)(3), 493.1810(c)(2)(ii), 493.1810(d), and 493.1834—Alternative Sanction: Civil Money Penalty (CMP) in the amount of $10,000 per day for each day of non-compliance effective five calendar days from the notice of imposition. If the laboratory requests a hearing, the CMP will not be collected until after the hearing decision is rendered. However, the $10,000/day will begin to accrue five (5) days from the notice of imposition and will continue to accrue until it can be verified that all the cited deficiencies have been corrected and the laboratory is in compliance with all Condition-level requirements or the laboratory's CLIA certificate is limited.
In determining the amount of the penalty, CMS has taken into account the following factors: (1) the laboratory was found to be out of compliance with five CLIA Condition-level requirements as well as numerous Standard-level CLIA requirements during the survey completed on December 23, 2015; (2) the deficiencies cited during the survey, specifically related to the Condition-level requirement set forth 42 C.F.R. § 493.1215, Hematology, were so serious as to result in the determination of immediate jeopardy to patient health and safety; (3) the laboratory failed to remove the jeopardy after being provided an opportunity to do so; (4) the laboratory failed to come into Condition-level compliance after being provided ample opportunity to do so; (5) the laboratory failed to meet all hematology requirements specified in 42 C.F.R. §§ 493.1230 through 493.1256, § 493.1269, and §§ 493.1281 through 493.1299, as is required by 42 C.F.R. 493.1215 for a laboratory providing services in the specialty of hematology; (6) the laboratory failed to meet all analytic system requirements specified in 42 C.F.R. §§ 493.1251 through 493.1283; (7) the laboratory failed to meet all requirements for a laboratory director of a laboratory performing high complexity testing, specified in 42 C.F.R. §§ 493.1443 and 493.1445; (8) the laboratory failed to meet all requirements for a technical supervisor overseeing high complexity testing specified in 42 C.F.R. §§ 493.1449 and 493.1451; (9) the laboratory failed to meet all requirements for testing personnel performing high complexity testing specified in 42 C.F.R. §§ 493.1489 and 493.1495; and (10) the laboratory has expressed no rational reasons for its failure to achieve compliance with all applicable Condition-level CLIA requirements.
- 42 C.F.R. §§ 493.1806(c)(1), 493.1832(b)(2), 493.1844(d)(1), and 493.1844(g)(1)—Alternative Sanction: Directed Portion of a Plan of Correction effective five calendar days from the notice of imposition. The laboratory will be directed to submit to this office within ten calendar days from the date of the notice of imposition of sanctions a list of the names and addresses of all physicians and other clients who have used some or all of the laboratory's services from January 2014 to the present date. This list may be used to advise the laboratory's clients of the nature of its non-compliance and the nature and effective date of any sanctions imposed against the laboratory. The effective date of this sanction will not be delayed due to the filing of a hearing request.
- 42 C.F.R. §§ 493.1804(b)(1)(ii), 493.1804(b)(2), 493.1807(b), 493.1808(b), 493.1826, 493.1844(d)(1), and 493.1844(h)(2)—Medicare Principal Sanction: Suspension of the laboratory's approval to receive Medicare payments for any services performed for the specialty of hematology on or after eight calendar days from the notice of imposition. As a consequence of the suspension of the approval to receive Medicare for services performed for the specialty of hematology, under Section 1902(a)(9)(C) of the Social Security Act and 42 C.F.R. § 440.30(c), payment under the Medicaid program, Title XIX of the Social Security Act, will no longer be available to the laboratory for all laboratory services performed for the specialty of hematology effective eight calendar days from the notice of imposition.
- 42 C.F.R. §§ 493.1807(a), 493.1808(a), 493.1842, and 493.1844(d)(3)—Principal Sanction: Cancellation of the laboratory's approval to receive Medicare payments for all laboratory services effective 60 calendar days from the notice of imposition. This sanction will be effectuated even if the laboratory files a timely appeal. Moreover, in accordance with Section 1902(a)(9)(C) of the Social Security Act and 42 C.F.R. § 440.30(c) and 493.1809, payment under the Medicaid program, Title XIX of the Social Security Act, will no longer be available to the laboratory for all laboratory services effective 60 calendar days from the notice of imposition. See 42 C.F.R. § 440.2(b).
The laboratory is advised that the above sanctions cannot be avoided by the closure of the laboratory, discontinuation of testing, voluntary withdrawal from the CLIA program, or changes in certificate to a lower level of testing.
When the laboratory's CLIA certificate is revoked, the laboratory will not be permitted to perform any testing, including waived testing and provider performed microscopy procedures, regardless of whether or not the laboratory charges for the testing.[2] When the laboratory's CLIA certificate is limited for the specialty of hematology, the laboratory will not be permitted to perform any hematology testing, including waived testing and provider performed microscopy procedures, regardless of whether or not the laboratory charges for the testing[3]. Also, upon revocation of a laboratory's CLIA certificate 42 U.S.C. § 263a(i)(3) and 42 C.F.R. § 493.1840(a)(8) prohibit the owners or operator(s) (including the laboratory director—see 42 C.F.R. § 493.2) from owning or operating (or directing) a laboratory for at least two years from the date of the revocation. This prohibition applies to the owner, operator, and laboratory director at the time the deficiencies which led to the proposal of sanctions were identified by CMS.
When the sanctions become effective as referenced above, in accordance with 42 C.F.R. § 493.1850(a)(2), information regarding the actions against the laboratory's CLIA certificate will appear in the Laboratory Registry for the calendar year in which the actions are imposed. In addition, pursuant to 42 C.F.R. § 493.1844(g)(1), CMS will notify the general public by means of a notice published in a local newspaper when these actions become effective as referenced above.
Instructions for Sending in Your Response
The laboratory has ten calendar days from the date of this notice, or until March 28, 2016 to submit in writing any evidence or information as to why the sanctions detailed above should not be imposed. If a response is not made, is untimely, or does not successfully rebut the bases for the proposed sanctions, we will notify the laboratory in writing that we will proceed to impose the above-referenced sanctions. We will provide information regarding the laboratory's hearing rights and a description of the appeals process at that time.
All responses, including written evidence or information as to why the proposed sanctions should not be imposed, as well as any future correspondence pertaining to this sanction action should be sent to:
Karen Fuller, Manager
State Oversight and CLIA Branch
Division of Survey and Certification
Centers for Medicare & Medicaid Services
90 7th Street, Suite 5-300 (5W)
San Francisco, CA 94103-6707
A copy of any response the laboratory makes to CMS' San Francisco Regional Office must also be sent to CMS' Central Office at the following address:
Division of Laboratory Services
Survey and Certification Group (SCG)
Center for Clinical Standards and Quality (CCSQ)
Centers for Medicare & Medicaid Services
7500 Security Blvd—Mail Stop C2-21-16
Baltimore, MD 21244
Attention: Sarah Bennett
If you have any questions regarding this notice, please call Gary Yamamoto of my staff at (415) 744-3738.
Sincerely,
Karen Fuller, Manager
State Oversight and CLIA Branch
Division of Survey and Certification
cc: California Department of Public Health, Laboratory Field Services
Sunil Dhawan, M.D.
East Bay Dermatology Medical Group
2557 Mowry Avenue, Suite 25
Fremont, CA 94538
- ↑ Theranos, Inc.'s February 12, 2016 submission states: "The laboratory directors during the period covered by the survey no longer hold any position with the lab." However, because Dr. Dhawan was the laboratory director at the time of the CLIA recertification and complaint survey concluded on December 23, 2015, we will continue to hold Dr. Dhawan responsible for all CLIA deficiencies cited. We will continue to address all notices related to the December 23, 2015 CLIA survey to Dr. Dhawan and Theranos, Inc. (We note that the February 12 submission indicates that Kingshuk Das, M.D., had since been appointed the laboratory's director.)
- ↑ The laboratory may continue to perform parallel testing on patient specimens if needed to implement corrective actions. However, the laboratory may not report any patient test results during the period when its CLIA certificate is revoked.
- ↑ The laboratory may continue to perform parallel testing on patient hematology specimens if needed to implement corrective actions. However, the laboratory may not report any patient hematology test results during the period when its CLIA certificate is limited for the specialty of hematology.
This work is in the public domain in the United States because it is a work of the United States federal government (see 17 U.S.C. 105).
Public domainPublic domainfalsefalse