Kaiku Health Real World Test Plan
2023 Performance Year
Justification for Real-World Testing Approach
Kaiku Health is a certified health information system that provides care teams with several tools for digital health interventions including a set of oncology-specific patient portal features that the certification focuses on. Combined with health information capture through patient-reported outcomes (PROs) and secure messaging, patients are provided timely access to view, download, and transmit their records.
This test plan is designed to test the functionality certified by the ONC Health IT Certification program as it is used in the real world. The following criteria are in the scope of this plan:
- § 170.315(e)(1) View, Download and Transmit to 3rd party
Overall Approach
Kaiku Health is a cloud-hosted health information system that is currently sold to Radiation Oncology and Medical Oncology care settings. Cloud hosting enables telemetry connectivity for all sold instances of the health information system. This provides remote event tracking capability which will be used for real world testing of the criterion which is in scope of this plan. This ensures testing results accurately reflect the real-world performance of the health information system.
The tested criterion requires that patients can view, download and transmit their health information. Remote tracking will therefore focus on patients' attempts to view, download, and transmit, their health information using the health information system. Additionally, patients need to be able to view the activity log, showing how their health information was accessed for view, download and transmit. Realworld error rates will be established for each of those functionalities.
Milestones
| Milestone | Care Setting | Timeframe |
|---|---|---|
| Development and testing of remote event and error tracking capabilities completed | Radiation Oncology Medical Oncology | January 15, 2023 |
| Baseline error rate collection period | Radiation Oncology Medical Oncology | January 16 - February 28, 2023 |
| Error rate tracking period | Radiation Oncology Medical Oncology | March 1 - December 31, 2023 |
| Analysis and report creation | Radiation Oncology Medical Oncology | January - February, 2024 |
| Analysis and report creation | Radiation Oncology Medical Oncology | January - February, 2024 |
| Submit Real World Testing report to ACB per instructions | Radiation Oncology Medical Oncology | February 15, 2024 |
Measures used in Overall Approach
Description of Measurement/Metric
| Measurement/Metric | Description |
|---|---|
| Error rate for View | Total number of errors observed in 'View' functionality, divided by total number of attempts for the 'View' functionality, expressed in %. Baseline error rate will be measured during a baseline collection period (see Milestones). Error rate will be tracked continuously after the baseline collection period. |
| Error rate for Download | Total number of errors observed in 'Download' functionality, divided by total number of attempts for the 'Download' functionality, expressed in %. Baseline error rate will be measured during a baseline collection period (see Milestones). Error rate will be tracked continuously after the baseline collection period. |
| Error rate for Transmit (unencrypted email) | Total number of errors observed in 'Transmit (unencrypted email)' functionality, divided by total number of attempts for the 'Transmit (unencrypted email)' functionality, expressed in %. Baseline error rate will be measured during a baseline collection period (see Milestones). Error rate will be tracked continuously after the baseline collection period. |
| Error rate for Transmit (secure) | Total number of errors observed in 'Transmit (secure)' functionality, divided by total number of attempts for the 'Transmit (secure)' functionality, expressed in %. Baseline error rate will be measured during a baseline collection period (see Milestones). Error rate will be tracked continuously after the baseline collection period. |
| Error rate for View Activity Log | Total number of errors observed in 'View Activity Log' functionality, divided by total number of attempts for the 'View Activity Log' functionality, expressed in %. Baseline error rate will be measured during a baseline collection period (see Milestones). Error rate will be tracked continuously after the baseline collection period. |
Associated Certification Criteria
| Measurement/Metric | Associated Certification Criteria | Relied Upon Software |
|---|---|---|
| Error rate for View | 170.315(e)(1): View, Download and Transmit to 3rd Party | MOSAIQ 2.82 (or higher) |
| Error rate for Download | 170.315(e)(1): View, Download and Transmit to 3rd Party | MOSAIQ 2.82 (or higher) |
| Error rate for Transmit (unencrypted email) | 170.315(e)(1): View, Download and Transmit to 3rd Party | MOSAIQ 2.82 (or higher) |
| Error rate for Transmit (secure) | 170.315(e)(1): View, Download and Transmit to 3rd Party | MOSAIQ 2.82 (or higher) |
| Error rate for View Activity Log | 170.315(e)(1): View, Download and Transmit to 3rd Party | MOSAIQ 2.82 (or higher) |
Justification for Selected Measurement/Metric
| Measurement/Metric | Justification |
|---|---|
| Error rate for View | Viewing patients' health information is required in §170.315(e)(1)(i)(A) |
| Error rate for Download | Downloading patients' health information is required in §170.315(e)(1)(i)(B)(1) |
| Error rate for Transmit (unencrypted email) | Transmitting patients' health information via unencrypted email is required in §170.315(e)(1) (i)(C)(1)(i) |
| Error rate for Transmit (secure) | Transmitting patients' health information via a secure channel is required in §170.315(e)(1) (i)(C)(1)(ii) |
| Error rate for View Activity Log | Viewing the Activity Log for access to patients' health information is required in §170.315(e)(1)(ii)(A) |
Care Settings
Elekta markets its products in the Radiation Oncology and Medical Oncology care settings. The measurements which are part of this plan will be tracked on all instances of Kaiku Health sold in the United States. This includes both care settings.
Expected Outcomes
| Measurement/Metric | Expected Outcomes |
|---|---|
| Error rate for View | Baseline error rate is expected to be low. Tracking error rate is expected to be comparable or lower than baseline error rate |
| Error rate for Download | Baseline error rate is expected to be low. Tracking error rate is expected to be comparable or lower than baseline error rate |
| Error rate for Transmit (unencrypted email) | Baseline error rate is expected to be low. Tracking error rate is expected to be comparable or lower than baseline error rate |
| Error rate for Transmit (secure | Baseline error rate is expected to be low. Tracking error rate is expected to be comparable or lower than baseline error rate |
| Error rate for View Activity Log | Baseline error rate is expected to be low. Tracking error rate is expected to be comparable or lower than baseline error rate |
Attestation
This Real World Testing plan is complete with all required elements, including measures that address all certification criteria and care settings. All information in the plan is up to date and fully addresses the health IT developer's Real World Testing requirements.
Authorized Representative Name: Lauri Sippola
Authorized Representative Email: lauri.sippola@elekta.com
Authorized Representative Phone: +358 40 661 7170
Authorized Representative Signature: Lauri Sippola
Date: 10 / 13 / 2022