Submission of comments on 'Reflection paper on establishing efficacy based on single-4 arm trials submitted as pivotal evidence in a marketing 5 authorisation.
General comments:
The reflection paper accurately describes the limitations, potential biases and mitigations associated with single arm trials (SAT). However we believe that the paper doesn’t address some difficult issues, discussed in the comments table below, such as:
- When SATs are acceptable
- Establishing external control arms
- Assessing safety
In addition, pivotal trials, including SATs, must also provide acceptable evidence for downstream health technology assessment (HTA). This is particularly important for rare diseases, where clinical trial populations are limited. It is essential that EMA coordinates with HTA bodies throughout Europe to ensure that advice given to companies regarding the acceptability of SATs is in line with HTA evidence requirements. This will avoid research waste, and clinical trials that do not generate the evidence necessary to bring the medicine all the way into the hands of patients. We suggest that sponsors seeking advice on clinical development plans including a pivotal SAT automatically undergo parallel scientific consultation from EMA and the European Network for Health Technology Assessment (EUnetHTA) 21 consortium. This could be integrated with the existing route of parallel scientific scientific and protocol assistance from EMA and the FDA, to achieve global consensus on clinical trial design for ultra-rare populations.
Specific comments on text:
When is a SAT suitable?
The paper states that the decision to rely on a single arm trial lies with the sponsor, that the sponsor should seek scientific advice, and that decisions are made on a case by case basis.
We believe that EMA must provide more extensive and specific guidance on how to determine when an SAT is necessary and appropriate. This is especially important to limit “creep” or generalized acceptance of single arm trials as providing sufficient evidence in other situations (such as label extensions into younger age groups).
Endpoints
We note that the reflection paper does not mention the use of patient-reported outcome measures (PROMs). We support the inclusion of suitably validated PROMs to reflect changes in how patients feel. Evidence from PROMS is also import for downstream HTA evaluation.
External controls
The reflection paper states that efficacy assessment can be via direct comparison against external clinical data “in exceptional circumstances.” Yet we note that EMA approvals of gene therapy products up the end of 2021 have, in several cases, included comparison against an external control.
We believe that any SAT serving a pivotal trial in a regulatory package must include an appropriate external control arm. This is especially important if the marketing authorization application relies on a single pivotal SAT.
We believe that a reflection paper for SATs is therefore incomplete without clear guidance on how to establish external control arms. We would like to see clear guidance on:
- Historical control arms using real world data from registries (such as the EMA-approved ECFSPR)
- Contemporaneous control data from concurrent clinical trials, and how to handle potential biases
- Historical control data from the control arms of clinical trials conducted by multiple commercial sponsors, and how issues around data protection and commercial sensitivities may be resolved. We believe that all stakeholders, especially patients, would benefit from breaking down data silos and re-utilizing data.
Safety
This reflection paper is focused on the challenges of establishing efficacy from SATs. While the introduction states that many of the challenges and considerations equally apply to the assessment of safety, we believe that efficacy must always be considered in tandem with safety. We suggest that further guidance on SATs should consider both efficacy and safety.