Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Inconsistent approaches of the G-BA regarding acceptance of primary study endpoints as being relevant to patients - an analysis of three disease areas: oncological, metabolic, and infectious diseases

BMC Health Services ResearchBMC series – open, inclusive and trusted201616:651

https://doi.org/10.1186/s12913-016-1902-8

Received: 18 October 2016

Accepted: 4 November 2016

Published: 14 November 2016

Abstract

Background

Previous evaluations of oncological medicines in the German early benefit assessment (EBA) procedure have demonstrated inconsistent acceptance of endpoints by regulatory authorities and the Federal Joint Committee (G-BA). Accepted standard endpoints for regulatory purposes are frequently not considered as patient-relevant in the German EBA system.

In this study the acceptance of clinically acknowledged primary endpoints (PEPs) from regulatory trials in EBAs conducted by the G-BA was evaluated across three therapeutic areas.

Methods

Medicines for oncological, metabolic and infectious diseases with EBAs finalised before 25 January 2016 were evaluated. Respective manufacturer’s dossiers, regulatory assessments, G-BA appraisals and oral hearing minutes were reviewed, and PEPs were examined to determine whether they were considered relevant to patients by the G-BA. Furthermore, the acceptance of symptomatic vs asymptomatic PEPs was also analysed.

Results

A total of 65 EBAs were evaluated. Mortality PEPs were widely accepted as patient-relevant but were only used in a minority of EBAs and exclusively in oncological diseases. Morbidity PEPs constituted around 72 % of assessed PEPs, but were excluded from the EBA in over half of the corresponding assessments as they were not considered patient-relevant. Symptomatic endpoints were largely deemed patient-relevant, whereas acceptance of asymptomatic endpoints varied between therapeutic areas.

Conclusions

This evaluation identified inconsistencies in patient relevance of morbidity-related PEPs as well as in acceptance of asymptomatic endpoints by the G-BA in all three disease areas examined. Better harmonisation between the regulatory authorities and the G-BA is still required after 5 years of AMNOG health technology assessment in Germany.

Keywords

Early benefit assessment HTA Marketing authorisation Primary endpoint AMNOG Morbidity

Background

Since 2011, all new medicines in Germany are required to undergo an early benefit assessment (EBA) in comparison to a pre-specified appropriate comparator [1]. At market entry, the pharmaceutical manufacturer (PM) needs to submit an EBA dossier evaluating the additional benefit of the drug, based on available clinical trial data, to the Federal Joint Committee (Gemeinsamer Bundesausschuss, G-BA).

The extent of additional benefit is classified by the G-BA as major (1), considerable (2), minor (3), non-quantifiable (4), no additional benefit (5), and less benefit (6) [2]. The benefit rating serves as the main basis for subsequent price negotiations for reimbursement between the PM and statutory health insurance providers.

For the determination of additional benefit the G-BA requests efficacy data in three different categories: mortality, morbidity and health-related quality of life (HRQoL) [2, 3]. However, the G-BA only takes into account endpoints deemed to be patient-relevant. Importantly, the opinion of the G-BA on patient relevance of endpoints frequently diverges from those of regulatory authorities [48].

A previous analysis of endpoints in oncology indicated acceptance by the G-BA of endpoints related to mortality, while morbidity endpoints were largely disregarded [4]. The current analysis extends the scope beyond oncology to evaluate if the identified trends also apply to other disease areas. Benefit assessments in the three most frequently assessed therapeutic areas (with ≥10 finalised and evaluable assessments) were analysed: oncological, metabolic, and infectious diseases. For all primary endpoints (PEPs), patient relevance was evaluated and used to determine whether these PEPs would be taken into consideration in an EBA by the G-BA.

Methods

Analysis set: EBA dossiers submitted by the PMs, the corresponding G-BA appraisals, and oral hearing minutes, all obtained from the G-BA website, were used as sources of data [9]. Dossiers were not included in the analysis if the G-BA’s view on patient relevance of endpoints was not discernible. This could be due to i) no dossier being submitted by the PM, ii) inability to evaluate acceptance of PEP by the G-BA (due to use of inappropriate comparator, inadequate indirect comparison, or dossiers/studies being incomplete), or iii) if a more recent assessment for the same medicine and indication was available.

Analysis of patient relevance of primary endpoints in the benefit assessment by the G-BA: The resulting dataset was used to analyse whether PEPs used in the clinical studies and reported in the PM dossiers went on to be accepted as patient-relevant in the G-BA appraisals. PEPs were recorded as:
  • patient-relevant when this was explicitly mentioned in the G-BA appraisal, or the corresponding data were used to justify an additional benefit,

  • partially patient-relevant when the PEP was a composite or co-primary endpoint with both patient-relevant and non-patient-relevant components, or

  • not relevant to patients when the G-BA clearly questioned the validity of the endpoint or the appraisal did not mention the PEP data or contained no clear statement of its inclusion.

Following determination of the G-BA’s view on patient relevance of each PEP, they were categorised as either symptomatic or asymptomatic. To ensure this categorisation of the endpoints was unbiased and meaningful, we utilised multiple sources of information integral to the EBA process to inform this procedure. Specifically, the categorisations were based on definitions and descriptions of the endpoints and the respective discussions within the dossiers, the assessments by the Institute for Quality and Efficiency in Healthcare (Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen, IQWiG), and the G-BA appraisals. The oral hearing at the G-BA aims to provide clarification on critical and controversial topics. Key representatives of the relevant medical and scientific societies are always attending those hearings. Thus, the oral hearing minutes served as a source of expert opinion which we referred to for categorisation of certain endpoints along with supporting scientific statements.

The medicines were divided into therapeutic areas, as categorised by the G-BA, and only therapeutic areas with ≥10 finalised and evaluable EBAs as of 25 January 2016 were considered, which led to the inclusion of oncological, metabolic and infectious diseases in the final analysis set.

Results

Analysis set

Inclusion of therapeutic areas with ≥10 finalised and evaluable EBAs (oncological, metabolic, and infectious disease areas) resulted in 97 EBAs which were considered for evaluation. Of these, 65 (67 %) were evaluable according to the inclusion criteria. Table 1 summarises the number of EBAs included in the analysis and the main reasons for exclusion of EBAs from the analysis set by therapeutic area.
Table 1

Inclusion vs exclusion of EBAs by therapeutic area

Therapeutic area

EBAs (n)

Assessments in analysis set (n)

Main reason for EBAs not being evaluablea

Included

Excluded

No dossier submitted

Data on PEP acceptance by G-BA not evaluable

Newer assessment available (same medicine and indication)

Oncological diseases

47

36

11

2

6

3

Metabolic diseases

34

15

19

3

14

2

Infectious diseases

16

14

2

0

2

0

Total

97

65

32

5

22

5

aEBAs were excluded from the analysis when no dossier was submitted by the PM, a newer assessment was available (same medicine and indication) or the G-BA’s viewpoint on the patient relevance of the PEP could not be discerned (due to incomplete dossiers or studies, use of inappropriate comparators, or inadequate indirect comparisons)

EBA Early benefit assessment, G-BA Federal Joint Committee, n Number of EBAs, PEP Primary endpoint, PM Pharmaceutical manufacturer

A high proportion of EBAs in metabolic diseases (19 of 34, 56 %) were not evaluable, mainly because the G-BA objected to the methodological execution of the studies for reasons including use of inappropriate comparator or indirect comparison, and incomplete dossiers or studies. A list of EBAs and the respective PEPs evaluated in the analysis is presented in Table 2.
Table 2

EBAs included in the analysis set and respective primary endpoints

Medicine

Indication

Primary endpoint (PEP)

Therapeutic expert panel present at oral hearing

Oncological diseases

 Abiraterone acetate

Prostate carcinoma

OS

BDU, DGHO, DVPZ

 Abiraterone acetatea

Prostate carcinoma

OS, rPFS

DGHO

 Afatinibc

Non-small cell lung cancer

PFS

DGHO, DKG (Working Group on Thoracic Oncology of the AIO)

 Aflibercept

Metastatic colorectal cancer

OS

DGVS

 Axitinib

Renal cell carcinoma

PFS

DGHO

 Bosutinib

Chronic myeloid leukaemia

MCR

DGHO

 Brentuximab vedotin

Hodgkin-Lymphoma, anaplastic large cell lymphoma

ORR

DGHO

 Cabazitaxel

Prostate carcinoma

OS

DGHO, DKG

 Cabozantinib

Thyroid gland neoplasia

PFS

DGE (thyroid gland section), DGHO, DGN

 Crizotinib

Non-small cell lung cancer

PFS, ORR

DGHO, POA, representatives from leading oncology centres

 Decitabine

Myeloid leukaemia

OS

DGHO

 Enzalutamide

Prostate carcinoma

OS

DGHO, DGU

 Enzalutamidea

Prostate carcinoma

OS, rPFS

DGHO

 Eribulin

Breast cancer

OS

DGHO

 Eribulinac

Breast cancer

OS, rPFS

DGHO

 Ibrutinib

Chronic lymphocytic leukaemia, relapsed or refractory mantle cell lymphoma (MCL)

ORR

DGHO

 Idelalisib

Chronic lymphocytic leukaemia, follicular lymphoma

PFS

DGHO, GLSG, representative from the University Hospital Gießen

 Ipilimumab

Melanoma

OS

DGHO

 Lenvatinib

Thyroid gland neoplasia

PFS

DGHO

 Nintedanib

Non-small cell lung cancer

PFS

DGHO, DKG (Working Group on Thoracic Oncology of the AIO), representative from the LungenClinic Grosshansdorf

 Nivolumab

Melanoma

OS, PFS

ADO, DGHO

 Obinutuzumab

Chronic lymphocytic leukaemia

PFS

DGHO

 Olaparib

Ovarian cancer

PFS

DGHO

 Pertuzumab

Breast cancer

PFS, ORR

AGO, DGHO

 Pomalidomide

Multiple myeloma

PFS

DGHO, representatives from the University Hospitals of Heidelberg, Tübingen and Würzburg

 Radium-223-dichloride

Prostate carcinoma

OS

DGHO

 Ramucirumab

Stomach cancer

OS

DGHO, DGVS

 Regorafenib

Colorectal cancer

OS

DGHO

 Ruxolitinibd

Chronic myeloproliferative diseases

≥35 % reduction in spleen volume

DGHO

 Ruxolitiniba

Polycythaemia vera

Haematocrit control without phlebotomy and ≥35 % reduction in spleen volume

DGHO

 Siltuximab

Multicentric Castleman’s disease

Durable tumour & symptomatic response (complete and partial response)

DGHO

 Sipuleucel-T

Prostate carcinoma

OS

DGHO, representative from the University Hospital Tübingen, Department of Urology

 Trastuzumab emtansine

Breast cancer

OS, PFS

AGO, DGHO

 Vandetanibb

Thyroid gland neoplasia

PFS

DGHO

 Vemurafenibc

Melanoma

OS, PFS

ADO, DGHO

 Vismodegib

Basal cell cancer

ORR

DGHO, DGMKG, German Society of Dermatology

Metabolic diseases

 Albiglutide

Diabetes mellitus type 2

HbA1c

DDG, Diabetes Research Group HZM, representative from the University Hospital Carl Gustav Carus (Dresden)

 Dulaglutide

Diabetes mellitus type 2

HbA1c

BVND, DDG, Diabetes Research Group HZM

 Eliglustat

Gaucher disease type 1

Stable health status (decrease in spleen and liver volume, Hb, thrombocytes) and %-change in spleen volume

ASIM, DGVS, representative from the Charité University Medicine Berlin

 Elosulfase alfa

Mucopolysaccharidose type IVA (Morquio A syndrome)

6MWT

ZSE Wiesbaden

 Insulin degludeca

Diabetes mellitus type 1

HbA1c

DDG

 Ivacaftor

Cystic fibrosis

FEV1%

-

 Ivacaftora

Cystic fibrosis

FEV1%

-

 Linagliptinb

Diabetes mellitus type 2

HbA1c

BVND, DDG, Diabetes Research Group HZM

 Pasireotide

Pituitary gland dysfunction

mUFC ≤ ULN

-

 Pasireotidea

Acromegaly

Biochemical control

-

 Saxagliptin

Diabetes mellitus type 2

HbA1c

DDG, Working Group on Pharmacoepidemiology, Diabetes Research Group HZM, Diabetes Centre Bad Lauterberg

 Saxagliptin/metformin

Diabetes mellitus type 2

HbA1c

BVND

 Sitagliptin

Diabetes mellitus type 2

HbA1c

DDG, Working Group on Pharmacoepidemiology, Diabetes Research Group HZM, Diabetes Centre Bad Lauterberg

 Sitagliptin/metformin

Diabetes mellitus type 2

HbA1c

DDG, Diabetes Centre Bad Lauterberg, Diabetes Research Group HZM, Working Group on Pharmacoepidemiology

 Vildagliptinb

Diabetes mellitus type 2

HbA1c

BVND, DDG

Infectious diseases

 Boceprevir

Chronic hepatitis C

SVR

bng, DGVS, German Liver Foundation

 Daclatasvir

Chronic hepatitis C

SVR

bng, dagnä, DGIM, DGVS

 Dasabuvir

Chronic hepatitis C

SVR

bng, dagnä, DGIM, DGVS

 Dolutegravir

HIV infection

VR

dagnä, DAIG

 Dolutegravir/abacavir/lamivudine

HIV infection

VR

-

 Elvitegravir/cobicistat/emtricitabin/tenofovir-disoproxil

HIV infection

VR

dagnä, DAIG

 Emtricitabine/rilpivirine/tenofovirdisoproxil

HIV infection

VR

dagnä, DAIG

 Fidaxomicin

Clostridium infection

Overall cure

DGHO, DGVS

 Ledipasvir/sofosbuvir

Chronic hepatitis C

SVR

bng, dagnä, DGIM, DGVS

 Ombitasvir/paritaprevir/ritonavir

Chronic hepatitis C

SVR

bng, dagnä, DGIM, DGVS

 Rilpivirine

HIV infection

VR

DAIG, dagnä

 Simeprevir

Chronic hepatitis C

SVR

bng, dagnä, DGI, DGIM, DGVS

 Sofosbuvir

Chronic hepatitis C

SVR

bng, dagnä, DGIM

 Telaprevir

Hepatitis C

SVR

bng, DGVS, German Liver Foundation

aNew therapeutic indication, bRe-assessment for the same indication, cRe-assessment after expiration of G-BA appraisal, dRuxolitinib is an orphan drug, but has undergone a regular EBA process after having reached sales of >€50 million per year

6MWT 6-minute walking test, ADO Working Group on Dermatological Oncology, AGO Working Group on Gynaecologic Oncology, AIO Working Group on Internal Oncology, ASIM Working Group on Congenital Metabolic Disorders in Internal Medicine, BDU Professional Association of German Urologists, bng Federal Association of Registered Gastroenterologists, BVND Federal Association of Registered Diabetologists, dagnä German Working Group of Registered Doctors in the Care of HIV-infected Persons, DAIG German AIDS Society, DDG German Diabetes Society, DGE German Society of Endocrinology, DGHO German Society of Hematology and Medical Oncology, DGI German Society of Infectious Diseases, DGIM German Society of Internal Medicine, DGMKG German Society of Oral and Maxillofacial Surgery, DGN German Society of Nuclear Medicine, DGU German Society of Urology, DGVS German Society of Gastroenterology, Digestive and Metabolic Diseases, DKG German Cancer Society, DVPZ Umbrella Organisation of Prostate Centres in Germany, EBA Early benefit assessment, FEV1 Forced expiratory volume in 1 second, GLSG German Low Grade Lymphoma Study Group, Hb Haemoglobin, HbA1c Glycated haemoglobin, HIV Human immunodeficiency virus, HZM Helmholtz Centre Munich, POA Working Group on Pulmonary Oncology, MCR Major cytogenic response, mUFC Median urinary free cortisol, ORR Objective response rate, OS Overall survival, PFS Progression-free survival, rPFS Radiographic progression-free survival, SVR Sustained viral response, ULN Upper limit of normal, VR Viral response, ZSE Centre for Rare Diseases

Analysis of patient relevance of primary endpoints in the benefit assessment by G-BA

All therapeutic areas

This analysis set consisted of 65 PEPs obtained from 65 EBA dossiers. The G-BA deemed the PEPs as relevant to patients in 31 (48 %) of 65 EBAs, not relevant to patients in 25 (38 %), and partially patient-relevant in nine (14 %) assessments (refer to sections on individual therapeutic areas for more details on specific PEPs).

Analysis by endpoint category showed that the majority of PEPs were morbidity endpoints (47 of 65, 72 %). Mortality PEPs were used in 11 of 65 PM submissions (17 %), all of which concerned oncological diseases and were deemed patient-relevant by the G-BA (Fig. 1a). Morbidity PEPs were considered patient-relevant in 20 of 47 (43 %) EBAs and partially patient-relevant in two cases (4 %). In seven EBAs the PEP was a composite endpoint of mortality and morbidity and in each case deemed partially patient-relevant. Six of these cases were in oncological diseases and one was in the metabolic disease area (see also sections on individual therapeutic areas). No assessment included HRQoL as a PEP.
Fig. 1

Patient relevance of PEPs according to the G-BA by endpoint dimension. a All three therapeutic areas, b oncological diseases, c metabolic diseases, and d infectious diseases. HRQoL was not used as PEP for any assessment, therefore it is not shown in the graphs

Oncological diseases

In oncological diseases 36 of 47 PM submissions were analysed (Table 1). Mortality PEPs were reported in 11 of the 36 EBAs (all overall survival [OS]) and regarded as patient-relevant in the benefit assessment by the G-BA in all cases (Fig. 1b). Fifteen of 19 (79 %) morbidity PEPs (mostly progression-free survival [PFS]) were deemed not patient-relevant. The three patient-relevant morbidity PEPs were major cytogenic response (MCR) (bosutinib, chronic myeloid leukaemia), reduction of spleen volume by >35 % (ruxolitinib, chronic myeloproliferative diseases), and haematocrit control without phlebotomy and reduction of spleen volume by >35 % (ruxolitinib, polycythaemia vera). For one composite morbidity endpoint, namely partial and complete durable tumour and symptomatic response, only complete response was considered patient-relevant and included in the G-BA appraisal (siltuximab, multicentric Castleman’s disease). Six assessments had co-primary endpoints, comprising mortality and morbidity (all OS and PFS). In all these cases only OS was considered in the appraisal.

Metabolic diseases

In metabolic diseases, 34 PM submissions were available, 15 of which were analysed (Table 1). All PEPs were morbidity endpoints, apart from one composite mortality/morbidity PEP (Fig. 1c) and three morbidity PEPs were deemed patient-relevant in the benefit assessments. One of these PEPs was glycated haemoglobin (HbA1c) in type 1 diabetes (insulin degludec), which was explicitly specified by the G-BA as a validated and patient-relevant endpoint. The others were 6-minute walk test (6MWT) in type IVA mucopolysaccharidose (elosulfase alfa), and median urinary free cortisol level (mUFC) in pituitary gland dysfunction (pasireotide).

In the ten EBAs where PEPs were not considered patient-relevant by the G-BA, PEPs were HbA1c and FEV1 (forced expiratory volume in 1 second) in type 2 diabetes and cystic fibrosis, respectively. In eight benefit assessments (albiglutide, dulaglutide, linagliptin [re-assessment], saxagliptin, saxagliptin/metformin, sitagliptin, sitagliptin/metformin, vildagliptin [re-assessment]) the PEP HbA1c was not explicitly mentioned as patient-relevant and not considered in the G-BA benefit evaluation, and therefore assumed to be not relevant to patients according to the evaluation criteria of this analysis stated above. In the two assessments of ivacaftor, FEV1 data was not taken into consideration by the G-BA as there were different opinions within the G-BA regarding its patient relevance.

In two EBAs the PEP was only partially accepted as patient-relevant by the G-BA. For eliglustat (Gaucher disease type 1) the PEP was a co-primary endpoint, consisting of two endpoints. The first was the composite morbidity endpoint stable health status (spleen volume, liver volume, haemoglobin and thrombocyte count) and the second percentage change in spleen volume, where only the change in spleen volume was deemed patient-relevant. For pasireotid (acromegaly) the PEP was biochemical control (defined as a combination of reduction of the mean growth hormone [GH] level below 2.5 μg/l and normalisation of the age- and sex-adjusted insulin-like growth factor 1 [IGF-1] level after 24 weeks). The PM defined biochemical control as a composite mortality/morbidity endpoint. However the G-BA did not accept the mortality component due to lack of validation.

Infectious diseases

In infectious diseases 14 of 16 PM submissions were included in the analysis set (Table 1). There were no mortality PEPs in infectious diseases. All 14 PEPs concerned morbidity (Fig. 1d), and all were accepted as patient-relevant by the G-BA. In most cases (13 of 14) the PEP was viral response (sustained viral response [SVR] in hepatitis C or viral response [VR] in HIV-infected patients), and in one case, the PEP was overall cure (fidaxomicin, clostridium infection).

Acceptance of symptomatic vs asymptomatic PEPs

PEPs were categorised as symptomatic or asymptomatic according to the definition of the PEP and based on the appraisal documents as well as scientific discussions during the hearing process. A list of therapeutic area expert panels attending the oral hearings is shown in Table 2. Table 3 summarises each evaluated PEP and the results of the categorisation by therapeutic area (see Additional file 1: Table S1 for a table listing the rationale of categorisation for each endpoint).
Table 3

G-BA acceptance of symptomatic vs asymptomatic morbidity PEPs as (A) patient-relevant and (B) non-patient-relevant

 

Oncological diseases

Metabolic diseases

Infectious diseases

(A) Morbidity PEPs accepted as patient-relevant by the G-BA

 Symptomatic

Complete durable tumour & symptomatic responsea

6MWT

Overall cure

≥35 % reduction in spleen volumea

Reduction in spleen volumea

-

 Asymptomatic

Haematocrit control without phlebotomya

HbA1c (Type 1 diabetes)

Viral response (VR, SVR)

MCR

mUFC

-

-

Biochemical control (mean GH <2.5 μg/L and normalisation of IGF-1)

-

(B) Morbidity PEPs not accepted as patient-relevant by the G-BA

 Symptomatic

Partial durable tumour & symptomatic responsea

-

-

 Asymptomatic

PFSa

HbA1c (Type 2 diabetes)

-

ORR

Haemoglobin levela

-

-

Thrombocyte counta

-

-

Reduction in liver volumea

-

-

FEV1

-

aThis PEP was a component of a co-primary endpoint in at least one dossier

6MWT 6-minute walking test, FEV1 Forced expiratory volume in 1 second, G-BA Federal Joint Committee, GH Growth hormone, HbA1c Glycated haemoglobin, IGF-1 Insulin-like growth factor 1, MCR Major cytogenic response, mUFC Median urinary free cortisol, ORR Objective response rate, OS Overall survival, PEP Primary endpoint, PFS Progression-free survival, SVR Sustained viral response, VR Viral response

Asymptomatic PEPs mostly comprised laboratory parameters (e.g. HbA1c in diabetes) or endpoints involving imaging outcomes (e.g. PFS or objective response rate [ORR]) [9]. Symptomatic endpoints included outcomes directly experienced by the patient, for example OS in oncological conditions and 6MWT in mucopolysaccharidose [9]. For other PEPs (complete/partial durable tumour and symptomatic response, overall cure), symptomaticity was self-explanatory by their definitions [9]. Regarding the symptomaticity of the PEPs related to reduction in spleen or liver volume, an independent Gaucher disease type 1 expert (a member of the German Working Group for Congenital Metabolic Disorders in Internal Medicine [Arbeitsgemeinschaft für angeborene Stoffwechselstörungen in der Inneren Medizin, ASIM]) stated in the minutes of the oral hearing procedure for eliglustat that the spleen volume in patients prior to treatment is typically increased 15-fold to ca. 2 L [9]. Since the measured reduction with eliglustat treatment was around 30 % and therefore easily palpable we assigned the reduction in spleen volume as a symptomatic endpoint. A similar reasoning applies to the reduction of ≥35 % of spleen volume in the EBA of ruxolitinib in the indications chronic myeloproliferative diseases and polycythaemia vera. In contrast, a closer analysis of the data for Gaucher disease showed that the relative enlargement and reduction in liver volume before and after eliglustat treatment were too small to be palpable and thus deemed not symptomatic.

Subsequently, we compared the consideration of symptomatic vs asymptomatic PEPs by the G-BA in evaluating additional benefit. For three PEPs the oral hearing minutes were consulted in order to categorise them as symptomatic or asymptomatic, as described above. Symptomatic endpoints were mostly regarded as patient-relevant (Table 3 A), apart from partial durable tumour and symptomatic response (siltuximab, multicentric Castleman’s disease) (Table 3 B). Asymptomatic endpoints (mainly laboratory parameters and endpoints assessed by imaging techniques) were largely deemed not relevant to patients. This had considerable impact on benefit assessment of oncology medications, since PFS and ORR are widely used PEPs which are generally accepted as patient-relevant by the European Medicines Agency (EMA), but were not deemed patient-relevant by the G-BA (Table 3 B). In metabolic diseases HbA1c was only considered patient-relevant in type 1 diabetes. In contrast, asymptomatic viral response endpoints (VR and SVR) in infectious diseases (mainly hepatitis C and HIV infection) were readily accepted as patient-relevant by the G-BA.

Discussion

This analysis demonstrates that specific morbidity PEPs from the EBA are categorically excluded by the G-BA across major disease areas, irrespective of indication and disease stage. Typically, these are PEPs used in pivotal studies for marketing authorisation. The respective clinical studies were specifically designed to show clinical benefit based on these endpoints. The aim of the EBA is to assess the additional benefit of a new medicine compared to the prespecified comparative treatment. This generally differs from the aim of regulatory authorities, which is to evaluate the benefit vs risk profile of a new medicine. Nevertheless, the methods and standards of evidence-based medicine apply to both evaluation procedures. Furthermore, primary endpoints of pivotal studies which led to EMA approval of the investigated drug should be accepted as patient-relevant, irrespective of whether ‘benefit’ or ‘additional benefit’ is under consideration. To suggest that these primary study endpoints are not relevant to patients, as the G-BA does, creates a major dilemma in clinical development and implicitly questions the ethical conduct of studies.

Our analysis by outcome dimension showed a strong dominance of morbidity over mortality PEPs (47 vs 11). While the mortality PEPs were accepted as patient-relevant without exception, less than half of the morbidity PEPs were deemed patient-relevant. Interestingly, the acceptance varied by therapeutic area, revealing a disadvantage for oncological and metabolic disease indications, where morbidity PEPs were predominantly regarded as not patient-relevant. Considering that the majority of these PEPs were accepted by regulatory authorities for marketing authorisation, the classification of such a large proportion of PEPs as not relevant to patients by the G-BA highlights marked differences in data interpretation by regulatory and health technology assessment (HTA) bodies. A similar finding was also demonstrated in a comparative analysis of parallel scientific advice of different European HTA bodies and the EMA [10].

The PEPs that were most frequently dismissed as not patient-relevant were PFS in oncological diseases and HbA1c in metabolic diseases (except for type 1 diabetes). Despite opposing views on patient relevance of PFS within the G-BA (e.g. afatinib, axitinib, crizotinib, eribuline) [9] data on this endpoint were systematically excluded from benefit assessments, based on the justification that PFS evaluation utilises asymptomatic endpoint assessment techniques such as imaging. Conversely, in infectious diseases, the PEP SVR was accepted as patient-relevant in hepatitis C despite being an asymptomatic surrogate endpoint lacking formal validation. PEPs in neurological diseases were also evaluated but ultimately excluded from the analysis set as only six assessments were evaluable. Here, morbidity PEPs were largely accepted as patient-relevant by the G-BA (5 of 6 EBAs).

The results of our analysis clearly demonstrate inconsistency in the G-BA’s approach to judging the patient relevance of PEPs between disease areas. For example, in infectious diseases, the G-BA has shown some flexibility in accepting the asymptomatic endpoint SVR as patient-relevant. On the other hand, in oncological and metabolic diseases some asymptomatic endpoints (PFS in cancers and HbA1c in type 2 diabetes) are categorically dismissed, without taking into consideration the different disease profiles of the indications. This is in contrast to the EMA, which adopts a broader approach when evaluating patient relevance and takes indication, difficulty in obtaining mortality data, and priority of accelerating patient access into account.

We welcome the flexibility the G-BA has shown regarding the acceptance of SVR as patient-relevant in hepatitis C, but identify a need to expand this flexibility to other endpoints. The general dismissal of PFS is questionable since in some cancers, such as ovarian cancer, it has been suggested that improved PFS is associated with clinical benefit and is a valid surrogate for extended OS [11]. Nevertheless the G-BA did not accept PFS as patient-relevant in the EBA of olaparib for the treatment of ovarian cancer [9].

The success of novel anticancer drugs in recent years has led to a classification shift of various oncological conditions from acutely fatal to chronic disease. Similar to chronic diseases such as diabetes, it is challenging to obtain mortality data showing significant differences in those oncological diseases with slow progression and low death rates. For these reasons, PFS has gained importance as a PEP in clinical trials because it is an early detectable and meaningful endpoint for disease progression [12, 13].

Similarly to PFS, HbA1c is routinely used as a surrogate endpoint in diabetes mellitus and widely accepted by regulatory authorities [14]. For example the EMA states in its “Note for guidance on clinical investigation of medicinal products in the treatment of diabetes mellitus” that HbA1c is the most widely accepted measure of overall, long-term blood glucose control in type 1 and type 2 diabetes, and therefore an appropriate primary study endpoint [15]. Moreover, this guidance states that reduction of HbA1c is directly related to a reduced risk of developing vascular complications. The G-BA, however, only accepts HbA1c as a patient-relevant endpoint in type 1 diabetes, but not in type 2.

For the patient relevance of morbidity endpoints, clarification of its definition and determination criteria by the G-BA are urgently needed, particularly in oncological diseases and type 2 diabetes, as currently no clear criteria for patient relevance are listed in the G-BA rules of procedure [2]. Representatives from patient advocacy groups and other external experts are consulted at several stages of the assessment procedure, potentially also concerning the patient relevance of endpoints, but their influence on the benefit appraisal is not transparent and has been criticised as insufficient [16, 17]. In a recent press release from the German Society for Hematology and Medical Oncology (Deutsche Gesellschaft für Hämatologie und Medizinische Onkologie, DGHO) a DGHO board member criticised the fact that market authorisation, EBA, and treatment guidelines often come to different conclusions, despite being based on the same clinical data, making it difficult for doctors to make treatment decisions [18].

Alignment of study requirements between the G-BA and regulatory authorities is necessary in order to optimise trial design and reduce patient sample sizes, particularly in rare indications, in order to allow timely access to new treatments. A recent publication has identified considerable heterogeneity in regulatory and HTA approaches, even among different European HTA bodies [10]. Regulatory authorities have adapted their approval pathways for innovative and promising new medicines to facilitate early patient access to new treatments, for example via conditional approval or adaptive pathways [10]. In order to effectively establish early access to medicines, HTA procedures need to follow the EMA’s footsteps and provide harmonised, transparent, flexible, conditional, and adaptive methods that adopt the level of evidence accepted by medicines agencies [10].

The UK National Institute for Health and Clinical Excellence (NICE) uses a less restrictive approach in their HTA procedure, where appropriate methods may be used to extrapolate data from less than ideal study designs regarding study type, study duration, patient population, choice of comparator, and type of outcomes [19]. In contrast to the approach adopted by the G-BA, where data that do not fully comply with requirements are largely rejected, NICE is more receptive to data derived from clinical trials which are for instance non-randomised, non-comparative or employ modelling.

Our analysis has revealed that asymptomatic endpoints were frequently disregarded by the G-BA, while symptomatic endpoints were largely deemed patient-relevant. However, acceptance of asymptomatic endpoints varied by therapeutic area. A major achievement in modern medicine is the ability to capture aspects of morbidity prior to their transformation in symptomatic disease in diagnostic and prognostic examinations. For example, in the assessment of axitinib for renal cell carcinoma, PFS was used to assess the delay in onset of metastases by imaging [9]. It is evident that early detection of metastases before the onset of symptomatic pain or even vertebral fractures is essential and of utmost relevance to the patient. In this case, the strict insistence on symptomatic endpoints may lead to a delay in complementary treatment options, such as bisphosphonates, which could prevent occurrence of vertebral fractures [20]. This would contradict the underlying ethical principles of diagnosis and therapy in this and many other oncological diseases.

Currently, the G-BA demands patient-relevant outcome data for the three endpoint categories mortality, morbidity and HRQoL, and weighs each with equal importance for all assessments. However, a number of reasons may lead to attenuation of mortality data, for instance, i) clinical advancement of available therapies, ii) change in disease status from ‘untreatable’ to ‘chronic’, and iii) differences in the course of diseases themselves. Given the differences in mortality and progression between different diseases, for example some forms of cancer vs non-fatal chronic diseases, equal weighting of endpoint categories is not informative. Instead, differential weighting of each endpoint category in the EBA, taking into account disease type and severity/stage, as outlined in Fig. 2 would be advisable.
Fig. 2

Adaptable and disease-specific weighting of endpoint categories

Limitation

For each EBA evaluated, only the respective PEP was included, as this is the parameter which provides the most clinically relevant and convincing evidence, based on the statistical principles of clinical trials [21], and clinical trials are explicitly designed to show efficacy in terms of the PEP. A more comprehensive approach would also include secondary endpoints. Nevertheless, our approach sufficiently highlights the clear inconsistencies that exist between endpoint acceptance by regulatory authorities and the G-BA.

Conclusions

There is a need for the G-BA to define the morbidity endpoints that are considered patient-relevant, particularly in indications such as diabetes and oncological diseases. This will allow PMs to target clinical trial designs towards requirements of the G-BA at an early clinical development stage to ensure an efficient pipeline for bringing innovative and efficacious treatments to patients. In addition, further harmonisation between regulatory bodies and G-BA with regards to acceptance of PEPs as relevant to patients is urgently required. In particular, this applies to morbidity-related PEPs that are not (yet) symptomatic and tangible to patients.

Abbreviations

6MWT: 

6-minute walking test

ADO: 

Working Group of Dermatological Oncology

AGO: 

Working Group of Gynaecological Oncology

AIO: 

Working Group of Internal Oncology

AMNOG: 

The Act on the Reform of the Market for Medicinal Products

ASIM: 

Working Group for Congenital Metabolic Disorders in Internal Medicine

BDU: 

Professional Association of German Urologists

bng: 

Federal Association of Registered Gastroenterologists

BVND: 

Federal Association of Registered Diabetologists

dagnä: 

German Working Group of Registered Doctors in the Care of HIV-infected Persons

DAIG: 

German AIDS Society

DDG: 

German Diabetes Society

DGE: 

German Society of Endocrinology

DGHO: 

German Society of Hematology and Medical Oncology

DGI: 

German Society of Infectiology

DGIM: 

German Society of Internal Medicine

DGMKG: 

German Society of Oral and Maxillofacial Surgery

DGN: 

German Society of Nuclear Medicine

DGU: 

German Society of Urology

DGVS: 

German Society of Gastoenterology, Digestive and Metabolic Diseases

DKG: 

German Cancer Society

DVPZ: 

Umbrella Organisation of Prostate Centres Germany

EBA: 

Early benefit assessment

EMA: 

European Medicines Agency

FEV1: 

Forced expiratory volume in 1 second

G-BA: 

Federal Joint Committee

GH: 

Growth hormone

GLSG: 

German Low Grade Lymphoma Study Group

Hb: 

Haemoglobin

HbA1c: 

Glycated haemoglobin

HIV: 

Human immunodeficiency virus

HRQoL: 

Health related quality of life

HTA: 

Health technology assessment

HZM: 

Helmholtz Centre Munich

IGF-1: 

Insulin-like growth factor 1

IQWiG: 

Institute for Quality and Efficiency in Healthcare

MCR: 

Major cytogenic response

mUFC: 

Median urinary free cortisol

NICE: 

National Institute for Health and Clinical Excellence

ORR: 

Objective response rate

OS: 

Overall survival

PEP: 

Primary endpoint

PFS: 

Progression-free survival

PM: 

Pharmaceutical manufacturer

POA: 

Working Group on Pulmonary Oncology

rPFS: 

Radiographic progression-free survival

SVR: 

Sustained viral response

ULN: 

Upper limit of normal

VR: 

Viral response

ZSE: 

Centre for Rare Diseases

Declarations

Acknowledgements

Manuscript preparation was supported by medical writing services from nspm ltd, Meggen, Switzerland, with financial support from Roche Pharma AG.

Funding

Not applicable.

Availability of data and material

Datasets were derived from the website of the G-BA (https://www.g-ba.de/informationen/nutzenbewertung/), a public domain resource. All data analysed during the current study are included in this published article and its supplementary information file.

Authors’ contributions

JR and TS designed the research and analysed the data. TS, GI, VA and JR participated in the interpretation of the data and the writing of the manuscript. TS, GI, VA and JR read and approved the final manuscript.

Competing interests

TS, GI and JR are employees of Roche Pharma AG, Grenzach-Wyhlen, Germany. VA has no conflicts of interest.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Not applicable.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Roche Pharma AG
(2)
Medical School of Hanover

References

  1. Bundesiministerium für Gesundheit. [Gesetz zur Neuordnung des Arzneimittelmarktes - AMNOG]. 2011. http://www.bundesgesundheitsministerium.de/service/begriffe-von-a-z/a/arzneimittelmarktneuordnungsgesetz-amnog.html. Accessed 15 Sep 2016.
  2. G-BA. [Verfahrensordnung des Gemeinsamen Bundesausschusses]. 2016. https://www.g-ba.de/downloads/62-492-1243/VerfO_2016-03-17_iK-2016-08-23.pdf. Accessed 15 Sep 2016.
  3. Deutscher Bundestag. [Sozialgesetzbuch (SGB) Fünftes Buch (V) - Gesetzliche Krankenversicherung - (Artikel 1 des Gesetzes v. 20. Dezember 1988, BGBl. I S. 2477, letzte Änderung durch Artikel 3 des Gesetzes vom 20. Dezember 2012 (BGBI. I S. 2781))]. 1988. http://www.gesetze-im-internet.de/bundesrecht/sgb_5/gesamt.pdf. Accessed 15 Sep 2016.
  4. Ruof J, Knoerzer D, Duenne AA, Dintsios CM, Staab T, Schwartz FW. Analysis of endpoints used in marketing authorisations versus value assessments of oncology medicines in Germany. Health Policy. 2014;118(2):242–54.View ArticlePubMedGoogle Scholar
  5. Ruof J, Fluckiger O, Andre N. Early benefit assessments in oncology in Germany: how can a clinically relevant endpoint not be relevant to patients? Drugs R D. 2015;15(3):221–6.View ArticlePubMedPubMed CentralGoogle Scholar
  6. Dabisch I, Dethling J, Dintsios CM, Drechsler M, Kalanovic D, Kaskel P, et al. Patient relevant endpoints in oncology: current issues in the context of early benefit assessment in Germany. Health Econ Rev. 2014;4(1):2–9.View ArticlePubMedPubMed CentralGoogle Scholar
  7. Kvitkina T, ten Haaf A, Reken S, McGauran N, Wieseler B. Patient-relevant outcomes and surrogates in the early benefit assessment of drugs: first experiences [Patientenrelevante Endpunkte und Surrogate in der frühen Nutzenbewertung von Arzneimitteln: erste Erfahrungen]. Z Evid Fortbild Qual Gesundhwesen. 2014;108(8-9):528–38.Google Scholar
  8. Hörn H, Nink K, McGauran N, Wieseler B. Early benefit assessment of new drugs in Germany - results from 2011 to 2012. Health Policy. 2014;116(2-3):147–53.View ArticlePubMedGoogle Scholar
  9. G-BA. Overview of products [Übersicht der Wirkstoffe]. 2016. http://www.g-ba.de/informationen/nutzenbewertung/. Accessed 15 Sep 2016.
  10. Tafuri G, Pagnini M, Moseley J, Massari M, Petavy F, Behring A, et al. How aligned are the perspectives of EU regulators and HTA bodies? A comparative analysis of regulatory-HTA parallel scientific advice. Br J Clin Pharmacol. 2016;82(4):965–73.View ArticlePubMedPubMed CentralGoogle Scholar
  11. Lebwohl D, Kay A, Berg W, Baladi JF, Zheng J. Progression-free survival: gaining on overall survival as a gold standard and accelerating drug development. Cancer J. 2009;15(5):386–94.View ArticlePubMedGoogle Scholar
  12. Sargent DJ, Hayes DF. Assessing the measure of a new drug: is survival the only thing that matters? J Clin Oncol. 2008;26(12):1922–3.View ArticlePubMedGoogle Scholar
  13. Axelrod C. The significance of progression-free survival as a clinical end point in oncology. Drug Benefit Trends. 2010;22:39–45.Google Scholar
  14. European Medicines Agency (EMA). Guideline on clinical investigation of medicinal products in the treatment or prevention of diabetes mellitus (CPMP/EWP/1080/00 Rev. 1). 2012. http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2012/06/WC500129256.pdf. Accessed 15 Sep 2016.
  15. Committee for proprietary medicinal products (CPMP). Note for guidance on clinical investigation of medicinal products in the treatment of diabetes mellitus (Report No.: CPMP/EWP/1080/00). 2002. http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003262.pdf. Accessed 15 Sep 2016.
  16. Bleß H-H, de Millas C, Kern B, Seidlitz C, Talaschus A. Auswirkungen von Stellungnahmen der Wissenschaft in der Nutzenbewertung von Arzneimitteln. 2016. https://www.mynewsdesk.com/material/document/55081/download. Accessed 15 Sep 2016.
  17. Deutsche Gesellschaft für Hämatologie und medizinische Onkologie (DGHO). Nutzenbewertung von Arzneimitteln der Onkologie und Hämatologie - eine Standortbestimmung. 2013. http://www.dgho.de/informationen/gesundheitspolitische-schriftenreihe/dgho_gpsr_arzneimittelnutzenbewertung.pdf. Accessed 15 Sep 2016.
  18. Deutsche Gesellschaft für Hämatologie und medizinische Onkologie (DGHO). DGHO erweitert Informationsplattform Onkopedia um den Bereich Arzneimittel Bewertung. 2016. https://www.dgho.de/informationen/presse/pressemitteilungen/dgho-erweitert-informationsplattform-onkopedia-um-den-bereich-arzneimittel-bewertung. Accessed 15 Sep 2016.
  19. Ivandic V. Requirements for benefit assessment in Germany and England - overview and comparison. Health Econ Rev. 2014;4(1):12.View ArticlePubMedPubMed CentralGoogle Scholar
  20. Novartis. Zometa Summary of Product Characteristics. 2016. http://www.ema.europa.eu/docs/en_GB/document_library/EPAR_-_Product_Information/human/000336/WC500051730.pdf. Accessed 15 Sep 2016.
  21. ICH Expert Working Group. ICH Harmonised tripartite guideline - Statistical principles for clinical trials - E9. 1998. http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E9/Step4/E9_Guideline.pdf. Accessed 15 Sep 2016.

Copyright

© The Author(s). 2016

Advertisement