What you don’t measure you don’t change! MS Services need to measured. #PolicticalSpeak #MSBlog #MSResearch
“Yesterday someone made the comment that metrics and league tables are a waste of time. We should simply get on with our day job. If this was true how come the government spends so much resource on the ONS (Office for National Statistics)? Most of us work in a bubble and without league tables and metrics how would we know if what we were doing was good, mediocre or possibly bad? The basic rule of economics is that people, markets, systems, etc. respond to sticks (disincentives) and carrots (incentives). Most people are aspirational, ambitious and competitive; being at the bottom of a league table focuses the mind. This is why I am so in favour of measurement and metrics; don’t underestimate the power of data to change things. If you can’t measure it you can’t change it.”
“As an example; the following is a graph from the EMSP’s MS Barometer of how we do in the EU in relation to DMT penetration in MS. Do you think this an appropriate position for us in the UK to be in? At least we don’t come last. I must remind you that when you rank the countries on GDP per capita we come 10th (see league table below); so this should not be about money.”
“Yesterday someone made the comment that metrics and league tables are a waste of time. We should simply get on with our day job. If this was true how come the government spends so much resource on the ONS (Office for National Statistics)? Most of us work in a bubble and without league tables and metrics how would we know if what we were doing was good, mediocre or possibly bad? The basic rule of economics is that people, markets, systems, etc. respond to sticks (disincentives) and carrots (incentives). Most people are aspirational, ambitious and competitive; being at the bottom of a league table focuses the mind. This is why I am so in favour of measurement and metrics; don’t underestimate the power of data to change things. If you can’t measure it you can’t change it.”
“As an example; the following is a graph from the EMSP’s MS Barometer of how we do in the EU in relation to DMT penetration in MS. Do you think this an appropriate position for us in the UK to be in? At least we don’t come last. I must remind you that when you rank the countries on GDP per capita we come 10th (see league table below); so this should not be about money.”
| ember states | GDP 2014 millions of euro |
Population in millions[4] |
GDP (PPP) per capita 2014 euro |
GDP (Nominal) per capita 2014 euro |
GDP (PPP) per capita 2014 EU28 = 100 |
Eurozone yes/no |
|---|---|---|---|---|---|---|
 |
13,920,541 | 507.4 | 27,300 | 27,300 | 100% | |
 |
2,903,790 | 80.8 | 33,800 | 35,200 | 124% | yes |
 |
2,222,361 | 64.3 | 29,600 | 34,400 | 108% | no |
 |
2,132,449 | 65.9 | 29,300 | 32,400 | 107% | yes |
 |
1,616,048 | 60.8 | 26,500 | 26,600 | 97% | yes |
 |
1,058,469 | 46.5 | 25,500 | 22,800 | 93% | yes |
 |
653,476 | 16.8 | 35,600 | 38,700 | 130% | yes |
 |
430,258 | 9.6 | 34,100 | 44,300 | 124% | no |
 |
413,133 | 38.5 | 18,600 | 10,700 | 68% | no |
 |
402,027 | 11.2 | 32,500 | 36,100 | 119% | yes |
 |
328,886 | 8.5 | 34,900 | 38,500 | 128% | yes |
 |
256,938 | 5.6 | 33,900 | 45,500 | 124% | no |
 |
204,015 | 5.5 | 30,000 | 37,400 | 110% | yes |
 |
185,412 | 4.6 | 36,100 | 40,200 | 132% | yes |
 |
179,081 | 11.0 | 19,600 | 16,300 | 72% | yes |
 |
174,384 | 10.4 | 21,400 | 16,800 | 78% | yes |
 |
154,939 | 10.5 | 23,000 | 14,700 | 84% | no |
 |
150,665 | 19.9 | 14,600 | 7,600 | 54% | no |
 |
103,303 | 9.9 | 18,500 | 10,500 | 68% | no |
 |
75,215 | 5.4 | 20,800 | 13,900 | 76% | yes |
 |
43,085 | 4.2 | 16,100 | 10,200 | 59% | no |
 |
45,288(2013) | 0.5 | 68,500(2013) | 83,100(2013) | 263% | yes |
 |
42,011 | 7.2 | 12,300 | 5,800 | 45% | no |
 |
37,246 | 2.1 | 22,600 | 18,100 | 83% | yes |
 |
36,288 | 2.9 | 20,100 | 12,400 | 74% | yes |
 |
24,058 | 2.0 | 17,600 | 12,100 | 64% | yes |
 |
19,526 | 1.3 | 19,900 | 14,800 | 73% | yes |
 |
17,506 | 0.9 | 23,400 | 20,500 | 85% | yes |
 |
7,962 | 0.4 | 23,100 | 18,600 | 85% | yes |
Source: Wikipedia
“I hope this justifies why I have include the penetration of DMTs and the proportion of MSers who are on DMTs that are on second-line and/or highly-active DMTs as part of the quality metrics we should be collecting. May be you still disagree?”
“The following is the EMSP’s MS barometer and summary sheets. I suggest you read them. What is interesting is how well, or not so well the UK does in collecting data about MS. This is why I am trying to make a call to action about the NICE MS Quality Statements, they don’t address the holistic management of MS and they set the bar so low that the are unlikely to make a massive difference to the majority of MSers living in the UK. We need the quality statements to go a lot further. May be statement number 1 should be about accurate data collection.”
I stick by my guns. This is a disease where people can become severely disabled and die well before the normal lifespan. I would rather you spend your time in the lab on the Proximus or Inspire trials rather than pouring over tables and charts produced by economists and statisticians. The numbers you should focus on are (I) number of neuroprotective agents available for use (2) number of repair agents available for use (3) percentage of MSers who live to the average lifespan… Big data has gone viral, but MSers quality of lives don't seem to be improving much eg. PPMS patients. Treatments not tables (data) should be the mantra.
But without data like this how do you get funders and interest for research and trials? Personally I find metrics useful, I don't put them above other things but I can see why they are needed to push things forward.
There are lies then there are statistics. Which ever way you look at it the UK does not seem to be doing very well. Recently I met some one with PPMS and I was shocked at the lack of symptom relief drugs she was receiving. OK part of the problem was that she was in denial but part of the problem was her lack of knowledge that there are things out there to make life easier and better. Give you a better QoL.If medical professionals and charities get the message out there then people become more knowledgeable and are in a position to ask/demand what is available.The problem does not lie with the boffins in the background but with those who meet and help MSers on a regular basis such as MS Nurses, local branches of the MS Society etc.
One of the issues the NHS suffers from is that the best medics get promoted and then spend more and more of their time doing management stuff including churning out stats and completing returns etc. Doctors should spend their time with patients. We've created an industry of standards, targets, reporting, auditing… The patient becomes a stat. All the patient wants is to get better. Endless graphics, data analysis, standards don't make me feel better.
Metrics are absolutely essential – otherwise how do we know whether a teatment is of benefit? If it is of benefit how do we know whether peopl who it would benefit are actually recieving the treatment? Epedemiological studies have exposed the connection with the global distribution of MS which has suggested causal factors – you never know where a link to a possible therapy could come from so inmy view keeping an open mind and researching on all fornts is essential.
If you don't really know where you want to get to then any road might take you there, but you may not end up at the destination you thought you would arrive at.No metrics = no mileposts on the journey – regardless of destination. It just becomes aimless wandering in the wilderness.
I wonder if there is a league table for MS research centres. There are quite a few in the UK – Cambridge, Bristol, London (various), Edinburgh… Which one is Chelsea and which one is QPR?
To Village People Go west singWe are Leeds and were proud of it!We are Leeds and were proud of it!We are Leeds and were proud of it!We are Leeds and were proud of it!:-)
Hopefully showing up at the MS Frontiers would be one the metrics. Supporting the hand that feeds you would be a nice place to start. Go East; I am West Ham supporter! Any Millwall or Crystal Palace supporters that I can kiss (Glaswegian style).
I think UCL are ahead in the league tables; number 2 in the world to Harvard.
I'd say Cambridge are top 4 Premier League. As Mouse alludes to above, Barts are Leeds United – mid-table Championship club. Prof G could be Premier League, but plays too many international fixtures.
This discussion misses the point of this post completely. It is not about us, the researchers, but about getting MSers better services and access to DMTs. In my opinion, the evidence is now overwhelming that access to early DMTs with effective and active management makes a huge difference to outcomes. Can we please focus on the task in hand? The UK is almost at the bottom of the DMT league when it comes to access to DMTs, this is the issue.
If you interested in academic league tables they exist already, we are judged on a national league table every 6 years. The process was called the Research Assessment Exercise (RAE) and more recently the Research Excellence Framework (REF). The problem is the government does not make the individual ranking of individual academics available. If you want you can use Google Scholar and print out our H-indexes and average them out. Some would say that the H-index doesn't capture our impact very well hence the need for alternative metrics, hence the surge in popularity of altmetrics, and similar indices. When it comes to making the lives of MSers better the H-index and Altmetric index are really worth zero.
Prof G, why is the UK so unwilling to prescribe new DMTs? Do they know something you don't?
Prof G,With respect, you are missing the point. If the UK is low down on the league table for accessing DMTs, surely neuros are to blame. The patient will be referred to a neuro for a diagnosis and then the neuro should be recommending a DMT and prescribing it. Are health authorities refusing to fund these DMTs? I had no trouble getting access to a DMT at my hospital trust. Not sure what the league table is telling us – neuros not prescribing, trusts not funding, or patients declining! UK always low down on many league tables – productivity, education… And high up on many – teenage pregnancies, obesity… Yet everyone wants to move here.
Prof G, I know there is lots of trial data showing highly effective dmts are reducing long term disability in trials but are there any epidemiology studies confirming this? I guess life expectancy improving is one but that might be down to better health care rather than slowing of disease progression? I remember reading your slides when I was first diagnosed and the graphs plateauing eds scores with highly effective dmts but this was all predictions based on a few years of trial data I'm presuming. if there was solid evidence I'm sure there would be much more impetus/money available to get people on dmts quick?
Re: "Prof G, I know there is lots of trial data showing highly effective dmts are reducing long term disability in trials but are there any epidemiology studies confirming this?"NICE approvals are based on 20 year timelines. The long-term data on natalizumab and alemtuzumab is pretty good. When you combine that with what is happening to brain atrophy rates in years 3 & 4 it is really a no-brainer. Progressive brain atrophy is really looking like it does what is say on the tin; if you are losing brain you are going to do badly. Finally, everyone seems to ignore the 21-year effect of IFN-beta on survival; just imagine scaling that up to the higher efficacy products? What people don't realise that to get to EDSS 10 you need to go through EDSS 4,5,6,7,8, 9 and 10 unless you die of another cause (suicide, MVA, etc.).
"Prof G, why is the UK so unwilling to prescribe new DMTs? Do they know something you don't?"One thing they do know is that they're too expensive. Coupled with ingrained treatment conservatism.
I find this quite shocking. I see Anon at 5:48 has listed some of the factors which could impact the league table, but I'm wondering whether the NICE guidelines for previous years have put people off accepting DMTs. The CRAB drugs all come with a long list of potential side effects and the major disadvantage of very regular injections, plus they aren't very effective. It's not very surprising that people may be less keen to start if that's all they were offered. Discounting Alemtuzumab for the minute (as from my experience, with a 'moderate' amount of lesions, neurologists haven't recommended it as first-line), it may be that now Tecfidera is on the market, the UK percentage of people on DMTs may increase. But if all options were available first-line, this could increase further. The thought of trading monthly injections (if I were to be JC-) for a highly effective DMT would be a no-brainer.
I think you may be right Jim. Also, the countries we are below in the DMT tables, don't provide the healthcare Britain does. They pay quite a lot for their medication, dental care etc. I think side effects play a huge part in this discussion. A Cancer patient will accept Chemotherapy, because the alternative is fatal, an MS patient has a more difficult decision, unless the DMT is guaranteed to stop relapses and progression. It's human nature.
'Progressive brain atrophy is really looking like it does what is say on the tin; if you are losing brain you are going to do badly. " The ASCEND trial is looking at reduction in brain atrophy therefore you believe Natalizumab is efficacious in SPMS?