Tag Archives: flawed research

ECT And Adolescents At The Mayo Clinic

BACKGROUND

At the APA annual convention in San Francisco in May of this year, Chad Puffer, DO, of the Mayo Clinic, presented a poster display titled ECT Use in Adolescents at the Mayo Clinic.  The poster was reported by Caroline Cassels of Medscape.com, a month later (ECT in Kids:  Safe, Effective, Robust and …Underutilized), and drew critical comment a week later (July 2) from Kelly Patricia O’Meara of CCHR International.

I didn’t spot any of this until last week, when it came up on the Twitter feed from DxRevision Watch and Peter Kinderman.

The research in question does not appear to have been written up in any journal – or if it has, I haven’t been able to find it.  All I’ve got to work with at this point is the APA ‘s abstract, which is reproduced below in full, and the Mayo Clinic’s poster, which was sent to me by Ms. Cassels.

APA Abstract NR7-34
ELECTROCONVULSIVE THERAPY USE IN ADOLESCENTS
AT MAYO CLINIC: A 20-YEAR PRACTICE
AND OUTCOMES REVIEW
Lead Author: Chad Puffer, D.O.
Co-Author(s): Christopher Wall, MD
Mark Frye, MD
SUMMARY:
Background: Electroconvulsive therapy (ECT ) remains a useful, yet infrequently employed treatment option in youth experiencing severe emotional illnesses. At Mayo Clinic, approximately 50 adolescents have been treated with ECT for a range of psychiatric illnesses over a 20-year span. Methods: This study reports a comprehensive practice and outcomes review of adolescents treated at Mayo Clinic with ECT. Treatment parameters including electrode localization, stimulus dosing, seizure duration and associated treatment complications are reported. Long-term follow-up clinical information regarding post-treatment outcomes as adults are also reported.
Conclusion: ECT use is a viable and appropriate treatment approach in youth experiencing severe, clinically debilitating illnesses that have been recalcitrant to other treatment options. Treatment parameters largely mirror the adult ECT practice, with some notable and important exceptions related to tolerability, seizure duration and variability of clinical outcomes.

CRITIQUE OF THE APA ABSTRACT

1.  “Electroconvulsive therapy (ECT) remains a useful, yet infrequently employed treatment option in youth experiencing severe emotional illnesses.”

Firstly, note the word “illness.”  They continue to refer to depression as an illness despite the lack of any evidence to support this position.

Secondly, note the assertion that shock treatment is a “useful” treatment option for youth.

Thirdly, note the term “yet infrequently applied…”  I think the implication here is that if more psychiatrists were aware of how great this treatment is, it would be used more widely

Fourthly, note the term “severe emotional illnesses,” implying, I believe, that the shock “treatment” in the study was used to address only emotional problems.  In fact, in the poster text it states clearly that the electric shocks were used to “treat” affective, psychotic, and  other  disorders (including anorexia nervosa!)

2.  “This study reports a comprehensive practice and outcomes review of adolescents treated at Mayo Clinic with ECT.”

The study in question was a fairly rudimentary and retrospective chart review, and characterizing it as “comprehensive” is quite a stretch.  There were only 46 participants over 20 years.  Twelve-month follow-up data was available for only 29 of these, and there was no control group.

3.  “Long-term follow-up clinical information regarding post-treatment outcomes as adults are also reported.”

The only reference to follow-up into adulthood mentioned on the poster is:

4.  “Adolescents receiving ECT were eventually diagnosed with personality disorders approximately 13% of the time upon reaching adulthood.”

5.  “Conclusion:  ECT use is a viable and appropriate treatment approach in youth experiencing severe, clinically debilitating illnesses that have been recalcitrant to other treatment options.”

In my view, this rather strong conclusion is not warranted by the evidence presented in the Mayo Clinic poster.  (More on this below).

CRITIQUE OF THE POSTER

The poster opens with some general comments as to the effectiveness of ECT in adults and in adolescents, and laments the fact that there is still some reluctance to use this treatment with children “despite its recognized efficacy.”  In support of the latter claim, the authors refer to an earlier (1993) Mayo Clinic record review of 20 young patients aged 18 or younger.  This earlier review was conducted by Schneekloth et al, and found “…ECT to be safe and effective in adolescents with severe and medication-resistant mental illnesses.”  Interestingly, Dr. Puffer’s poster study draws almost identical conclusions:

“Despite controversy on ECT use in adolescents, these data suggest it is a safe, reasonably well-tolerated and effective treatment for the most severely ill adolescents resistant to pharmacotherapy and psychotherapy.”

The poster study records search identified 46 pediatric patients who had received ECT at the Mayo Clinic between 1993 and 2012.

“Each patient’s medical chart was reviewed to verify ECT administration during their care at Mayo Clinic. All available treatment settings, side effects, medications and diagnoses were recorded. Where possible, pre-ECT and post-ECT clinical data were recorded.”

Note the words “where possible,” which suggests that some of the pre and post assessments were missing.  Since these assessments are the essence of the matter, it is critical to know the extent of, and the reason for these omissions.  But the poster gives no other information.

“Most adolescents who received ECT were taking one less medication one year after ECT than they were at initiation of ECT, and the average change in number of medications prescribed pre- and one year post-ECT was -0.74.”

This is interesting, but hardly constitutes a great improvement, especially as one of the stated criteria for ECT was a lack of response to pharmacotherapy.

The most important item under the “results” heading was:

“ECT appeared to be effective in reducing symptoms of affective (major depressive disorder, bipolar affective disorder), psychotic (psychosis NOS, schizophrenia, steroid-induced psychosis), and other disorders (schizoaffective disorder, depression with psychotic features, catatonia, anorexia nervosa) as measured by Clinical Global Impression scales of Symptoms and Improvement upon independent retrospective analysis by a board-certified Child and Adolescent Psychiatrist and a PGY-2 Psychiatry resident.”

The words “appeared to be” seem noteworthy, and imply a good deal more caution than the conclusion given in the APA abstract.

The criterion for improvement is also noteworthy:

“…as measured by CGI scales…”

CGI stands for Clinical Global Impression.  This is a subjective assessment instrument used by psychiatrists to rate severity of a client’s problem immediately after a clinical interview.  It is widely used in clinical practice and in research.  It is subjective, but it does bring an element of uniformity to these kinds of assessments.  The CGI has been in use since 1976.

There are 3 CGI scales:

S – assesses the severity of the “mental illness.”
I – assesses any improvement since last assessment
E – assesses the therapeutic effect of the intervention and any side effects.

Back to the poster.  What Dr. Puffer and his colleagues seem to be saying is that ECT appeared to be effective with these various disorders, as measured by CGI-S and CGI-I scales – and this is where it gets murky – “…upon independent retrospective analysis by…” two qualified psychiatrists.

It’s the word “retrospective” that I’m having trouble with, and the poster does not make it clear what this means.  Under the heading “discussion,” it says:

“Retrospective CGI-S and CGI-I analysis showed significant improvement in ECT-treated adolescents.”

But again, the term retrospective is not clarified.  Two phone calls and an email to Dr. Puffer at the Mayo Clinic asking for clarification have gone unanswered.

It looks like this is what happened.  Between 1993 and 2012, 46 young people (ranging in age from 12 to 19, incidentally) received ECT.  The psychiatrist administering the shocks wrote up his notes in the clinical record in the normal way, but didn’t do a CGI assessment, either pre or post.  Then in 2012, the two expert psychiatrists retrospectively completed the CGI-S and the CGI-I pre and post- ECT based on the notes in the treatment record.  This is a very questionable research procedure, and it is with considerable hesitation that I even suggest it. But I can see no other interpretation.  If the CGI’s had been done at the time of the ECT, reading and transcribing the scores would have been a simple clerical task.  It certainly would not have required two independent psychiatrists.

If my interpretation is correct, then no safe conclusions can be drawn from the study.  A retrospective analysis of a psychiatrist’s record of his own work is fraught with possibilities for error.  This is why we have double-blinded, randomized, controlled trials!

Also of note is the fact that the third CGI scale – the efficacy index – was not used.  In fact, it says under “discussion” that “Systematic measurement of cognitive side-effects of ECT was not available.”  This presumably means:  not available in the record; which presumably means that no assessment of cognitive side effects was done either at the time of the ECT or at the one-year follow-up.  Or, I suppose it could mean that these assessments were done, but were so damaging that they were omitted from the write-up.

The lack of cognitive side effects assessment is particularly disturbing, in that the American Academy of Child and Adolescent Psychiatry in their 2002 Practice Parameter for Use of Electroconvulsive Therapy With Adolescents  state under the heading “cognitive assessment”:

“Every adolescent undergoing ECT must have a memory assessment before treatment, at treatment termination, and at an appropriate time after treatment (usually between 3 and 6 months post-treatment) [MS].”

MS means “minimal standard,” meaning that this is an absolute requirement.  The parameter was approved and published in 2002, so it was in place for the entire second half of the study (2002-2012).  In this light, an admission by the authors that measurements of side effects were not available is, in my opinion at least, a matter of grave concern.

OTHER ISSUES

Each adolescent received, on average, 10.4 shocks.  The range was 4 to 27.

The average seizure duration was 84.4 seconds.  The range is 16 seconds to 272 seconds (4 minutes and 32 seconds).  Sixteen participants had seizures lasting more than three minutes.

GENERAL

The Mayo Clinic is a prestigious medical center with a world-wide reputation for medical excellence.  I am truly surprised that they allowed this document to be displayed under their name and logo.

In my view, this is another flawed study in a long line of flawed studies that purport to demonstrate the efficacy of ECT.  If ECT proponents want to be taken seriously, they need to take on board Johnstone et al’s 1980 Northwick Park electroconvulsive therapy trial finding that sham ECT (where the patient is anesthetized but not shocked) is just as effective as real ECT.  Until then, it’s just more chaff in the wind.