Spurious Hurricane Trends in the Pacific

"Hurricane Patricia October 23, 2015, GOES-15 visible animation" by National Oceanic and Atmospheric Administration (GOES-15 satellite); animation provided via the University of Miami's Rosenstiel School of Marine and Atmospheric Science - http://andrew.rsmas.miami.edu/bmcnoldy/tropics/patricia15/23Oct_VIS.gif. Licensed under Public Domain via Commons.

Modern times are characterized by an immense amount of scientific information.  Journals abound. Many experts maintain a blog. Popular press publish niche magazines for those that just can't get enough from their day job.

This reflects a (sometimes heated) conversation about what to believe about how the world works.  As well, the scientific debate is a process of checks and balances.  Sometimes scientists make mistakes.  Sometimes the best way to analyze data is not clear, obvious, or agreed upon by everyone.  The literature is better understood as a long drawn out discussion.

Often, this is taken a step further from what scientists argue is true to the inherently political realm of why it matters.  Therefore individual scientific publications cannot or perhaps, ought not, be divorced from the broader argument in which they are situated.

Recently, Nature, featured a debate about the predictability of tropical cyclones based on the ENSO index predictor (El Niño Southern Oscillation).  ENSO has two phases: El Niño (warmer than average) and La Niña (cooler than average).  The different phases, especially EL Niño, garner international attention because they play a significant role in climate variability and thus, the occurrence of extreme events such as...tropical cyclones.

In the three part correspondence, scientists debate methodological integrity.  This matters for what findings imply about the predictability of ENSO for TC activity.  This further matters for how society regards activity predictions which are used everywhere from the nightly news to policy advocacy groups to managing financial regimes and establishing insurance contracts.

(Despite its importance in society and policy, Nature is a subscription service.)

First: Research Paper

Jin, F-F., Boucharel, J., and Lin, I-I.  2014. Eastern Pacific tropical cyclones intensified by El Niño delivery of subsurface ocean heat. Nature. 516: 82-85.  (hereafter as JBL14)

The core of the JBL14, publication is as follows (from the abstract):

Here we show that El Niño—the warm phase of an ENSO cycle—effectively discharges heat into the eastern North Pacific basin two to three seasons after its wintertime peak, leading to intensified TCs. This basin is characterized by abundant TC activity and is the second most active TC region in the world. As a result of the time involved in ocean transport, El Niño’s equatorial subsurface ‘heat reservoir’, built up in boreal winter, appears in the eastern North Pacific several months later during peak TC season (boreal summer and autumn). By means of this delayed ocean transport mechanism, ENSO provides an additional heat supply favourable for the formation of strong hurricanes. This thermal control on intense TC variability has significant implications for seasonal predictions and long-term projections of TC activity over the eastern North Pacific.

Currently, the world is experiencing El Nino conditions- of the most intense on record.   Much research suggests that El Nino slightly increases hurricane activity in the Pacific Ocean basin but decreases it slightly in the Atlantic Basin.  It follows that the excitement of this summer's hurricane activity has been in the Pacific.  The season has produced the most intense western hemisphere TC on record, Hurricane Patricia (image above).

JBL14 find large correlations between ENSO and measures of tropical cyclone activity (i.e. ACE = accumulated cyclone energy) in the Eastern North Pacific (pictured here) 

Observed ENSO signals in the winter are thus good indicators of TC activity during the subsequent summer in the central to eastern North Pacific, with the potential to capture about 40–70% of the yearly ACE variability. Because of the environmental and societal impacts of intense hurricanes, and even though the individual TC tracking still remains a considerable challenge, this high predictability of extreme hurricane activity may be valuable for surrounding regions. 

Thus, the implications of JBL14 is that the worst is yet to come.  While this summer may have been exciting, just wait till next summer...  

El Niño events usually peak around Christmas time; warm T105 [top 105m ocean temperatures] anomalies discharged from the Equator as the aftermath of El Niño events will therefore peak during the following boreal summer and autumn, just in time for the active hurricane season in the Northern Hemisphere.

And of course, (as is standard fare) in closing, JBL14 give a plug for what this means for a the future under climate change predictions.  

Second: Comment

Moon, II-Ju, Kim, S-H., and Wang, C. 2015. El Niño and intense tropical cyclones. Nature. 526: E4-E5. (hereafter, MKW15)

Overall, MKW15 agree to the general theoretical conclusion of JBL14: "Specific big El Nino events" influence tropical cyclone activity.  But they challenge the significance of the applicability of the findings, "The connections are not robust enough to apply for the seasonal prediction for all types of ENSO events."

 The challenge is based on a questionable integrity of the analysis...

(1) the correlation between subsurface ocean heat delivered by El Niño and tropical cyclone activity is statistically exaggerated; and (2) wintertime ENSO conditions, which are claimed to have predictive value, are not strongly correlated with tropical cyclone activity during the subsequent summer. 

The first issue is one of data smoothing.  

Data smoothing is useful for identifying patterns in otherwise volatile data.  It is an attempt to identify the signal in the midst of the noise.  But interpreting smoothed data has caveats.

For one, it can create the impression of more dramatic trends than actually exist.  For instance, consider the two images below created from Google's Ngram tool using the phrase "political risk."  The one on the left has no smoothing.  The one in the middle has smoothing of 3.  The one on the right has smoothing of 40.

Clearly the one on the right makes things look forever increasingly politically risky.  This is because the smoothing process amplifies high or low values in subsets (i.e. in the above 3 years or 40 years), while minimizing annual variability.    

Which one is best or most true has to do with logic and the standards and general practices of your colleagues.

In addition, and most relevant for this discussion, is that smoothed data can create correlations that are simply statistical artifacts.  This occurs because the data is no longer independent.  Independence is a bedrock assumption for statistics and correlation calculations.  But I'll let other tech savvy experts and bloggers explain this here and here and here.

How smoothing is used in analysis matters for conclusions.  Smoothing and spurious correlations underly the controversy surrounding the infamous climate "hockey stick."  The core of the debate is whether or not the increased trend (that is, the part of the hockey stick that bats around the puck) is simply statistical artifact.  The validity of the statistics in that image has implications for its meaning and value in debates about climate policy.  See peer reviewed work on it here. 

Hence, MKW15 call out JBL14 for botching their smoothing analysis.  

[JBL14] used a three-year smoothing, which is a suitable technique when the physical variations being examined are multiannual. However, the use of three-year smoothing is not appropriate in this case because [JBL14] examined interannual variations of tropical cyclone activity, focusing on interseasonal connections between wintertime ENSO and summertime tropical cyclones. It turns out that the smoothing significantly increased the correlation between subsurface ocean heat delivered by El Niño (based on the principal component of the second empirical orthogonal function mode in ref. 4, PC2) and tropical cyclone activity from 0.29 to 0.62. The smoothing also enhanced the correlation of a bilinear regression model of [JBL14] from 0.37 to 0.64. (I removed references to publication images for legibility here)

In summary, while a correlation still exists it is about half as large as JBL14 report.

MKW15 also call out JBL14 for cherry picking data.  When comparing activity between El Nino and La Nina years, they used substantially unequal data sets: 43 months (El Nino) to 25 months (La Nina).  This matters because a core of the analysis is a comparison of total number of storms in the two data sets.

This is an unfair comparison, leading naturally to a higher number of tropical cyclones in the high-heat-content periods. An impartial comparison should examine the differences in terms of mean values for each month rather than the total number of tropical cyclones.

Finally, the MKW15 challenge the underlying assumption throughout JBL15's story.  

[JBL14] argued that observed ENSO signals (the Niño index) in the winter are good indicators of tropical cyclone activity during the subsequent summer in the eastern North Pacific. However, the correlation between the Niño index in the winter and ACE during the subsequent summer is very low (r = 0.18), which implies that the subsurface ocean heat delivered by El Niño has very little contribution (~3%) to the total variations of tropical cyclone activity in the subsequent summer.

Third: Reply

Jin, F-F., Boucharel, J., and Lin, I-I.  2015. Jin, et al. reply. Nature. 526: E5-E6. (hereafter JBL15)

JBL15 rationalize their smoothing process.  Remember above that the acceptability of smoothing techniques largely has to do with who your friends are.  So, JBL14 feel justified whereby MKW15 find the technique irresponsible.  

The main point of disagreement is the logic of the time frame of analysis.  JBL14 look at activity rates from year to year.  That is El Nino in year one will affect tropical cyclone activity in year 2.  However, applying a three year smoothing technique sort of "hides" these year to year changes in favor of broader brush stroke (multiyear) changes.  While MKW15 thinks that this sort of hiding is a problem, JBL15 says hiding the effects is exactly the point.  

I think MKW15 has a more convincing argument.  If one wants to know the impact of events in one year on the next, then seeking to hide those years where their is little to no effect sounds like cheating.

On the second critique, JBL15 argue that removing the months of no hurricane activity is important for accurate counting.  

[MKW15] argue that this significance is severely degraded when their ‘accurate’ counting of total number per month is used. However, we believe that their counting ignored one important fact: there are many months without any tropical cyclone (hurricane) occurrence in the record. We argue that those ‘hurricane-empty’ months should be removed for a truly accurate counting.      

I think that there is this thing that happens sometimes among people: those years in which a phenomena of interest does not occur are therefore, irrelevant.  

This is like my saying that my life is characterized by sadness and hardship based on a handful of unfortunate events rather than acknowledging the many years and of joy and fulfilling opportunities.  

Focusing on a subset of events does not characterize the whole story.  It's not accurate in the sense that it is not the whole story.  

Finally, JBL15 take up the accusation that their findings are not all that useful.  Their argument is largely theoretical.  In effect: The hight of El Nino is measured in winter over December.  The heat from strong El Nino's has to go somewhere.  The heat is discharged into the eastern Pacific.  Warm water is significant for tropical cyclone activity.  So, December El Nino measurements are predictive of next year's activity. 

However, MKW15 argue that the findings are not useful because the observations do not support the theory.  

The thing with tropical cyclone data is that it is limited.  It is difficult to say that one theory is more correct than another because many theories and often conflicting theories are supported by the limited amount of data.  Data is limited in all basins but ever more so in basins outside the Atlantic.   

So, predictability is perhaps itself, spurious.  What is most important is for what purpose these theories are being put forward and the wisdom of inserting them into specific decision making contexts.  

Is it for, Academic debate?  Beefing up emergency management preparations?  Changing insurance rates?   These each have very different real world outcomes.  

-----------

So, there you have it.  Debate, discussion and potentially, mistakes and inaccuracies in analysis.  

This is important for understanding what goes on in scientific journals and the significance of a body of work as compared to any individual article.  Picking a journal article out of the mix to represent significant advancements in knowledge is misleading at best.