This is part two in a two-part series. For part one click here.
The stories that a country tells itself, and others, helps to create and define its self identity. Part one of this paper (please see here) provided a theoretical and methodological overview of the use of sentiment analysis across country pairs to understand these stories. It also established that this analysis has the potential to highlight when stories might boil over to drive action. Significant challenges to self-identity have been shown to drive countries to take action to defend these identities, even if these actions create physical and economic security risks.
Part two of this paper, below, presents detailed analysis of country pairs and suggests potential conclusions. This work leverages the Global Database of Events, Language, and Tone (GDELT). The full dataset developed from GDELT for this paper can be found in the Harvard Dataverse here, allowing for replicability of all analysis and charts shown in part one and part two of this paper.
The temperature now
Russia / Ukraine conflict
Figure 1 below analyzes volume intensity weighted tone (VIWT, please see part one for a detailed description and calculation) for Ukrainian press published in Ukrainian discussing Russia (blue line), Russian press published in Russian discussing Ukraine (orange line) and the difference between the two (grey line) from Jan 1, 2017 through Dec. 12, 2024. Several observations can be made:
- Russian VIWT towards Ukraine became more negative in October 2021, a full four-plus months prior to the actual invasion. Trends during the period between January-February 2022 clearly highlighted that Russian narratives had moved into new and, at the time, uncharted negative territory and could have been used as a warning for those forecasting little chance of an invasion.
- A second and more pronounced negative movement in VIWT for Russian press discussing Ukraine occurred in early 2023, around the same time that Russia was pursuing a “shadow mobilization” to increase the number of troops available in Ukraine.
- Smaller VIWT negative movements were seen in April 2024, when Ukrainian forces attacked the Tatarstan region of Russia with large drone attacks and the UK announced its largest military aid package for Ukraine to date; and in October 2024, when the US and France stepped up aid and North Korea committed to deploying 12,000 troops to assist with Russian operations in Ukraine.
At present, dyadic VIWT have normalized to their previous steady state levels, post a small dip concurrent with the US presidential election.
Figure 1: Volume intensity weighted tone for Russia (published in Russian) and Ukraine (published in Ukrainian) toward each other.
Source: GDELT Database http:/www.gdeltproject.org
South-North Korea relations
VIWT for South Korean (ROK) media discussing North Korea (DPRK) in both Korean and English have fallen precipitously into negative territory, as shown in Figure 2 below. VIWT for English language stories is currently at its most negative level since July 2018 (US Secretary of State Pompeo visited Pyongyang), while Korean-language VIWT is at the lowest level by far in the data set (stretching back to 1 Jan 2017). This significant negative sentiment shift occurs concurrent with the DPRK amending its constitution to designate ROK as a “hostile state” and abandoning aspirations for reconciliation, DPRK destruction of road and rail links with ROK, DPRK ratification of a mutual defense treaty with Russia, and DPRK troops serving alongside Russia troops in Ukraine.
Figure 2: Volume intensity weighted tone South Korea on North Korea (published in Korean vs. English).
Source: GDELT Database http:/www.gdeltproject.org
Additional analysis was conducted to compare South Korean and North Korean sentiment changes over time, shown in Figure 3 below. A review of DPRK sources in GDELT show that almost all data comes from NK News.org. The limited number of sources available reduces the reliability of analysis of DPRK sentiment and narrative shifts.
Figure 3: Volume intensity weighted tone for South Korea vs. North Korea toward each other.
Source: GDELT Database http:/www.gdeltproject.org
Taiwan-China relations
VIWT for Taiwanese and Chinese press discussing each other moves in ways consistent with real world events, lending credence to the underlying analysis. Significant negative moves in dyadic VIWT can be seen centered around specific events inclusive of COVID in 2020, the transit of Taiwan’s vice president through the US and the subsequent “warning” air and sea exercises held by the People’s Liberation Army (PLA) and People’s Liberation Army Navy (PLAN), former Taiwan President Ma’s visit to Beijing, and Taiwan President Tsai’s meeting with then US House of Representatives Speaker Kevin McCarthy in California.
Figure 4: Volume intensity weighed tone for Taiwan and China toward each other (published in Mandarin).
Source: GDELT Database http:/www.gdeltproject.org
The most important trend to highlight, however, is the volatile but consistently more positive trend of VIWT for China speaking about Taiwan (orange line) relative to the flattish VIWT for Taiwan speaking about China. This analysis presents no evidence that Chinese authorities are preparing narratives that would justify physical activities against Taiwan and the Taiwanese people, with no use of dehumanizing language of the type that prepared Russian citizens for Ukrainian casualties. The data is consistent with the statement that China is not preparing for kinetic activity against Taiwan that results in casualties.
Drawing conclusions from expressed sentiment from Taiwan is more complex. This approach arguably is most effective when there is confidence that the views expressed in the media closely mirror that of the government. In this case, political control over the media is in a way helpful. Therefore, the fact that Reports without Borders rates China 172nd out of 180 countries in terms of press freedom drives higher confidence that sentiment measured is the sentiment of the government, vs. commercial organizations trying to maximize readership. Taiwan, rated 27th in press freedom, is arguably less suited to this type of analysis than China. For reference, other countries referenced in this paper rank 55th (USA), 62nd (South Korea), 70th (Japan), 162nd (Russia), and 177th (North Korea).
United States-China relations
Conclusions driven by the application of this tool to the US-China dyad include: 1) US sentiment has been significantly more negative than China’s expressed sentiment for the duration of the sample set. It is unclear how much of this is driven by a more commercially oriented press, which is arguable incentivized to be sensational. 2) China VIWT tone dipped negative from neutral in early 2018, concurrent with US Administration tariff escalation vs. China. 3) COVID was the nadir of dyadic VIWT, but statistics generally recovered by 2021. 4) China VIWT regarding the US improved in the days immediately following the recent US election and turned positive for only the third time since 2019, peaking in late November 2024.
Figure 5: Volume Intensity Weighted Tone for the US (published in English) and China (published in Mandarin) toward each other.
Source: GDELT Database http:/www.gdeltproject.org
Figure 6: Volume intensity weighted tone US (published in English) and China (published in Mandarin) Towards each other
Source: GDELT Database http:/www.gdeltproject.org
Japan-China relations
Japan and China have a long history of disputes inclusive of sovereignty issues around the Senkaku/Diaoyu islands. Recent diplomatic efforts inclusive of high-level summits have been highlighted by academics as a potential thaw in the bilateral relationship. This is clearly reflected in the VIWT analysis shown in Figure 7 with positive values post the Kishida/Xi Summit in September 2023. This trend has continued (with oscillation) despite China’s protests regarding Japan’s remilitarization.
Figure 7: Volume intensity weighted tone Japan (published in Japanese) and China (published in Mandarin) towards each other.
Source: GDELT Database http:/www.gdeltproject.org
Future areas of research
The analysis shown above appears to be a promising technique to begin to quantify signals that presage conflict. It is not meant to “prove” anything, however, but seeks to highlight changing levels of risk, and force the challenging of consensus based on data. There are several additional areas of enquiry that can improve this analysis.
“Freedom feedback loops”
The impact of a free press on the validity of this analysis deserves greater study. Theoretically, the less free a press is, and the more it is the direct mouthpiece of the government in question, the “cleaner” the signal. Press in countries like North Korea (ranked 177th out of 180 for free press), China (172nd), Russia (162nd) might be better targets for this analysis relative to Taiwan (27th) or the US (55th). Additional work analyzing the differential applicability across media markets is likely valuable.
“It’s all social now”
This analysis focused on resources available in the GDELT dataset. This dataset incorporates 3.2 trillion data points. Despite that, there is data that is not included, which for the purposes of this analysis is largely centered in the social media world. Analysis that examines the different narratives and tones in social media vs. traditional media could be instructive, in particular to examine the existence and directionality of any feedback loop between the two data sets (who is responding to who’s story?).
James R. Sullivan, CFA ([email protected]) is an Adjunct Fellow at Pacific Forum and an External Associate in the Economic Competition and Conflict Research Group at King’s College London.
PacNet commentaries and responses represent the views of the respective authors. Alternative viewpoints are always welcomed and encouraged.
Photo Credit: International Crisis Group
If you want to see more insightful analysis and impactful events in the Indo-Pacific, support us by donating. Your contribution fuels our mission for a secure and cooperative future: Donate Here