oa Four best practices for measuring news sentiment using ‘off-the-shelf’ dictionaries: a large-scale p-hacking experiment

References

1. Barberá, P., Boydstun, A. E., Linn, S., McMahon, R., & Nagler, J.(2020). “Automated Text Classification of News Articles: A Practical Guide.”Political Analysis, June, 1–24. https://doi.org/10.1017/pan.2020.8
   [Google Scholar]
2. Barberá, PBarberá, P., Boydstun, A., Linn, S., McMahon, R., & Nagler, J.(2016). “Methodological Challenges in Estimating Tone: Application to News Coverage of the Us Economy.” In Meeting of the Midwest Political Science Association, Chicago, Il.
   [Google Scholar]
3. Benoit, K., Watanabe, K., Wang, H., Nulty, P., Obeng, A., Müller, S., & Matsuo, A.(2018). “Quanteda: An R Package for the Quantitative Analysis of Textual Data.”Journal of Open Source Software3 (30): 774. https://doi.org/10.21105/joss.00774
   [Google Scholar]
4. Bishop, D. V., & Thompson, P. A.(2016). “Problems in Usingp-Curve Analysis and Text-Mining to Detect Rate of P-Hacking and Evidential Value.”PeerJ4 (February): e1715. https://doi.org/10.7717/peerj.1715
   [Google Scholar]
5. Boukes, M., Van de Velde, B., Araujo, T., & Vliegenthart, R.(2019). “What’s the Tone? Easy Doesn’t Do It: Analyzing Performance and Agreement Between Off-the-Shelf Sentiment Analysis Tools.”Communication Methods and Measures14 (2): 83–104. https://doi.org/10.1080/19312458.2019.1671966
   [Google Scholar]
6. Boumans, J. W., & Trilling, D.(2015). “Taking Stock of the Toolkit.”Digital Journalism4 (1): 8–23. https://doi.org/10.1080/21670811.2015.1096598
   [Google Scholar]
7. Bradley, M. M., & Lang, P. J.(1999). “Affective Norms for English Words (ANEW): Instruction Manual and Affective Ratings.”Technical report C-1, the center for research in psychophysiology.
   [Google Scholar]
8. Clifford, S., & Jerit, J.(2013). How words do the work of politics: Moral foundations theory and the debate over stem cell research. The Journal of Politics, 75 (3), 659–671. https://doi.org/10.1017/s0022381613000492
   [Google Scholar]
9. Cohen, J. E.(2004). If the news is so bad, why are presidential polls so high? Presidents, the news media, and the mass public in an era of new media. Presidential Studies Quarterly, 34 (3), 493–515. https://doi.org/10.1111/j.1741-5705.2004.00209.x
   [Google Scholar]
10. Dainas, A., Munot, V., & Tsutsui, S.(2015). The moral foundations in new york times. https://pdfs.semanticscholar.org/0c08/ab050e941e57de95433722895d8c1abd9064.pdf.
    [Google Scholar]
11. Diesner, J., & Evans, C. S.(2015). Little bad concerns: Using sentiment analysis to assess structural balance in communication networks. Proceedings of the 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2015 – ASONAM ’15. https://doi.org/10.1145/2808797.2809403
    [Google Scholar]
12. Dodds, P. S., & Danforth, C. M.(2009). Measuring the happiness of large-scale written expression: Songs, blogs, and presidents. Journal of Happiness Studies, 11 (4), 441–456. https://doi.org/10.1007/s10902-009-9150-9
    [Google Scholar]
13. Eshbaugh-Soha, M.(2010). The tone of local presidential news coverage. Political Communication, 27 (2), 121–140. https://doi.org/10.1080/10584600903502623
    [Google Scholar]
14. Fogel-Dror, Y., Shenhav, S. R., Sheafer, T., & Van Atteveldt, W.(2018). Role-based association of verbs, actions, and sentiments with entities in political discourse. Communication Methods and Measures, 13 (2), 69–82. https://doi.org/10.1080/19312458.2018.1536973
    [Google Scholar]
15. Fu, K.-w., & Chan, C.-h.(2013). Analyzing online sentiment to predict telephone poll results. Cyberpsychology, Behavior, and Social Networking, 16 (9), 702–707. https://doi.org/10.1089/cyber.2012.0375
    [Google Scholar]
16. Gilbert, C., & Hutto, E.(2014). Vader: A parsimonious rule-based model for sentiment Analysis of Social Media Text.In Eighth International Conference on Weblogs and Social Media (icwsm-14)., 81:82. comp.social.gatech.edu/papers/icwsm14.vader.hutto.pdf.
    [Google Scholar]
17. Gonzalez-Bailon, S., De Francisci Morales, G., Mendoza, M., Khan, N., & Castillo, C.(2014). Cable news coverage and online news stories: A large-scale comparison of media bias. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.2389525
    [Google Scholar]
18. González-Bailón, S., & Paltoglou, G.(2015). Signals of public opinion in online Communication. The ANNALS of the American Academy of Political and Social Science659 (1): 95–107. https://doi.org/10.1177/0002716215569192
    [Google Scholar]
19. Graham, J., Haidt, J., & Nosek, B. A.(2009). Liberals and Conservatives Rely on Different Sets of Moral Foundations. Journal of Personality and Social Psychology96 (5): 1029–46. https://doi.org/10.1037/a0015141
    [Google Scholar]
20. Grimmer, J., & Stewart, B. M.(2013). Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts. Political Analysis21 (3): 267–97. https://doi.org/10.1093/pan/mps028
    [Google Scholar]
21. Haidt, J.(2012). The righteous mind: Why good people are divided by politics and religion. Vintage.
    [Google Scholar]
22. Haselmayer, M., & Jenny, M.(2016). Sentiment Analysis of Political Communication: Combining a Dictionary Approach with Crowdcoding. Quality & Quantity51 (6): 2623–46. https://doi.org/10.1007/s11135-016-0412-4
    [Google Scholar]
23. Hu, M., & Liu, B.(2004). Mining and summarizing customer reviews. Proceedings of the 2004 ACM SIGKDD International Conference on Knowledge Discovery and Data Mining – KDD ’04. https://doi.org/10.1145/1014052.1014073
    [Google Scholar]
24. Ji, Q., Raney, A. A., Janicke-Bowles, S. H., Dale, K. R., Oliver, M. B., Reed, A., . . .Raney, A.A.2018. Spreading the Good News: Analyzing Socially Shared Inspirational News Content. Journalism & Mass Communication Quarterly96 (3): 872–93. https://doi.org/10.1177/1077699018813096
    [Google Scholar]
25. Lee, H. S.(2014). Analyzing the Multidirectional Relationships Between the President, News Media, and the Public: Who Affects Whom?Political Communication31 (2): 259–81. https://doi.org/10.1080/10584609.2013.815295
    [Google Scholar]
26. Leetaru, K.(2011). Culturomics 2.0: Forecasting large-scale human behavior using global news media tone in time and space. First Monday, August. https://doi.org/10.5210/fm.v16i9.3663
    [Google Scholar]
27. Liu, B.2010. Sentiment Analysis and Subjectivity. Handbook of Natural Language Processing2 (2010): 627–66.
    [Google Scholar]
28. Mohammad, S. M., & Turney, P. D.(2012). Crowdsourcing a word-emotion association Lexicon. Computational Intelligence29 (3): 436–65. https://doi.org/10.1111/j.1467-8640.2012.00460.x
    [Google Scholar]
29. Munezero, M., Montero, C. S., Sutinen, E., & Pajunen, J.(2014). Are They Different? Affect, Feeling, Emotion, Sentiment, and Opinion Detection in Text. IEEE Transactions on Affective Computing5 (2): 101–11. https://doi.org/10.1109/taffc.2014.2317187
    [Google Scholar]
30. Naveed, N., Gottron, T., Kunegis, J., & Alhadi, A. C.(2011). Bad News Travel Fast. Proceedings of the 3rd International Web Science Conference on – WebSci ’11. https://doi.org/10.1145/2527031.2527052
    [Google Scholar]
31. Pang, B., & Lee, L.(2008). Opinion Mining and Sentiment Analysis. Foundations and Trends® in Information Retrieval2 (1–2): 1–135. https://doi.org/10.1561/1500000011
    [Google Scholar]
32. Pennebaker, J. W., Boyd, R. L., Jordan, K., & Blackburn, K.(2015). The Development and Psychometric Properties of Liwc2015
    [Google Scholar]
33. Puschmann, C., & Powell, A.(2018). Turning Words into Consumer Preferences: How Sentiment Analysis Is Framed in Research and the News Media. Social Media + Society4 (3): 205630511879772. https://doi.org/10.1177/2056305118797724
    [Google Scholar]
34. Rauh, C.(2018). Validating a Sentiment Dictionary for German Political Language—a Workbench Note. Journal of Information Technology & Politics15 (4): 319–43. https://doi.org/10.1080/19331681.2018.1485608
    [Google Scholar]
35. Ribeiro, F. N., Araújo, M., Gonçalves, P., André Gonçalves, M., & Benevenuto, F.2016. SentiBench – a Benchmark Comparison of State-of-the-Practice Sentiment Analysis Methods. EPJ Data Science5 (1). https://doi.org/10.1140/epjds/s13688-016-0085-1
    [Google Scholar]
36. Rooduijn, M., & Pauwels, T.(2011). Measuring Populism: Comparing Two Methods of Content Analysis. West European Politics34 (6): 1272–83. https://doi.org/10.1080/01402382.2011.616665
    [Google Scholar]
37. Rudkowsky, E., Haselmayer, M., Wastian, M., Jenny, M., Emrich, Š., & Sedlmair, M.(2018). More Than Bags of Words: Sentiment Analysis with Word Embeddings.Communication Methods and Measures12 (2-3): 140–57. https://doi.org/10.1080/19312458.2018.1455817
    [Google Scholar]
38. Schubert, J. N., Stewart, P. A., & Curran, M. A.(2002). A Defining Presidential Moment: 9/11 and the Rally Effect. Political Psychology23 (3): 559–83. https://doi.org/10.1111/0162-895x.00298
    [Google Scholar]
39. Simmons, J. P., Nelson, L. D., & Simonsohn, U.(2011). False-Positive Psychology. Psychological Science22 (11): 1359–66. https://doi.org/10.1177/0956797611417632
    [Google Scholar]
40. Song, H., Tolochko, P., Eberl, J.-M., Eisele, O., Greussing, E., Heidenreich, T., . . .Boomgaarden, H. G.(2020). In Validations We Trust? The Impact of Imperfect Human Annotations as a Gold Standard on the Quality of Validation of Automated Content Analysis. Political Communication37 (4): 550–72. https://doi.org/10.1080/10584609.2020.1723752
    [Google Scholar]
41. Soroka, S. N.(2002). Agenda-Setting Dynamics in Canada. UBC press.
    [Google Scholar]
42. Stone, P. J., & Hunt, E. B.(1963). A Computer Approach to Content Analysis.”Proceedings of the May 21-23, 1963, Spring Joint Computer Conference on – AFIPS ’63 (Spring). https://doi.org/10.1145/1461551.1461583
    [Google Scholar]
43. Tausczik, Y. R., & Pennebaker, J. W.(2009). The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods. Journal of Language and Social Psychology29 (1): 24–54. https://doi.org/10.1177/0261927x09351676
    [Google Scholar]
44. The American Presidency Project. (n.d.) Presidential Job Approval. https://www.presidency.ucsb.edu/statistics/data/presidential-job-approval
    [Google Scholar]
45. Van Atteveldt, W., & Peng, T.-Q.(2018). When Communication Meets Computation: Opportunities, Challenges, and Pitfalls in Computational Communication Science. Communication Methods and Measures12 (2-3): 81–92. https://doi.org/10.1080/19312458.2018.1458084
    [Google Scholar]
46. Vosoughi, S., Roy, D., & Aral, S.(2018). The Spread of True and False News Online. Science359 (6380): 1146–51. https://doi.org/10.1126/science.aap9559
    [Google Scholar]
47. Walgrave, S., Soroka, S., & Nuytemans, M.(2007). The Mass Media’s Political Agenda-Setting Power. Comparative Political Studies41 (6): 814–36. https://doi.org/10.1177/0010414006299098
    [Google Scholar]
48. Walter, S.(2019). Better Off Without You? How the British Media Portrayed Eu Citizens in Brexit News. The International Journal of Press/Politics24 (2): 210–32. https://doi.org/10.1177/1940161218821509
    [Google Scholar]
49. Whissell, C.(2008). Emotional Fluctuations in Bob Dylan’s Lyrics Measured by the Dictionary of Affect Accompany Events and Phases in His Life. Psychological Reports102 (2): 469–83. https://doi.org/10.2466/pr0.102.2.469-483
    [Google Scholar]
50. Whissell, C. M.(1989). The Dictionary of Affect in Language. The Measurement of Emotions, 113–31. https://doi.org/10.1016/b978-0-12-558704-4.50011-6
    [Google Scholar]
51. Young, L., & Soroka, S.(2012). Affective News: The Automated Coding of Sentiment in Political Texts.Political Communication29 (2): 205–31. https://doi.org/10.1080/10584609.2012.671234
    [Google Scholar]