Analyzing U.S. Army Officer Evaluation Reports with Natural Language Processing: A Log-Odds and Latent Dirichlet Allocation Exploration

  • Heidy Shi
  • John Caddell
  • Julia Lensing
Keywords: Text Mining, Topic Modeling, Latent Dirichlet Allocation, Officer Evaluation Reports (OERs)

Abstract

Each job field (branch) in the Army requires a unique set of skills and talents of the officers assigned. Officers who demonstrate the required skills are often more successful in their assigned branch. To better understand how success is described across branches, research was conducted using text mining and text analysis of a data set of Officer Evaluation Reports (OERs). This research looked for common trends and discrepancies across varying branches and like groups of branches by analyzing the narrative portion of OERs. Text analysis methods examined words and bigrams commonly used to describe varying degrees of performance by officers. Topic modeling using Latent Dirichlet Allocation (LDA) was also conducted on top rated narratives to investigate trends and discrepancies in clustering narratives. Findings show that qualitative narratives for the top two performance designations fail to differentiate between officers’ varying levels of performance regardless of branch.

References

Aggarwal, C. C., & Zhai, C. (2012). Mining Text Data. New York, NY: Springer Science + Business Media, LLC.
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of Machine Learning Research, 3(Jan), 993-1022.
Calarusso, M. J., Heckel, K. G., Lyle, D. S., & Skimmyhorn, W.L. (2016). Starting Strong: Talent-Based Branching of Newly Commissioned U.S. Army Officers. Strategic Studies Insitute and U.S. Army War College Press, 9(1), 1-62.
Department of the Army. (2015). Evaluation Reporting System. (Army Regulation 623-3). Washington DC: U.S. Government Printing Office.
Han, J., Kamber, M., & Pei, J. (2011). Data Mining Concepts and Techniques. Waltham, MA: Elsevier Inc.
Kite, D. P. (1998). The US Army Officer Evaluation Report: Why are We Writing to Someone Who Isn’t Reading? Air Command and Staff College Air University.
Lopez, T. C. (2013, March 29). New Army OER Means Fewer Boxes, More Accountability for Raters. Army News Service. Retrieved from: https://www.jble.af.mil/News/Article-Display/Article/257909/new-army-oer-means-fewer-boxes-more-accountability-for-raters/
Milkovich, G. T., & Boudreau, J.W. (1997). Human Resource Management. Boston, MA: Times Mirror Higher Education Group.
Morrisey, G. L. (1983). Performance Appraisals for Business & Industry. Reading, MA: Addison-Wesley Publishing Company.
Pfeffer J., & Sutton R. I. (2000). The knowing–doing gap: How smart companies turn knowledge into action. Boston: Harvard Business School Press.
Reese, T. R. (2002). Transforming the Officer Evaluation System: Using a 360 Degree Feedback Model. US Army War College, Pennsylvania.
Silge, J., & Robinson, D. (2019). Text Mining with R: A Tidy Approach. Sebastopol, CA: O’Reilly Media, Inc.
Straffon, N. (1997). Promotion Boards. Army Reserve Magazine (Winter), 19.
Welch, J. F. (2001). Jack: Straight from the gut. New York: Warner Books.
Published
2019-12-31
How to Cite
Shi, H., Caddell, J., & Lensing, J. (2019). Analyzing U.S. Army Officer Evaluation Reports with Natural Language Processing: A Log-Odds and Latent Dirichlet Allocation Exploration. Industrial and Systems Engineering Review, 7(1), 44-55. https://doi.org/10.37266/ISER.2019v7i1.pp44-55