Examining the Internal Structure and Measurement Invariance of the Organizational Justice Scale in a Sample of Employees in Puerto Rico




Palabras clave:

Confirmatory Factor Analysis, exploratory factor analysis, internal structure, measurement invariance, Organizational Justice, Psychometric Properties


The purpose of current study was to examine the internal structure, psychometric properties, and measurement invariance of the Organizational Justice Scale (OJS) in a sample of people employed in Puerto Rico. A total of 1,099 employed people who worked at least 20 hours per week and were 21 years of age or older participated in the study. Item analysis by dimension, exploratory factor analysis, and confirmatory factor analysis were performed using structural equation modeling to examine the internal structure of the OJS. The revised version of the OJS (OJS-R) was made up of 12 items, three for each of the dimensions of distributive justice and formal procedural, and six for the dimension of interactional justice. The results of the factor analysis, especially the confirmatory one, support the three-dimensional internal structure of the OJS-R. In terms of reliability, Cronbach's alpha and McDonald's omega coefficients ranged from .793 to .927. The new version of the OJS-R seems to be invariant by gender, age, job position and type of organization. In this way, the results provide evidence of the validity and reliability of the OJS-R to measure the perception of organizational justice.

Biografía del autor/a

Lillian V. Rovira-Millán, University of Puerto Rico at Cayey

Psicóloga Clínica, Investigadora y Catedrática en Psicología, Departamento de Ciencias Sociales, Universidad de Puerto Rico, Cayey, Puerto Rico

Rafael A. Blanco-Rovira, Albizu University

Estudiante Doctoral en Psicología Industrial/Organizacional en la Universidad Albizu y Asistente de Investigación

Ana C. López-Iglesias, Albizu University

Graduada del Programa de Biología de la UPR y Aceptada en el Programa de Maestría en Psicología Industrial/Organizacional de la Universidad Albizu y Asistente de Investigación


Adams, J.S. (1963). Toward an understanding of inequity. Journal of Abnormal and Social Psychology, 67, 422-436.

Adams, J.S. (1965). Inequity in social exchange. In L. Berko-witz (Ed.), Advances in experimental social psychology (Vol. 2, pp.267-299). Academic Press.

Ambrose, M.L. & Arnaud, A. (2005). Are procedural justice and distributive justice conceptually distinct? In J. Greenberg & J.A. Colquitt (Eds.), Handbook of organiza-tional justice (pp. 59-84). Lawrence Erlbaum Associates Publishers.

Ato, M., López-García, J. J., & Benavente, A. (2013). Un-sistema de clasificación de los diseños de investigación en psicología. Anales de Psicología, 29(3),1038-1059. https://doi.org/10.6018/analesps.29.3.178511

Bies, R.J. (1987). The predicament of injustice: The manage-ment of moral outrage. In L.L. Cummings & B.M. Staw (Eds.), Research in organizational behavior (Vol. 9, pp. 289-319. JAI Press.

Bies, R.J. & Moag, J.F. (1986). Interactional justice: Commu-nication criteria of fairness. In R.J. Lewicki, B.H. Shep-pard, & M.H. Bazerman (Eds.), Research on negotiations in organizations (Vol. 1, pp.43-55). JAI Press.

Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen and J. S. Long (Eds.), Testing structural equation models (pp. 136-162). Sage.

Browne, M. W., & Cudeck, R. (1989). Single sample cross-validation indices for covariance structures. Multivariate Behavioral Research, 24(4), 445-455. https://doi.org/10.1207/s15327906mbr2404_4

Byrne, B. M. (2016). Structural equation modeling with AMOS: Basic concepts, applications, and programming. Routledge.

Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14, 464-504. https://psycnet.apa.org/doi/10.1080/10705510701301834

Cohen-Charash, Y. & Spector, P.E. (2001). The role of justice in organizations: A meta-analysis. Organizational Behavior and Human Decision Processes, 86(2), 278-321. http://dx.doi.org/10.1006/obhd.2001.2958

Colquitt, J. A., Conlon, D. E., Wesson, M. J., Porter, C. O. L. H., & Ng, K. Y. (2001). Justice at the millennium: A me-ta-analytic review of 25 years of organizational justice re-search. Journal of Applied Psychology, 86(3), 425-445. https://doi.org/10.1037/0021-9010.86.3.425

Colquitt, J. A., Scott, B. A., Rodell, J. B., Long, D. M., Zapata, C. P., Conlon, D. E., & Wesson, M. J. (2013). Justice at the millennium, a decade later: A meta-analytic test of social exchange and affect-based perspectives. Journal of Applied Psychology, 98(2), 199-236. https://doi.org/10.1037/a0031757

Colquitt, J. A. & Shaw, J.C. (2005). How should organiza-tional justice be measured? In J. Greenberg & J.A. Colquitt (Eds.), Handbook of organizational justice (pp. 113-152). Lawrence Erlbaum Associates Publishers.

Comrey, A. L., & Lee, H. B. (1992). A first course in factor anal-ysis. Lawrence Erlbaum Associates.

Cropanzano, R., Byrne, Z. S., Bobocel, D. R., & Rupp, D. E. (2001). Moral virtues, fairness heuristics, social entities, and other denizens of organizational justice. Journal of Vocational Behavior, 58(2), 164-209. https://doi.org/10.1006/jvbe.2001.1791

Cropanzano, R. & Wright, T.A. (2011). The impact of organi-zational justice on occupational health. In J.C. Quick & L.E. Tetrick (Eds.), Handbook of Occupational Health Psy-chology (pp. 205-219). American Psychological Associa-tion.

de Rooij, M. & Weeda, W. (2020). Cross-validation: A meth-od every psychologist should know. Advances in Methods and Practices in Psychological Science, 3(2), 248-263. https://journals.sagepub.com/doi/10.1177/2515245919898466

Deressa, B., Adugna, K., Bezane, B., Jabessa, M., Wayessa, G., Kebede, A., Tefera, G., & Demissie, Y. (2022). The Rela-tionship Between Organizational Commitment and Or-ganizational Justice Among Health Care Workers in Ethiopian Jimma Zone Public Health Facilities. Journal of healthcare leadership, 14, 5-16. https://doi.org/10.2147/JHL.S345528

DeVellis, R. F. (2017). Scale Development: Theory and Applica-tions (4th ed.). SAGE Publications.

Deutsch, M. (1975). Equity, equality, and need: What de-termines which value will be used as the basis for dis-tributive justice? Journal of Social Issues, 31, 137-149.

Dunn, T.J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychol-ogy, 105(3), 399-412. https://doi.org/10.1111/bjop.12046

Eib, C., Bernhard-Oettel, Magnusson Hanson, L.I., Lein-eweber, C. (2018). Organizational justice and health: Studying mental preoccupation with work and social support as mediators for lagged and reversed relation-ships. Journal of Occupational Health Psychology, 23(4), 553-567. http://dx.doi.org/10.1037/ocp0000115

Fabrigar, L.R., Wegener, D.T., MacCallum, R.C., & Strahan, E.J. (1999). Evaluating the use of exploratory factor anal-ysis in psychological research. Psychological Methods, 4(3), 272-299. https://doi.org/10.1037/1082-989X.4.3.272

Finkelstein, R., Minibas-Poussard, J., & Bastounis, M. (2009). The moderating role of seeking social support on coping styles and perceptions of organizational justice: A study with French and Turkish students. Social Behavior and Personality: An International Journal, 37(6), 845-862. https://doi.org/10.2224/sbp.2009.37.6.845

Fornell, C., & Bookstein, F. L. (1982). Two structural equation models: LISREL and PLS applied to consumer exit-voice theory. Journal of Marketing Research, 19(4), 440-452. https://doi.org/10.2307/3151718

Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and meas-urement error. Journal of Marketing Research, 18(1), 39-50. https://doi.org/10.2307/3151312

Gefen, D., Straub, D., Boudreau, M-C. (2000). Structural equation modeling and regression: guidelines for re-search practice. Communications of the AIS, 4, 1-78. https://doi.org/10.17705/1CAIS.00407

Gignac, G. E. (2016). The higher-order model imposes a pro-portionality constraint: that is why the bifactor model tends to fit better. Intelligence, 55, 57–68. https://doi.org/10.1016/j.intell.2016.01.006

Green, S.B. & Yang, Y. (2009). Reliability of summed item scores using structural equation modeling: An alterna-tive to coefficient alpha. Psychometrika, 74, 155-167. https://doi.org/10.1007/s11336-008-9099-3

Greenberg, J. (1987). A taxonomy of organizational justice theories. Academy of Management Review, 12, 9-22. https://doi.org/10.5465/amr.1987.4306437

Greenberg, J. (1993). The social side of fairness: Interpersonal and informational classes of organizational justice. In R. Cropanzano (Ed.), Justice in the workplace: Approaching fairness in human resource management (pp. 79-103). Law-rence Erlbaum Associates.

Greenberg, J. (2010). Organizational injustice as an occupa-tional health risk. The Academy of Management Annals, 4(1), 205-243. https://psycnet.apa.org/doi/10.1080/19416520.2010.481174

Hair, J., Black, W., Babin, B., & Anderson, R. (2019). Multi-variate Data Analysis. Cengage Learning EMEA.

Hauenstein, N.M.A., McGonigle, T. & Flinder, S.W. (2001). A Meta-Analysis of the Relationship Between Procedural Justice and Distributive Justice: Implications for Justice Research. Employee Responsibilities and Rights Journal, 13, 39-56. https://doi.org/10.1023/A:1014482124497

Hayes, A. F. & Coutts, J. J. (2020). Use omega rather than cronbach’s alpha for estimating reliability. But…. Com-munication Methods and Measures, 14(1), 1-24. https://doi.org/10.1080/19312458.2020.1718629

Homans, G.C. (1961). Social behaviour: Its elementary forms. Routledge & Kegan Paul.

Hu, L.-T., & Bentler, P. M. (1995). Evaluating model fit. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, is-sues, and applications (pp. 76-99). Sage Publications.

Hu, L.-t., & Bentler, P. M. (1999). Cutoff criteria for fit in-dexes in covariance structure analysis: Conventional cri-teria versus new alternatives. Structural Equation Model-ing, 6(1), 1-55. https://doi.org/10.1080/10705519909540118

IBM Corp. (2021). IBM SPSS Statistics for Windows, Version 28.0. IBM Corp.

Imran, R., & Allil, K. (2016). The impact of organizational justice on employee retention: Evidence from Oman. In-ternational Review of Management and Marketing, 6(2), 246-249. https://www.econjournals.com/index.php/irmm/article/view/2014

Kline, P. (1994). An easy guide to factor analysis. Routledge.

Kline, R.B. (2016). Principles and practice of structural equation modeling. The Guilford Press.

Levanthal, G.S. (1976). The distribution of rewards and re-sources in groups and organziations. In L. Berkowitz & W. Walster (Eds.), Advances in experimental social psycholo-gy (Vol 9, pp. 91-131). Academic Press.

Levanthal, G.S. (1980). What should be done with equity theory? New approaches to the study of fairness in so-cial relationships. In K. Gergen, M. Greenberg, & R.Willis (Eds.), Social exchange: Advances in theory and research (pp. 27-55). Plenum Press.

Levanthal, G.S., Karusa, J., & Fry, W.R. (1980). Beyonf fair-ness: A theory of allocations preferences. In G. Mikula (Ed.), Justice and social interaction (167-218). Springer-Verlag.

Li, C.H. (2016a). Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diago-nally weighted least squares. Behavioral Research Methods, 48(3), 936-949. https://doi.org/10.3758/s13428-015-0619-7

Li, C.-H. (2016b). The performance of ML, DWLS, and ULS estimation with robust corrections in structural equation models with ordinal variables. Psychological Methods, 21(3), 369-387. https://doi.org/10.1037/met0000093

Littlewood Zimmerman, H.F. & Bernal García, E.R. (2011). Mi primer modelamiento de ecuación estructural: LISREL. Medellín, Colombia: Centro de Investigación en Com-portamiento Organizacional (CINCEL).

Loi, R., Yang, J., & Diefendorf, J. M. (2009). Four-factor jus-tice and daily job satisfaction: A multi-level investiga-tion. Journal of Applied Psychology, 94(3),770-781. https://doi.org/10.1037/a0015714

Lotfi, M.H. & Pour, M.S. (2013). The relationship between organizational justice and job satisfaction among the employees of Tehran Payame Noor University. Procedia Social and Behavioral Sciences, 93, 2073-2079.

MacCallum, R.C., Browne, M.W., & Sugawara, H.M. (1996). Power analysis and determination of sample size for co-variance structure modeling. Psychological Methods, 1(2), 130-149. https://psycnet.apa.org/doi/10.1037/1082-989X.1.2.130

Martocchia, J. J., & Judge, T. A. (1995). When we don't see eye to eye: Discrepancies between supervisors and sub-ordinates in absence disciplinary procedures. Journal of Management, 21(2), 251-278. https://doi.org/10.1177/014920639502100205

Marsh, H. W., Balla, J. R., & Hau, K. T. (1996). An Evaluation of Incremental Fit Indexes: A Clarification of Mathemat-ical and Empirical Properties. In G. A. Marcoulides, & R. E. Schumacker (Eds.), Advanced structural equation model-ing techniques (pp. 315-353). Lawrence Erlbaum.

Martínez-Tur, V., Peiró, J.M., Ramos, J., & Moliner, C. (2006). Justice Perceptions as Predictors of Customer Satisfac-tion: The Impact of Distributive, Procedural, and Inter-actional Justice. Journal of Applied Social Psychology, 36(1), 100-119. https://doi.org/10.1111/j.0021-9029.2006.00005.x

Mengstie, M.M. (2020). Perceived organizational justice and turnover intention among hospital healthcare workers. BMC Psychology, 8, 19. https://doi.org/10.1186/s40359-020-0387-8

Meredith, W., & Teresi, J. A. (2006). An Essay on Measure-ment and Factorial Invariance. Medical Care, 44(11, Suppl 3), S69-S77. https://doi.org/10.1097/01.mlr.0000245438.73837.89

Moliner, C., Martínez-Tur, V., Peiró, J. M., Ramos, J., & Cro-panzano, R. (2005). Relationships between organizational justice and burnout at the work-unit level. International Journal of Stress Management, 12(2), 99–116. https://doi.org/10.1037/1072-5245.12.2.99

Muthén, L. K. & Muthén, B. O. (1998–2012). Mplus User’s Guide. Muthén & Muthén.

Ndjaboué, R., Brisson, C., & Vézina, M. (2012). Organisation-al justice and mental health: A systematic review of pro-spective studies. Occupational and Environmental Medicine, 69(10), 694-700. http://dx.doi.org/10.1136/oemed-2011-100595

Ponterotto, J.G. & Ruckdeschel, D.E. (2007). An overview of coefficient alpha and a reliability matrix for estimating adequacy of internal consistency coefficients with psy-chological research measures. Perceptual and Motor Skills, 105(3), 997-1014. https://doi.org/10.2466%2Fpms.105.3.997-1014

Rahim, M.A., Magner, N.R., & Shapiro, D.L. (2000). Do jus-tice perceptions influence styles of handling conflict with supervisors? What justice perceptions, precisely. International Journal of Conflict Management, 11(1), 9-31. https://doi.org/10.1108/eb022833

Reise S. P. (2012). Invited paper: the rediscovery of bifactor measurement models. Multivariate behavioral research, 47(5), 667-696. https://doi.org/1.1080/00273171.2012.715555

Rigdon, E. E. (1995). A necessary and sufficient identifica-tion rule for structural models estimated in practice. Multivariate Behavioral Research, 30(3), 359-383. https://doi.org/10.1207/s15327906mbr3003_4

Robbins, J. M., Ford, M. T., & Tetrick, L. E. (2012). Perceived unfairness and employee health: A meta-analytic inte-gration. Journal of Applied Psychology, 97(2), 235–272. http://dx.doi.org/10.1037/a0025408

Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016a). Evalu-ating bifactor models: Calculating and interpreting sta-tistical indices. Psychological methods, 21(2), 137-150. https://doi.org/10.1037/met0000045

Rodríguez, A., Reise, S., & Haviland, M. (2016b). Applying bifactor statistical indices in the evaluation of psycholog-ical measures. Journal of Personality Assessment, 98(3), 223-237. https://doi.org/1.1080/00223891.2015.1089249

Rodríguez Rosa, J. (2003). Los efectos de la satisfacción laboral, compromise organizacional, justicia organizacional, contrato psicológico y variables sociodemográficas en la manifestación de comportamientos relacionados a la ciudadanía organi-zacional [Unpublished master's thesis]. Pontifical Catho-lic University of Puerto Rico.

Rosario-Hernández, E. & Rovira Millán, L.V. (2006). Posibles predictores de las conductas de ciudadanía organi-zacional en una muestra de empleados en Puerto Rico. Revista Interamericana de Psicología Ocupacional, 25(2), 71-86.

Rosario-Hernández, E. & Rovira-Millán, L.M. (2007). Desar-rollo y validación de la Escala de Justicia Organizacional. Ciencias de la Conducta, 22(1), 61-78.

Rosario-Hernández, E. & Rovira Millán, L.V. (2011). La rela-ción entre las manifestaciones de conductas de ciuda-danía organizacional con los cinco factores de la person-alidad, satisfacción laboral, compromiso organziacional, justicia organizacional y contrato psicológico. Revista Puertorriqueña de Psicología, 22, 72-98.

Rosario-Hernández, E. & Rovira Millán, L.V. (2014). Posibles predictors de las conductas discrecionales en el lugar de trabajo en una muestra de personas empleadas en Puer-to Rico. Revista Puertorriqueña de Psicología, 25(2), 202-223.

Rosario-Hernández, E., Rovira Millán, L.V., & Merino-Soto, C. (2021). Review of the Internal Structure, Psychomet-ric Properties, and Measurement Invariance of the Work-Related Rumination Scale – Spanish Version. Fron-tiers in Psychology, 12. https://www.frontiersin.org/article/10.3389/fpsyg.2021.774472

Rosseel, Y. (2012). lavaan: An R Package for Structural Equa-tion Modeling. Journal of Statistical Software, 48(2), 1-36. https://www.jstatsoft.org/v48/i02/

Rupp, D. E., Shao, R., Jones, K. S., & Liao, H. (2014). The utility of a multifoci approach to the study of organiza-tional justice: A meta- analytic investigation into the consideration of normative rules, moral accountability, bandwidth-fidelity, and social exchange. Organizational Behavior and Human Decision Processes, 123(2), 159-185. http://dx.doi.org/10.1016/j.obhdp.2013.10.011

Skarlicki, D.P. & Folger, R. (1997). Retaliation in the work-place: The roles of distributive, procedural, and interac-tional justice. Journal of Applied Psychology, 82(3), 434-443. https://psycnet.apa.org/doi/10.1037/0021-9010.82.3.434

Spector, P. E. (1992). Summated rating scale construction: An introduction. SAGE Publications.

Stark, S., Chernyshenko, O. S., & Drasgow, F. (2006). Detect-ing differential item functioning with confirmatory fac-tor analysis and item response theory: Toward a unified strategy. Journal of Applied Psychology, 91(6), 1292-1306. https://doi.org/10.1037/0021-9010.91.6.1292

Sweeney, P.D. & McFarlin, D.B. (1997). Worker’s evaluations of the “ends” and the “means”: An examination of four models of distributive and procedural justice. Organiza-tional Behavior and Human Decision Processes, 55(1), 23-40. https://doi.org/10.1006/obhd.1993.1022

Thibaut, J. & Walker, L. (1975). Procedural justice: A psycholog-ical analysis. Lawrence Erlbaum Associates.

Tucker, L. R., & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38(1), 1-10. https://doi.org/10.1007/BF02291170

Vandenberg, R. J., & Lance, C. E. (2000). A review and syn-thesis of the measurement invariance literature: Sugges-tions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4-69. https://doi.org/10.1177/109442810031002

Wang, X., Liao, J., Xia, D., & Chang, T. (2010). The impact of organizational justice on work performance: Mediating effects of organizational commitment and leader-member exchange. International Journal of Manpower, 31(6), 660-677. https://doi.org/10.1108/01437721011073364

Wang, J., & Wang, X. (2012). Structural Equation Modeling: Applications using Mplus. John Wiley & Sons.

Welbourne, T.M., Balkin, D.B., & Gómez-Mejía, L.R. (1995). Gainsharing and mutual monitoring: A combined agen-cy-organizational justice interpretation. Academy of Man-agement Journal, 38, 881-899. https://doi.org/10.5465/256750

Zayas Ortiz, M. (2011). Relación entre las manifestaciones de ciudadanía organizacional con el compromiso organizacional, la justicia organizacional y la satisfacción laboral en una muestra de empleados de la banca privada [Disertación in-edita]. Universidad Interamericana de Puerto Rico, Cen-tro de Estudios Graduados en Desarrollo Empresarial y Gerencial, San Germán, Puerto Rico




Cómo citar

Rosario-Hernández, E., Rovira-Millán, L. V., Blanco-Rovira, R. A., & López-Iglesias, A. C. (2023). Examining the Internal Structure and Measurement Invariance of the Organizational Justice Scale in a Sample of Employees in Puerto Rico. Revista Caribeña De Psicología, 7(1), e8059. https://doi.org/10.37226/rcp.v7i1.8059



Artículos Originales

Artículos más leídos del mismo autor/a