Preliminary Analysis	Procedure	Separate	Simultaneous
Step 1: Exploring factor structure	When the initial structure is unknown, one should conduct exploratory analysis treating all observations as independent, that is, ignoring the nested data structure. It should be recognized that the standard errors and fit statistics may be biased because cross-level dependencies are not modeled. The aim is to determine the number of factors and preliminary data structure. This step may be skipped if researchers are seeking to test a specific factor structure.		Step 1
Step 2: Indicator intraclass correlation coefficient (ICC)	Estimate the ICC for each item. The ICCs represent the variance attributable to between-group differences. If the ICCs are small, there may not be a need to conduct a multilevel factor analysis and there is little basis for isomorphism. As a rule of thumb, ICCs greater than .05 indicate reasonable amount of variability across groups (Dyer, Hanges, & Hall, 2005).
Step 3: Ascertaining multilevel structure	Determine whether a multilevel model fits better than a levels-free model (in practice, this step is usually combined with the test of strong configural isomorphism).
Testing Isomorphism	Procedure	Separate	Simultaneous
Weak configural isomorphism: Same number of factors holds between levels. Are the numbers of descriptive dimensions equivalent across levels?	Factor retention techniques applied only to higher-level units for referent-shift composition and compared to number of factors for self-referent lower-level units. For lower-level data, there are two ways of obtaining the number of factors. One can run factor analysis (a) ignoring nested structure or (b) on the pooled within-groups covariance matrix. Dimensions are generally shown to be indexed by similar indicators across levels without requiring specific loadings to be fixed at zero.	Step 1
Strong configural isomorphism: Same number of factors holds across levels and the pattern of zero and nonzero factor loadings holds between levels. Does the factor structure of descriptive dimensions hold across levels?	Separate estimation: Confirmatory factor analysis applied to higher-level units for referent-shift composition; compared to confirmatory factor analysis for self-referent lower-level units. Standard goodness-of-fit indices at each level are examined. For lower-level data, one can run confirmatory factor analysis (a) ignoring nested structure, (b) on the pooled within-groups covariance matrix, or (c) using multilevel confirmatory factor analysis. Simultaneous estimation: One can examine the fit of a factor model with fixed zero and nonzero loadings across levels. This is applicable only when there is more than one factor.	Step 2	Step 2
Weak metric isomorphism: Relative ordering of factor loadings/item discriminations holds between levels. Are descriptive dimensions characterized by construct indicators in a similar manner across levels?	Application of congruence coefficient between loadings obtained from higher- and lower-level units. Based on preliminary simulations, we recommend that the rank-order correlation between factor loadings larger than .50 indicates weak metric isomorphism.6	Step 3	Step 3
Strong metric isomorphism: Magnitude of factor loadings/item discriminations holds between levels. This is a stronger assumption than weak metric isomorphism of having similar rank-ordering. The multilevel factor has variability divided into a within component and a between component; therefore we can dimensionalize individuals and groups in the same way. Are descriptive dimensions dimensionalized in a similar manner across levels?	In FA or IRT, one would constrain the loadings to be equal across levels. This means that within-group and between group covariances are proportional across levels. The proportionality is the ratio of the factor standard deviations. A better fit for the constrained model (compared to the unconstrained model) would indicate strong metric isomorphism.		Step 4
*Special case: Weak/strong metric isomorphism for comparable factors: Different numbers of factors hold across levels but metric isomorphism occurs between some factors. Is there a correspondence among construct indicators in some descriptive dimensions across levels?	Weak metric isomorphism: For comparable factors, one may obtain the congruence coefficient between loadings from higher- and lower-level units. Strong metric isomorphism: For comparable factors, one may constrain the loadings to be equal across levels. We recommend examining the fit of the constrained and constrained model. A better fit for the constrained model would indicate strong metric isomorphism for the comparable factors.	*If needed, one can test for weak metric isomorphism	*If needed, one can test for both weak and/or strong metric isomorphism