Below we state how we planned and preregistered to analyze the data in more detail.

Factorial validity

Preregistered procedure

We will inspect the factorial validity of the privacy and personality variables as follows.

  • We run a model with all personality and privacy variables as we originally configured them in a confirmatory factor analysis.
  • If model fit is below the criteria, we will first inspect modification indices, potentially allowing covariances or cross-loadings if theoretically plausible.
  • If these changes do not yield sufficient fit, we will drop malfunctioning items, while having at least three items per dimension/facet.
  • If this should not work out, we will run Exploratory Structural Equation Modelling using oblimin rotation, allowing all personality items to load on the same factors, while allowing all privacy items to load on the same factors.
  • If fit is still subpar, we will conduct in-depth exploratory factor analyses (EFA) to assess the underlying factor structure.
  • EFAs will be run using maximum likelihood estimation and oblimin rotation (Osborne and Costello 2004, 7).
  • If more than one dimension is revealed, we will implement bifactor model solutions.1
  • Bifactor models retain a general measure of the variable, without introducing novel potentially overfitted subdimensions.
  • If no adequate bifactor model can be found, we will proceed by deleting items with low loadings on the general factor and/or the specific factors.
  • If also after deletion of individual items no bifactor solution should emerge, we will use a subset of the items to extract a single factor with sufficient factorial validity.

Implementation and Deviation

When analyzed individually, most measures showed satisfactory model fit, not requiring any changes. Some measures showed satisfactory model fit after small adaptions, such as allowing items to covary. Half a dozen measures were then still marginally outside of the preregistered thresholds (e.g., liveliness with a RMSEA of .12). We could’ve now substantially altered the factor structure to see if this leads to sufficient model fit. However, we considered it more cogent to rather accept this then to fundamentally alter scales.

We did not explicate a minimum thresholf of reliability. Most measures showed satisfactory results (i.e., reliability above .70). However, some measures such as altruism, unconventionality, or anonymity showed insufficient reliability. Again, instead of strongly adapting measures (as suggested above), we decided to maintain the initial factor structure and did not delete any items and we did not introduce substantial changes to the factors.

Analysis

Preregistered Procedure

Because the analyses are complex, it might be that we need to simplify the model. We will proceed as follows.

  • Instead of a fully latent structural regression model, we will then conduct a partially latent structural regression model, in which the predictor variables will be modeled as single indicators while controlling for measurement error (Kline 2016, 214).
    • To get high-quality single indicators of the predictors, we will compute the average of the model predicted values/latent factor scores, which can be extracted from the CFAs.
    • If the CFAs show a unidimensional solution, we will use the model predicted values for this latent factor; if the CFAs produce a multidimensional solution, we will use the model predicted values for the general latent factor.

Implementation and Deviation

Although individually most of the measures showed good fit, when analyzed together fit decreased substantially, below acceptable levels. This problem maintained when trying to model the results using single indicator of the predictors with factor scores. As a result, we conservatively decided to analyze our data using the variables’ observed mean scores.

Kline, Rex B. 2016. Principles and Practice of Structural Equation Modeling. 4th ed. Methodology in the Social Sciences. New York, NY: The Guilford Press.
Osborne, Jason W., and Anna B. Costello. 2004. “Sample Size and Subject to Item Ratio in Principal Components Analysis.” Practical Assessment, Research & Evaluation 9 (11).

  1. Bifactor models feature one factor that explains the variance in all items (the so-called general factor or g-factor). In addition, at least two additional factors are implemented that explain the variance in a subset of the items. The general factor and the specific factors are orthogonal. Bifactor models are nested within hierarchical models. For more information on bifactor models, see Kline (2016), p. 319. Note that we will not specify a bifactor model of all items measuring need for privacy, because we are explicitly interested in the relations between the personality facets and the respective dimensions of need for privacy.↩︎