Hypothesis: Do students who show higher overall life satisfaction and sense of meaning also show higher reading performance?

The following Data-Mining brief shows our team's process of selecting indicators from the PISA 2018 dataset, utilizing supervised and unsupervised machine-learning methods to provide foundational descriptive data analysis (that may be used and improved upon for potential future research). We included our iterative processes below, using the Python application Orange for exploratory data mining to serve our hypothesis.

Iteration 1: Variable selection based on substantive interpretations

We first selected variables based on our exploratory data analysis, selecting several indicators based on a review of research literature:

ST185Q01HA: Agree: My life has clear meaning or purpose.
ST185Q02HA: Agree: I have discovered a satisfactory meaning in life.
ST185Q03HA: Agree: I have a clear sense of what gives meaning to my life.
ST186Q05HA: Thinking about yourself and how you normally feel: how often do you feel as described below? Happy
EUDMO: Eudaemonia: meaning in life (WLE)
ST016Q01NA: Overall, how satisfied are you with your life as a whole these days?

The target outcome variable: READ: PISA Reading Scores

To begin our exploratory process, we refined our initial selection of two indicators of student purpose and meaning in life: “Eudaemonia” and “Life Satisfaction.” We believed that structuring our following analysis by selecting numeric predictors would result in more meaningful association analyses going forward.

Predictor Variables:

EUDMO “Eudaemonia”: Meaning in life (range: 1.0 - 60.0)
ST016Q1NA “Life Satisfaction”: Overall, how satisfied are you with your life as a whole these days? (range: 1.0 - 11.0)

Outcome Variable:

READ Reading Scores: Range 157.30 - 810.49; 50% bin median threshold = 504.84

Bivariate Classification: Exploratory Data Mining

We trained our learner model on several classification tests (Random Forest, Logistic Regression, kNN) to determine the highest number of correctly classified indicators of higher or lower PISA reading scores for various combinations of variables. Random Forest outputs yielded the highest proportions of “true” classified instances based on cross-validation using 10 folds.

Findings: Classification (Random Forest)

These outputs were satisfactory, though higher proportions of correctly classified items would have been preferable.

We then used Principal Components Analysis (unsupervised learning) to create a new factored predictor of our two selected numeric variables, based on the proportion of correctly-classified data from our previous Random Forest test. We used Principal Component 1 (PC1, which accounted for 75% of variance) and visualized the distribution of student Reading Scores based on their PC1 scores.

For our purposes, PC1 can be known as “life satisfaction.” Based on this visualization, it appears that students who score higher in Reading (red) also more frequently scored higher on PC1.

Iteration 2: Variable selection based on feature importance algorithms

[Note: These tests were utilized for exploratory purposes, and do not necessarily reflect a statistically robust process of accurate & precise key feature selection.]

To inform our selection of additional potential indicators, we again used Random Forest classification of over 500 PISA features to predict our reading scores (on a dichotomous scale). Of the 500 features, the Rank (kNN) widget indicated that 24 features yielded non-zero Random Forest outputs.

Of these 24 features, we selected:

BELONG: Subjective well-being, sense of belonging to school
MASTGOAL: Mastery Goal Orientation
GFOFAIL: General Fear of Failure

Limitations:

Some of the output values from Rank and Feature Importance widgets did not reveal a clear "cut-off" to indicate which indicators we should prioritize, and thus were excluded here.
The models used (particularly Random Forest) are "black boxes" – meaning we can't "see" the internal process of classifying variables. This makes it difficult to know if the outputs are valid or reliable.

Final Results

To conclude our process, we ran a final Principal Components Analysis using a combination of our generated (Iteration 2) and hand-selected (Iteration 1) indicators. We opted to include GFOFAIL as a potential indicator of “success,” though we acknowledge potential substantive arguments against the inclusion of this variable.

We first used our classification learning model to identify correctly-classified data. From this set, the classified data was modeled using PCA to create our final visualization.

The final variables selected were as follows:

EUDMO: Eudaemonia: meaning in life (WLE)
ST016Q01NA: Overall, how satisfied are you with your life as a whole these days?
BELONG: Subjective well-being, sense of belonging to school

MASTGOAL: Mastery Goal Orientation
GFOFAIL: General Fear of Failure

We opted to use the Random Forest classification model based on highest proportions of correctly-classified features (compared to outputs calculated using Logistic Regression). Then, our PCA was run on correctly classified features:

We used Principal Component 1 (PC1, which accounted for 43% of variance) and visualized the distribution of student Reading Scores based on their PC1 scores.

For our purposes, PC1 can be known as “life satisfaction.”

Based on this visualization, it appears that students with “higher” Reading scores (red) constituted a higher proportion of those who scored the highest in “life satisfaction” (PC1). The relative distributions for the remaining “life satisfaction” values seemed relatively uniform. Additionally, the probability of having a “lower” Reading scores (blue) was most concentrated at the lowest “life satisfaction” score.

Limitations:

As mentioned previously, the Random Forest model is a "black box." This becomes highly relevant when considering the findings re: our Predictions tests. It is unclear how our models generated these predicted outputs.
Training and test datasets were not isolated; it is uncertain how this may have produced skewed, biased, or incorrect outputs.

PISA 2018: Reading Scores and "Life Satisfaction" - Educational Data Mining