Ex) Article Title, Author, Keywords
Ex) Article Title, Author, Keywords
2021; 31(3): 299-320
Published online August 31, 2021 https://doi.org/10.29275/jerm.2021.31.3.299
Copyright © Korea Society of Education Studies in Mathematics.
Correspondence to:†Jihyun Hwang, jihyun-hwang@kangwon.ac.kr
ORCID: https://orcid.org/0000-0003-3107-7142
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0), which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
We examined the relationships between mathematics achievement and collaborative problem solving, which has been emphasized as the 21st century skills. Focusing on moderating effects of attitude toward working with others, we analyzed the PISA 2015 data for binational comparison between Korean and the U.S. Random-intercept linear models - where the intercepts vary across schools while slopes are fixed across schools - were applied to the data including 5,545 students in Korea and 5,564 in the U.S. The findings showed the positive linear relationships between mathematics achievement and collaborative problem solving as expected, but the slope was remarkably higher in the Korean model. However, we found no significant moderating effects of attitude toward working with others on the relationship between mathematics achievement and collaborative problem solving. Particularly, students in Korea and the U.S. are likely to have similarly low scores in mathematics achievement when their collaborative problem solving scores are low. However, Korean students would have even higher mathematics achievement scores if students in Korea and the U.S. have similarly high collaborative problem solving scores. The findings suggest the necessity for further scrutinies on attitude toward working with others and research on teaching and learning the 21st century skills in an integrated way.
Keywordscollaborative problem solving, mathematics achievement, PISA 2015, International Comparison
Mathematics educators have increasingly emphasized mathematical competency in the Fourth Industrial Revolution for enriched lives integrated with advanced technology. Firsthand experiences of rapid changes in the new era advise that learning mathematics is critical to understand and use new technology like machine learning and artificial intelligence. However, governments and academic organizations have identified only domain-general competencies required in the 21st century, which requires reframing mathematical competency for the 21st century. Bridging the 21st century competency (domain-general) and mathematics competency (domain-specific), students could be offered best opportunities to be prepared for the current and next generations through learning mathematics. Particularly, we are interested in connecting learning mathematics to collaborative problem solving, which both the World Economic Forum (2015) and the National Research Council (2012) suggested as interpersonal competencies for the 21st Century.
Collaborative problem solving is one of the essential skills that is projected to be essential skill in developing societies and workforce. Collaborative problem solving refers to “the capacity of an individual to effectively engage in a process whereby two or more agents attempt to solve a problem by sharing the understanding and effort required to come to a solution and pooling their knowledge, skills and efforts to reach that solution” (OECD, 2017, p. 134). By its definition, collaborative problem solving involves two aspects: the cognitive aspect of problem solving and social components aspect of collaboration. In disciplinary learning, including mathematics, collaborative problem solving is understood not only as a skillset that young learners are expected to develop through K-12 and higher education curriculum, but also as a way of learning subject materials in K-12 classrooms. When engaged in collaborative problem-solving activities, students can learn disciplinary concepts and practices through explaining concepts to peers and communicating their ideas while solving problems together (Langer-Osuna, 2016; Moss & Beatty, 2006). In mathematics classrooms, students naturally interweave mathematical terms, numbers, and symbols in their interactions during collaborative problem solving (Moss & Beatty, 2006).
As a way of learning mathematics, activities that involve collaborative problem solving can be a setting for an activity where students learn mathematical knowledge and practices. For example, collaborative problem solving, as a setting for learning activities, has been used to support student learning of mathematical concepts and practices. Research studies have examined how students can engage in collaborative problem solving activities to learn mathematical concepts and practices, such as patterning in early algebra (Moss & Beatty, 2006), middle school students’ mathematical generalization (Ellis, 2011), and college calculus (Reisel et al., 2014). In turn, mathematics is one of the curricular subjects that provide students with opportunities to learn knowledge and thinking that contribute to the development of collaborative problem solving. For instance, the standards for mathematics practices, including those (National Governors Association Center for Best Practices [NGACBP] & Council of Chief State School Officers [CCSSO], 2010) suggest that students can communicate their mathematical ideas with others and that they can also understand and compare others’ problem-solving approaches. Such ways of learning mathematics can foster students’ competence in collaborative problem solving. This relationships between mathematics learning and collaborative problem solving can be thought of as bidirectional and cyclical (see
As seen in
With such perspective of collaborative problem solving and learning mathematics, we explored in a descriptive manner how collaborative problem solving is related to mathematical competence. Specifically, we investigated the relationship between mathematics achievement - an outcome of learning mathematics - and collaborative problem solving as measured in the Program for International Student Assessment (PISA) 2015. Considering that collaborative problem solving can be a way of learning mathematics (Ellis, 2011; Moss & Beatty, 2006; Reisel et al., 2014), it would be expected to see a positive relationship between mathematics achievement and collaborative problem solving. However, empirical evidence for this relationship has not been well-documented. Thus, the purpose of this study is to provide a broad picture of the relationship between mathematics achievement and collaborative problem solving through secondary analysis with the PISA 2015 data.
For the research purpose, we focused on (1) moderating effect of attitude toward working with others and (2) binational comparison between South Korea and the U.S. First, we argue the necessity to include additional factors in analysis for better understanding of the relationship. We tested moderating effects of attitude toward cooperation, particularly valuing and enjoying cooperation1) measured in the PISA 2015. Considering problem-solving component of collaborative problem solving (Hesse et al., 2015; OECD, 2017a), attitudes toward cooperation could have significant roles in students’ development of collaborative problem solving and its connection to other skills such as mathematics achievement. However, there are few studies that explore the role of attitude toward cooperation in relation to collaborative problem-solving ability or to mathematics competence.
Second, this secondary-analysis research is binational comparison between South Korea and the U.S. The primary rationale for binational comparison is to take cultural difference into consideration in studying the relationship between mathematics achievement and collaborative problem solving. From the results of PISA 2015, both South Korea and the U.S. performed above the OECD average in collaborative problem solving, while their mathematics achievement results were significantly different. This indicates possible difference in the relationships between mathematics achievement and collaborative problem solving that is from different cultures of the two countries. Considering the social aspect of collaborative problem solving, cultural differences may be an important factor that impact the relationship between mathematics achievement and collaborative problem solving (Forman & McPhail, 1996; OECD, 2017). Thus, binational comparison between South Korea and the U.S. helps us describe how cultural differences are possibly related to mathematics achievement and collaborative problem solving.
Collaborative problem solving refers to “a joint activity where dyads or small groups execute a number of steps in order to transform a current state into a desired goal state (Hesse et al., 2015). Though problem solving ability somewhat relies on one’s domain knowledge (Funke & Frensch, 2007), collaborative problem solving ability refers to a domain-general ability that enable an individual effectively to engage in collaborative problem-solving activities across different subject domains (OECD, 2017). Answering to shifting requirements of workplace and society, educators in K-12 and higher education have been paying attention to identification of relevant 21st century skills and the ways of teaching and learning these skills. As one of the 21st century skills, collaborative problem solving has been emphasized by employers and educators as a skill to be taught and developed, rather than a skill that develops naturally through everyday life (Hesse et al., 2015). Accordingly, educational research and practices have come to pursue the development of collaborative problem solving as an educational goal (Gasser, 2011; Larson & Miller, 2011; Lloyd, 1999). Developing competence in collaborative problem solving is important for students not only for future workplace capacity, but also for them to be productive learners in K-12 education. With more awareness of the importance of collaborative problem solving, researchers have been attempting to define collaborative problem-solving ability as a teachable and measurable concept (Hesse et al., 2015; Rosen & Foltz, 2014). Particularly, problem solving skills and knowledge acquired through mathematics learning may manifest in other problem situations including collaborative problem solving with domain-general content. This is based on the theoretical assumption that conceptual knowledge and problem-solving skills that students use and learn in mathematics are advantageous to the development of collaborative problem solving and vice versa.
Teaching and evaluating students’ collaborative problem solving involve two aspects that reflect the nature of the concept: a cognitive aspect of problem solving and a social aspect of collaboration (Andrews-Todd & Forsyth, 2020; Graesser et al., 2018). According to the 21st century skills framework of Binkley et al. (2012), problem solving is one of the ways of thinking, while collaboration is one of the ways of working. The PISA framework of problem solving defines the problem solving aspect of collaborative problem solving by identifying elements of problem solving: problem as a discrepancy between current state and goal state, problem space as a mental representation of the problem states and of the steps to close the discrepancy, a plan for steps to approach nearer to the goal state, execution of the steps, and monitoring and reflecting the problem-solving processes (OECD, 2017). The difference between collaborative and individual problem solving is that these processes are often internal in individual settings, while problem-solving processes become observable and communicative in collaborative problem solving. When engaging in collaborative problem solving, participants exchange and share their thoughts to identify problem space, steps to go through, and monitoring of executing the steps.
The aspect of collaboration describes the process of collaborative problem solving where members of a group need to be able to collaborate with one another in order to coordinate problem-solving processes among multiple agents. Collaboration includes communication (the exchange of knowledge and opinions), cooperation (an agreed division of work), and responsiveness for active and insightful participation (Hesse et al., 2015). Based on these elements of collaboration, collaborative skills that students are expected to develop are specified as establishing and maintaining shared understanding, taking appropriate action to solve the problem, and establishing and maintaining team organization (OECD, 2017). Though the ability of collaborative problem solving is argued to be domain-general, tasks and activities for collaborative problem solving can be content-rich, involving specific concepts and practices learned in different subject domains including mathematics.
Collaborative problem solving is understood not only as a critical ability for complex work and team collaboration, but also as a way for students to learn subject materials in K-12 classrooms (Graesser et al., 2018). As noted, collaborative problem solving involves the cognitive aspect of problem solving and the social aspect of collaboration. The cognitive aspect of collaborative problem solving is thinking processes of problem solving, and problem solving is one of the ways of thinking mathematically (Binkely et al., 2012; Schoenfeld, 1992). Considering both aspects of collaborative problem solving, problem solving and collaboration, we argue that collaborative problem-solving and mathematics learning are interconnected in several ways: 1) learning mathematics can develop students’ competence in collaborative problem solving, 2) collaborative problem solving activities have been implemented as a way of learning mathematics (e.g., Harding et al., 2017; Hurme & Järvelä, 2005; Sears & Reagin, 2013), and 3) the content of mathematics has been utilized to design collaborative problem-solving tasks (Care et al., 2015).
First, learning mathematical concepts and practices can contribute to the development of collaborative problem-solving ability (Gasser, 2011; Larson & Miller, 2011). This is based on the problem-solving aspect of collaborative problem solving that is prevalent in mathematics problem solving. Mathematics problem solving frameworks identify cognitive processes of problem solving as sequential phases (Schoenfeld, 1983). Phases of mathematics problem solving include reading, analysis, exploration, planning/implementation, and verification. Identifying these phases of mathematics problem solving allows for assessing students’ problem solving skills, teaching students how to approach a problem, and supporting students to use problem solving strategies in a wider range of problem situations (Wu & Adams, 2006). Likewise, the problem solving component of collaborative problem solving involves similar processes: explore and understand the problem, represent and formulate the problem situation, plan and execute, and monitor and reflect (OECD, 2017a). Either individual or collaborative, understanding problem situations, planning problem solving steps, and monitoring and reflecting on those steps are critical cognitive processes for both collaborative problem solving and mathematical problem solving.
Secondly, engaging collaborative problem-solving process can be a way of learning mathematical concepts and practices. In mathematics classrooms, collaborative activities were found to productively support student engagement in cognitive processes of problem solving and their learning of mathematical concepts and practices cooperatively with peers (National Council of Teachers of Mathematics [NCTM], 2000). For this reason, collaborative problem-solving tasks have been implemented for learning mathematics (e.g., Harding et al., 2017; Hurme & Järvelä, 2005; Sears & Reagin, 2013). Specifically, communication, which is one of the elements of collaboration, is the core practice that enables students to share mathematical ideas and to experience essential process for developing students’ understanding of mathematical concepts through participating in small-group or classroom discussion (Hatano & Inagaki, 1991). Moreover, it is critical for students to have opportunities to participate in dialogical interactions with classmates and instructors, because those dialogical interactions are critical for students to learn how to make and refine their arguments and gleam the perspective of others (see Hershkowitz et al., 2007; Lampert, 1990). Negotiating ideas through dialogical interactions in discussions enables students to generate their own knowledge and ideas instead of replicating knowledge taught by teachers and textbooks (Ford & Forman, 2006). In essence, collaboration, communication, and dialogical interactions are critical processes in mathematics learning as well as in collaborative problem solving.
Finally, mathematics provides content for designing collaborative problem-solving tasks. Collaborative problem-solving tasks are categorized as content-free or content-rich (Care et al., 2015). Content-rich, or content-dependent, tasks employ skills and knowledge that is drawn from curriculum-based learning, whereas content-free tasks mainly require application of reasoning, not demanding prerequisite knowledge taught in curriculum-based subjects (Care et al., 2015). According to this categorization, tasks that involve mathematical concepts and situations are content-rich tasks. As most disciplinary curricula, concepts and knowledge in mathematics become the content basis for the problems to be solved collaboratively. Those content-rich tasks are presented to students to measure students’ collaborative problem-solving ability as well as develop the skills that collaborative problem-solving ability entails (Poon et al., 2015).
Cultural aspects could also influence engagement in collaboration as sociocultural views on the value of collaboration could orient student collaborative problem-solving development. Different cultures provide unique opportunities for students to observe and participate in essential sociocultural practices, including economic, political, instructional, and recreational activities (Forman & McPhail, 1996). Participating in these activities, young individuals internalize their affective, social, and intellectual meaning, and this represents a dialectic relation between cultural practice and individual experience (Cobb et al., 1996; Forman & McPhail, 1996). In the same vein, students from different cultural context may have different norms of and attitudes toward collaborative activities. For example, in a competitive situation as those of East Asian countries, students might believe that learning individually is more important for high-stake assessments (Zhu et al., 2009). However, cultural characteristics such as how taking initiative is valued and taking different roles during collaborative work were not considered to be different across different countries in the assessment of collaborative problem solving in PISA 2015 (OECD, 2017a).
Because the problem solving component is the core cognitive ability that comprises collaborative problem solving, it can be hypothesized that mathematical problem-solving ability is positively related to collaborative problem-solving ability. However, this has not been empirically shown in previous research, for which one of the reasons is that large-scale, standardized assessment of collaborative problem solving is a relatively new test administered in PISA 2015.
In this study, the relationships between mathematics achievement and collaborative problem solving are scrutinized using the PISA 2015 data of South Korea and the U.S. The PISA 2015 included collaborative problem solving as one of its subject areas, providing researchers with a unique opportunity to study collaborative problem solving as a measurable construct. Because it is almost impossible to evaluate human-to-human interactions in a large-scale assessment, the assessment of collaborative problem solving in the PISA 2015 is realized as a computer-based assessment. This approach uses programming computer agents to control external factors in collaborative contexts, including problem situations and actions of virtual team member(s) with whom a test taker collaborates (OECD, 2017a). Based on the characteristics of the assessment, it is expected that the assessment of collaborative problem solving measures a distinctive construct that captures additional information about students’ learning that is hardly captured by existing achievement indices. Therefore, as a type of an individual’s competence, collaborative problem solving is assumed to be related to other intellectual measures that involve individuals’ problem-solving abilities, such as mathematical literacy.
Like the previous conceptualization of collaborative problem solving, the framework in the PISA 2015 conceptualized collaborative problem solving as a two-dimensioned competence that integrates individual problem-solving processes and collaboration processes (OECD, 2017a). In its framework, the PISA 2015 specifically emphasizes the skills that constitutes collaborative work such as group thinking and communication skills based on the PISA 2012’s framework of problem solving. Given that collaboration comprises one of two facets of collaborative problem solving, it is assumed that collaborative problem solving is influenced not only by the intellectual ability of students, but also by their perceptions of working with others. Hence, the PISA 2015 data on students’ attitude toward collaborative activities, such as individuals valuing collaborative problem solving and enjoying cooperation, is one of the key factors that impact how one engages in collaborative processes.
In this study, we investigated the relationships between mathematics achievement and collaborative problem solving in the PISA 2015 data of South Korea and the U.S students. In doing so, we examined the role of attitude toward collaborative activities in the relationships between mathematics achievement and collaborative problem solving.
The main question guiding this research is, what is the relationship between mathematics achievement and collaborative problem solving in the PISA 2015 data of South Korea and the U.S.? With this overarching question, we investigated the following sub-questions in this study:
Do relationships between mathematics achievement and collaborative problem solving differ in the PISA 2015 data of Korean and U.S. students?
Are the relationships between students’ mathematics achievement and collaborative problem solving moderated by how students value/enjoy cooperation?
Does the moderating effect of valuing and enjoyment of cooperation differ in the South Korean and U.S. data?
This research investigated the relationship between mathematics achievement (MATH), collaborative problem solving (CLPS), and attitudes toward working with others (enjoying cooperation and valuing collaborative problem solving) using the PISA 2015 data. We merged data of South Korea and the U.S. to address the research questions. We applied multilevel modeling, considering the nature of data structure of the PISA 2015.
We collected South Korean and U.S. students’ data from the PISA 2015 international database. The original data include 5,581 Korean students across 168 schools and 5,712 U.S. students across 176 schools. The target population of the PISA assessment is students who were aged between 15 years and 3 months and 16 years and 2 months and attended an educational institution with grades 7 and higher. We acknowledged that some of the sample students did not complete the student surveys, and those students did not have available scores of attitudes toward cooperation and collaborative problem solving (COOPERATIVE and CPSVALUE). Those students missing scores of attitudes toward cooperation and collaborative problem solving take up 0.6% in Korean data and 2.6% in the U.S. data. Given that the proportions of students missing those attitudes variables in the entire sample are very small, we determined deleting the data missing some of the variables should have little impact on overall patterns in the findings in this study. As a result, the final numbers of students in the data were 5,545 for Korea and 5,564 for the U.S.
We collected 10 sets of plausible values (PV1MATH-PV10MATH) representing students’ mathematical literacy scores (MATH) and 10 sets of plausible values (PV1CLPS-PV10CLPS) representing CLPS respectively from the PISA 2015 database. These two collections of the plausible values were constructed by assessment developers for the optimal representation of population estimates. The results of this research are based on the 100 (10 by 10) different pairs of the plausible values of MATH and CLPS, following the guidelines provided by Chaney et al. (2001). First, we estimated all coefficients in the multilevel model and their error variances using all pairs of MATH and CLPS. After all pairs of plausible values were analyzed, we computed the final estimate (labeled with t*) of all coefficients and the standard error of t* (labeled with SE(t*)). In the result section, we will report the average of the error variances (labeled with U*) and the variations among all estimates of the coefficients (labeled with B), which were used to calculate SE(t*).
We also collected the data of students’ COOPERATE and CPSVALUE, each of which was constructed by applying the item response theory (IRT) to the statements as seen in
Table 1
PISA 2015 variable code | Statement |
---|---|
Question: To what extent do you disagree or agree with the following statements about yourself? | |
COOPERATE | |
ST082Q02 | I am a good listener. |
ST082Q03 | I enjoy seeing my classmates be successful. |
ST082Q08 | I take into account what others are interested in. |
ST082Q12 | I enjoy considering different perspectives. |
CPSVALUE | |
ST082Q01 | I prefer working as part of a team to working alone. |
ST082Q09 | I find that teams make better decisions than individuals. |
ST082Q13 | I find that teamwork raises my own efficiency. |
ST082Q14 | I enjoy cooperating with peers. |
We applied multilevel modeling, particularly random-intercept linear modeling to the Korean and U.S data, where the intercepts vary across schools while slopes are fixed across schools. We determined to implement the random-intercept modeling with consideration of the two-stage stratified sampling design in the PISA 2015. Another rationale for multilevel analysis was intraclass correlation coefficients (ICCs) greater than 0.25 (Kim et al., 2009). ICCs from the first (PV1MATH) to the tenth set of plausible values (PV10MATH) were 0.296, 0.295, 0.313, 0.307, 0.313, 0.304, 0.296, 0.304, 0.303, and 0.299, whose average was 0.303. In the multilevel regression model, random slopes were not considered because there was no statistical difference in the slopes across the schools. In other words, including random slopes in the model could cause collinearity issues. All results were weighted by W_FSTUWT in the PISA 2015 database.
To include moderation effects of COOPERATE and CPSVALUE respectively and examine differences in the models between Korea and the U.S., we included three-way interaction terms in the full multilevel model. We used the R package
1) Model including moderation of COOPERATE
2) Model including moderation of CPSVALUE
where MATH
After the regression analysis, we calculated and visualized moderating effects of COOPERATE or CPSVALUE by fixing its values in each country model. We chose values of –1.5, 0, and 1.5 representing 1.5 standard deviation below average, average, and 1.5 standard deviation above average. By doing this, the results will show the relationships between MATH and CLPS of students who have different degrees of enjoying or valuing collaborative problem solving.
Table 2
Variable | Korea (Z=1) | United States (Z=0) | |||
---|---|---|---|---|---|
Weighted average | Standard deviation | Weighted average | Standard deviation | ||
COOPERATE | –0.017 | 0.938 | 0.127 | 1.002 | |
CPSVALUE | 0.142 | 0.910 | 0.056 | 1.026 | |
PV1CLPS | 537.800 | 83.079 | 521.554 | 107.077 | |
PV2CLPS | 539.456 | 85.365 | 524.532 | 106.423 | |
PV3CLPS | 538.610 | 82.942 | 523.859 | 109.509 | |
PV4CLPS | 536.848 | 82.728 | 523.870 | 107.815 | |
PV5CLPS | 538.731 | 84.075 | 520.640 | 105.874 | |
PV6CLPS | 537.490 | 84.474 | 523.832 | 107.336 | |
PV7CLPS | 538.095 | 84.774 | 524.185 | 106.248 | |
PV8CLPS | 537.892 | 83.668 | 522.931 | 107.472 | |
PV9CLPS | 539.101 | 83.465 | 523.231 | 106.905 | |
PV10CLPS | 541.069 | 83.920 | 522.847 | 106.339 | |
PV1MATH | 524.550 | 99.794 | 472.756 | 87.876 | |
PV2MATH | 523.487 | 99.487 | 472.711 | 88.785 | |
PV3MATH | 525.199 | 98.732 | 470.653 | 88.132 | |
PV4MATH | 526.082 | 99.272 | 470.519 | 88.148 | |
PV5MATH | 523.703 | 100.876 | 472.094 | 86.772 | |
PV6MATH | 522.861 | 99.938 | 471.433 | 86.589 | |
PV7MATH | 523.708 | 99.165 | 471.195 | 88.064 | |
PV8MATH | 522.967 | 99.650 | 472.833 | 86.535 | |
PV9MATH | 526.155 | 99.986 | 471.670 | 87.870 | |
PV10MATH | 523.270 | 100.137 | 472.795 | 89.288 |
Because we applied multilevel analysis using 100 pairs of MATH and CLPS plausible values to find the final estimates (t*) of regression coefficients, it is impossible to report all results in detail due to limited space. We provide results using the first sets of plausible values of MATH and CLPS as examples (see
Table 3
Fixed Effect | Estimate | Std. Error | df | t | Pr(>|t|) |
---|---|---|---|---|---|
Intercept, β0 | 161.100 | 3.566 | 2409 | 45.174 | <0.001 |
PV1CLPS, β1 | 0.596 | 0.006 | 11070 | 103.025 | <0.001 |
COOPERATE, β2 | 1.895 | 2.815 | 10840 | 0.673 | 0.501 |
Z, β3 | –71.690 | 10.560 | 7760 | -6.788 | <0.001 |
PV1CLPS×COOPERATE, β4 | –0.005 | 0.005 | 10840 | -0.973 | 0.331 |
PV1CLPS×Z, β5 | 0.212 | 0.019 | 10510 | 11.238 | <0.001 |
COOPERATE×Z, β6 | 13.680 | 9.576 | 10960 | 1.429 | 0.153 |
PV1CLPS×COOPERATE×Z, β7 | –0.018 | 0.018 | 10950 | -1.007 | 0.314 |
Random effect | Variance | Std. Dev | |||
Intercept, | 595.7 | 24.41 | |||
Residual, | 1058935.4 | 1029.05 |
Table 4
Fixed Effect | Estimate | Std. Error | df | t | Pr(>|t|) |
---|---|---|---|---|---|
Intercept, β0 | 164.300 | 3.555 | 2348 | 46.217 | <0.001 |
PV1CLPS, β1 | 0.589 | 0.006 | 11070 | 103.046 | <0.001 |
CPSVALUE, β2 | 6.749 | 2.840 | 10860 | 2.377 | 0.018 |
Z, β3 | –79.780 | 10.520 | 7641 | –7.585 | <0.001 |
PV1CLPS×CPSVALUE, β4 | –0.020 | 0.005 | 10850 | –3.906 | <0.001 |
PV1CLPS×Z, β5 | 0.228 | 0.019 | 10480 | 12.103 | <0.001 |
CPSVALUE×Z, β6 | 17.170 | 10.100 | 10960 | 1.700 | 0.089 |
PV1CLPS×CPSVALUE×Z, β7 | –0.025 | 0.019 | 10950 | –1.327 | 0.185 |
Random Effect | Variance | Std. Dev | |||
Intercept, | 602.7 | 24.55 | |||
Residual, | 1052086.4 | 1025.71 |
Table 5
t* | U* | B | V(t*) | SE(t*) | |
---|---|---|---|---|---|
Intercept β0 | 180.427 | 3.773 | 59.126 | 63.491 | 7.968 |
CLPS β1 | 0.554 | 0.006 | 0.000 | 0.006 | 0.080 |
COOPERATE β2 | 5.043 | 2.979 | 7.919 | 10.977 | 3.313 |
Z β3 | –57.884 | 11.150 | 92.283 | 104.356 | 10.215 |
CLPS×COOPERATE β4 | –0.008 | 0.006 | <0.001 | 0.006 | 0.075 |
CLPS×Z β5 | 0.190 | 0.020 | <0.001 | 0.020 | 0.142 |
COOPERATE×Z β6 | 4.470 | 10.225 | 32.322 | 42.871 | 6.548 |
CLPS×COOPERATE×Z β7 | –0.005 | 0.019 | <0.001 | 0.019 | 0.138 |
Note: t* represents the final estimate of the coefficients. U* is the average of the error variances and B indicates the variations among all estimates of the coefficients. V(t*) and SE(t*) are the error variance and standard error of t*.
Table 6
t* | U* | B | V(t*) | SE(t*) | |
---|---|---|---|---|---|
Intercept β0 | 182.722 | 3.761 | 56.206 | 60.529 | 7.780 |
CLPS β1 | 0.550 | 0.006 | 0.000 | 0.006 | 0.079 |
CPSVALUE β2 | 6.481 | 2.986 | 7.028 | 10.085 | 3.176 |
Z β3 | –63.285 | 11.054 | 98.067 | 110.101 | 10.493 |
CLPS×CPSVALUE β4 | –0.020 | 0.006 | <0.001 | 0.006 | 0.074 |
CLPS×Z β5 | 0.201 | 0.020 | <0.001 | 0.020 | 0.142 |
CPSVALUE×Z β6 | 7.178 | 10.703 | 35.796 | 46.857 | 6.845 |
CLPS×CPSVALUE×Z β7 | –0.006 | 0.020 | <0.001 | 0.020 | 0.140 |
Note: t* represents the final estimate of the coefficients. U* is the average of the error variances and B indicates the variations among all estimates of the coefficients. V(t*) and SE(t*) are the variance and standard error of t*.
Using t* in
To answer the research question on binational comparison, we found that the slopes of Korean linear models are higher than those of the U.S. models. This finding indicates that Korean students have more increase in MATH as CLPS increases by 1. In specific, when Korean and U.S. students are in the range of lower CLPS scores (around 300), students in both countries are likely to have similar MATH scores. When students have CLPS scores above 500, Korean students are likely to have higher MATH than the U.S. students with the same CLPS scores. It is also interesting that there are many Korean students scored outstandingly high in MATH, above 700. On the other hand, very few U.S students were found to have high scores in MATH, though they have similarly high CLPS scores. In contrast to MATH, there are many of U.S. students whose CLPS are greater than 700, while only few Korean students scored high in CLPS. These findings are in parallel with U.S. students’ large variations in CLPS and Korean students’ large variation in MATH (see
This research investigated the relationships between mathematics achievement and collaborative problem solving using the PISA 2015 data. Based on the two aspects of collaborative problem solving - problem solving/cognitive and collaboration/social, we examined the potential moderating effects of attitude toward working with others (enjoying cooperation and valuing collaborative problem solving) through binational comparison between Korea and the U.S. Positive linear relationships between mathematics achievement and collaborative problem solving were found in both countries, while the slope of the Korean linear model is notably greater than that of the U.S. model. However, there were no significant moderating effects of attitude toward working with others found in both Korea and the U.S.
With regards to the first research question (Do the relationships between mathematics achievement and collaborative problem solving differ in the PISA 2015 data for Korean and U.S. students?), we conjectured using the average scores of collaborative problem solving and mathematics achievement released by the PISA that the two constructs were positively related to each other. This was also consistent with the findings of the regression analysis. This positive relationship between mathematics achievement and collaborative problem solving found from our results show that being competent in collaborative problem solving is associated with higher mathematics achievement. Further, this finding also indicates that collaborative problem solving may be one of the critical ways of learning mathematics.
Aside from finding a positive association between collaborative problem solving and mathematics achievement, we found differences between those of South Korean and U.S. students, as evident by somewhat difference in their slopes. Specifically, the increase in mathematics achievement as collaborative problem solving increased by 1 was greater for Korean students. This means that for those with low scores in collaborative problem solving, for both Korean and U.S. students, they are also likely to have similarly low scores in mathematics achievement. However, for students with high collaborative problem solving, there was a gap in mathematics achievement scores between Korean and U.S. students. This suggests that the relationship between collaborative problem solving and mathematics achievement is stronger in Korean students’ data, meaning that Korean students have more increase in mathematics achievement per unit increase in collaborative problem solving. This finding seems to be linked with different score patterns of students who had high scores in collaborative problem solving and mathematics achievement. This might be related to Korean students’ hidden curriculum from home, such as extra lessons and parental support and guidance that is based on higher respect for education (Paik, 2004). In addition, it might be that learning classrooms with more collaborative activities strengthened the relationship between collaborative problem solving and mathematics achievement in Korean data. In fact, Korean classrooms were studied to have more collaborative activities compared to other countries such as Mexico (Castro, 2014).
As demonstrated through different slopes in the regression models as well, Korean and U.S. students had different patterns in the highest score range of collaborative problem solving and mathematics achievement. Specifically, there were U.S. students who had very high scores in collaborative problem solving, but did not show remarkably high mathematics achievement scores. On the other hand, we found that despite Korean students scoring very high in mathematics achievement scores, they did not have such high scores in collaborative problem solving. This finding indicates that collaborative problem solving could be necessary for mathematics achievement in that the two had positive relationship across all ranges of collaborative problem solving score. However, this does not mean that competence in collaborative problem solving is sufficient for high mathematics achievement, given that patterns in high-score ranges of collaborative problem solving and mathematics achievement appeared to be different in the two countries. Thus, we argue that high mathematics achievement requires collaborative problem-solving ability as well as other mathematics practices such as mathematical proof, abstract reasoning, and mathematical modeling. In sum, collaborative problem solving scores are required to develop mathematics achievement as a way of learning mathematics, but it is not sufficient for high mathematics achievement.
Although it is reasonable that these findings are related to cultural differences, we cannot determine what cultural aspects directly cause these differences in the findings because of the limited scope of the secondary analysis. Though the PISA 2015 framework acknowledged possible cultural differences, such as the perceived value of taking initiative when students are from different social classes, these are not considered in the assessment (OECD, 2017a). Even so, we suggest some possible cultural influences that could account for the different scores. For example, South Korean cultural views on education and this influence on educational institutions could be considered for implications and future research. One consideration is that given the documented competitiveness of the South Korean educational system, Korean students might have higher mathematics standards and extra time for studying mathematics during private lessons outside of the public curriculum than the U.S. students did (Kaiser et al., 1999; Paik, 2004). For this reason, Korean students could have been exposed to more opportunities to experience more amount of as well as various types of mathematical practices. Particularly, in the cases of high-achieving students, a greater experience of various mathematical practices may have contributed to higher achievement of Korean students than U.S. students when having similar collaborative problem solving scores.
The second and third research questions focused on the moderating effects of attitude toward working with others and differences in the effects between Korea and U.S. scores (Are the relationships between students’ mathematics achievement and collaborative problem solving moderated by how students value collaborative problem solving and enjoy cooperation? And does the moderating effect of valuing and enjoyment of cooperation differ in Korean and U.S. data?). We expected to find significant moderating effects of attitude toward working with others in the relationship between collaborative problem solving and mathematics achievement. However, the findings showed that the effects of attitude were minimal, and this was consistent in the moderation models of Korea and the U.S. Interpreting the result of minimal moderating effects of attitude toward collaborative problem solving is challenging, and we reassessed the fundamental question: should it moderate the relationship? Reflecting upon previous research on attitude towards mathematics, the findings could be attributed to an overly simplified definition of attitude towards working with others (valuing collaborative problem solving and enjoying cooperation). First, cooperation, defined as simply working together, is a part of collaboration, but this is not directly associated with collaborative problem solving or mathematics achievement. This is because collaboration requires symmetrical structure in knowledge levels, equitable roles, and common group goals across group members, which is more than merely working together through division of labor (Dillenbourg, 1999; OECD, 2017a). Second, attitude towards mathematics is multi-dimensional and self-efficacy is one of the most influential dimension on mathematics achievement (Hwang et al., 2017; Liang, 2010). When self-efficacy is controlled, valuing mathematics and interest in mathematics have no or very small effects on mathematics achievement. Analogously, it is possible that valuing collaborative problem solving and enjoying cooperation could be insignificant, when examined directly with collaborative problem-solving ability or mathematics achievement as in this study. In this sense, another possible explanation is that without the intermediate variables, such as engagement in and exposure to collaborative problem-solving activities in general and in mathematics classrooms, it is difficult to reveal the moderating role of attitude towards working with others in the relationship between collaborative problem solving and mathematics achievement. In other words, valuing and enjoying collaborative activities may not necessarily be associated with higher ability in collaborative problem solving.
Considering previous studies on attitude toward mathematics and variables included in this study, it is reasonable to expect that there might have been other critical factors that influence the relationship between collaborative problem solving and mathematics achievement. Though only perceptions about working with others were included as variables in this study, students’ learning also depends on students’ beliefs about what mathematics is, as well as how to learn mathematics, which are collectively called their epistemological beliefs (Muis, 2004). Students’ beliefs about mathematics (what it is) and collaboration (how to learn) could simultaneously and collectively have an influence on students’ development of understanding of mathematical concepts and collaborative problem-solving skills. Future studies on the topic can take these complex relationships into consideration, so as to identify those teaching and learning practices and students’ attitude that are beneficial to learning mathematics and collaborative problem solving.
It should be noted that a secondary analysis was conducted to provide a broad picture of the relationship between mathematics achievement and collaborative problem solving. Providing detailed explanations of each part of the findings is beyond the scope of this study. Thus, more future studies, both in qualitative and quantitative methods, can provide further evidence to argue any possible effects of third variables, such as cultural characteristics, beliefs, and attitudes, on mathematics achievement and collaborative problem solving.
Current society is changing rapidly with the Fourth Industrial Revolution, in which various disciplines like mathematics, statistics and computer science, have been integrated (Beane, 1995; European Science Foundation, 2010). Those different disciplines can impact all aspects of students’ everyday life both inside and outside classrooms. This is why 21st century skills such as collaborative problem solving are called crosscutting competencies (Pellegrino & Hilton, 2013). On one hand, elements of these crosscutting competencies are present in various disciplinary classrooms. However, for the same reason, those competencies are not unique to or distinctively expressed within a specific discipline (Pellegrino & Hilton, 2013). From the perspective of disciplinary education of mathematics, it is necessary for educators and researchers to delve into the relationships among those competencies and mathematics learning. As a preliminary step, we provided a descriptive association between collaborative problem solving as one of the 21st century competencies and mathematics achievement. With further effort on the topic, we will be able to support students to learn such 21st century skills including collaborative problem solving in an integrated way in mathematics classrooms.
No potential conflict of interest relevant to this article was reported.
2021; 31(3): 299-320
Published online August 31, 2021 https://doi.org/10.29275/jerm.2021.31.3.299
Copyright © Korea Society of Education Studies in Mathematics.
1Graduate Student, Department of Educational Psychology and Learning Sciences, University of Iowa, USA, 2Assistant Professor, Department of Mathematics Education, Kangwon National University, South Korea
Correspondence to:†Jihyun Hwang, jihyun-hwang@kangwon.ac.kr
ORCID: https://orcid.org/0000-0003-3107-7142
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0), which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
We examined the relationships between mathematics achievement and collaborative problem solving, which has been emphasized as the 21st century skills. Focusing on moderating effects of attitude toward working with others, we analyzed the PISA 2015 data for binational comparison between Korean and the U.S. Random-intercept linear models - where the intercepts vary across schools while slopes are fixed across schools - were applied to the data including 5,545 students in Korea and 5,564 in the U.S. The findings showed the positive linear relationships between mathematics achievement and collaborative problem solving as expected, but the slope was remarkably higher in the Korean model. However, we found no significant moderating effects of attitude toward working with others on the relationship between mathematics achievement and collaborative problem solving. Particularly, students in Korea and the U.S. are likely to have similarly low scores in mathematics achievement when their collaborative problem solving scores are low. However, Korean students would have even higher mathematics achievement scores if students in Korea and the U.S. have similarly high collaborative problem solving scores. The findings suggest the necessity for further scrutinies on attitude toward working with others and research on teaching and learning the 21st century skills in an integrated way.
Keywords: collaborative problem solving, mathematics achievement, PISA 2015, International Comparison
Mathematics educators have increasingly emphasized mathematical competency in the Fourth Industrial Revolution for enriched lives integrated with advanced technology. Firsthand experiences of rapid changes in the new era advise that learning mathematics is critical to understand and use new technology like machine learning and artificial intelligence. However, governments and academic organizations have identified only domain-general competencies required in the 21st century, which requires reframing mathematical competency for the 21st century. Bridging the 21st century competency (domain-general) and mathematics competency (domain-specific), students could be offered best opportunities to be prepared for the current and next generations through learning mathematics. Particularly, we are interested in connecting learning mathematics to collaborative problem solving, which both the World Economic Forum (2015) and the National Research Council (2012) suggested as interpersonal competencies for the 21st Century.
Collaborative problem solving is one of the essential skills that is projected to be essential skill in developing societies and workforce. Collaborative problem solving refers to “the capacity of an individual to effectively engage in a process whereby two or more agents attempt to solve a problem by sharing the understanding and effort required to come to a solution and pooling their knowledge, skills and efforts to reach that solution” (OECD, 2017, p. 134). By its definition, collaborative problem solving involves two aspects: the cognitive aspect of problem solving and social components aspect of collaboration. In disciplinary learning, including mathematics, collaborative problem solving is understood not only as a skillset that young learners are expected to develop through K-12 and higher education curriculum, but also as a way of learning subject materials in K-12 classrooms. When engaged in collaborative problem-solving activities, students can learn disciplinary concepts and practices through explaining concepts to peers and communicating their ideas while solving problems together (Langer-Osuna, 2016; Moss & Beatty, 2006). In mathematics classrooms, students naturally interweave mathematical terms, numbers, and symbols in their interactions during collaborative problem solving (Moss & Beatty, 2006).
As a way of learning mathematics, activities that involve collaborative problem solving can be a setting for an activity where students learn mathematical knowledge and practices. For example, collaborative problem solving, as a setting for learning activities, has been used to support student learning of mathematical concepts and practices. Research studies have examined how students can engage in collaborative problem solving activities to learn mathematical concepts and practices, such as patterning in early algebra (Moss & Beatty, 2006), middle school students’ mathematical generalization (Ellis, 2011), and college calculus (Reisel et al., 2014). In turn, mathematics is one of the curricular subjects that provide students with opportunities to learn knowledge and thinking that contribute to the development of collaborative problem solving. For instance, the standards for mathematics practices, including those (National Governors Association Center for Best Practices [NGACBP] & Council of Chief State School Officers [CCSSO], 2010) suggest that students can communicate their mathematical ideas with others and that they can also understand and compare others’ problem-solving approaches. Such ways of learning mathematics can foster students’ competence in collaborative problem solving. This relationships between mathematics learning and collaborative problem solving can be thought of as bidirectional and cyclical (see
As seen in
With such perspective of collaborative problem solving and learning mathematics, we explored in a descriptive manner how collaborative problem solving is related to mathematical competence. Specifically, we investigated the relationship between mathematics achievement - an outcome of learning mathematics - and collaborative problem solving as measured in the Program for International Student Assessment (PISA) 2015. Considering that collaborative problem solving can be a way of learning mathematics (Ellis, 2011; Moss & Beatty, 2006; Reisel et al., 2014), it would be expected to see a positive relationship between mathematics achievement and collaborative problem solving. However, empirical evidence for this relationship has not been well-documented. Thus, the purpose of this study is to provide a broad picture of the relationship between mathematics achievement and collaborative problem solving through secondary analysis with the PISA 2015 data.
For the research purpose, we focused on (1) moderating effect of attitude toward working with others and (2) binational comparison between South Korea and the U.S. First, we argue the necessity to include additional factors in analysis for better understanding of the relationship. We tested moderating effects of attitude toward cooperation, particularly valuing and enjoying cooperation1) measured in the PISA 2015. Considering problem-solving component of collaborative problem solving (Hesse et al., 2015; OECD, 2017a), attitudes toward cooperation could have significant roles in students’ development of collaborative problem solving and its connection to other skills such as mathematics achievement. However, there are few studies that explore the role of attitude toward cooperation in relation to collaborative problem-solving ability or to mathematics competence.
Second, this secondary-analysis research is binational comparison between South Korea and the U.S. The primary rationale for binational comparison is to take cultural difference into consideration in studying the relationship between mathematics achievement and collaborative problem solving. From the results of PISA 2015, both South Korea and the U.S. performed above the OECD average in collaborative problem solving, while their mathematics achievement results were significantly different. This indicates possible difference in the relationships between mathematics achievement and collaborative problem solving that is from different cultures of the two countries. Considering the social aspect of collaborative problem solving, cultural differences may be an important factor that impact the relationship between mathematics achievement and collaborative problem solving (Forman & McPhail, 1996; OECD, 2017). Thus, binational comparison between South Korea and the U.S. helps us describe how cultural differences are possibly related to mathematics achievement and collaborative problem solving.
Collaborative problem solving refers to “a joint activity where dyads or small groups execute a number of steps in order to transform a current state into a desired goal state (Hesse et al., 2015). Though problem solving ability somewhat relies on one’s domain knowledge (Funke & Frensch, 2007), collaborative problem solving ability refers to a domain-general ability that enable an individual effectively to engage in collaborative problem-solving activities across different subject domains (OECD, 2017). Answering to shifting requirements of workplace and society, educators in K-12 and higher education have been paying attention to identification of relevant 21st century skills and the ways of teaching and learning these skills. As one of the 21st century skills, collaborative problem solving has been emphasized by employers and educators as a skill to be taught and developed, rather than a skill that develops naturally through everyday life (Hesse et al., 2015). Accordingly, educational research and practices have come to pursue the development of collaborative problem solving as an educational goal (Gasser, 2011; Larson & Miller, 2011; Lloyd, 1999). Developing competence in collaborative problem solving is important for students not only for future workplace capacity, but also for them to be productive learners in K-12 education. With more awareness of the importance of collaborative problem solving, researchers have been attempting to define collaborative problem-solving ability as a teachable and measurable concept (Hesse et al., 2015; Rosen & Foltz, 2014). Particularly, problem solving skills and knowledge acquired through mathematics learning may manifest in other problem situations including collaborative problem solving with domain-general content. This is based on the theoretical assumption that conceptual knowledge and problem-solving skills that students use and learn in mathematics are advantageous to the development of collaborative problem solving and vice versa.
Teaching and evaluating students’ collaborative problem solving involve two aspects that reflect the nature of the concept: a cognitive aspect of problem solving and a social aspect of collaboration (Andrews-Todd & Forsyth, 2020; Graesser et al., 2018). According to the 21st century skills framework of Binkley et al. (2012), problem solving is one of the ways of thinking, while collaboration is one of the ways of working. The PISA framework of problem solving defines the problem solving aspect of collaborative problem solving by identifying elements of problem solving: problem as a discrepancy between current state and goal state, problem space as a mental representation of the problem states and of the steps to close the discrepancy, a plan for steps to approach nearer to the goal state, execution of the steps, and monitoring and reflecting the problem-solving processes (OECD, 2017). The difference between collaborative and individual problem solving is that these processes are often internal in individual settings, while problem-solving processes become observable and communicative in collaborative problem solving. When engaging in collaborative problem solving, participants exchange and share their thoughts to identify problem space, steps to go through, and monitoring of executing the steps.
The aspect of collaboration describes the process of collaborative problem solving where members of a group need to be able to collaborate with one another in order to coordinate problem-solving processes among multiple agents. Collaboration includes communication (the exchange of knowledge and opinions), cooperation (an agreed division of work), and responsiveness for active and insightful participation (Hesse et al., 2015). Based on these elements of collaboration, collaborative skills that students are expected to develop are specified as establishing and maintaining shared understanding, taking appropriate action to solve the problem, and establishing and maintaining team organization (OECD, 2017). Though the ability of collaborative problem solving is argued to be domain-general, tasks and activities for collaborative problem solving can be content-rich, involving specific concepts and practices learned in different subject domains including mathematics.
Collaborative problem solving is understood not only as a critical ability for complex work and team collaboration, but also as a way for students to learn subject materials in K-12 classrooms (Graesser et al., 2018). As noted, collaborative problem solving involves the cognitive aspect of problem solving and the social aspect of collaboration. The cognitive aspect of collaborative problem solving is thinking processes of problem solving, and problem solving is one of the ways of thinking mathematically (Binkely et al., 2012; Schoenfeld, 1992). Considering both aspects of collaborative problem solving, problem solving and collaboration, we argue that collaborative problem-solving and mathematics learning are interconnected in several ways: 1) learning mathematics can develop students’ competence in collaborative problem solving, 2) collaborative problem solving activities have been implemented as a way of learning mathematics (e.g., Harding et al., 2017; Hurme & Järvelä, 2005; Sears & Reagin, 2013), and 3) the content of mathematics has been utilized to design collaborative problem-solving tasks (Care et al., 2015).
First, learning mathematical concepts and practices can contribute to the development of collaborative problem-solving ability (Gasser, 2011; Larson & Miller, 2011). This is based on the problem-solving aspect of collaborative problem solving that is prevalent in mathematics problem solving. Mathematics problem solving frameworks identify cognitive processes of problem solving as sequential phases (Schoenfeld, 1983). Phases of mathematics problem solving include reading, analysis, exploration, planning/implementation, and verification. Identifying these phases of mathematics problem solving allows for assessing students’ problem solving skills, teaching students how to approach a problem, and supporting students to use problem solving strategies in a wider range of problem situations (Wu & Adams, 2006). Likewise, the problem solving component of collaborative problem solving involves similar processes: explore and understand the problem, represent and formulate the problem situation, plan and execute, and monitor and reflect (OECD, 2017a). Either individual or collaborative, understanding problem situations, planning problem solving steps, and monitoring and reflecting on those steps are critical cognitive processes for both collaborative problem solving and mathematical problem solving.
Secondly, engaging collaborative problem-solving process can be a way of learning mathematical concepts and practices. In mathematics classrooms, collaborative activities were found to productively support student engagement in cognitive processes of problem solving and their learning of mathematical concepts and practices cooperatively with peers (National Council of Teachers of Mathematics [NCTM], 2000). For this reason, collaborative problem-solving tasks have been implemented for learning mathematics (e.g., Harding et al., 2017; Hurme & Järvelä, 2005; Sears & Reagin, 2013). Specifically, communication, which is one of the elements of collaboration, is the core practice that enables students to share mathematical ideas and to experience essential process for developing students’ understanding of mathematical concepts through participating in small-group or classroom discussion (Hatano & Inagaki, 1991). Moreover, it is critical for students to have opportunities to participate in dialogical interactions with classmates and instructors, because those dialogical interactions are critical for students to learn how to make and refine their arguments and gleam the perspective of others (see Hershkowitz et al., 2007; Lampert, 1990). Negotiating ideas through dialogical interactions in discussions enables students to generate their own knowledge and ideas instead of replicating knowledge taught by teachers and textbooks (Ford & Forman, 2006). In essence, collaboration, communication, and dialogical interactions are critical processes in mathematics learning as well as in collaborative problem solving.
Finally, mathematics provides content for designing collaborative problem-solving tasks. Collaborative problem-solving tasks are categorized as content-free or content-rich (Care et al., 2015). Content-rich, or content-dependent, tasks employ skills and knowledge that is drawn from curriculum-based learning, whereas content-free tasks mainly require application of reasoning, not demanding prerequisite knowledge taught in curriculum-based subjects (Care et al., 2015). According to this categorization, tasks that involve mathematical concepts and situations are content-rich tasks. As most disciplinary curricula, concepts and knowledge in mathematics become the content basis for the problems to be solved collaboratively. Those content-rich tasks are presented to students to measure students’ collaborative problem-solving ability as well as develop the skills that collaborative problem-solving ability entails (Poon et al., 2015).
Cultural aspects could also influence engagement in collaboration as sociocultural views on the value of collaboration could orient student collaborative problem-solving development. Different cultures provide unique opportunities for students to observe and participate in essential sociocultural practices, including economic, political, instructional, and recreational activities (Forman & McPhail, 1996). Participating in these activities, young individuals internalize their affective, social, and intellectual meaning, and this represents a dialectic relation between cultural practice and individual experience (Cobb et al., 1996; Forman & McPhail, 1996). In the same vein, students from different cultural context may have different norms of and attitudes toward collaborative activities. For example, in a competitive situation as those of East Asian countries, students might believe that learning individually is more important for high-stake assessments (Zhu et al., 2009). However, cultural characteristics such as how taking initiative is valued and taking different roles during collaborative work were not considered to be different across different countries in the assessment of collaborative problem solving in PISA 2015 (OECD, 2017a).
Because the problem solving component is the core cognitive ability that comprises collaborative problem solving, it can be hypothesized that mathematical problem-solving ability is positively related to collaborative problem-solving ability. However, this has not been empirically shown in previous research, for which one of the reasons is that large-scale, standardized assessment of collaborative problem solving is a relatively new test administered in PISA 2015.
In this study, the relationships between mathematics achievement and collaborative problem solving are scrutinized using the PISA 2015 data of South Korea and the U.S. The PISA 2015 included collaborative problem solving as one of its subject areas, providing researchers with a unique opportunity to study collaborative problem solving as a measurable construct. Because it is almost impossible to evaluate human-to-human interactions in a large-scale assessment, the assessment of collaborative problem solving in the PISA 2015 is realized as a computer-based assessment. This approach uses programming computer agents to control external factors in collaborative contexts, including problem situations and actions of virtual team member(s) with whom a test taker collaborates (OECD, 2017a). Based on the characteristics of the assessment, it is expected that the assessment of collaborative problem solving measures a distinctive construct that captures additional information about students’ learning that is hardly captured by existing achievement indices. Therefore, as a type of an individual’s competence, collaborative problem solving is assumed to be related to other intellectual measures that involve individuals’ problem-solving abilities, such as mathematical literacy.
Like the previous conceptualization of collaborative problem solving, the framework in the PISA 2015 conceptualized collaborative problem solving as a two-dimensioned competence that integrates individual problem-solving processes and collaboration processes (OECD, 2017a). In its framework, the PISA 2015 specifically emphasizes the skills that constitutes collaborative work such as group thinking and communication skills based on the PISA 2012’s framework of problem solving. Given that collaboration comprises one of two facets of collaborative problem solving, it is assumed that collaborative problem solving is influenced not only by the intellectual ability of students, but also by their perceptions of working with others. Hence, the PISA 2015 data on students’ attitude toward collaborative activities, such as individuals valuing collaborative problem solving and enjoying cooperation, is one of the key factors that impact how one engages in collaborative processes.
In this study, we investigated the relationships between mathematics achievement and collaborative problem solving in the PISA 2015 data of South Korea and the U.S students. In doing so, we examined the role of attitude toward collaborative activities in the relationships between mathematics achievement and collaborative problem solving.
The main question guiding this research is, what is the relationship between mathematics achievement and collaborative problem solving in the PISA 2015 data of South Korea and the U.S.? With this overarching question, we investigated the following sub-questions in this study:
Do relationships between mathematics achievement and collaborative problem solving differ in the PISA 2015 data of Korean and U.S. students?
Are the relationships between students’ mathematics achievement and collaborative problem solving moderated by how students value/enjoy cooperation?
Does the moderating effect of valuing and enjoyment of cooperation differ in the South Korean and U.S. data?
This research investigated the relationship between mathematics achievement (MATH), collaborative problem solving (CLPS), and attitudes toward working with others (enjoying cooperation and valuing collaborative problem solving) using the PISA 2015 data. We merged data of South Korea and the U.S. to address the research questions. We applied multilevel modeling, considering the nature of data structure of the PISA 2015.
We collected South Korean and U.S. students’ data from the PISA 2015 international database. The original data include 5,581 Korean students across 168 schools and 5,712 U.S. students across 176 schools. The target population of the PISA assessment is students who were aged between 15 years and 3 months and 16 years and 2 months and attended an educational institution with grades 7 and higher. We acknowledged that some of the sample students did not complete the student surveys, and those students did not have available scores of attitudes toward cooperation and collaborative problem solving (COOPERATIVE and CPSVALUE). Those students missing scores of attitudes toward cooperation and collaborative problem solving take up 0.6% in Korean data and 2.6% in the U.S. data. Given that the proportions of students missing those attitudes variables in the entire sample are very small, we determined deleting the data missing some of the variables should have little impact on overall patterns in the findings in this study. As a result, the final numbers of students in the data were 5,545 for Korea and 5,564 for the U.S.
We collected 10 sets of plausible values (PV1MATH-PV10MATH) representing students’ mathematical literacy scores (MATH) and 10 sets of plausible values (PV1CLPS-PV10CLPS) representing CLPS respectively from the PISA 2015 database. These two collections of the plausible values were constructed by assessment developers for the optimal representation of population estimates. The results of this research are based on the 100 (10 by 10) different pairs of the plausible values of MATH and CLPS, following the guidelines provided by Chaney et al. (2001). First, we estimated all coefficients in the multilevel model and their error variances using all pairs of MATH and CLPS. After all pairs of plausible values were analyzed, we computed the final estimate (labeled with t*) of all coefficients and the standard error of t* (labeled with SE(t*)). In the result section, we will report the average of the error variances (labeled with U*) and the variations among all estimates of the coefficients (labeled with B), which were used to calculate SE(t*).
We also collected the data of students’ COOPERATE and CPSVALUE, each of which was constructed by applying the item response theory (IRT) to the statements as seen in
Table 1 .
PISA 2015 variable code | Statement |
---|---|
Question: To what extent do you disagree or agree with the following statements about yourself? | |
COOPERATE | |
ST082Q02 | I am a good listener. |
ST082Q03 | I enjoy seeing my classmates be successful. |
ST082Q08 | I take into account what others are interested in. |
ST082Q12 | I enjoy considering different perspectives. |
CPSVALUE | |
ST082Q01 | I prefer working as part of a team to working alone. |
ST082Q09 | I find that teams make better decisions than individuals. |
ST082Q13 | I find that teamwork raises my own efficiency. |
ST082Q14 | I enjoy cooperating with peers. |
We applied multilevel modeling, particularly random-intercept linear modeling to the Korean and U.S data, where the intercepts vary across schools while slopes are fixed across schools. We determined to implement the random-intercept modeling with consideration of the two-stage stratified sampling design in the PISA 2015. Another rationale for multilevel analysis was intraclass correlation coefficients (ICCs) greater than 0.25 (Kim et al., 2009). ICCs from the first (PV1MATH) to the tenth set of plausible values (PV10MATH) were 0.296, 0.295, 0.313, 0.307, 0.313, 0.304, 0.296, 0.304, 0.303, and 0.299, whose average was 0.303. In the multilevel regression model, random slopes were not considered because there was no statistical difference in the slopes across the schools. In other words, including random slopes in the model could cause collinearity issues. All results were weighted by W_FSTUWT in the PISA 2015 database.
To include moderation effects of COOPERATE and CPSVALUE respectively and examine differences in the models between Korea and the U.S., we included three-way interaction terms in the full multilevel model. We used the R package
1) Model including moderation of COOPERATE
2) Model including moderation of CPSVALUE
where MATH
After the regression analysis, we calculated and visualized moderating effects of COOPERATE or CPSVALUE by fixing its values in each country model. We chose values of –1.5, 0, and 1.5 representing 1.5 standard deviation below average, average, and 1.5 standard deviation above average. By doing this, the results will show the relationships between MATH and CLPS of students who have different degrees of enjoying or valuing collaborative problem solving.
Table 2 .
Variable | Korea (Z=1) | United States (Z=0) | |||
---|---|---|---|---|---|
Weighted average | Standard deviation | Weighted average | Standard deviation | ||
COOPERATE | –0.017 | 0.938 | 0.127 | 1.002 | |
CPSVALUE | 0.142 | 0.910 | 0.056 | 1.026 | |
PV1CLPS | 537.800 | 83.079 | 521.554 | 107.077 | |
PV2CLPS | 539.456 | 85.365 | 524.532 | 106.423 | |
PV3CLPS | 538.610 | 82.942 | 523.859 | 109.509 | |
PV4CLPS | 536.848 | 82.728 | 523.870 | 107.815 | |
PV5CLPS | 538.731 | 84.075 | 520.640 | 105.874 | |
PV6CLPS | 537.490 | 84.474 | 523.832 | 107.336 | |
PV7CLPS | 538.095 | 84.774 | 524.185 | 106.248 | |
PV8CLPS | 537.892 | 83.668 | 522.931 | 107.472 | |
PV9CLPS | 539.101 | 83.465 | 523.231 | 106.905 | |
PV10CLPS | 541.069 | 83.920 | 522.847 | 106.339 | |
PV1MATH | 524.550 | 99.794 | 472.756 | 87.876 | |
PV2MATH | 523.487 | 99.487 | 472.711 | 88.785 | |
PV3MATH | 525.199 | 98.732 | 470.653 | 88.132 | |
PV4MATH | 526.082 | 99.272 | 470.519 | 88.148 | |
PV5MATH | 523.703 | 100.876 | 472.094 | 86.772 | |
PV6MATH | 522.861 | 99.938 | 471.433 | 86.589 | |
PV7MATH | 523.708 | 99.165 | 471.195 | 88.064 | |
PV8MATH | 522.967 | 99.650 | 472.833 | 86.535 | |
PV9MATH | 526.155 | 99.986 | 471.670 | 87.870 | |
PV10MATH | 523.270 | 100.137 | 472.795 | 89.288 |
Because we applied multilevel analysis using 100 pairs of MATH and CLPS plausible values to find the final estimates (t*) of regression coefficients, it is impossible to report all results in detail due to limited space. We provide results using the first sets of plausible values of MATH and CLPS as examples (see
Table 3 .
Fixed Effect | Estimate | Std. Error | df | t | Pr(>|t|) |
---|---|---|---|---|---|
Intercept, β0 | 161.100 | 3.566 | 2409 | 45.174 | <0.001 |
PV1CLPS, β1 | 0.596 | 0.006 | 11070 | 103.025 | <0.001 |
COOPERATE, β2 | 1.895 | 2.815 | 10840 | 0.673 | 0.501 |
Z, β3 | –71.690 | 10.560 | 7760 | -6.788 | <0.001 |
PV1CLPS×COOPERATE, β4 | –0.005 | 0.005 | 10840 | -0.973 | 0.331 |
PV1CLPS×Z, β5 | 0.212 | 0.019 | 10510 | 11.238 | <0.001 |
COOPERATE×Z, β6 | 13.680 | 9.576 | 10960 | 1.429 | 0.153 |
PV1CLPS×COOPERATE×Z, β7 | –0.018 | 0.018 | 10950 | -1.007 | 0.314 |
Random effect | Variance | Std. Dev | |||
Intercept, | 595.7 | 24.41 | |||
Residual, | 1058935.4 | 1029.05 |
Table 4 .
Fixed Effect | Estimate | Std. Error | df | t | Pr(>|t|) |
---|---|---|---|---|---|
Intercept, β0 | 164.300 | 3.555 | 2348 | 46.217 | <0.001 |
PV1CLPS, β1 | 0.589 | 0.006 | 11070 | 103.046 | <0.001 |
CPSVALUE, β2 | 6.749 | 2.840 | 10860 | 2.377 | 0.018 |
Z, β3 | –79.780 | 10.520 | 7641 | –7.585 | <0.001 |
PV1CLPS×CPSVALUE, β4 | –0.020 | 0.005 | 10850 | –3.906 | <0.001 |
PV1CLPS×Z, β5 | 0.228 | 0.019 | 10480 | 12.103 | <0.001 |
CPSVALUE×Z, β6 | 17.170 | 10.100 | 10960 | 1.700 | 0.089 |
PV1CLPS×CPSVALUE×Z, β7 | –0.025 | 0.019 | 10950 | –1.327 | 0.185 |
Random Effect | Variance | Std. Dev | |||
Intercept, | 602.7 | 24.55 | |||
Residual, | 1052086.4 | 1025.71 |
Table 5 .
t* | U* | B | V(t*) | SE(t*) | |
---|---|---|---|---|---|
Intercept β0 | 180.427 | 3.773 | 59.126 | 63.491 | 7.968 |
CLPS β1 | 0.554 | 0.006 | 0.000 | 0.006 | 0.080 |
COOPERATE β2 | 5.043 | 2.979 | 7.919 | 10.977 | 3.313 |
Z β3 | –57.884 | 11.150 | 92.283 | 104.356 | 10.215 |
CLPS×COOPERATE β4 | –0.008 | 0.006 | <0.001 | 0.006 | 0.075 |
CLPS×Z β5 | 0.190 | 0.020 | <0.001 | 0.020 | 0.142 |
COOPERATE×Z β6 | 4.470 | 10.225 | 32.322 | 42.871 | 6.548 |
CLPS×COOPERATE×Z β7 | –0.005 | 0.019 | <0.001 | 0.019 | 0.138 |
Note: t* represents the final estimate of the coefficients. U* is the average of the error variances and B indicates the variations among all estimates of the coefficients. V(t*) and SE(t*) are the error variance and standard error of t*..
Table 6 .
t* | U* | B | V(t*) | SE(t*) | |
---|---|---|---|---|---|
Intercept β0 | 182.722 | 3.761 | 56.206 | 60.529 | 7.780 |
CLPS β1 | 0.550 | 0.006 | 0.000 | 0.006 | 0.079 |
CPSVALUE β2 | 6.481 | 2.986 | 7.028 | 10.085 | 3.176 |
Z β3 | –63.285 | 11.054 | 98.067 | 110.101 | 10.493 |
CLPS×CPSVALUE β4 | –0.020 | 0.006 | <0.001 | 0.006 | 0.074 |
CLPS×Z β5 | 0.201 | 0.020 | <0.001 | 0.020 | 0.142 |
CPSVALUE×Z β6 | 7.178 | 10.703 | 35.796 | 46.857 | 6.845 |
CLPS×CPSVALUE×Z β7 | –0.006 | 0.020 | <0.001 | 0.020 | 0.140 |
Note: t* represents the final estimate of the coefficients. U* is the average of the error variances and B indicates the variations among all estimates of the coefficients. V(t*) and SE(t*) are the variance and standard error of t*..
Using t* in
To answer the research question on binational comparison, we found that the slopes of Korean linear models are higher than those of the U.S. models. This finding indicates that Korean students have more increase in MATH as CLPS increases by 1. In specific, when Korean and U.S. students are in the range of lower CLPS scores (around 300), students in both countries are likely to have similar MATH scores. When students have CLPS scores above 500, Korean students are likely to have higher MATH than the U.S. students with the same CLPS scores. It is also interesting that there are many Korean students scored outstandingly high in MATH, above 700. On the other hand, very few U.S students were found to have high scores in MATH, though they have similarly high CLPS scores. In contrast to MATH, there are many of U.S. students whose CLPS are greater than 700, while only few Korean students scored high in CLPS. These findings are in parallel with U.S. students’ large variations in CLPS and Korean students’ large variation in MATH (see
This research investigated the relationships between mathematics achievement and collaborative problem solving using the PISA 2015 data. Based on the two aspects of collaborative problem solving - problem solving/cognitive and collaboration/social, we examined the potential moderating effects of attitude toward working with others (enjoying cooperation and valuing collaborative problem solving) through binational comparison between Korea and the U.S. Positive linear relationships between mathematics achievement and collaborative problem solving were found in both countries, while the slope of the Korean linear model is notably greater than that of the U.S. model. However, there were no significant moderating effects of attitude toward working with others found in both Korea and the U.S.
With regards to the first research question (Do the relationships between mathematics achievement and collaborative problem solving differ in the PISA 2015 data for Korean and U.S. students?), we conjectured using the average scores of collaborative problem solving and mathematics achievement released by the PISA that the two constructs were positively related to each other. This was also consistent with the findings of the regression analysis. This positive relationship between mathematics achievement and collaborative problem solving found from our results show that being competent in collaborative problem solving is associated with higher mathematics achievement. Further, this finding also indicates that collaborative problem solving may be one of the critical ways of learning mathematics.
Aside from finding a positive association between collaborative problem solving and mathematics achievement, we found differences between those of South Korean and U.S. students, as evident by somewhat difference in their slopes. Specifically, the increase in mathematics achievement as collaborative problem solving increased by 1 was greater for Korean students. This means that for those with low scores in collaborative problem solving, for both Korean and U.S. students, they are also likely to have similarly low scores in mathematics achievement. However, for students with high collaborative problem solving, there was a gap in mathematics achievement scores between Korean and U.S. students. This suggests that the relationship between collaborative problem solving and mathematics achievement is stronger in Korean students’ data, meaning that Korean students have more increase in mathematics achievement per unit increase in collaborative problem solving. This finding seems to be linked with different score patterns of students who had high scores in collaborative problem solving and mathematics achievement. This might be related to Korean students’ hidden curriculum from home, such as extra lessons and parental support and guidance that is based on higher respect for education (Paik, 2004). In addition, it might be that learning classrooms with more collaborative activities strengthened the relationship between collaborative problem solving and mathematics achievement in Korean data. In fact, Korean classrooms were studied to have more collaborative activities compared to other countries such as Mexico (Castro, 2014).
As demonstrated through different slopes in the regression models as well, Korean and U.S. students had different patterns in the highest score range of collaborative problem solving and mathematics achievement. Specifically, there were U.S. students who had very high scores in collaborative problem solving, but did not show remarkably high mathematics achievement scores. On the other hand, we found that despite Korean students scoring very high in mathematics achievement scores, they did not have such high scores in collaborative problem solving. This finding indicates that collaborative problem solving could be necessary for mathematics achievement in that the two had positive relationship across all ranges of collaborative problem solving score. However, this does not mean that competence in collaborative problem solving is sufficient for high mathematics achievement, given that patterns in high-score ranges of collaborative problem solving and mathematics achievement appeared to be different in the two countries. Thus, we argue that high mathematics achievement requires collaborative problem-solving ability as well as other mathematics practices such as mathematical proof, abstract reasoning, and mathematical modeling. In sum, collaborative problem solving scores are required to develop mathematics achievement as a way of learning mathematics, but it is not sufficient for high mathematics achievement.
Although it is reasonable that these findings are related to cultural differences, we cannot determine what cultural aspects directly cause these differences in the findings because of the limited scope of the secondary analysis. Though the PISA 2015 framework acknowledged possible cultural differences, such as the perceived value of taking initiative when students are from different social classes, these are not considered in the assessment (OECD, 2017a). Even so, we suggest some possible cultural influences that could account for the different scores. For example, South Korean cultural views on education and this influence on educational institutions could be considered for implications and future research. One consideration is that given the documented competitiveness of the South Korean educational system, Korean students might have higher mathematics standards and extra time for studying mathematics during private lessons outside of the public curriculum than the U.S. students did (Kaiser et al., 1999; Paik, 2004). For this reason, Korean students could have been exposed to more opportunities to experience more amount of as well as various types of mathematical practices. Particularly, in the cases of high-achieving students, a greater experience of various mathematical practices may have contributed to higher achievement of Korean students than U.S. students when having similar collaborative problem solving scores.
The second and third research questions focused on the moderating effects of attitude toward working with others and differences in the effects between Korea and U.S. scores (Are the relationships between students’ mathematics achievement and collaborative problem solving moderated by how students value collaborative problem solving and enjoy cooperation? And does the moderating effect of valuing and enjoyment of cooperation differ in Korean and U.S. data?). We expected to find significant moderating effects of attitude toward working with others in the relationship between collaborative problem solving and mathematics achievement. However, the findings showed that the effects of attitude were minimal, and this was consistent in the moderation models of Korea and the U.S. Interpreting the result of minimal moderating effects of attitude toward collaborative problem solving is challenging, and we reassessed the fundamental question: should it moderate the relationship? Reflecting upon previous research on attitude towards mathematics, the findings could be attributed to an overly simplified definition of attitude towards working with others (valuing collaborative problem solving and enjoying cooperation). First, cooperation, defined as simply working together, is a part of collaboration, but this is not directly associated with collaborative problem solving or mathematics achievement. This is because collaboration requires symmetrical structure in knowledge levels, equitable roles, and common group goals across group members, which is more than merely working together through division of labor (Dillenbourg, 1999; OECD, 2017a). Second, attitude towards mathematics is multi-dimensional and self-efficacy is one of the most influential dimension on mathematics achievement (Hwang et al., 2017; Liang, 2010). When self-efficacy is controlled, valuing mathematics and interest in mathematics have no or very small effects on mathematics achievement. Analogously, it is possible that valuing collaborative problem solving and enjoying cooperation could be insignificant, when examined directly with collaborative problem-solving ability or mathematics achievement as in this study. In this sense, another possible explanation is that without the intermediate variables, such as engagement in and exposure to collaborative problem-solving activities in general and in mathematics classrooms, it is difficult to reveal the moderating role of attitude towards working with others in the relationship between collaborative problem solving and mathematics achievement. In other words, valuing and enjoying collaborative activities may not necessarily be associated with higher ability in collaborative problem solving.
Considering previous studies on attitude toward mathematics and variables included in this study, it is reasonable to expect that there might have been other critical factors that influence the relationship between collaborative problem solving and mathematics achievement. Though only perceptions about working with others were included as variables in this study, students’ learning also depends on students’ beliefs about what mathematics is, as well as how to learn mathematics, which are collectively called their epistemological beliefs (Muis, 2004). Students’ beliefs about mathematics (what it is) and collaboration (how to learn) could simultaneously and collectively have an influence on students’ development of understanding of mathematical concepts and collaborative problem-solving skills. Future studies on the topic can take these complex relationships into consideration, so as to identify those teaching and learning practices and students’ attitude that are beneficial to learning mathematics and collaborative problem solving.
It should be noted that a secondary analysis was conducted to provide a broad picture of the relationship between mathematics achievement and collaborative problem solving. Providing detailed explanations of each part of the findings is beyond the scope of this study. Thus, more future studies, both in qualitative and quantitative methods, can provide further evidence to argue any possible effects of third variables, such as cultural characteristics, beliefs, and attitudes, on mathematics achievement and collaborative problem solving.
Current society is changing rapidly with the Fourth Industrial Revolution, in which various disciplines like mathematics, statistics and computer science, have been integrated (Beane, 1995; European Science Foundation, 2010). Those different disciplines can impact all aspects of students’ everyday life both inside and outside classrooms. This is why 21st century skills such as collaborative problem solving are called crosscutting competencies (Pellegrino & Hilton, 2013). On one hand, elements of these crosscutting competencies are present in various disciplinary classrooms. However, for the same reason, those competencies are not unique to or distinctively expressed within a specific discipline (Pellegrino & Hilton, 2013). From the perspective of disciplinary education of mathematics, it is necessary for educators and researchers to delve into the relationships among those competencies and mathematics learning. As a preliminary step, we provided a descriptive association between collaborative problem solving as one of the 21st century competencies and mathematics achievement. With further effort on the topic, we will be able to support students to learn such 21st century skills including collaborative problem solving in an integrated way in mathematics classrooms.
No potential conflict of interest relevant to this article was reported.
Table 1
PISA 2015 variable code | Statement |
---|---|
Question: To what extent do you disagree or agree with the following statements about yourself? | |
COOPERATE | |
ST082Q02 | I am a good listener. |
ST082Q03 | I enjoy seeing my classmates be successful. |
ST082Q08 | I take into account what others are interested in. |
ST082Q12 | I enjoy considering different perspectives. |
CPSVALUE | |
ST082Q01 | I prefer working as part of a team to working alone. |
ST082Q09 | I find that teams make better decisions than individuals. |
ST082Q13 | I find that teamwork raises my own efficiency. |
ST082Q14 | I enjoy cooperating with peers. |
Table 2
Variable | Korea (Z=1) | United States (Z=0) | |||
---|---|---|---|---|---|
Weighted average | Standard deviation | Weighted average | Standard deviation | ||
COOPERATE | –0.017 | 0.938 | 0.127 | 1.002 | |
CPSVALUE | 0.142 | 0.910 | 0.056 | 1.026 | |
PV1CLPS | 537.800 | 83.079 | 521.554 | 107.077 | |
PV2CLPS | 539.456 | 85.365 | 524.532 | 106.423 | |
PV3CLPS | 538.610 | 82.942 | 523.859 | 109.509 | |
PV4CLPS | 536.848 | 82.728 | 523.870 | 107.815 | |
PV5CLPS | 538.731 | 84.075 | 520.640 | 105.874 | |
PV6CLPS | 537.490 | 84.474 | 523.832 | 107.336 | |
PV7CLPS | 538.095 | 84.774 | 524.185 | 106.248 | |
PV8CLPS | 537.892 | 83.668 | 522.931 | 107.472 | |
PV9CLPS | 539.101 | 83.465 | 523.231 | 106.905 | |
PV10CLPS | 541.069 | 83.920 | 522.847 | 106.339 | |
PV1MATH | 524.550 | 99.794 | 472.756 | 87.876 | |
PV2MATH | 523.487 | 99.487 | 472.711 | 88.785 | |
PV3MATH | 525.199 | 98.732 | 470.653 | 88.132 | |
PV4MATH | 526.082 | 99.272 | 470.519 | 88.148 | |
PV5MATH | 523.703 | 100.876 | 472.094 | 86.772 | |
PV6MATH | 522.861 | 99.938 | 471.433 | 86.589 | |
PV7MATH | 523.708 | 99.165 | 471.195 | 88.064 | |
PV8MATH | 522.967 | 99.650 | 472.833 | 86.535 | |
PV9MATH | 526.155 | 99.986 | 471.670 | 87.870 | |
PV10MATH | 523.270 | 100.137 | 472.795 | 89.288 |
Table 3
Fixed Effect | Estimate | Std. Error | df | t | Pr(>|t|) |
---|---|---|---|---|---|
Intercept, β0 | 161.100 | 3.566 | 2409 | 45.174 | <0.001 |
PV1CLPS, β1 | 0.596 | 0.006 | 11070 | 103.025 | <0.001 |
COOPERATE, β2 | 1.895 | 2.815 | 10840 | 0.673 | 0.501 |
Z, β3 | –71.690 | 10.560 | 7760 | -6.788 | <0.001 |
PV1CLPS×COOPERATE, β4 | –0.005 | 0.005 | 10840 | -0.973 | 0.331 |
PV1CLPS×Z, β5 | 0.212 | 0.019 | 10510 | 11.238 | <0.001 |
COOPERATE×Z, β6 | 13.680 | 9.576 | 10960 | 1.429 | 0.153 |
PV1CLPS×COOPERATE×Z, β7 | –0.018 | 0.018 | 10950 | -1.007 | 0.314 |
Random effect | Variance | Std. Dev | |||
Intercept, | 595.7 | 24.41 | |||
Residual, | 1058935.4 | 1029.05 |
Table 4
Fixed Effect | Estimate | Std. Error | df | t | Pr(>|t|) |
---|---|---|---|---|---|
Intercept, β0 | 164.300 | 3.555 | 2348 | 46.217 | <0.001 |
PV1CLPS, β1 | 0.589 | 0.006 | 11070 | 103.046 | <0.001 |
CPSVALUE, β2 | 6.749 | 2.840 | 10860 | 2.377 | 0.018 |
Z, β3 | –79.780 | 10.520 | 7641 | –7.585 | <0.001 |
PV1CLPS×CPSVALUE, β4 | –0.020 | 0.005 | 10850 | –3.906 | <0.001 |
PV1CLPS×Z, β5 | 0.228 | 0.019 | 10480 | 12.103 | <0.001 |
CPSVALUE×Z, β6 | 17.170 | 10.100 | 10960 | 1.700 | 0.089 |
PV1CLPS×CPSVALUE×Z, β7 | –0.025 | 0.019 | 10950 | –1.327 | 0.185 |
Random Effect | Variance | Std. Dev | |||
Intercept, | 602.7 | 24.55 | |||
Residual, | 1052086.4 | 1025.71 |
Table 5
t* | U* | B | V(t*) | SE(t*) | |
---|---|---|---|---|---|
Intercept β0 | 180.427 | 3.773 | 59.126 | 63.491 | 7.968 |
CLPS β1 | 0.554 | 0.006 | 0.000 | 0.006 | 0.080 |
COOPERATE β2 | 5.043 | 2.979 | 7.919 | 10.977 | 3.313 |
Z β3 | –57.884 | 11.150 | 92.283 | 104.356 | 10.215 |
CLPS×COOPERATE β4 | –0.008 | 0.006 | <0.001 | 0.006 | 0.075 |
CLPS×Z β5 | 0.190 | 0.020 | <0.001 | 0.020 | 0.142 |
COOPERATE×Z β6 | 4.470 | 10.225 | 32.322 | 42.871 | 6.548 |
CLPS×COOPERATE×Z β7 | –0.005 | 0.019 | <0.001 | 0.019 | 0.138 |
Note: t* represents the final estimate of the coefficients. U* is the average of the error variances and B indicates the variations among all estimates of the coefficients. V(t*) and SE(t*) are the error variance and standard error of t*.
Table 6
t* | U* | B | V(t*) | SE(t*) | |
---|---|---|---|---|---|
Intercept β0 | 182.722 | 3.761 | 56.206 | 60.529 | 7.780 |
CLPS β1 | 0.550 | 0.006 | 0.000 | 0.006 | 0.079 |
CPSVALUE β2 | 6.481 | 2.986 | 7.028 | 10.085 | 3.176 |
Z β3 | –63.285 | 11.054 | 98.067 | 110.101 | 10.493 |
CLPS×CPSVALUE β4 | –0.020 | 0.006 | <0.001 | 0.006 | 0.074 |
CLPS×Z β5 | 0.201 | 0.020 | <0.001 | 0.020 | 0.142 |
CPSVALUE×Z β6 | 7.178 | 10.703 | 35.796 | 46.857 | 6.845 |
CLPS×CPSVALUE×Z β7 | –0.006 | 0.020 | <0.001 | 0.020 | 0.140 |
Note: t* represents the final estimate of the coefficients. U* is the average of the error variances and B indicates the variations among all estimates of the coefficients. V(t*) and SE(t*) are the variance and standard error of t*.
pISSN 2288-7733
eISSN 2288-8357
Frequency : Quarterly