
More detailed information on each course assessment, including the rubric, is located in the Appendix A. Thus, there is a possibility of 0-2 points per task aspect. Task aspects are scored based on a rubric where 0 = non-competent, 1 = approaching competence, and 2 = competent. Therefore, task aspects serve as items in the current analysis. It is also worth noting that every PA task has a written communications aspect (e.g., interpretation of findings to a certain audience) due to its importance in the program. For example, in the case of data analytics, one aspect may correspond with the ability to clean and organize data, another refers to the subsequent analysis performed, and others focus on proficient use of the software used to perform data collection, cleaning, and analysis. All PAs in this course series have just one task but with multiple criteria or “aspects” which correspond with specific skills associated with the overall competency as reflected in the rubric (Baryla, Shelley, & Trainor, 2012). PAs are graded individually by human raters and based on a rubric rather than a cut score. The assessments are also performance-based (PAs).

Thus, all summative assessments are completed online and can be completed at any time in the course (once the student has exhausted and/or mastered application of the skills and competencies inherent in the content, which is done on their own time and at their own pace). All courses are online as part of a competency-based higher education institute, Western Governors University. Data covers a 1-year period and thus includes 2 terms for each student. All students completed one of two possible course tracks for the data analytics concentration of the data analytics program in the College of Information Technology. The current study included a sample of n=922 students. Results supported validity at the course level as well as the presence of a higher-order factor (program-level competency) for the fast course track but not the slow track.

Moreover, the presence of program-level competencies was investigated using hierarchical factor analysis for two groups: a faster, shorter course track and a slow, longer course track. The purpose of this study was to use factor analysis to test the internal and structural validity of course-level performance assessment scores for a group of courses taken as part of a data analytics program in an OCBHE institution. Thus, for students within a given program or major, it is ideal for scores from course assessments with overlapping content to correlate and indicate higher-order skills or competencies. AbstractPrograms in an online competency-based higher education (OCBHE) institute will focus on a set of skills and competencies that form a theme throughout multiple courses, where one course builds upon another in terms of increasing the strength or depth of competency.
