The distribution of learning activities to achieve the ILOs is arranged in the form of courses. Each course has a course learning outcome (CLO), which supports a particular ILO.
The distribution is arranged so that all ILO Study Programs have been supported by the courses shown in Standard 1.
Evaluation of student achievement is carried out in two types: evaluating student performance on a subject and evaluating the ILO program's achievements. Assessments are conducted per course learning outcome by structured homework, quizzes, midterm exams, final exams, or seminars collected and result in the CLO achievement. Every assessment element is weighted for the final score. It is informed at the beginning of the semester and stated in the student handbook module. The course description contains the learning objectives, terms, contents, assessment component, reading reference, evaluation procedure, teaching methods, and assessment standards.
Student course score ranges from 0 to 100, which is an aggregate of the scores of various assessment components that have been designed for the course. Commonly for the class lecture, the largest portion of scores is from the midterm and final exam scores. The percentage composition is varied between course and lecturer, but known by the students joining the class. For practical work and seminars, the final score is the aggregate of the work scores in the laboratory/class and independent work with the same weighting. The final score is calculated based on equation (3.1).
1
1
Final score = ,
n
i i i i i i
i
n
i i i
i
Bt Nt Bq Nq Bm Nm BaNa BpNp Bt Bq Bm Ba Bp
=
=
+ + + +
+ + + +
(3.1)with
: the weight of the - homework score, : the weight of the - quizzes score, : the weight of the - midterm score, : the weight of the - final exam score, : the weight of the
i i
Bt i th
Bq i th
Bm i th
Ba i th
Bp - practice score,
, , , , are the score of each assessment.
i th Nt Nq Nm Na Np
For Community Service (KKN), the assessment is carried out by the supervisor of the student's work at the KKN site, referring to the provisions of the Universitas Brawijaya. The
28
final score of Field Work Practice (PKL) is a combination of field supervisors' scores in industry/institutions and the supervisor's value of the Study Program based on reports, presentation, and other necessary exams per case by case. According to the University academic rules, the aggregate value is converted into letter grades, as shown in Table 3.2.
The results of student achievement towards CLO in all courses are combined according to the formulation shown in Figure 3.1 The student ILOs is calculated using the CLO and related assessment score. The study program ILOs achievement is calculated based on the algorithm in Figure 3.2. The calculation process was done automatically using in-house developed software
Figure 3.1 Algorithm of conversion from the student's assessment scores into student's CLO and ILO scores of a single course
29
Figure 3.2 Algorithm of conversion from ILO scores of all courses into study program's aggregate ILO scores
3.2. Structured of Exam: Cover Intended Learning Outcomes (Knowledge, Skills, Competencies, and Attitude)
The ILOs, which are supported by courses, are classified into four aspects, i.e., knowledge, skills, competence, and attitude. Every aspect is assessed through a combination of examination types, as described in Table 3.1.
Table 3.1 The type of exam used in Mathematics
Exam Type Learning Outcome
Knowledge Skill Competence Attitude Multiple-choice questions ✓
Best short answer ✓
Essay ✓ ✓
Demonstration/presentation ✓ ✓ ✓
Practice ✓ ✓ ✓
Academic document writing and
seminar (for PKL and Thesis) ✓ ✓
𝑃𝐼𝐿𝑂,𝑗 =σ𝐾𝑖=1𝑋𝐼𝐿𝑂, 𝑖𝑗𝑇𝑖𝑍𝐼𝐿𝑂,𝑖𝑗 𝑊𝐼𝐿𝑂, 𝑗 𝑊𝐼𝐿𝑂, 𝑗 =𝑋𝐼𝐿𝑂, 𝑖𝑗𝑇𝑖
𝐾
𝑖=1
•ZILO,ij : average j-th ILO Score of i-th course
•𝑇𝑘 : ECTS weight of k-th course
•XILO, ij : weight of j-th ILO of i-th course
PILO, j : aggregate j-th ILO of study program
WILO, j : weight of j-th ILO of study program
30
3.3 Module Handbook, Distribution and Exam Organization
At the beginning of the semester, the lecturer coordinator makes a lecture plan containing learning objectives, terms, descriptions, contents, assessment components, reading reference, evaluation procedure, teaching methods, and assessment standards (see handbook module). The plan is given to students at the beginning of the class as a lecture contract. Assessment scores are filled in the grading system portal (SIADO, https://siado.ub.ac.id/dosen) by the class coordinator. The student can see their score on their academic record portal (SIAM, http://siam.ub.ac.id). The final score hard copies archived in the Department office.
The university arranges exam schedules for the courses following the academic calendar. Midterm and final exams are scheduled for each semester. Quizzes, assignments, class presentations, and laboratory practice are conducted independently by the lecturer throughout the semester. The typical schedule of these exams are:
1. Midterm Examination: 8th week of the semester 2. Final Examination: 16th week of the semester
The regulations and implementation of midterm and final examinations are stated in the quality manual document https://matematika.ub.ac.id/en/quality-manual/. Exam dates and times are openly announcing on the Faculty website. The management and process processes of the midterm and final examination run well and do not impede the learning process.
A couple of weeks before the examination time, the students can see the schedule. A student may ask to change the schedule if burdening students simultaneously; otherwise, the schedule is determined to be the semester's exam schedule. Schedule rearrangement may apply based on student objection due to their load and announced, at least, a week before the time.
The management encourages lecturers to provide feedback on each student assessment for student improvement. However, there is still a problem with the implementation. Some lecturers have done this. Whilst some others did irregularly, and few did not.
Lecturers have one week after the exam submit all the assessment scores for the final student grade. The dateline was set by the faculty based on the academic calendar. The system will automatically set the student score to "B" for the overdue submission of the lecturer.
Grades included in the degree are listed on the student’s transcript (Kartu Hasil Studi).
The description of the scoring system is given in Table 3.2.
Table 3.2 The conversion of the exam score to the letter grade 0-100 Scale Letter Grade Quality Index Description
>80 – 100 A 4 Excellent
>75 – 80 B+ 3.5 In between Excellent and good
>69 – 75 B 3 Good
>60 – 69 C+ 2.5 In between Good and Fair
>55 - 60 C 2 Fair
>50 - 55 D+ 1.5 In between Fair and Poor
>44 - 50 D 1 Poor
0-44 E 0 Fail
31
After the final grades are announced, students are encouraged to fill the class feedback and evaluation by filling the online questionnaire at http://siam.ub.ac.id. The questionnaire covers the course teaching and learning process, teaching method, clarity of teaching material delivery, assessment transparency, and conformity of the material with the handbook module.
Students are also able to provide written advice and criticism on the satisfaction of their learning experience. Filling out the questionnaire and giving suggestions is the students' requirements to fill out a study plan in the following semester.
The resulting scores and feedback are used to write the course portfolio in reflections on the lecture implementation and suggestions for future lectures. The portfolios are handed to the head of the study program. An evaluation meeting is conducted at the end of each semester for improvements.
Students may repeat taking a course or a semester break course as they like. The semester break program is carried out between even and odd semesters, where the credit load and evaluation are the same as the regular semester. Semester break provides students opportunities to improve the grades of courses taken to increase the cumulative grade point index, shorten the study period and avoid dropping out of the study. The student can take only a limited number of courses. At the semester break program, the classes are intensified.
3.4 Attending the Exam
Students must attend the midterm and final exam if they pass the minimum attendance of the class. A minimum attendance of 80% is required. Students can request a follow-up exam if they cannot follow the specified schedule due to acceptable reason. Students may ask for the follow-up exams in case of sick (proven by doctor’s certificate), accident that prevents attending exam (proven by police and hospital certificate), parents or siblings passed away (a certificate from the community leader (RT)). The vice dean for academic affairs will consider other force majeure with strong evidence. The rule for a follow-up exam is:
1. The student reports their absence from the exam and the desire to take the follow-up exam no later than 3 (three) days after the exam in the course concerned, except in force majeure conditions
2. Submit a letter of application to the Vice Dean for Academic Affairs by enclosing evidence stating the reason for not attending the exam
3. The proof letter must be sent no later than 7 (seven) days after the exam of the course concerned except in force majeure conditions
4. Reason and proof document
Academic attitude violations during the exam or classes will be penalized according to the Faculty Academic regulation (https://mipa.ub.ac.id/en/academics/)
32
3.5 Exam Result Transparency
Students can see their score at the academic portal. Students have the right to ask for their scores based on the exam answer sheets assessed. Students who are not satisfied with the assessment results can ask for exam re-corrections from the class lecturer. Students are also entitled to obtain information about applying the exam criteria to achieve their learning outcomes.
The scoring mechanism for each type of student exam is carried out transparently. It is guided by the student exam assessment rubric (the ILO rubric is given in https://matematika.ub.ac.id/wp-content/uploads/sites/3/dokumen/pedoman-
akademik/Pedoman-akademik-sarjana-FMIPA-EN.pdf. The weight of each assessment has been determined in the Module Handbook. Besides, for courses with parallel classes, the midterm and final exam questions are the same. It is to minimize the learning outcome evaluation discrepancies. The correction mechanism is carried out crosswise between lecturers, using standard answers that already have a score. Also, the scoring process uses the same answer and grading references for essay questions.
However, if there are still complaints from students about the final score of the course that has been taken, FMIPA UB has rules that are given in the Academic Guidebook of FMIPA UB. The student may complain to the lecturer, asking and provides proof of the wrong score or writing the issue to the head of the study program if s/he is not satisfied with the grade given by the lecturer, assuming there is unfair judgment. The head of the study program should handle the issue. These rules provide an opportunity to revise student grades that have been uploaded on the system, with a maximum time limit of 1 week after the grades are uploaded.
Until now, there have been no complaints about student scores that have not been resolved through the existing procedures..
3.6 Field Work Practice (PKL) and Community Service (KKN)
Some courses are implemented outside the campus are Field Work Practice (PKL) and Community Service (KKN). These courses' purpose is to make students learn how they apply their knowledge and skills in the real world. It is also intended that the student gain valuable experience that boosts their confidence and increases working readiness.
The implementation of KKN is coordinated at the faculty and university level through an institution called the Institute of Research and Community Services (LPPM). Field supervisors assess student KKN activities with an assessment component of student abilities: attitude, time management, communication, teamwork, solve problems using the scientific knowledge learned, and build relationships.The achievement of these aspects is a component of the study program learning objectives formulated in the the ILO 7 and 8. The final score of students KKN from field supervisors is then inputted into the value portal through the academic section of the Faculty of Mathematics and Natural Sciences UB..
33
In implementing PKL, students are given the freedom to choose PKL locations to follow the scientific field occupied on campus. The process of selecting a PKL location begins with submitting a proposal to the desired PKL location. Administratively, the department will provide cover letters to students to submit PKL applications to industry/institutions. In addition, the department (based on the study program request) appoints a lecturer to guide the student from the preparation process to the end of the PKL implementation.
The learning process is usually carried out for one to 2 months. The final score for PKL is the aggregate value of field supervisors in industry/institutions and supervisors from Study Programs with a certain weight. The final score of students PKL is a combination of field supervisors' score in industry/institutions and the score of supervisors from the study program.
Assessment of the results of student PKL activities is based on student abilities in the following aspects: adapting to a dynamic and heterogeneous environment, attitudes, time management, communication, teamwork, solve problems in the field using the scientific knowledge learned and build networking. The achievement of these aspects is a component of the study program learning objectives formulated in the ILO 7 and 8. However, the assessment standard for field supervisors is not yet available. So far, the implementation of KKN and PKL has been running well and has resulted in the achievements of the program study ILO related to good results (Standard 6)
3.7 Final Project
Completing the final undergraduate project is a must for the Bachelor's Degree according to Regulation of Rector of Universitas Brawijaya Number 223/PER/2010. The final project is a scientific work that can be written based on independent research or other equivalent work determined by faculty, such as literature studies, fieldwork practices, or internships. The undergraduate thesis is designed to be completed by students during one semester. Guidelines and writing of proposals and thesis are on the Faculty of Mathematics and Natural Sciences website (https://mipa.ub.ac.id/en/academics/).
Assessments for the final project are proposal, work result and process, presentation, and comprehensive examination. Proposal seminars and result seminars are carried out openly, while the exam thesis is close. Overall assessment includes writing, the subject's difficulty level, mastery of the subject, and thesis presentation. The students must present and defend the thesis in front of the board of examiners.
In Mathematics, the thesis topic is generally associated with theoretical investigation, empirical study, and literature studies. The topics and content of the thesis are free, where students have consulted with their advisor beforehand. One advisor guides the thesis writing.
The thesis's evaluation is done through 3 (three) stages: proposal seminar, result seminar, and thesis exam, with the weight of each component, which is 20%, 50%, and 30%. All the implementations of these stages are regulated in the Standard Operating Procedure (SOP) of the Mathematics Department ( https://matematika.ub.ac.id/s1-matematika/en/sop/).
34
3.8 Plagiarism
To prevent plagiarism, UB develops service facilities to check the originality of documents with Turnitin software. For undergraduate students, checking the plagiarism of their thesis / scientific paper is still optional. However, several final project supervisors have implemented rules for conducting plagiarism checks for their students, especially for students whose scientific papers are published at the national and international levels. Also, there is an obligation that at the beginning of the final project paper, students are required to provide a statement stating that the results of their research are not plagiarism. All references used in the manuscript must be written in the bibliography, and the bibliography must be cited correctly.
If plagiarism is found, then the final project is canceled. These regulations are listed in the Faculty of Mathematics and Natural Sciences Academic Guidance (https://mipa.ub.ac.id/en/academics/). So far, there were no students penalized due to the final project work plagiarism.
3.9 Strength, Weakness, and Area of Improvement
Table 3.3 Strength, Weakness, and Area of Improvement
Strength Weakness
• The exam system is well established and can measure the LO achievement qualitatively and quantitatively.
• The assessment standards have been well formulated to minimize bias towards the assessment.
• There is a transparency of student scores with a standardized system.
These scores can be accessed through the academic record portal (SIAM).
• The duration of 1 week provided for lecturers to correct the student work and upload grades is insufficient so that the lecturer does not have enough time to provide feedback to students.
• There is no standard guideline for assessing PKL courses for field supervisors.
Area for Improvement
• Reevaluate the rules for the duration of time provided between the exam and the upload of student test scores on the academic portal to give lecturers the opportunity to provide feedback to students
• Build communication with industry to formulate student PKL assessment standards.