Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=vjeb20
Journal of Education for Business
ISSN: 0883-2323 (Print) 1940-3356 (Online) Journal homepage: http://www.tandfonline.com/loi/vjeb20
Effectiveness of Web-Based Courses on Technical
Learning
Monica Lam
To cite this article: Monica Lam (2009) Effectiveness of Web-Based Courses on Technical Learning, Journal of Education for Business, 84:6, 323-331, DOI: 10.3200/JOEB.84.6.323-331 To link to this article: http://dx.doi.org/10.3200/JOEB.84.6.323-331
Published online: 07 Aug 2010.
Submit your article to this journal
Article views: 40
View related articles
he Web influences every aspect of life, including how individu-als learn. At present, individuindividu-als can earn a full degree by way of theWeb. TraditionaluniversitiesalsoofferWeb-based (WB) courses to enhance their deliverychannels.Inthepresentstudy, I investigated the effectiveness of WB courses on students’ technical learning as measured by students’ final exami-nation scores. Along with the deliv-ery method factor (WB or traditional classroom[TC]courses),Ialsoadopted students’cumulativeGPA(foracademic standing),gender,ethnicity,andevalua-tion method (multiple-choice or prob-lem-solvingquestions)asthe predictor variables. I applied multiple regression analysestotheentiredatasetandsub-setsofdata.
LiteratureReview
The WB learning phenomenon has become increasingly prevalent and sig-nificant. Many indicators, including the percentage of colleges that offer WB learning, expenditure on WB learning technology,WB course enrollment, and online tuition and fees earned by edu-cational institutes, show the dramatic upward trends of WB learning and its variants (Quinn et al., 2006; Symonds, 2001).WBlearninghasalsopenetrated traditional brick-and-wall campuses, which are proud of their classroom
teaching. In the United States, the Uni-versity of Maryland, the largest state university, offers students more than 70 different degree and certificate options bywayoftheWeb.Professionaldegrees, which rely on discussion, interaction, networking,andcasestudiesasprimary learning techniques, are no exception. Forexample,ConcordLawSchooloffers onlinelawdegrees.DukeUniversityhas a global executive MBA program that allowsworkingexecutivestofinish65% ofthecurriculumovertheWeb(Arbaugh, 2000; McCallister & Matthews, 2001; Symonds).Manymoreequivalentexam-plescanalsobefound.
WB learning has its advantages and disadvantages.Onthepositiveside,WB learning has no classroom restrictions. Studentscanlearnattheirownpaceand ataconvenienttimeandplace.Thisis especially important for working indi-vidualsandnontraditionalstudentswho arephysicallyseparatedfromcampuses or cannot frequently commute to cam-puses.WBlearningalsohasthebenefit of transferring the control to students (Kochtanek&Hein,2000;Lin&Hsieh, 2001).Studentscanmovebackandforth between Web pages, spend as much timeasnecessaryonacertaintopic,and revisit pages for difficult topics. WB courses also allow instructors to orga-nize the course content into a logical and written format that is beneficial to studentswhodonothavegoodlistening
EffectivenessofWeb-BasedCourseson
TechnicalLearning
MONICALAM
CALIFORNIASTATEUNIVERSITY SACRAMENTO,CALIFORNIA
T
ABSTRACT.Theauthorinvestigated theeffectivenessofWeb-basedcourseson technicallearning.Theregressionresults showthatthedeliveryformat(Web-based ortraditionalclassroomcourses)hasno significanteffectonstudentperformance. However,althoughgenderisasignificant predictorintraditionalclassroomcourses, itseffectdisappearsinWeb-basedcourses. ThereisevidencethatWeb-basedcourses canbeconducivetotheleaningprocessof technicalknowledgeforfemalestudents. Forthehigh-GPAsubgroup,thepredictors ofethnicity,GPA,andproblem-solving questionsasanevaluationmethodwere positivelyassociatedwithperformance.
Keywords:ethnicity,evaluationmethod, genderdifference,studentperformance, Web-basedlearning
Copyright©2009HeldrefPublications
skills.Alternatively,WBcoursesrequire studentstohavegoodreadingskillsand self-discipline. Students may also have isolation problems and technical diffi-culties(Palloff&Pratt,1999;Sweeney & Ingram, 2001). The basic question in the present study is whether WB courses are better than TC courses in termsofdeliveringtechnicalknowledge and,ifso,whatkindsofstudentsbenefit the most. In the remainder of this sec-tion,Ireviewrelevantresearchfindings work include subject matter, measures forcourseeffectiveness,studentcharac-teristics,andresearchresults.Ipresent a brief explanation of the discussion frameworkbeforetheliteraturereview. Thereviewrevealswhichaspectsofthe research question in the present study remainunanswered.
Thefirstfactorinthediscussionframe-work is subject matter. In their meta-analysis of WB instruction, Sitzmann, Kraiger, Stewart, and Wisher (2006) differentiated between declarative and proceduralknowledgetoinvestigatethe effectivenessofWBinstruction. Declar-ativeknowledgereferstothememoryof facts,principles,andrelationsofknowl-edge elements and cognitive strategies for accessing and applying knowledge. Alternatively, procedural knowledge referstohowtoperformatask,includ-ing compilation steps, traversal strate-gies, and optimization methods. In the presentstudy,myaimwastounderstand the effect of WB courses on technical learning, which falls into the domain of procedural knowledge. The second factorofeffectivenessinthediscussion framework can be classified into the twocategoriesofperformanceandper-ception.Performanceisstudents’actual learningresultsfromobservablebehav-ior,suchasexaminationscores. Percep-tion, on the contrary, is opinion-based andsubjectiveconcerningstudents’sat-isfaction and perceived usefulness of WBcourses.Imeasuredstudents’actual performancefromacomprehensivefinal examinationinaprogrammingclass.For studentcharacteristics,thethirdfactorin the discussion framework, there were
variablessuchaslearningstyle,ethnic-ity, gender, age, prior knowledge, and learningskills.Theresearchrationalefor investigatingstudentcharacteristicswas based on the assumption that students with different profiles would respond differently to WB courses, leading to different degrees of performance and perception. Regarding research results as the fourth factor in the discussion framework,theoutcomesareasfollows: WBcoursesaremoreeffectivethanTC courses, TC courses are more effective than WB courses, or TC courses have thesameeffectivenessasWBcourses.
NoPerformanceDifferenceBetween WBandTCStudents
Six recent studies (Friday, Friday-Stroud, Green, & Hill, 2006; Jones, Moeeni,&Ruby,2005;Piccoli,Ahmad, & Ives, 2001; Priluck, 2004; Scheines, Leinhardt,Smith,&Cho,2005;Smea-ton&Keogh,1999)reportedthatthere was no student performance difference betweenWBandTCcourses.Smeaton and Keogh, Piccoli et al., and Jones et al. had IT-related courses as the sub-ject matter, whereas Priluck, Scheines etal.,andFridayetal.useddeclarative knowledge as the subject matter. Pri- luckalsoinvestigatedstudents’satisfac-tion,whichwashigherforTCthanfor WBcoursesintermsofteambuilding, criticalthinking,oralandwrittencom-munications, global perspective, and social interaction. That provides some evidencethathighsatisfactiondoesnot necessarily lead to high performance inWBorTCcourses.Lee(2001)also foundthatstudents’satisfactionhadno relation to self-reported achievement in WB courses, which reinforces the claim that performance, self-reported or actual, is not necessarily related to satisfactionwithWBcourses.
WBLearningBetterThanTCLearning There are studies showing that stu- dentsinWBcoursesachievebetterper-formance than in TC courses (Bryan, Campbell, & Kerr, 2003; Chou & Liu, 2005; Lockyer, Patterson, & Harper, 2001).Lockyeretal.studiedtheeffect ofWBlearninginundergraduatehealth care courses and concluded that WB
studentsscoredhigherthandidTCstu-dents. Regarding student participation, Lockyer et al. found thatWB students generated higher quality discussion by providing more in-depth content and references,whereasTCstudentsgener-ated a higher quantity of discussion. For Bryant et al., the subject matter was an introductory information sys-tem course for undergraduate students. That study concluded that WB learn-ing was significantly better than TC learning for concept tests,TC learning was marginally better than WB learn-ingforgroupproject,andWBlearning wasjustaseffectiveasTClearningfor activity folio. Chou and Liu measured differencesinstudentperformanceand perception between WB learning and TClearningforahighschoolIT-related better performance from TC learning compared with WB learning. Sweeney and Ingram (2001) investigated mar-keting students’ perceived learning effectiveness in WB and TC learning. Sweeney and Ingram found that TC learning was perceived to have higher learning effectiveness in a tutorial set-ting.Bryanetal.(2003)foundthatTC learningwasmarginallybetterthanWB learning for group projects. Maki and Maki (2002) compared the test scores betweenWBandTClearninginintro- ductorypsychologycourses,asmoder-atedbystudents’comprehensionskills. Maki and Maki determined that stu-dentswithhighercomprehensionskills performed better in WB than in TC courses.However,comprehensionskills were not significantly associated with satisfaction,andallstudents,regardless oftheircomprehensionskills,preferred TCtoWBcourses.
EffectsofStudentCharacteristics
I report recent research findings of the effects of student characteristics on performance and perception for WB and TC learning. Wang and Newlin’s (2000)studyaboutpsychologystudents revealed no student demographic dif-ferencesbetweenWBandTClearning.
Bryan et al.’s (2003) study determined that academic standing (measured by tertiary entrance score) significantly affected performance. Alternatively, Jonesetal.(2005)foundthatacademic standing (as measured by GPA), age, workhours,andpreviouscourseshadno effectonstudents’performanceforWB and TC learning in an undergraduate telecommunication course. Jones et al. didnotconsidergenderandethnicityas predictors. As suggested by theoretical analyses (Prinsen, Volman, & Terwel, 2007)andexperimentalresearchresults (Arbaugh 2000; Lu,Yu, & Liu, 2003; Wallace & Clariana, 2005; Wernet, Olliges, & Delicath, 2000), gender and ethnicity may have significant effects onperformancedifferencebetweenWB andTClearninginregardtoprocedural knowledge, which deserves research attention. In a study comparing perfor-mance difference between a paper test andanonlinetestforanundergraduate informationsystemscourse(Wallace& Clariana), researchers discovered that non-White female participants scored the lowest for online tests on the mid-term,butthehighestforonlinetestsfor thefinal.However,WallaceandClariana onlyinvestigatedthedifferencebetween paper and online tests, not the differ-ence betweenWB and TC learning. In other words, the treatment difference was not aboutWB and TC instruction, butjustaboutthemediumfortestdeliv-ery. When the investigation target was WB learning only (i.e., no comparison between WB and TC learning), Lu et al. concluded that learning style, gen-der, age, job status, year of admission, onlineexperience,andMISpreparation hadnoeffectonstudentperformancein an MBA or MIS course. In Lu et al.’s study,theonlystudentcharacteristicthat had a significant effect was ethnicity, whichidentifiedWhitesashavingbetter performancethanBlacksinWBcours-es.InanotherWB-onlystudy,Arbaugh showed that older female students may havestrongerleaningexperiencesinWB courses. Nevertheless, Arbaugh only covered students’ perception, not their actualperformance.Wernetetal.studied socialworkstudents’satisfactioninWB andTCcoursesandconcludedthatolder nontraditionalstudentsfoundWBlearn-
ingmoreappealingandhadhighersat-isfaction with it.A meta-analysis study regarding gender differences on com-puter-supported collaborative learning concluded that WB courses provided a more equitable discussion environment inwhichwomenfeltmorecomfortable participating(Prinsenetal.).
Insummary,themajorityofresearch results from existing literature, which indicates thatWB learning is as effec-tive as or equivalent to TC learning forstudentperformance.However,itis not clear for most studies whether the subjectmatterisdeclarativeknowledge, proceduralknowledge,oramixtureof both.WhenTClearningwasconsidered as more effective than WB learning, it was for perceived learning, group project,andsatisfaction.Regardingstu-dent characteristics, some studies have shownthatage(Jonesetal.,2005;Luet faction (Arbaugh, 2000; Wernet et al., 2000).Forothercharacteristics,suchas ed it to be insignificant. Similarly, for ethnicity as a performance predictor, Lu et al. reported it to be significant, but Wang and Newlin (2000) reported it to be insignificant. As for gender asaperformancepredictor,studiesfor gendereffecthavefocusedonWebstu-dentsonly(Luetal.),testmediumonly (Wallace&Clariana,2005),ordeclara-tive knowledge only (Scheines et al., 2005).The effect of gender on perfor-mancedifferencebetweenWBandTC learning for technical subjects has not beendetermined.
StatementoftheProblem
In the present research, I investigat-ed the effect of student characteristics (GPA,ethnicity,gender,hitrate,andread rate),deliverymodes(WBorTC),and evaluation methods (problem-solving or multiple-choice questions) for the overall performance of students (final exam score) in technical
undergradu-atecourses.Thepredictorvariablesand dependentvariablearefullydescribedin theMethodsection.Thesignificanceof this research is threefold. First, educa-torsneedtounderstandtheeffectiveness of WB learning for different students in different subjects. The present study sheds light on the effectiveness ofWB learning on technical subjects such as programming and information system development.Technical learning, by its nature, fits the definition of procedural knowledge. There are many aspects of technical learning that can be difficult to deliver by way of Web pages. For example, debugging, as an important skillforstudentstomasterinprogram-ming classes, may be more effective-ly taught in a face-to-face laboratory environment. Alternatively, the logical understandingofprogrammingfallsinto the area of higher order skills, which is claimed to be an effective learning objective forWB courses (Kao, 2002). Inthepresentstudy,Iaimedtofindthe overalleffectivenessofWBlearningfor technicalsubjects,whichhavebothpros and cons forWB delivery. Second, the presentstudyprovidesguidanceforthe pedagogy of WB courses for techni-cal subjects. Programming courses are usually difficult for students to mas-ter. Understanding how different ques-tion formats can promote learning is beneficialtoinstructors’teachingplans. The treatment of evaluation method in the present study provides information about this issue. The third contribution ofthepresentresearchisunderstanding theeffectofGPA,gender,andethnicity on performance difference for proce-dural knowledge between WB and TC learning,whichhasnotbeenconfirmed intheliterature.
METHOD
The present research is an empirical study using data from an undergradu-ate elective programming course that I taughtfor2yearsandapplicationdevel-opment using Visual Basic. I collected data from class records and students’ recordsintheuniversitysystem.Thepre-dictor variables included students’ gen-der, ethnicity, GPA, hit rate, read rate, delivery mode, and evaluation method. The dependent variable was students’
final examination scores at the end of a semester. I used analyses of variance (ANOVAs) and regression analyses as thestatisticaltoolsforthisresearch.
Participants
Participants were 364 students from sixWBandthreeTCcoursesin2aca-demic years (221 WB students, 143 TC students). Tables 1–3 provide the descriptivestatisticsforparticipants,by ethnicity, gender, delivery method, and evaluation method. The course was an undergraduate elective programming classforMISmajorsusingASP.Netto develop Web applications. I designed andtaughtalltheWBandTCsections. The prerequisite for enrolling in the targeted class was a passing grade in an advanced Visual Basic class. The only requirement for enrolling in WB courses was Internet access. Students chose freely whether to enroll in WB orTCcourses.Duringthestudyperiod, there were no cases in which students couldnotenrollinaWBorTCcourse becausethesectionwasfull.
Treatment
I designed the WB course using WebCT. The main page of the WB coursehastheiconsofsyllabus,assign-ments, PowerPoint slides, exams and quizzes,discussionboard,e-mail,sam- pleprograms,labexercises,andacalen-dar.TheWBcoursedividedintoseven sections. Students were advised to fin-ish each section following a flowchart thatoutlinedstepbystepwhattodoby Quiz records show that approximately 95%ofstudentsfinishedtheirquizzeson time. Students accessed quiz questions and their answers after the deadline, eventhoughtheydidnottakethequiz. Laboratory exercises were designed for students to practice programming concepts and techniques. WB students
were advised to do their lab exercises in one of the student labs any time by a certain deadline.Although there was no way for me to check whether stu-dents finished their laboratory exercis-es, that follow-up quiz questions were partially based on laboratory exercises providedincentiveforstudentstowork on laboratory exercises on their own. WB students were encouraged to e-mail the instructor whenever they had questionsorproblemswiththeWebCT system.Somestudentscomplainedthat theWebCTsystemdidnotrecordquiz scores correctly or the screen froze up after a few questions. I allowed WB students to delete approximately 15% ofthelowestquizscorestocompensate for any technical problems that were outoftheircontrol.WBstudentswere requiredtocomebacktotheclassroom totakeallexaminationsinperson.
TheTCsectionofthecoursehadall thesameteachingmaterialsastheWB section except for a few differences. First,IpresentedallPowerPointslides intheclassroom,andsecond,Iadmin-istered all quizzes in the classroom. Quiz questions were also randomized in the lecture section. Third, students
didtheirexercisesinalaboratoryenvi-This section describes GPA, eth-nicity, gender, hit rate, and read rate as the predictor variables for student performance. The ethnicity predictor has Black, White, Hispanic, Indian or MiddleEastern,andAsianasvariables in the present study. I observed that in technical subjects, students with the sameethnicitytendedtocollaborateand assistoneanother.Thefactorofunder-graduate classes is likely to intensify the phenomenon of mutual assistance because undergraduate students have more opportunities to stay on campus thandograduatestudents.BecauseWB learningisnotyetcommononthecam- pusonwhichthisexperimentwascar-ried out, undergraduate students who enrolledinWBcoursesstillhadplenty aforementioned issues, I hypothesized
TABLE1.DescriptiveStatistics forCategoricalVariables(N= 364)
Variable n
Ethnicity
Black 12
White 133
Hispanic 40
IndianorMiddleEastern 15
Asian 164
Gender
Male 209
Female 155
Deliverymethod
Web-based 221
Traditionalclassroom 143 Evaluationmethod
Problemsolving 150 Multiplechoice 214
TABLE2.Students,byDelivery andEvaluationMethods
Evaluationmethod Delivery Problem Multiple method solving choice
Web-based 110 114 Traditional
classroom 40 103
TABLE3.EthnicityandGender DistributionforWeb-based (WB)andTraditionalClass-room(TC)Students
Variable TC(%) WB(%)
Ethnicity
Black 2.79 3.61 White 31.46 39.81 Hispanic 10.48 11.31 IndianorMiddle
Eastern 4.89 3.61 Asian 50.34 41.62 Gender
Male 56.64 57.91 Female 43.36 42.08
thatethnicity would have a significant effect on undergraduate technical WB learning.Regarding gender as a pre-dictor for performance, the literature has not provided a complete picture. I hypothesized there would be a sig-nificant gender effect on performance betweenWBandTCstudents.
GPAisstudents’cumulativeGPAup tothesemestertheytooktheprogram-ming course under investigation. GPA measures students’ overall academic standing on the basis of all courses that they have taken at the university. Overall academic standing indicates a student’s discipline, study habit, and performance,whichusuallyisareliable predictor for new academic endeavor. HitandreadratesarevariablesforWB students only.Hit rate is the number ofaWBstudent’slog-onsessions,and
read rate is the number of teaching materials a WB student retrieves to readfromtheWBcourse.Hitandread rates measure a student’s enthusiasm to access course materials on theWeb. I hypothesized that GPA, hit rate, and readratewouldbesignificantpredictors forperformanceinWBcourses.
HypothesisforDeliveryMode andEvaluationMethod
The delivery mode of WB or TC learning is a major treatment in the present study. I hypothesized that dif-ferent delivery modes would generate differentperformance.Thepredictorof evaluationmethod,asanothertreatment in the present study, determines the question format in the midterm: mul-tiple-choice or problem-solving ques-tions. Midterms prepare students for theirfinalcomprehensiveexamination. Because problem-solving questions involve in-depth analysis of problems and practice on solution development, I hypothesized that problem-solving questions would be associated with higheroverallperformancethanwould multiple-choicequestions.
OverallPerformanceDifference The total score in the final compre-hensive examination was used as the dependentvariableinthepresentstudy. The final comprehensive examination was the same for all students in all
semesters. The final comprehensive examination had a variety of question formats including multiple choice, problem solving, programming, and concept definitions. I chose to use the totalscoreratherthanthefinalgradeas thedependentvariablebecausethetotal scorewasmorecompatibleamongdif-ferentsectionsthanwasthefinalgrade, which may have to be normalized in somesections.
AnalyticalProcedure
Thefirststepinthedataanalysispro-cess was to check student distribution intermsofgender,ethnicity,GPA,and totalscorebetweenWBandTClearn-ing.IappliedanANOVAtodetermine whether there was a significant GPA andtotalscoredifferencebetweenWB and TC learning. In the second step, I applied multiple regression analyses to theentiredatasetwithallvariables.In thethirdstep,Iappliedmultipleregres-sionanalysestothedatasubsetsofWB and TC learning. In the fourth step, I applied multiple regression analyses to thestudentsubgroupsoflow,medium, and high GPA. All normal probability plots for regression models showed no signofassumptionviolationandprob-lematicresidues.
RESULTS
GeneralDataDistribution
Tables 1–3 show the student distri-bution by ethnicity, gender, delivery method, and evaluation method in the entire data set. There were 364 stu-dentsintheentiredatasetinwhichthe majority was Asian (164) and White (133).Thereweremoremale(n=209) andWB (n = 221) students than there werefemale(n=155)andTC(n=143) students in the entire data set. As for the evaluation method shown in Table 2, the data set has more students with multiple-choice questions (n = 217) than with problem-solving questions (n = 150) in midterm examinations. Comparingethnicityandgenderdistri-butionforWBandTCstudentsinTable 3, the main difference is the higher percentage of White students and the lower percentage of Asian students in WB courses. Also, as shown in Table
1,thepresentstudyhadasmallsample sizeforBlack(n=12),Hispanic(n= 40),andIndianorMiddleEastern(n= 15) students. To check for significant mean differences for GPA and total score betweenWB andTC students, I performed anANOVA. Table 4 shows the maximums, minimums, standard deviations, and means for GPA and totalscore,asclassifiedbyWBandTC learning.Themeandifferencesbetween WBandTClearningforGPAandscore are not significant. The insignificant totalscoredifferencebetweenWBand TC learning confirms those research resultsfromtheliteratureclaimingWB learning as equivalent to TC learning for learning procedural knowledge. Table5showsthedescriptivestatistics
Table 6 shows the regression results forallstudentsinthedataset.Model1.1 inTable6hasallthepredictorvariables.
TABLE4.DescriptiveStatistics forGPA(Predictor)andScore (DependentVariable)
Statistics GPA Totalscore
Maximum 4.00 102.10 Minimum 1.67 28.33 SD 0.52 12.53 M 2.86 71.23 MDifference
WB 2.85 71.36 TC 2.86 71.01
p .82 .79
Note. WB=Web-based;TC=traditional classroom.
TABLE5.DescriptiveStatistics forHitandReadVariablesfor Web-basedStudents
Statistics Hit Read
Maximum 583.00 62.00 Minimum 33.00 0.00 SD 179.34 36.89 M 87.62 16.27
BecauseGPAisthedominantpredictor, IremoveditfromModel1.2torevealthe significanceforothervariables.Models 1.1 and 1.2 are significant, but Model 1.2hadasmalladjustedR2(.037).Only
predictorswithasignificancelevelof≤ .05werelistedinregressionresults.In Model1.1,whichisthemodelwithall predictorsforallstudents,onlyGPAisa significantpredictoratthesignificance levelofzero,andithasastandardized coefficientof.57.InModel1.2,which hasallpredictorsexceptGPAforallstu-dents,Whitehaspositiveeffect.Inother words,whenGPAisignored,Whitestu-dentstendtohavehigherscores.
Table 7 shows the regression results forWBstudents.InModel2.1(withall predictors),themodelissignificant,F(1)
=0,p =0.Again,GPAistheonlysignif-icantpredictoratasignificancelevelof zero.TheresultsfromModel2.1(WB) aresimilartothosefromModel1.1(all students) with GPA as the only highly significantandpositivepredictor.Model 2.2 (without GPA for WB students) is alsosimilartoModel1.2(withoutGPA forallstudents),ignoringtheBlackand IndianorMiddleEasterneffectsbecause oftheirsmallsamplesizes.
Table 8 shows the regression results forTCstudents.Model3.1isthemodel withallpredictorsforTCstudents.The model is highly significant,F(3) = 0 (differentpredictorshavedifferentp val-ues; see Table 8), with a high adjusted R2(.462).Female,problemsolving,and
GPA are the three significant predic-tors, which have –.16, .267, and .612 as the standardized coefficients, respec-tively. The standardized coefficients in Model 3.1 indicate the following: (a) Female students in TC courses tend to havelowscores;(b)problemsolvingas theevaluationmethodisassociatedwith highscores;and(c)GPAisstillthebest predictor for score. Comparing Model 3.1 (all predictors for TC) with Model 2.1 (all predictors for WB students), I noticedthatthepredictionpowerofGPA is stronger (β = .612) in TC courses thaninWB(β =.549)courses.Where-as female and problem solving have no significant effect on WB learning, they respectively have significantly negative andpositiveeffectsonTClearning.The insignificance of female and problem solvingonWBlearningmaybebecause
oftheflexibilityoflearningpaceinWB courses, which eliminates the learn-ing difficulties for some students inTC courses.InModel3.2,femaleandprob- lemsolvingarestillthesignificantpre-dictorsforTClearningaftertheremoval oftheGPApredictor.
PredictorPowerofHigh,Medium, andLowGPA
Because GPA is a highly significant predictor, I classified students into high-, medium-, and low-GPA groups to perform further analyses: high GPA
≥ 3.1; medium GPA≥ 2.5, but≤ 3.1; and low GPA≤ 2.5. Table 9 shows
theregressionresultsforthehigh-GPA (Model4.1),medium-GPA(Model4.2), and low-GPA (Model 4.3) groups. All models are highly significant, but only thehigh-GPAgrouphasadecentadjust-edR2(.302).Thesignificantpredictors
inthehigh-GPAgroupareAsian,prob-lem solving, and GPA. All significant predictorsinthehighGPAgrouphave positive standardized coefficients. In otherwords,inthehigh-GPAgroup,the predictorsofAsian,GPA,andproblem solving as the evaluation method have asignificantimpactonperformance.In the medium-GPA group, no predictor is significant. In the low-GPA group, no predictor has a positive effect, but
TABLE6.RegressionResultsforAllStudents
Model Standardized
Model significance(F) df AdjustedR2 Variable,*p coefficients
1.1(withall 0.000 1 .323 GPA,0 .57 variables)
1.2(without 0.001 3 .037 Black,.031 –.113
GPA) White,.008 .140
IndianorMiddle .109
Eastern,.037
*p≤.05
TABLE7.RegressionResultsforWeb-basedStudents
Model Standardized
Model significance(F) df AdjustedR2 Variable,*p coefficients
2.1(withall 0.000 1 .298 GPA,0 .549 variables)
2.2(without 0.004 2 .041 Black,.071 –.121
GPA) White,.012 .17
*p≤.05
TABLE8.RegressionResultsforTraditionalClassroomStudents
Model Standardized
Model significance(F) df AdjustedR2 Variable,*p coefficients
3.1(withall 0.000 3 .462 Women,.011 –.160 variables) Problemsolving,0 .267
GPA,0 .612
3.2(without 0.001 2 .086 Women,.05 –.159 GPA) Problemsolving,.001 .264
*p≤.05
problemsolvinghasanegativeeffecton score.Figures1and2summarizethose importantfindings.
DISCUSSION
WBasaViableAlternativetoTC LearningforTechnicalCourses
In the present study, I found that deliverymode,WBorTClearning,has noeffectonstudents’technicallearning performance,whichissupportedbyan ANOVA (see Total Score in Table 4) and regression analyses (see Table 6). Regression analyses show that GPA is the only significant predictor for all students. It indicates that students’ academic standing mostly determined their technical learning performance no matter which delivery mode they were subject to. This finding assures educators of the equivalence of WB and TC learning in technical knowl-edgedelivery.Underbudgetconstraints andmanpowershortages,WBlearning is a viable alternative to TC learning. However,educatorsneedtopayspecial attentiontothedesignofWBcoursesto makesurethatstudentsreceiveappro-priatefeedbacktoguidetheirself-study process. Technical subjects that have a high concentration of procedural knowledge are especially suitable for WBdeliverybecausethelearningpro-cess can be modularized, organized, and documented into written materials tobedeliveredusingWebmedia.
EnhancingTCLearningbyWB Learning
Evaluationmethod,problemsolving, ormultiplechoiceasquestionformatin midtermexaminationshasnosignificant effectontechnicallearningperformance forWBlearning(seeTable6).However, forTClearning(seeTable8),problem solvinghasasignificantpositiveeffect ontechnicallearningperformance.This findingpartiallysupportsmyhypothesis thatproblem-solvingquestionsarebet-ter than multiple-choice questions for preparing students for the final com-prehensive exam. The phenomenon of WBstudentsbeingunaffectedbyevalu-ation method may be explained by the learning process in WB courses. WB TABLE9.RegressionResultsforAllStudents,ClassifiedbyGPA
Model Standardized
Model significance(F) df AdjustedR2 Variable,*p coefficients
34.1(highGPA; 0.000 4 .302 IndianorMiddle .219 GPA≥3.1; Eastern,.007
n=118) Asian,.008 .22
Problemsolving,.001 .281
GPA,0 .451
4.2(medium 0.066 2 .027 None GPA;2.5≤
GPA<3.1; n=131)
4.3(lowGPA; 0.013 1 .045 Problemsolving,.013 –.231 GPA<2.5;
n=115)
*p≤.05
FIGURE1.DeliveryModeforTechnicalLearningPerformance. Delivery
mode
OnlyGPAisthe significant predictorfor performance
GPA(+),problem solving(+),and
female(–)are significantpredictors
forperformance Traditionalclassroom Web-based
FIGURE2.GPA’sPredictionPoweronTechnicalLearningPerformance. GPA(+),Indianor
MiddleEastern(+), Asian(+),and problemsolving(+)are
significantpredictors fortechnicallearning
performance
Thereisnosignificant predictorwithp<.05 fortechnicallearning
performance
Problemsolving(–) istheonlysignificant predictorfortechnical learningperformance High-GPA
students (GPA≥3.1)
Medium-GPA students (2.5≤GPA<3.1)
Low-GPA students (GPA<2.5) GPAmeasuring
students’academic standing
studentsneedtolearnthematerialsstep bystep,followwritteninstruction,test theirknowledge,processfeedback,and correct their understanding. The WB learningprocessalreadyprovidesprac-ticetoWBstudentsforproblemsolving. Alternatively,becauseTCstudentsrely ontheinstructortoguidethemthrough the learning process, problem solving as an evaluation method can provide practiceopportunitiesforlearningrein-forcement.Thisfindingcertainlyscores anotherpointforWBintermsofinstill-ing and reinforcanotherpointforWBintermsofinstill-ing problem-solvanotherpointforWBintermsofinstill-ing procedureasalearningprocess.
In light of the aforementioned find-ings, TC learning can benefit from encouraging students to take partial responsibilityfortheirlearningprocess. Insteadofrelyingonclassroomtimefor knowledge delivery, TC learning can be supplemented by more outside-the-classroom activities for self-evaluation and learning process reinforcement. This recommendation is in line with hybridcoursesconsistingofWBandTC elements. Although there are research studies showing the promising poten-tialofhybridcourses(Riffell&Sibley, 2005), the correct design to retain the benefitsofWBandTCcoursesstillhas tobecarefullydeveloped.
AssistingDisadvantageous StudentsbyWB
Among those predictor variables, only GPA, gender, and ethnicity show influence over technical learning per-formance. From Tables 6 and 7, in whichthedominantpredictorGPAwas deletedfromanalysismodels,ethnicity is revealed as the next significant pre-dictor. In WB courses, White students tended to achieve high scores. Alter-natively, in TC learning (see Table 8), ethnicityisnotasignificantpredictor.It seemsthatface-to-facecommunication isimportanttocertainethnicitygroups foreffectivetechnicallearning.
Gender appears as a significant pre-dictoronlyinTCcourses(seeTable8). Female students tended to have lower technical learning performance in TC courses.However,thegenderdifference disappearsinWBcourses.Thisfinding supportsthespeculationthatWBcours-esprovidealeveledgroundforfemale
students to excel in technical learning. This may be because female students canaskquestionsfreelyinWBcourses without feeling embarrassed or intimi-dated by their fellow students, such as thecasemaybeforfemalestudentsin TCcourses.
AccommodatingHighAchievers andUnmotivatedStudentsby EvaluationMethods
Regarding GPA effect, in Table 9, GPA has no prediction power for the medium-andlow-GPAcategorybutis asignificantpredictorforthehigh-GPA category. It seems that there is a cut-off point for GPA to be a significant predictorfortechnicallearningperfor-mance. GPA has no prediction power when it is lower than a certain value (3.1 in the present study). Moreover, problemsolvingasanevaluationmeth-od has the progressive effect of nega-tive, none, and positive on low-GPA, medium-GPA, and high-GPA students, respectively.Forlow-GPAstudents,the problem-solving question format may confuse them rather than help them to prepare for the final comprehensive exam.Forhigh-GPAstudents,because theycanmasterlow-levellearningsuch asconceptunderstandingontheirown, advanced-level learning opportunities providedbyaproblem-solvingevalua-tionmethodbenefitthemsignificantly. This finding can remind educators of the importance of balancing out dif-ferent evaluation methods to support different students’ learning profiles. In termsofethnicity,Asianstudentswith high GPAs performed better than did other students in the high-GPA group. Regression analysis for high-GPA stu-dentsistheonlyevidenceofAsianasa significantpredictor.InTable6(forall students),Whiteispositivelyassociated withperformance.Theaforementioned findingseemstosuggestthatethnicity does affect performance in different situations but requires more research toconfirm.
The research results of the pres-ent study support the recommenda-tions as follows. First, WB learning can be accepted as an equivalent to TC learning, given suitable course design and implementation. Second,
for disadvantaged students who can-not perform well in TC courses for technical subjects, flexibility, consis- tency,andthenonthreateningenviron-ment of WB learning may improve their performance. WB learning can be adopted as a tool to improve dis-advantaged students’ competitiveness. Third, educators should accommodate students of different academic stand-ings by adopting different evaluation methods. The present study provides evidence that problem-solving ques-tionsimproveperformanceforstudents with high academic standing but not forstudentswithlowacademicstand-ing,inTClearningenvironments.
Conclusion
The present research study investi-gated the effectiveness of WB courses ontechnicallearning.Icollectedstudent data on ethnicity, gender, read rate, hit rate, GPA, evaluation method, commu-nicationmode,andtotalscorefromboth anWBandTCundergraduateprogram-ming course. I analyzed data using an ANOVA and multiple regression mod-els. TheANOVA reveals that GPA and totalscorebetweenWBandTCstudents do not have significant differences. In terms of student profile, WB courses have more White students and fewer Asian students. The regression analy-ses reveal the following findings. First, thedeliveryformat(WBorTC)hasno significanteffectonthedependentvari-abletotalscore.Noneoftheregression models have WB or TC as significant predictors. This finding provides some assurance for the effectiveness of WB courses for technical learning, or WB learning is at least as effective as TC learning. Second, the read and hit rates forWBstudentsarenotsignificantpre-dictors for score. Third, whereas there are significant performance differences betweenmaleandfemalestudents,and between problem-solving and multiple-choice question format in lecture sec-tions,thosedifferencesdisappearinWB courses.ItseemsthatWBlearningpro-videsalearningenvironmentthatallows disadvantagedstudentstocompetemore effectively than in TC learning. Also, evaluation method (problem-solving or multiple-choicequestions)hasaneffect
only on TC, and the effect is positive. Fourth, medium or low GPA has no effectonscore.Inthehigh-GPAgroup, Asianstudents,problem-solvingevalua-tionmethod,andGPAtendtoassociate withhighscore.Insummary,thepresent study provides supportive evidence for theeffectivenessofWBcoursesfortech-nical learning. I also identified female and problem solving as the significant predictors for TC students; however, they were not significant predictors for WB students. For students with high GPA (> 3.1), problem solving as an evaluation method tended to allow stu-dentstoachievehighscore.Asrevealed in the present research study, problem-solving questions in technical subjects allowhigh-GPAstudentstoexpresstheir creativity and knowledge in a free for-mat, which leads to better performance in class. As for students in the low-GPA group, problem-solving questions in technical subjects did not seem to help them improve their performance. One limitation of the present study is the unbalanced sample sizes for differ-entethnicitygroups.Futurestudiescan explore the ethnicity effects on perfor-mance difference betweenWB and TC courses for procedural and declarative knowledge.Thedesignofhybridcourses toachievetheadvantagesofWBandTC coursesalsodeservesmoreresearch.
NOTE
Monica Lam is a professor of data mining, data warehouse, accounting information system, and project management. Her research interests focusonfinancialapplicationsofneuralnetworks, systemdesignandmethodologies,andWebtech-nologiesforeducation.
Correspondence concerning this article should be addressed to Monica Lam, CSUS-MIS, 6000 J Street, Sacramento, CA 95819-6088, USA. E-mail:lamsm@csus.edu
REFERENCES
Arbaugh, J. B. (2000). How classroom environ-ment and student engageenviron-ment affect learning inInternet-basedMBAcourses. BusinessCom-municationQuarterly,63(4),9–26.
Bryan, K., Campell, J., & Kerr, D. (2003). ImpactofWebbasedflexiblelearningonaca-demic performance in information systems.
Journal of Information Systems Education,
14,41–50.
Chou,S.W.,&Liu,C.H.(2005).Learningeffec-tiveness in aWeb-based virtual learning envi-ronment:Alearnercontrolperspective.Journal ofComputerAssistedLearning,21,65–76. Friday,E.,Friday-Stroud,S.S.,Green,A.L.,&
Hill, A. Y. (2006). A multisemester compari-son of student performance between multiple traditionalandonlinesectionsoftwomanage-mentcourses.JournalofBehavioral&Applied Management,8(1),66–82.
Jones, K. R., Moeeni, F., & Ruby P. (2005). Comparing Web-based content delivery and instructor-ledlearninginatelecommunications course. JournalofInformationSystemsEduca-tion,16,265–271.
Kao,K.N.(2002). Levelsofcognitionofinstruc-tion and of students’ reflective thinking in a selected Web-enhanced course. Unpublished doctoral dissertation, Ohio State University, Columbus,OH.
Kochtanek,T.R.,&Hein,K.K.(2000).Creating and nurturing distributed asynchronous learn-ing environments.Online Information Review,
24,280–295.
Lee,M.-G.(2001).Profilingstudents’adaptation styles in Web-based learning.Computers & Education,36,121–132.
Lin,B.,&Hsieh,C.-T.(2001).Web-basedteach-ing and learner control: A research review.
Computers&Education,37,377–386. Lockyer, L., Patterson, J., & Harper, B. (2001).
ICTinhighereducation:Evaluatingoutcomes for health education.Journal of Computer AssistedLearning,17,275–283.
Lu,J.,Yu,C.S.,&Liu,C.(2003).Learningstyle, learningpatterns,andlearningperformancein a WebCT-based MIS courses.Information & Management,40,497–507.
Maki, W. S., & Maki, R. H. (2002). Multi-media comprehension skill predicts differ-ential outcomes of Web-based and lecture courses.JournalofExperimentalPsychology,
8,85–98.
McCallister,L.,&Matthews,L.(2001).Electron-icMBAs:Thefutureishere.S.A.M.Advanced ManagementJournal,66(1),41–47.
Palloff,R.M.,&Pratt,K.(1999). Buildinglearn-
ingcommunitiesincyberspace:Effectivestrat-egiesfortheonlineclassroom.SanFrancisco: Jossey-Bass.
Piccoli,G.,Ahmad,R.,&Ives,B.(2001).Web-basedvirtuallearningenvironments:Aresearch framework and a preliminary assessment of effectiveness in basic it skills training.MIS Quarterly,25,401–426.
Priluck,R.(2004).Web-assistedcoursesforbusi-nesseducation:Anexaminationoftwosections ofprinciplesofmarketing. JournalofMarket-ingEducation,26,161–173.
Prinsen, F. R., Volman, M. L. L., & Terwel, J. (2007). Gender-related differences in com-puter-mediated communication and computer-supported collaborative learning.Journal of ComputerAssistedLearning,23,393–409. Quinn, B., Barroca, L., Muselibeh, B.,
Fernan-dez-Ramil, J., Rapanotti, L., Thomas, P., et al. (2006). Learning software engineering at a distance.IEEESoftware,23,36–43.
Riffell, S., & Sibley, D. (2005). Using Web- basedinstructiontoimprovelargeundergradu-atebiologycourses:Anevaluationofahybrid course format.Computers & Education, 44, 217–235.
Scheines, R., Leinhardt, G., Smith, J., & Cho K. (2005). Replacing lecture with Web-based coursematerials. JournalofEducationalCom-putingResearch,32(1),1–26.
Sitzmann,T.,Kraiger,K.,Stewart,D.,&Wish-er, R. (2006). The comparative effectiveness of Web-based and classroom instruction: A meta-analysis. Personnel Psychology,59, 623–664.
Smeaton,A.F.,&Keogh,G.(1999).Ananalysis oftheuseofvirtualdeliveryofundergraduate lectures.Computers&Education,32,83–94. Sweeney, J. C., & Ingram, D. (2001).A com-parison of traditional andWeb-based tutori-als in marketing education: An exploratory study.Journal of Marketing Education, 23, 55–62.
Symonds, W. C. (2001, December 3). Giving it the old online try.Business Week,3760, 76–80.
Wallace,P.,&Clariana,R.B.(2005).Testmode familiarity and performance: Gender and race comparisonsoftestscoresamongcomputer-lit-eratestudentsinadvancedinformationsystems courses.Journal of Information Systems Edu-cation,16,177–182.
Wang, A.Y., & Newlin, M. H. (2000). Charac-teristics of students who enroll and succeed in psychology Web-based classes.Journal of EducationalPsychology,92,137–143. Wernet, S. P., Olliges, R. H., & Delicath, T.
A. (2000). Postcourse evaluations of WebCT classes by social work students.Research on SocialWorkPractice,10,487–504.
