Support vector machine applications in the field of hydrology

ContentslistsavailableatScienceDirect

Applied Soft Computing

jo u r n al h om ep a g e :w w w . e l s e v i e r . c o m / l o c a t e/ a s o c

Review Article

Support vector machine applications in the field of hydrology:

A review

Sujay Raghavendra. N

^∗

, Paresh Chandra Deka

DepartmentofAppliedMechanicsandHydraulics,NationalInstituteofTechnologyKarnataka,Surathkal,India

a r t i c l e i n f o

Articlehistory:

Received23September2013

Receivedinrevisedform9December2013 Accepted2February2014

Availableonline4March2014 Keywords:

Supportvectormachines Hydrologicalmodels Statisticallearning Optimizationtheory

a b s t r a c t

Intherecentfewdecadestherehasbeenverysignificantdevelopmentsinthetheoreticalunderstand- ingofSupportvectormachines(SVMs)aswellasalgorithmicstrategiesforimplementingthem,and applicationsoftheapproachtopracticalproblems.SVMsintroducedbyVapnikandothersintheearly 1990saremachinelearningsystemsthatutilizeahypothesisspaceoflinearfunctionsinahighdimen- sionalfeaturespace,trainedwithoptimizationalgorithmsthatimplementsalearningbiasderivedfrom statisticallearningtheory.Thispaperreviewsthestate-of-the-artandfocusesoverawiderangeofappli- cationsofSVMsinthefieldofhydrology.TouseSVMaidedhydrologicalmodels,whichhaveincreasingly extendedduringthelastyears;comprehensiveknowledgeabouttheirtheoryandmodellingapproaches seemstobenecessary.Furthermore,thisreviewprovidesabriefsynopsisofthetechniquesofSVMsand otheremergingones(hybridmodels),whichhaveprovenusefulintheanalysisofthevarioushydrolog- icalparameters.Moreover,variousexamplesofsuccessfulapplicationsofSVMsformodellingdifferent hydrologicalprocessesarealsoprovided.

©2014ElsevierB.V.Allrightsreserved.

Contents

1. Introduction... 373

2. Theoryofsupportvectormachines... 373

2.1. Supportvectorclassification... 373

2.1.1. ClusteringusingSVM... 374

2.2. Supportvectorregression... 375

2.2.1. Nonlinearsupportvectorregression–thekerneltrick... 375

2.3. Variantsofsupportvectormachines... 376

3. Applicationsofsupportvectormachinesinhydrology ... 376

3.1. Applicationsinthefieldofsurfacewaterhydrology ... 377

3.1.1. Rainfallandrunoffforecasting... 377

3.1.2. Streamflowandsedimentyieldforecasting... 377

3.1.3. Evaporationandevapotranspirationforecasting ... 378

3.1.4. Lakeandreservoirwaterlevelprediction... 379

3.1.5. Floodforecasting... 379

3.1.6. Droughtforecasting... 379

3.2. Applicationsinthefieldofgroundwaterhydrology... 380

3.2.1. Groundwaterlevelprediction ... 380

3.2.2. Soilmoistureestimation... 380

3.2.3. Groundwaterqualityassessment... 383

3.3. ApplicationsofhybridSVMmodels... 383

4. Selectionofoptimizationalgorithm,kernelsandrelatedhyper-parameters... 384

5. Advantagesofsupportvectormachines... 384

6. Disadvantagesofsupportvectormachines... 384

∗Correspondingauthor.Tel.:+919035219180.

E-mailaddress:sujayraghavendran@ymail.com(S.Raghavendra.N).

1568-4946/$–seefrontmatter©2014ElsevierB.V.Allrightsreserved.

http://dx.doi.org/10.1016/j.asoc.2014.02.002

7. Conclusions... 384 8. Futurescopeofwork... 385 References... 385

1. Introduction

Hydrological models provide us a wide range of significant applicationsinthemulti-disciplinarywaterresourcesplanningand management activities. Hydrologicalmodels can beformulated using deterministic,probabilistic and stochastic approaches for thecharacterizationofsurfaceandgroundwatersystemsincon- junctionwithcoupledsystemsmodellingsuchashydro-ecology, hydro-geologyandclimate.However,duetoresourceconstraints andtheconfinedscopeofavailablemeasurementtechniques,there arelimitationstotheavailabilityofspatial-temporaldata[1];hence aneedexiststogeneralizedataprocuredfromtheavailablemea- surementsinspaceandtimeusingspecialtechniqueslikesupport vectormachinesanditshybridmodels.

Hydrologicalmodelapplicationshaveawidevarietyofobjec- tives,dependingontheproblemthatneedstobeinvestigated[1].

Toanalyzehydrologicdatastatistically,theusermustknowbasic definitions andunderstandthepurposeand limitationsof SVM analysis.TheapplicationofSVMmethodsofhydrologicalanaly- sesrequiresmeasurementofphysicalphenomena.Themodeller hastoevaluatetheaccuracyofthedatacollectedandshouldhave abriefknowledgeonhowthedataareaccumulatedandprocessed beforetheyareusedinmodellingactivities.Someofthecommonly useddatainhydrologicstudiesincluderainfall,snowmelt,stage, streamflow,temperature,evaporation,andwatershedcharacteris- tics[2].

SVMshavebeenrecentlyintroducedrelativelynewstatistical learningtechnique.Duetoitsstrongtheoreticalstatisticalframe- work,SVMhasprovedtobemuchmorerobustinseveralfields, especiallyfornoisemixeddata,thanthelocalmodelwhichutilizes traditionalchaotictechniques[3].TheSVMhasbroughtforthheavy expectationsinrecentfewyearsastheyhavebeensuccessfulwhen appliedinclassificationproblems,regressionandforecasting;as theyincludeaspectsandtechniquesfrommachinelearning,statis- tics,mathematicalanalysisandconvexoptimization.Apartfrom possessingastrongadaptability,globaloptimization,andagood generalizationperformance,theSVMsaresuitableforclassifica- tionofsmallsamplesofdataalso.Globally,theapplicationofthese techniquesinthefieldofhydrologyhascomealongwaysincethe firstarticlesbeganappearinginconferencesinearly2000s[4].

Thispaperaims atdiscussing thebasictheorybehindSVMs andexistingSVMmodels,reviewingtherecentresearchdevelop- ments,andpresentingthechallengesforthefuturestudiesofthe hydrologicalimpactsofclimatechange.

2. Theoryofsupportvectormachines

Thesupportvectormachines(SVMs)aredevelopedbasedon statisticallearningtheoryandarederivedfromthestructuralrisk minimizationhypothesistominimizebothempiricalriskandthe confidenceinterval ofthelearning machineinorder toachieve a goodgeneralization capability.SVMs havebeenproven tobe anextremelyrobustandefficientalgorithmforclassification[5]

andregression[6,7].CortesandVapnik[8]proposedthecurrent basicSVMalgorithm.Thebeautyofthisapproachistwofold:itis simpleenoughthatscholarswithsufficientknowledgecanreadily understand,yetitispowerfulthatthepredictiveaccuracyofthis approachoverwhelmsmanyothermethods,suchasnearestneigh- bours,neuralnetworksanddecision tree.Thebasicideabehind

SVMsistomaptheoriginaldatasetsfromtheinputspacetoa highdimensional,oreveninfinite-dimensional featurespaceso thatclassificationproblembecomessimplerinthefeaturespace.

SVMshavethepotentialtoprocreatetheunknownrelationship presentbetweenasetofinputvariablesandtheoutputofthesys- tem.Themainadvantageof SVMis that,it useskerneltrick to buildexpertknowledgeaboutaproblemsothatbothmodelcom- plexityandpredictionerroraresimultaneouslyminimized.SVM algorithmsinvolvetheapplicationofthethreefollowingmathe- maticalprinciples:

•PrincipleofFermat(1638)

•PrincipleofLagrange(1788)

•PrincipleofKuhn–Tucker(1951)

2.1. Supportvectorclassification

The preliminary objective of SVM classification is to estab- lishdecisionboundariesinthefeaturespacewhichseparatedata pointsbelongingtodifferentclasses.SVMdiffersfromtheother classificationmethodssignificantly.Itsintentistocreateanopti- malseparatinghyperplanebetweentwoclassestominimizethe generalization error and thereby maximize the margin. If any twoclassesareseparablefromamongtheinfinitenumberoflin- earclassifiers,SVMdeterminesthathyperplanewhichminimizes thegeneralizationerror(i.e.,errorfor theunseen testpatterns) andconverselyifthetwoclassesarenon-separable,SVMtriesto searchthathyperplanewhichmaximizesthemarginandatthe sametime,minimizesaquantityproportionaltothenumberof misclassificationerrors.Thus, theselectedhyperplanewillhave themaximummarginbetweenthetwoclasses,wheremarginis definedasasummation ofthedistancebetweentheseparating hyperplaneandthenearestpointsoneithersideoftwoclasses [5].

SVMclassificationandtherebyitspredictivecapabilitycanbe understoodbydealingwithfourbasicconcepts:(1)theseparation hyperplane,(2)thehard-marginSVM(3)thesoft-marginSVMand (4)kernelfunction[9].

SVMmodelswereoriginallydevelopedfortheclassificationof linearlyseparableclassesofobjects.ReferringtoFig.1.Consider

Fig.1. Maximumseparationhyperplane.

Adaptedfrom[61].

-1

-1

Input Space Feature Space

+1

-1-1

-1

H

-1

-1 -1

-1

-1 -1 -1

-1

-1 -1

-1 -1 +1 +1 +1 +1 +1 +1 +1

+1 +1 +1

+1 +1

+1 +1 +1

+1 +1

-1 -1 -1

φ

Fig.2. Linearseparationinfeaturespace.

Adaptedfrom[61].

atwo-dimensionalplaneconsistinglinearlyseparableobjectsof twoseparateclasses{class(+)andclass(*)}.Thegoalistofind aclassifier which separates themperfectly.Therecan bemany possiblewaystoclassify/separatethoseobjectsbutSVMtriesto findauniquehyperplanewhichproducesmaximummargin(i.e., SVM maximizes the distance between the hyperplane and the nearestdatapointofeachclass).Theclass(+)objectsareframed behindhyperplaneH1,whilethehyperplaneH2framestheclass(*) objects.Theobjectsofeitherclasswhichexactlyfalloverthehyper- planesH1andH2aretermedassupportvectors.Most“important”

trainingpointsaresupportvectors;theydefinethehyperplaneand havedirectbearingontheoptimumlocationofthedecisionsurface.

Themaximummarginobtainedisdenotedası.Whenonlysupport vectorsareusedtospecifytheseparatinghyperplane,sparseness ofsolutionemergeswhendealingwithlargedatasets.

Inrealtimeproblemsitisnotpossibletodetermineanexactsep- aratinghyperplanedividingthedatawithinthespaceandalsowe mightgetacurveddecisionboundaryinsomecases.HenceSVM canalsobeusedasaclassifier fornon-separableclasses(Fig.2, left).Insuchcases,theoriginalinputspacecanalwaysbemapped tosomehigher-dimensionalfeaturespace(Hilbertspace)using non-linearfunctionscalledfeaturefunctions(aspresented in Figs.2and3).Eventhoughfeaturespaceishighdimensional,it couldnotbepracticallyfeasibletousedirectlythefeaturefunctions forclassificationofthehyperplane.Soinsuchcases,nonlinear mappinginducedbythefeaturefunctionsisusedforcomputation usingspecialnonlinearfunctionscalledkernels.

Soundselectionof kernelsprovidesusfor theexcellentgen- eralization performance. The correlation or similarity between two differentclasses of data pointsis depictedutilizing kernel functions.Kernelshave theadvantageof operatingintheinput space,wheretheresultoftheclassificationproblemisaweighted sumof kernel functions evaluated at the support vectors. The KerneltrickallowsSVM’stoformnonlinearboundariesThereare quiteagoodnumberofkernelsthatcanbeusedinSVMs.These includelinear,polynomial,radialbasisfunctionandsigmoid.The

Fig.3. Supportvectormachinesmaptheinputspaceintoahigh-dimensionalfea- turespace.

Adaptedfrom[61].

role of nonlinear kernels providesthe SVM with theability to modelcomplicatedseparatinghyperplanes.

K(x,x)=

⎧ ⎪

⎪ ⎪

⎪ ⎨

⎪ ⎪

⎪ ⎪

⎩

(x^T·x) Linear (x^T·x+1)^d Polynomial exp(−||x−x||²) RBF tanh(x·x+C) Sigmoidal

UniquecharacteristicsofSVMclassificationandkernelmeth- odsinclude:(1)theirperformanceisguaranteedastheyarepurely basedontheoreticalexamplesofstatistical learning;(2) search space has a unique minimal; (3) training is extremely robust and efficient;(4) thegeneralizationcapability enablestrade-off betweenclassifiercomplexityanderror.

DetailedtheoreticalbackgroundandbasicSVM’sformulaecan befoundinmanyreferencesespeciallyin[5,9].

Some of the SVM-classifiers include: hard-margin/maximal marginSVM;softmarginSVM;k-nearestneighbourSVM,radial basisfunction-SVM,translationalinvariantkernels-SVM,tangent distancekernels-SVM.BriefexplanationsrelatedtotheseSVMclas- sificationtechniquescanbefoundinShigeo[10].

2.1.1. ClusteringusingSVM

Clusteringisoftenformulatedasadiscreteoptimizationprob- lem.Themainobjectiveofclusteranalysisistosegregateobjects intogroups,suchthatobjectsofeachgrouparemore“alike”to eachotherthantheobjectsofothergroups(usuallyrepresentedas avectorofmeasurements,orapointinamultidimensionalspace).

Clusteringisunsupervisedlearningofahiddendataconcept.The applicationsofclusteringoftendealwithlargedatasetsanddata withmanyattributes.Abriefintroductiontoclusteringinpattern recognitionisprovidedinDudaandHart[11].EnhancingtheSVM trainingprocessusingclusteringorsimilartechniqueshasbeen examinedwithnumerousvariationsinYuetal.[12]andShihetal.

[13].Cluster-SVM acceleratesthetraining processbyexploiting thedistributionalpropertiesofthetrainingdata.Thecluster-SVM algorithm,firstpartitionsthetrainingdataintonumerouspairwise separateclustersandthentherepresentativesoftheseclustersare utilizedtotrainaninitialSVM,basedonwhichthesupportvectors andnon-supportvectorsareidentifiedapproximately(referFig.4).

Theclusterscomprisingonlynon-supportvectorsarereplacedwith theirrepresentativestoreducethenumberoftrainingdatasignif- icantlyandtherebythespeedofthetrainingprocessisenhanced.

Fig.4.SVM-clustering.

Basedonahierarchicalmicro-clusteringalgorithm,Yuetal.[12]

putforwardascalablealgorithmtotrainSVMswithalinearkernel.

TheSVM-Internalapproachtoclusteringwasinitiallyproposed byWinters-Hiltetal.[14].Datapointsaremappedtoahighdimen- sional feature space by means of a kernel where the minimal enclosingsphereisexplored.Thewidthofthekernelfunctiongov- ernsthemeasureatwhichthedataisprobedincontrastthesoft marginconstantassiststosuperviseoutliersandover-lappingclus- ters.Thestructureofadatasetissurveyedbyalteringthesetwo parameters,soastopreserveaminimalnumberofsupportvectors forsubstantiatesmoothclusterboundaries.SincetheSVM-Internal Clusteringisfaintlybiasedtowardstheshapeoftheclustersinfea- turespace(thebiasisforsphericalclustersinthekernelspace),it lacksrobustness.Inmostofthereal-timedatasetswhereinstrongly overlappingclustersareseen,theinternalSVM–internalclustering algorithmcanonlydelineatethereasonablysmallclustercores.

Otherdrawbackslikeinitialchoiceofkernel,excessivegeometric constraints,etc.makewayfortheuseofSVM-externalclustering algorithmthatclustersdatavectorswithoutanypriorknowledgeof data’sclusteringcharacteristics,orthenumberofclusters.K-means clusteringtechniquewithSVMisanexampleofSVM-externalclus- tering.

2.2. Supportvectorregression

Letusconsiderasimplelinearregressionproblemtrainedon dataset={u_i,v_i; i=1,...,n}withinputvectorsu_iandlinked targetsvi.Afunctiong(u)hastobeformulatedapproximatelyin order tolink up inheritedrelations between thedata sets and therebyitcanbeusedinthelaterparttoinfertheoutputvfor anewinputdatau.

StandardSVMregressionusesalossfunctionLε(v,g(u))which describesthedeviationoftheestimatedfunctionfromtheoriginal one.Severaltypesoflossfunctionscanbeminedintheliterature e.g.,linear,quadratic,exponential,Huber’slossfunction,etc.Inthe presentcontextthestandardVapnik’s–εinsensitivelossfunction isusedwhichisdefinedas

Lε(v,g(u))=

0 for|v−g(u)|≤ε

|v₋g(u)|−ε otherwise (1) Usingε-insensitivelossfunction,onecanfindg(u)thatcanbet- terapproximatetheactualoutputvectorvandhastheatmost errortoleranceεfromtheactualincurredtargetsv_iforalltraining data,andconcurrentlyasflataspossible.Considertheregression functiondefinedby

g(u)=w·u+b (2)

wherew∈,istheinputspace;b∈Risabiastermand(w·u)is dotproductofvectorswandu.flatnessinEq.(2)referstoasmaller valueofparametervectorw.Byminimizingthenorm||w||²flatness canbeascertainedalongwithmodelcomplexity.Thusregression problemcanbestatedasthefollowingconvexoptimizationprob- lem.

min

w,b,,∗

1

2||w||²+C

n i=1

(_i+_i^∗)

subjectto

vi−(w·u_i+b)≤ε+_i (w·u_i+b)−v_i≤ε+_i^∗ _i,^∗_i≥0, i=1,2,...,n

(3)

where_iand_i^∗areslackvariablesintroducedtoevaluatethedevi- ationoftrainingsamplesoutsideε-insensitivezone.Thetrade-off betweentheflatnessofgandthequantityuptowhichdeviations greaterthanεaretoleratedisdepictedbyC>0.Cisapositivecon- stantinfluencingthedegreeofpenalizinglosswhenatrainingerror

occurs.Underfittingandoverfittingoftrainingdataareavoidedby minimizationoftheregularizationtermw²/2alongwiththetrain- ingerrortermC

n

i=1(_i−^∗_i)inEq.(3).Theminimizationproblem inEq.(3)representstheprimalobjectivefunction.

NowtheproblemisdealtbyconstructingaLagrangefunction fromtheprimalobjective functionbyintroducinga dualsetof variables,˛-ⁱand˛¯_iforthecorrespondingconstraints.Optimality conditionsareexploitedatthesaddlepointsofaLagrangefunction leadingtotheformulationofthedualoptimizationproblem:

max

˛-^i,˛¯i

−1 2

n i,j=1

˛-ⁱ−˛¯i(˛-^j−˛¯j)ui·uj−ε

n i=1

(˛-ⁱ+˛¯i)+

n i=1

vi(˛-ⁱ−˛¯i)

subjectto

n i=1

(˛-ⁱ−˛¯i)=0 0≤˛-ⁱ≤C, i=1,2,...,n 0≤˛¯i≤C, i=1,2,...,n

(4)

AfterdeterminingLagrangemultipliers,˛-ⁱand˛¯_i;theparameter vectorswandbcanbeevaluatedunderKarush–Kuhn–Tucker(KKT) complementarityconditions[15],whicharenotdiscussedherein.

Therefore,thepredictionisalinearregressionfunctionthatcanbe expressedas

g(u)=

n i=1

˛_i−˛¯_iu_i·u+b (5)

ThusSVMregressionexpansionisderived;wherewisdepicted asalinearcombination ofthetraining patternsviand bcanbe foundusingprimaryconstraints.For|g(u)|≥εLagrangemultipliers maybenon-zeroforallthesamplesinsidetheε-tubeandthese remainingcoefficientsaretermedassupportvectors.

2.2.1. Nonlinearsupportvectorregression–thekerneltrick NowformakingSVMregressiontodealwithnon-linearcases;

pre-processingoftrainingpatternsu_ihastodonebymappingthe input space into somefeature spaceI usingnonlinear func- tionϕ=−→Iandisthenappliedtothestandardsupportvector algorithm.Alsothedimensionalityofϕ(x)canbeveryhuge,mak- ing‘w’hardtorepresentexplicitlyinmemory,and hardforthe quadraticprogrammingoptimizertosolve.Therepresentertheo- remKimeldorfandWabha[74]showsthat:w=

n

i=1˛_i·(x_i)for somevariables˛.Insteadofoptimizing‘w’wecandirectlyopti- mize˛.Therebythedecisionfunctionis obtained(kerneltrick 1).Therepresentertheoremisexploitedtoexaminethesensitiv- itypropertiesofε-insensitiveSVRandintroduce theconceptof approximatedegreesoffreedom.Thedegreesoffreedomplaya vitalroleintheassessmentoftheoptimism–i.e.,thedifference betweentheexpectedin-sampleerrorandtheexpectedempirical risk.

Letuibemappedintothefeaturespacebynonlinearfunction ϕ(u)andhencethedecisionfunctionisgivenby

g(w,b)=w·(u)+b (6)

This nonlinear regression problem can be expressedas the followingoptimizationproblem.Fig.5depictstheconceptofnon- linearSVregressioncorrespondingtoEq.(7)

min

w,b,,∗

1

2||w||²+C

n i=1

(_i+_i^∗)

subjectto

v_i−(w·(u_i)+b)≤ε+_i (w·(u_i)+b)−vi≤ε+_i^∗ _i,^∗_i≥0, i=1,2,...,n

(7)

wherewisthevectorofcoefficients,_iand^∗_i arethedistancesof thetrainingdatasetpointsfromtheregionwheretheerrorsless

Error

g = w (u) + b

Margin SVs Non-SVs

Error SVs Loss

i

i

i

Fig.5. NonlinearSVRwithVapnik’sε-insensitivelossfunction.

Adaptedfrom[41].

thanεareignoredandbisaconstant.Theindexilabelsthe‘n’train- ingcases.They∈±1istheclasslabelsandu_iistheindependent variable.

Then,thedualformofthenonlinearSVRcanbeexpressedas max

˛-^i,˛¯i

−1 2

n i,j=1

(˛-ⁱ−˛¯_i)(˛-^j−˛¯_j)(u_i)·(u_j)

−ε

n i=1

(˛-ⁱ+˛¯_i)+

nl i=1

v_i(˛-ⁱ−˛¯_i)

subjectto

n i=1

˛-ⁱ−˛¯_i=0

0≤˛-ⁱ≤C,i=1,2,...,n 0≤˛¯_i≤C,i=1,2,...,n

(8)

The“kerneltrick”K(u_i,u_j)=(u_i),(u_j)isusedforcomputa- tionsininputspacetofetchtheinnerproductsintofeaturespace I.AnyfunctionsatisfyingMercer’stheorem[16]shouldbeusedas kernels.

Finally, the decision function of nonlinear SVR with the allowanceofthekerneltrickisexpressedasfollows.

g(u)=

l i,j=1

(˛-ⁱ−˛¯_i)Ku_i·u+b (9)

Theparametersthatimpactovertheeffectivenessthenonlinear SVRarethecostconstantC,theradiusoftheinsensitivetubeε,and thekernelparameters.Theseparametersaremutuallydependent overoneanotherandhencealteringthevalueofoneparameter affectstheotherlinkedparametersalso.TheparameterCchecksfor thesmoothness/flatnessoftheapproximationfunction.Asmaller valueofCyieldsalearningmachinewithpoorapproximationdue tounderfittingoftrainingdata.AgreaterCvalueoverfitsthetrain- ingdataandsetsitsobjectivetominimizeonlytheempiricalrisk makingwayformorecomplexlearning.Theparameterεisrelated withsmootheningthecomplexityoftheapproximationfunction andcontrolsthewidthoftheε-insensitivezoneusedforfitting thetrainingdata.Theparameterεinfluencesoverthenumberof supportvectors,andthenboththecomplexityandthegeneraliza- tioncapabilityoftheapproximationfunctionisdependentupon itsvalue.Italsogovernstheprecisionoftheapproximationfunc- tion.Smallervaluesofεleadtomorenumberofsupportvectors

( Hidden Nodes ) Support vectors

(Lagrange multipliers) Weights Bias b

Input vector x Output y

) , (xx₁ K

) , (xx₂ K

) , (xx₃ K

) , (xx_m K

1

3

m 2

Fig.6.NetworkarchitectureofSVM.

Adaptedfrom[44,45].

andresultsincomplexlearningmachine.Greaterεvaluesresultin moreflatestimatesoftheregressionfunction.Determiningappro- priatevaluesofCandεisoftenaheuristictrial-and-errorprocess.

Fig.6showsthegeneralnetworkarchitectureofSVM.

2.3. Variantsofsupportvectormachines

•Least-squaressupportvectormachines.

•Linearprogrammingsupportvectormachines.

•Nu-supportvectormachines.

Least-squaressupportvectormachines(LS-SVM)proposedby Suykensetal.[17]isawidelyapplicableandfunctionalmachine learning technique for both classification and regression. The solutionofLS-SVMisderivedoutoflinearKarush–Kuhn–Tucker equationsinsteadofaquadraticprogrammingproblemoftradi- tionalSVM.AmajordrawbackofLS-SVMisthattheuseofasquared lossfunctionwithoutanyregularizationleadstolessrobustesti- mates.In order toavoid this, theweightedLS-SVM is adopted whereinsmallweightsareassignedtothedataandatwo-stage trainingmethodisproposed.

Originally,Vapnikproposedlinear-programmingsupportvec- tormachines(LP-SVM)basedontheheuristicthatanSVMwith lesssupportvectorshasthebettergeneralizationability.Support vectormachinesusinglinearprogrammingtechniquehavebeen demonstratedtoachievesparsenessandbeenextensivelystudied intheframeworkoffeatureselection[18–20],scalabilityofopti- mization[21],marginmaximization,errorboundofclassification andconvergencebehaviouroflearning,etc.Comparedtothecon- ventionalQP-SVM,theLP-SVMlacksmanyfeaturesthatareunique totheQP-SVMsuchasasolidtheoreticalfoundationandanelegant dualtransformationtointroducekernelfunctions.

TheNu-Supportvectormachines(␯-SVM)hasbeenderivedso thatthesoftmarginhastolieintherangeofzeroandone[22], wheretheSVMemploysinhomogeneousseparatinghyperplanes, thatis,itdonotessentiallyincludestheorigin.Theparameter‘’

doesnotgovernthetrade-offbetweenthetrainingerrorandthe generalizationerrorbutinsteaditnowplaystworoles;firstlyitis theupperboundonthefractionofmarginerrorsandsecondlyit isthelowerboundonthefractionofsupportvectors.Parameter

‘’inNu-SVC/one-classSVM/Nu-SVRapproximatesthefractionof trainingerrorsandsupportvectors.

3. Applicationsofsupportvectormachinesinhydrology AlotofexamplesthatdiscussdifferentapplicationsofSVMs for hydrologicalprediction and forecastingcan befoundinthe literature.Thesectionsbelowillustrateafewexamplesofvarious

applications of SVMs in the field of surface and ground water hydrologyrespectively.

3.1. Applicationsinthefieldofsurfacewaterhydrology 3.1.1. Rainfallandrunoffforecasting

Several methods/models have been developed by various researchersinordertosimulatetherainfallandrunoffprocesses.

Hencethereisaneedfortheselectionofasuitablerobustmodel forefficientplanningandmanagementofwatersheds.Theinten- tionofthepresentstudyistosurveythemajorproposalsinrainfall andrunoffforecastingmethodologyusingSVMs.

ManyresearchershaveinvestigatedthepotentialofSVMsin modellingwatershedrunoffbasedonrainfallinputs.Dibikeetal.

[23]investigatedonusingSVMsforrainfall-runoffmodelling.The dailyrainfall,evaporation,andstream-flow dataofthreediffer- entcatchmentsofdifferentrainfallintensitieswerepre-processed toobtaindataformatappropriatetoSVMandANN.DuringSVM training, three types of kernel functions namely – polynomial kernel, RBF kernel and neuralnetwork kernel were employed.

Bytrial-and-errorprocess,theparameterεcorrespondingtothe error insensitivezone, the capacity factor C, and other kernel- specificparametersweresettooptimalvalues.OnanaverageSVM techniqueachieveda15%increaseofaccuracyintheestimation ofrunoffduringtheverificationperiodwhencomparedtoANN model.Theyconcludebyremarkingthehardshipinvolvedduring establishingtheoptimalvaluesforparametersCandεandcallit asa“heuristicprocess” andrecommendsforautomationofthis process.

BrayandHan[24]emphasizeusingSVMsforidentificationof asuitablemodelstructureanditsrelevantparametersforrainfall runoffmodelling.Theirtestingandtrainingdatawerecomposed usingrainfallandflowdataseriesoftheBirdCreekcatchment.Scal- ingfactorswereappliedtorainfallandflowvariablesastheyhad varyingunitsandmagnitudes.Theyhaveproposedflowchartsfor modelselectionandadaptationofLIBSVMsoftware.Anattempt hasbeenmadetoexploretherelationshipsamongvariousmodel structures(-SVor-SVregression),kernelfunctions(linear,poly- nomial,radialbasisandsigmoid),scalingfactor,modelparameters (costC,epsilon)andcompositionofinputvectors.Theirtraining resultsdepictthat,SVMlearntwellwithallrainandflowinput combinationseventhoughtherewereslightunderestimatesinele- vatedflowsandtheirdurationinallcases.Theyachievedoptimized resultswiththreerainobservationsandfourflowobservationsin eachinputvector.

Tripathietal.[25]demonstratedanSVMapproachforstatistical downscalingofprecipitationatmonthlytimescale.Theapproach wasappliedtometeorologicalsub-divisions(MSDs)inIndiaand testedforitseffectiveness.ForeachMSD,SVM-baseddownscaling model(DM)wasdevelopedforseason(s)withsignificantrainfall usingprincipalcomponentsextractedfromthepredictorsasinput andthecontemporaneousprecipitationobservedattheMSDasan output.Themulti-layerbackpropagationtechniqueverifiedthat theproposedDMperformedwellthantheconventionaldownscal- ingmodels.Laterontoobtainthefutureprojectionsofprecipitation fortheMSDs,theSVM-basedDMwereemployedwhichutilizedthe secondgenerationcoupledglobalclimatemodel(CGCM2).SVMs provedtobemuchmoresuperiortoconventionalartificialneu- ralnetworksinthefieldofstatisticaldownscalingforconducting climateimpactstudies.Theyconcludedthat,SVMswasperfectly suitedforthedownscalingtaskbecauseoftheirgoodgeneraliza- tionperformanceincapturingnon-linearregressionrelationships betweenpredictorsandpredictand,inspiteoftheirincapabilityto understandproblemdomainknowledge.

AnirudhandUmesh[26]designedanSVMmodelforthepre- dictionofoccurrenceofrainydays.Thedatacollectedforovera

timeperiodof2061daysencompassingdailyvaluesofmaximum andminimumtemperatures,panevaporation,relativehumidity, windspeedwereconsideredtobemodelinputsandcorresponding daysofrainfallwereregardedastargetvariable.TheydeployedLin- earandGaussiankernelfunctionsformappingthenonlinearinput spacetoahigherdimensionalfeaturespace.Theirstudystressed theimportanceofthemodelparametersnamelycostparameterC andgammaparameterforeffectiveSVMclassification.Satisfac- toryperformancewasderivedoutofSVMmodelandtheGaussian kerneloutperformedthelinearone.

Chenetal. [27]attemptedtoevaluatetheimpactofclimate changeonprecipitationutilizingthepotentialofsmoothsupport vectormachine(SSVM)methodindownscalinggeneralcirculation model(GCM)simulationsintheHanjiangBasin.Theperformance ofSSVMwascomparedwiththeANNduringthedownscalingof GCMoutputs todaily precipitation.NCEP/NCAR reanalysisdata wereusedtoestablishthestatisticalrelationshipbetweenlarge- scale circulation and precipitation.Thedaily precipitation from 1961to2000wasusedaspredictandand aregressionfunction wasdevelopedoverit.ThedeterminingparametersofSSVMwere:

thewidthofRadialbasisfunction( )andthepenaltyfactor(C).

ThedevelopedSSVMdownscalingmodelshowedsatisfactoryper- formanceintermsofdailyandmonthlymeanprecipitationinthe testingperiods.However,theyconcludebystatingthedrawbacks oftheSSVMmethodinreproducingextremedailyprecipitationand standarddeviation.

3.1.2. Streamflowandsedimentyieldforecasting

Thesimulationofshort-termandlong-termforecastsofstream- flow and sediment yield plays a significant role in watershed managementdealingwithnatureofthestormandstreamflowpat- terns,assessmentofonsiteratesoferosionandsoillosswithina drainagebasin,measuresforcontrollingexcessrunoff,etc.Fore- castingstreamflowhours,daysandmonthswellinadvanceisvery importantfortheeffectiveoperationofawaterresourcessystem.

Theilleffectsofnaturaldisastersuchasfloodscanbeprevented bypredictionofaccurateoratleastreasonablyreliablestreamflow patterns.

Jianetal.[28]noticedtheimportanceofaccuratetime-and site-specificforecastsofstreamflowandreservoirinflowforeffec- tivehydropowerreservoirmanagementandscheduling.Theyused monthlyflowdataoftheManwanReservoirspanningoveratime periodfromJanuary1974toDecember2003;thedatasetfrom January1974toDecember1998wereusedfortrainingandthedata setsfromJanuary1999toDecember2003servedforvalidation.

Theyalsohadtheobjectiveoffindingoptimalparameters–C,εand ofRBFkernel.So,ashuffledcomplexevolutionalgorithm(SCE-UA algorithm)involvingexponentialtransformationwasdetermined toidentifyappropriateparametersrequiredfortheSVMprediction model.TheSVMmodelgaveagoodpredictionperformancewhen comparedwiththoseofARMAandANNmodels.Theauthorspoint out,SVMsdistinctcapabilityandadvantagesinidentifyinghydro- logicaltimeseriescomprisingnonlinearcharacteristicsandalsoits potentialinthepredictionoflongtermdischarges.

Mesut[29]utilized SVMsforpredictingsuspendedsediment concentration/load in rivers. The method was applied to the observedstreamflowandsuspendedsedimentdataoftworivers in theUSA.Thepredictedsuspended sediment values obtained fromtheSVMmodelwerefoundtobeingoodagreementwiththe originalestimatedones.Negativesedimentestimates,whichwere encounteredinthesoftcomputingcalculationsusingFuzzylogic, ANNandNeuro-fuzzy,werenotproducedbySVMprediction.The bestsuspendedsedimentestimatesaccordingtotheerrorcriteria wereobtainedwiththeSVMprocedurepresentedinthisstudy.The suitableselectionhyper-parametersandfindingouttheirlimiting valuesisimportanttomodelanytimeseries.

ArunandMahesh[30]researchedonpredictingthemaximum depthofscourongrade-controlstructureslikesluicegates,weirs andcheckdams,etc.Fromtheavailablelaboratoryandfielddata ofearlierpublishedstudies,theyattemptedtoexplorethepoten- tialofSVMsinmodellingthescour.Polynomialkernel-basedSVMs providedencouragingperformanceinupscalingresultsoflarge scaledataset,severaltrialswereconductedtoanalyzetheeffectof variousinputparametersinpredictingthescourdepthforMissiga streamdata.TheresultsfromtheSVMbasedmodellingapproach indicatedanimprovedperformanceincontrasttoboththeempiri- calrelationandbackpropagationneuralnetworkapproachwith thelaboratorydata.Itseffectivenesswasalsoexaminedin pre- dictingthescourfromareducedmodelscale(laboratorydata)to thelarge-scalefielddata.TheyconcludedthatboththeRBFand polynomialkernel-basedSVMsperformwellincomparisontoback propagationANN.

Debasmitaetal.[31]emphasizedtheuseofSVMstosimulate runoffandsedimentyieldfromwatersheds.Theysimulateddaily, weekly,andmonthlyrunoff andsediment yieldfromanIndian watershed,withmonsoonperioddata,usingSVM,arelativelynew pattern-recognitionalgorithm.Inordertoevaluatemodelperfor- mancetheyusedcorrelationcoefficientforassessingvariability;

coefficientof efficiency for assessing efficiency; and the differ- enceofslope ofa best-fit linefrom observed-estimatedscatter plotsto1:1lineforassessing predictability.Theresultsof SVM werecompared withthose ofANN. Runoff estimationwasalso carriedout through themultipleregressive patternrecognition technique(MRPRT).TheresultsofMRPRTdidnotshowanysignif- icantimprovementwhencomparedtothatofSVM;hence,itwas ignoredforthesimulationofsedimentyield.Theyobservedsig- nificantimprovementinperformanceoftheproposedSVMmodel intraining,calibrationandvalidationascomparedtoANN.They acknowledgethatSVMservestobeanefficientalternativetoANN forrunoffandsedimentyieldpredictions.

Hazietal.[32]proposedanSVMmodeltopredictsedimentloads inthreeMalaysianrivers–Muda,Langat,andKurauwhereSVM wasemployedwithoutanyrestrictiontoan extensivedatabase compiledfrommeasurementsintheserivers.TheyemployedNeu- rosolutions5.0toolbox,developedbyNeurodimension,Inc.forthe developmentofSVM model.Theyinitiallyfixedmodelparame- ters˛iandεtobe1and0respectively.Theoptimalvalueofε wasobtainedusing a geneticalgorithm. Their model produced goodresultscomparedtoothertraditionalsediment-loadmethods.

Theperformance oftheSVMmodeldemonstrateditspredictive capabilityandthepossibilityofthegeneralizationofthemodelto nonlinearproblemsforriverengineeringapplications.Theywere successfulinpredictingtotalloadtransportinagreatvarietyofflu- vialenvironments,includingbothsandandgravelriversusingSVM.

Thepredictedmeantotalloadwasalmostinperfectagreement withthemeasuredmeantotalload.

ZahrahtulandAni[33]investigatedthepotentialoftheSVM modelforstreamflowforecastingatungaugedsitesandcompared itsperformancewithotherstatisticalmethodofmultiplelinear regression(MLR).Theyusedtheannualmaximumflowseriesof 88waterlevelstationsinPeninsularMalaysiaasdata.Radial-basis function(RBF)kernelwasusedduringtrainingand testingasit providedmoreflexibilitywithfewerparametersalongwithmap- pingnonlineardataintoapossiblyinfinitedimensionalspace.The performancesofbothmodelswereassessedbythreequantitative standardstatisticalindices–meanabsoluteerror(MAE),rootmean square error(RMSE) and Nash-Sutcliffe coefficientof efficiency (CE).TheirstudyresultsrevealthatSVMmodeloutperformedthe predictionabilityof thetraditionalMLR model underallofthe designatedreturnperiods.

Paragetal.[34]proposedthepotentialofleastsquare-support vectorregression(LS-SVR)approachtomodelthedailyvariation

ofriver flow.Theirmain intentionbehindusingLS-SVM wasit offeredahighercomputationalefficiencythanthatoftraditional SVMmethodandthetrainingofLS-SVMrequiredonlythesolution ofasetoflinearequationsinsteadofthelongandcomputation- allydemandingquadratic programmingproblemasinvolved in thetraditionalSVM.ByexploitingthecapabilityofLS-SVR,acou- pleofissueswhichhinderedthegeneralizationoftherelationship betweenpreviousandforthcomingriver flow magnitudeswere dealt.TheLS-SVRmodeldevelopmentwascarriedoutwithpre- constructionofriverflowregimeanditsperformancewasassessed forbothpre-andpost-constructionofthedamforanyperceivable difference.Amultistep-aheadpredictionwascarriedoutinorder toinvestigatethetemporalhorizonoverwhichthepredictionper- formancewasrelieduponandtheLS-SVRmodelperformancewas foundtobesatisfactoryupto5-day-aheadpredictionseventhough theperformancewasdecreasingwiththeincreaseinlead-time.The LS-SVRresultswereoutperformingwhencomparedtothatofthe ANNsmodelforalltheleadtimes.

3.1.3. Evaporationandevapotranspirationforecasting

Evaporationtakesplacewheneverthereisavapourpressure deficitbetweenawatersurfaceandtheoverlyingatmosphereand sufficientenergyisavailable.Themostcommonandimportantfac- torsaffectingevaporationaresolarradiation,temperature,relative humidity,vapourpressuredeficit,atmosphericpressure,andthe windspeed.Evaporationlossesshouldbeconsideredinthedesign ofvariouswaterresourcesandirrigationsystems.In areaswith littlerainfall,evaporationlossescanrepresentasignificantpart ofthewaterbudgetfor alakeorreservoir,and maycontribute significantlytotheloweringofthewatersurfaceelevation.Evapo- transpiration(ET)isdefinedasthelossofwatertotheatmosphere bythecombinedprocessesofevaporationfromthesoilandplant surfacesandtranspirationfromplants.Itisnecessarytoquantify ETforworkdealingwithwaterresourcemanagementorenviron- mentalstudies.

Moghaddamniaet al.[35] presented a preliminarystudy on evaporationandinvestigatedtheabilitiesofSVMstoenhancethe predictionaccuracyofdailyevaporationintheChahnimehreser- voirsofZabolinthesoutheastofIran.TheSVMtechniquedeveloped forsimulatingevaporationfromthewatersurfaceofareservoirwas evaluatedbasedoncriteriasuchasrootmeansquareerror(RMSE);

meanabsoluteerror(MAE);meansquareerror(MSE)andcoeffi- cientofdetermination(R²).AtechniquecalledGammaTestwas employedfortheselectionofrelevantvariablesintheconstruction ofnon-linearmodelsfordaily(global)evaporationestimations.Lin- ear,polynomial,RBF,andsigmoidkernelfunctionswereusedinthe studyanditwasobservedthatRBFkerneloutperformedinallthe cases.TheycomparedthreetypesofSVMsnamelyone-classSVM, epsilon-SVRandnu-SVR;theresultsrevealthat,epsilon-SVRmodel revealedbetterresults.

Eslamianetal.[36]constitutedanSVMandanANNmodelfor theestimationofmonthlypanevaporationusingmeteorological variableslikeairtemperature,solarradiation,windspeed,rela- tivehumidityandprecipitation.HeretheSVM formulatedusing RBFkernelpreferredagridsearchforcross-validationtoprevent overfittingproblem.Windspeedandairtemperaturewereiden- tifiedtobethemostsensitivevariablesduringthecomputation.

Majorobservationsinclude:(1)thecomputationeffortwassignifi- cantlysmallerwithSVMsthantheANNsalgorithm;(2)basedonthe valuesofmeansquareerror(MSE),theSVMapproachperformed betterthanANN.

Pijush[37]adoptedleastsquaresupportvectormachine(LS- SVM) for prediction of Evaporation Losses (EL) in Anand Sagar reservoir,Shegaon(India).Meanairtemperature(^◦C),averagewind speed (m/s),sunshine hours(h/day),andmean relativehumid- ity(%)wereusedasinputsofLSSVMmodel.Themainobjectives

putforthbytheauthorincludes–developinganequationforthe predictionofEL;determinationoftheerrorbarofpredictedEL;

performingsensitivityanalysisforinvestigatingtheimportanceof eachoftheinputparameters.Acomparativestudyhasbeenpre- sentedbetweenLSSVMandANNmodels.ThedevelopedLSSVM provedtobeapowerfulrobusttoolinestimationofELandalso depictedsome prediction uncertainty. Sensitivity analysis indi- catedthattemperaturehadthemaximumeffectonEL.

Hosseinetal.[38]studiedthepotentialofSVM,adaptiveneuro- fuzzyinferencesystem(ANFIS),multiplelinearregression(MLR) andmultiplenon-linearregression(MNLR)forestimatingETo.In addition,fourtemperature-basedandeight radiation-basedETo equationsweretestedagainstthePMF-56model.Theperformance ofSVMandANFISmodelsforEToestimationwerebetterthanthose achievedusingtheregressionandclimatebasedmodelsandthis confirmedtheabilityofthesetechniquesinETomodellinginsemi- aridenvironments.Someprominentobservationsinclude–SVM’s werecapablein selectingthekey vectorsasitssupportvectors duringthetrainingprocessandremovedthenon-supportvectors automaticallyfromthemodel.Thismadethemodelcopeupwell withnoisyconditions.

3.1.4. Lakeandreservoirwaterlevelprediction

KhanandCoulibaly[39]examinedthepotentialofthesupport vectormachine(SVM)inlong-termpredictionoflakewaterlevels.

LakeEriemeanmonthlywaterlevelsfrom1918to2001wereused topredictfuturewaterlevelsupto12monthsahead.Heretheopti- mizationtechnique(linearlyconstrainedquadraticprogramming function)usedintheSVMforparameterselectionhasmadeitsupe- riortotraditionalneuralnetworks.TheyfoundRBFkerneltobethe mostappropriateandadoptsitwithacommonwidthof(=0.3)for allpointsinthedataset.Furthertheysetthevaluesforregulariza- tionparameterC=100andε-insensitivelossfunctionwithε=0.005 chosenbythetrial-and-errorapproach.80–90%oftheinputdata wereidentifiedtobesupportvectorsinthemodel.Theperformance wascomparedwithamultilayerperceptron(MLP)andwithacon- ventionalmultiplicativeseasonalautoregressivemodel(SAR).On thewhole,SVMshowedgoodperformanceandwasprovedtobe competitivewiththeMLPandSARmodels.Fora3-to12-month- ahead prediction, the SVM model outperformed the two other modelsbasedontheroot-meansquareerrorandcorrelationcoef- ficientperformancecriteria.Furthermore,theSVMdemonstrated inherentadvantagesinformulatingcostfunctionsandofquadratic programmingduringmodeloptimization.

Afiqetal.[40]investigatedonusingSVMtoforecastthedaily dam water level of the Klang reservoir, located in peninsular Malaysia.Inordertoascertainthebestmodeltheauthorsincorpo- ratedfourdifferentcategories,firstlythebestinputscenariowas appliedbyconsideringbothrainfallR(t−i)andthedamwaterlevel L(t−i);secondlythe-SVMregressionwasselectedasthebest regressiontype,thirdlythe5-foldcross-validationwasconsidered toproducethemostaccurateresultsandfinally,alltheresultswere combinedtodeterminethebesttimelag.Theperformanceofthe -SVMmodelwasassessedandcomparedwiththatofANFISmodel usingstatisticalparameterslikeRMSE,MAEandMAPE.Theirstudy resultsrevealthat-SVMmodeloutperformedthepredictionsof ANFIS.

3.1.5. Floodforecasting

Floodforecasting is asignificantnon-structuralapproach for floodmitigation.Floodsgenerallydevelopoveraperiodofdays, whenthereistoomuchrainwatertofitintheriversandwater spreadsoverthelandnexttoit(the‘floodplain’).Inpractice,the riverstageistheparameterthatisconsideredinfloodforecasting studies.Thecurrentpaperpresentstheexamplesofreal-timeflood forecastingmodelsusingSVMs.

Yu etal. [41] establisheda real-time flood stageforecasting modelbasedonSVRemployinghydrologicalconceptofthetime ofresponse forrecognizinglags associated withtheinputvari- ables. Their input vector accounted for both rainfall and river stage, toforecast the hourlystages of theflash flood. The RBF KernelwasemployedduringmodellingofnonlinearSVR.Forfind- ing optimalparameters ofSVR, a two-step gridsearch method wasemployed.Twomodelstructures,withdifferentrelationships betweentheinputandoutputvectorswereformulatedtopredict one-tosix-hour-aheadstageforecastsinrealtime.Boththemodel structuresyieldedalmostsimilarforecastingresultsbasedonvali- dationevents.TheirstudydepictedtherobustperformanceofSVR instageforecastingandpresentedasimpleandsystematicmethod forselectionoflagsofthevariablesandthemodelparametersfor floodforecasting.

Hanetal.[42]appliedSVMtoforecastfloodoverBirdCreek catchmentandfocusesonsomesignificantissuesrelatedtodevel- opingandapplyingSVMinfloodforecasting.TheyusedLIBSVM formodellingalong with‘Gunn’toolboxfordatanormalization.

They demonstrated onhow tooptimally device a model from amongalargenumberofvariousinputcombinationsandparam- eters in real-timemodelling. Theirresultsshowthat thelinear andnon-linearkernelfunctions(i.e.,RBF)cangeneratesuperior performanceagainsteachotherunderdifferentcircumstancesin thesamecatchment.TheirprominentobservationisthatSVMalso suffersfromover-fittingandunder-fittingproblemsandtheover- fittingismoredamagingthanunder-fitting.

Wei et al.[43] examined thepotentialof a Distributed SVR (D-SVR) model for river stage prediction, the commonly used parameterinfloodforecasting.TheproposedD-SVRimplemented alocalapproximationtotrainingdata,sincepartitionedoriginal trainingdatawereindependentlyfittedbyeachlocalSVRmodel.

Tolocatetheoptimaltriplets(C,ε,)atwo-stepGeneticAlgorithm wasemployedtomodelD-SVR.Inordertoassesstheperformance of D-SVR, prediction was also arrived at via LR (linear regres- sion), NNM (Nearest-neighbourmethod), and ANN-GA(genetic algorithm-basedANN).ThevalidationresultsoftheproposedD- SVRmodelrevealedthattheriverflowpredictionwerebetterin comparisonwithothers,andthetrainingtimewasconsiderably reducedwhencompared withtheconventionalSVRmodel.The keyfeaturesinfluencingovertheperformanceofD-SVRwasthatit implementedalocalapproximationmethodinadditiontoprinciple ofstructuralriskminimization.

ChenandYu[44,45]representedSVMasanetworkarchitecture resemblingartificialneuralnetworks(multilayerperceptron)and wasprunedtoacquiremodelparsimonyorimprovedgeneraliza- tion.Theirstudyestablishedtwoapproaches/methodsofpruning thesupportvectornetworks,andemployingthemtoacasestudy of real-time flood stage forecasting. The first approach/method prunedtheinputvariablesbythecross-correlationmethod,while theotherapproachprunedthesupportvectorsaccordingtothe derivedrelationshipamongSVMparameters.Thesepruningmeth- odsdidnotaltertheSVMalgorithm,andhencetheprunedmodels stillhadtheoptimalarchitecture.Thereal-timefloodstagefore- castingperformancepertainingtotheprimaryandtheprunedSVM modelswereevaluated,andthecomparisonresultsindicatedthat thepruningreducedthenetworkcomplexitybutdidnotdegrade theforecastingability.

3.1.6. Droughtforecasting

Waterstressanddroughtareconditionsincreasinglyfacedin manypartsoftheworld.Droughtoccurrencevariesinspatialcover- age,frequency,intensity,severityanddurationasitistheextended periodofdryseasonorshortfallinthemoisturefromthesurround- ingenvironment.Superficiallyonecansaythatdroughtperiods areassociatedwithperiodsofanomalousatmosphericcirculation

pattern,whichfinallyinducesrisks.Theproblemofdroughtfore- castingiscloselyrelatedtotheproblemoflong-termforecastingof temperaturesandprecipitation.Droughtimpactsbothsurfaceand groundwaterresourcesandcanleadtoreductionsinwatersupply, diminishedwaterquality,cropfailure,diminishedpowergenera- tionandaswellashostofothereconomicandsocialactivities.

Shahbazietal.[46]appliedtheSVMtechniquetodevelopmod- elsforforecastingseasonalstandardizedprecipitationindex(SPI).

TheirintentwastoevaluatetheperformanceofSVMinrecognizing repetitivestatisticalpatternsinvariationsofmeteorologicalvari- ablesin areasonably largeareasurroundingIran,which would alsomake way for enhancing the prediction capability of sea- sonalvaluesofSPI-indicatorofmeteorologicaldroughtseverity.

Theyusedhistoricaltimeseriesofthemeteorologicalvariablescol- lectedfromNCEP/NCARreanalysisdatasetasthemodelpredicts.

MutualInformation(MI)indexwasutilizedasfeatureselectionfil- tertodecreasetheSVMinputspacedimension.Themodelwas trainedtopredictseasonalSPIsinautumn,winter,andspringsea- sons.Theperformanceoftheproposedmodeldemonstratedthat theseasonalSPIvalues canbepredictedwith2–5monthslead- timewithenoughaccuracywhich canbeutilized forlong-term waterresources planningandmanagement. SVM outperformed ANNin almostall scenarios indicating higher accuracy of SVM predictions.Theirmajorobservationwasthatlinearkernelfunc- tionoutperformedtheotherkernelfunctionsintheSPIprediction.

Theyconcludedthat,thepredictionlead-timeshouldbetakeninto accountwhileevaluatingtheperformanceoftheproposedmodels inthepredictionofseasonalSPI’s.

BelaynehandAdamowski[47]comparedtheeffectivenessof differentdata-drivenmodelslikeartificialneuralnetworks(ANN), waveletneuralnetworks(WNN), andsupportvector regression (SVR)forforecastingdroughtconditionsintheAwashRiverBasin ofEthiopia.A3-monthstandardprecipitationindex(SPI)andSPI 12wereforecastedoverleadtimesof1and6monthsineachsub- basin.Theauthorseffectivelyillustratethechangesintheseasonal trendsinprecipitationusing3-monthSPI,agoodindicatorofagri- culturaldrought.Theperformanceofallthemodelswasassessed andcomparedusingRMSEandcoefficientofdetermination-R².The WNNforecastsofSPI3and12inallthesubbasinswerefound tobeveryeffective.WNN modelsrevealedsuperiorcorrelation betweenobservedandpredictedSPIcomparedtosimpleANNsand SVRmodels.

Ganguli and Reddy [48] studied and analyzed the climate teleconnectionswithmeteorologicaldroughtstodevelopensem- bledroughtpredictionmodelsutilizing supportvectormachine (SVM) – copula approach over Western Rajasthan (India). The standardized precipitationindex (SPI) wasused toidentifythe meteorologicaldroughts.Regionaldroughtsexhibitedinterannual aswellasinterdecadalvariabilityduringtheanalysisoflarge-scale climateforcingcharacterizedbyclimateindicessuchasElNi ˜no SouthernOscillation,IndianOceanDipoleModeandAtlanticMul- tidecadalOscillation.SPI-baseddroughtforecastingmodelswere developedwithupto3monthsleadtimetakingintoconsideration thatpotentialteleconnectionsexistbetweenregionaldroughtsand climateindices. Since thetraditional statistical forecast models failedtocapturenonlinearityandnonstationaritycorrelatedwith droughtforecasts,supportvectorregression(SVR),wasadoptedto predictthedroughtindex,andthecopulamethodwasutilizedto simulatethejointdistributionofobservedandpredicteddrought index.Theensemblesofdroughtforecastaremodelledutilizing thecopula-basedconditionaldistributionofanobserveddrought indextrainedoverthepredicteddroughtindex.Theauthorsdevel- oped two variantsof drought forecast models,namelya single modelforalltheperiodsinayearandseparatemodelsforeachof thefourseasonsinayearandsubsequentlymentionedthatthere wasanenhancementinensemble predictionofdroughtindices

incombinedseasonalmodeloverthesinglemodelwithoutsea- sonalpartitions.Theperformanceofdevelopedmodelsisvalidated forpredictingdroughttimeseriesfor10yearsdata.Theauthors concludethattheproposedSVM–copulaapproachimprovesthe droughtpredictioncapabilityandcatersassessmentofuncertainty associatedwithdroughtpredictions.

3.2. Applicationsinthefieldofgroundwaterhydrology 3.2.1. Groundwaterlevelprediction

Thewatertableisthesurfaceofthesaturatedzone,belowwhich allsoilporesorrockfracturesarefilledwithwater.Preciseground- waterlevelmodellingandforecastinginfluencedeepfoundation designs,landuseandgroundwaterresourcesmanagement.Simu- latingofgroundwaterlevelbymeansofnumericalmodelsrequires varioushydrologicaland geologicalparameters.In thesepapers below,applicationofSVMindevelopinga reliablegroundwater levelfluctuationforecastingsystemtogeneratetrendforecastsis beingdiscussed(Table1).

Jin et al. [49] proposed SVM based dynamic prediction of groundwater level. Taking into account of groundwater level dynamicserieslengthandpeakmutationcharacters,theyproposed the least squares support vector machine (LS-SVM) arithmetic basedonchaosoptimizationpeakvalueidentification.Considering theeffectivegeneralizationcapabilityofthesigmoidRBFfunction, itwasemployedasakernelfunctionintheirstudy.Theirmodel performanceillustratedthat;thefittedvalues,thetestedvalues andthepredictedvalueshadsmallerdifferencefromtheirrealval- ues.Theirsimulationresultsdepictedthat,theC-_ii-SVMmodel wasveryeffectiveinreflectingthedynamicevolutionprocessof groundwaterlevelwell.

Mohsenetal.[50]madeanattemptwithSVMsandANNsfor predictingtransientgroundwaterlevelsinacomplexgroundwa- tersystemundervariablepumpingandweatherconditions.Their objectiveofmodellingwastopredictthewaterlevelelevationsat differenttimehorizons,i.e.,daily,weekly,biweekly,monthly,and bimonthly.Byminimizingthe10-foldcrossvalidationestimatesof thepredictionerror(arrivedbyutilizinggridsearch),theoptimal setofhyper-parametersweredecided.Radialbasisfunction(RBF) wasidentifiedtobeaproperkernelfunctionfortheirstudy.Itwas foundthatSVMoutperformedANNintermsofpredictionaccuracy andgeneralizationforlongerpredictionhorizonsevenwhenfewer dataeventswereavailableformodeldevelopment.Ahighconsis- tencywasseenintrainingandtestingphasesofSVMmodelling whencomparedtoANN.Theyconcludethatingroundwaterprob- lems,particularlywhendataarerelativelysparse,SVMoffereda betteralternativetoANNwithitsglobaloptimizationalgorithm.

Heesungetal.[51]developedtwononlineartime-seriesmod- elsforpredictingthegroundwaterlevel(GWL)fluctuationsusing ANNsandSVMs.Theobjectiveoftheirstudywastodevelopand comparedata-basedtime-seriesforecastingmodelsforshort-term GWLfluctuationsinacoastalaquiferduetorechargefrompre- cipitationand tidal effect. Thedilemma of salt water intrusion necessitatedfortheGWLpredictioninacoastalaquifer.Theinput variableslikepastGWL,precipitation,andtidelevelswereutilized fortheGWLpredictionoftwowellsofacoastalaquifer.Themodel performanceimplicatedthatSVMwasbetterthanANNduringboth modeltraining and testing phases.TheSVM model generalized wellwithinputstructuresandleadtimesthananANNmodel.The uncertaintyanalysisofmodelparametersdetectedanequifinality ofmodelparametersetsandhigheruncertaintyintheANNmodel thanSVMinthiscase.

3.2.2. Soilmoistureestimation

Gilletal.[52]appliedSVM’stostudythedistributionandvari- ation of soilmoisturewhich is advantageous inpredicting and