12 2
1401
358 - 349
DOI:10.22059/jwim.2022.341294.981
: ! " #$
!"#$
%
*
! "!#
$ %!#
&
' ( ' .
:+, - ./ 0 1 18
/ 01 / 1401 +, - 7 89 0 1
: 10 / 04 / 1401
!
" # $ % & ! % '
() * + . -
. / 0
1 ! 2 3 45 6&7 $ ( ! 89 : .2 - * + ; < = > ' ?& &9
@ ' 6&7 - 89 ; A
& B -
.
* + ; 1 ! C . : 1< & -
&7 89 6
1<
D " &EF / 0 . . 3 45
GHI
!C) 8 . .2 ! A ( I 3 45
6 () ;
() * + &J K L)
M N ! & 89 D 10
;2 #
!
" #
2 Q .2 - A =
!&H-
!C) 8 R .2 - ' 89 &S T ;2 D 9 D 1- & !:H- U+ ! ;
;2 - * + () 2I ! D U
! V W1< > ! +
-
! 1< V
9 / 0 . - 6 .& Z C ; :
[ 5 ! D &
650 . :
89:
&
;
; 2
; J&
.
Determining operational patterns considering operator’s error in structures settings in irrigation networks
Kazem Shahverdi*
Assistant Professor, Department of Water Science Engineering, Faculty of Agriculture, Bu-Ali Sina University, Hamedan, Iran.
Received: April 07, 2022 Accepted: July 01, 2022
Abstract
Structures operation in traditional water conveyance and distribution canals is manually done using operators’ experience. Determining operational patterns in these canals is an important issue done in recent studies using artificial intelligence. One of the errors occurring during the settings of the structure is the operators’ error applying some errors as they operate the determined setting. This issue and its effect hasn’t been investigated in the previous research so far. In this research, the reinforcement learning model was used to determine the operational patterns considering the operator errors of five percent and 10 percent applied randomly. A non-linear model of the studied canal that is the E1R1 canal as a part of Dez network located in the north of Khuzestan was employed to simulate. The results showed that reinforcement learning can accurately determine the operational patterns with a maximum iteration of 650 so that the action values are more than 0.9 in most cases.
Keywords: Canal, Operator Error, Water Management, Reinforcement Learning.
!:H- ; ; H &( !'&^ U: &
_ = N ! W L (/ ? )
B . < &(
. 1< D
!
" #
& T B
. - V & ; Z ! !Q
B ! G L ! b 2 N
!c 6&7 " &G1< ; -
$ %
)
Savari et al., 2016
( ! 6&7 &- . 2I .
89 / ;D d< . ( & ! % &- . 2I 1 Ee
. -
:G1< H
!:H- ;
3 45 'G +
!Q 2 ! A J I
1<
D
& D " ; G L
! D <
5 7
! & 2 Q H
:G1<
' ; 9 .2 ! A J I
G L 6%N D ?&
!Q 2 ! A J I
)
Sothea et al., 2013
( . U 45 W L 2 Q !
89 2 - $ % ! f <
:G1< ! A !C 89 &S D
.2 -
. R 3 45 ; ' ! D U
_ = .& W L 2) < 3 ?A ! % D J
2 - )
Anwar et al., 2016
(
! & 6 () .
D J 5 !C Q
! 7 W L D 3 ?A
2A !C I D .&Z
6 () . ! D U R .2A J 2<
5 - (1 ! & g ! 2&HM )
Liu et al., 2018
( .
&- ! 'C0 g ! !Q ; Q & B
! J 3 45 'G +
W1C) () * + 2 Q
>
' (2 9 b N ) A
() .2 ! A J " # < = >
* + -
!
" # I & & 9 ) Q
;2 &J h
!V Q
$ % ; A > ! 2 i ! $ f . -
! !Q G L D
" # jA ! A J
.&&C U .&
. -
.&) 3 45 () * + ! g
6 () ; < = > '
&J
! 1
D <
!9 - : 5
A ; < = >
- '
I
&J D ! 2A J ()
&(
! kG + ! J Q " #
2 0 R $ % .10 . - W# N 3+
.
! ;
! 2G<
b 6&7 D & D " ;
U / U ?A * ! g W I Q 2A 3
23 N ! 2<
5 / 5
& 2<
)
Shahverdi et al., 2016
( ! no .
>
- $ % > ' &J ! > . f 2I ( & !
; >
! 4 6 B - -
)
Savari et
al., 2016
(.
: 6
6 () ? .
; &J D
;2 &J R 6&1C
p 9 B - Z &J ' ! C . () * + () ;
8 -
! 2 / 0 . .
3 45
Shahverdi et al.
) (2020
R
.2 - > ?J ;f 2I 6 () ' 2 9 b N >
D (Z !C Q
!Q ! A J I )
Fatemeh et al.,
(2020
2& N W&GL n5 ;
! 2 D
H1 B - ;6 () 5 ! &
2&G I - U&5 ! D U R .2 - ! !c
H1 B - N
.
* + () '
' -
&J T ! U
/ 0 3 45 < = > ! & (
- )
Shahverdi, 2022
( ; J4 . .
I . Z ; () T '
6 () J4 . R . 6&7 D R jA ?& &J
( U ! 2 ! A J I I - : &J 6 () > D . -
3&
! () !
()
! &J : ; &J 1-
;
! 3 45
Shahverdi & Javad Monem
) ( 2022
2< 3 ?A ! % 2A J I
. - &J
3 45
Savari & Monem
) (2022
;
!:H- T& q 6 () ' ( 1- < = H=<
!
" # 2A !C H&
!G1Q W L kG +
! ! <
3- ! D U R . - ! <
!C r9 - H ! I ;! A W L
. - D 1 U ! V ; -
3 45 'G +
6&7 D D ?& .&&C ! &
&9
. .2 - $ % ; N
6 D Z " 1&7
! 2G<
29 H
? b &J
1 I ;j&I D
' &(
$ % & =+- ! % ( _ V . -
! !Q ! Z 3 45
* + ; &9
()
'
>
>
$ % < = () j&I * + ! % ! -
" 1&7 . 1< ; -
! 2G<
D H 9
!:H- ; 6 D Z $ % & B
!%& . -
6&7 89 1 !%&
. . Q G L 2&G I 2 & ;
6 () - 89 K L) < = >
0 ! &J I ; & :G1<
3 45 .2 0 N
!" #$ %
&J ; &J >
h 6 3
2&C0 ) B&L (!s 2 N) W < T .& )
&C0 ; : ! 2 ( :&) &
j1<)
(!s J - ) G1< - U ( 2 f 1< ( 0 ) B&L ! + . -
$ % W1< ! 2 D > - - 5 -
W1<
" G !H L
. - W1< !
&5 G8 - M N > 5 ;
3&
.
W1< >
; & . - 9 W# N 2
tf 1C > 5 h
u8 u8 89 D ?&
k C _ > . A G# B . -
!
" # ) B
1 ) ( 3 ( W:-) - 1
.(
m
j k
ij i k
ij i ij
Q a Q
p
1
) exp(
/ ) exp(
)
(
(1 !8 )
)
* 2 exp(
/ 2
1 t target
k y y
r
(2 !8 )
← . (3 !8 )
) B 1
) ( 3
' 5 (
-
!
" # . -
) (a :p - W1< + 1 N
:
!Q j 8 D
+ W1<
'# .& ) -
(T
:Q - W1< >
: 5
:r - W1< > 5
yt
: j1<
et
ytarg
: j1<
_
:
&J w
J - !&) Z ;
! D <
k C ! !&) ) !8 ' . -
1 (
J - T + 1 N !H L
1 N ! G1< 2 3&
; - ! -
0 !s - +
;
!
.
;3 45 :& &
; - ICSS
1<
. -
!1 > ! 2 i ! $ f
J - + D : J -
!
" # A = ; &J ! . -
> G8 J - 3&
b
D + 1 N 3&
j1< ! !Q . - 9
! 2 ) !8 ' > 5 ; 2
!H L (
) !8 ' W1< > 2 - 3
(
!H L . &J > _ . -
2
. M N &J A V > ! 1< + ! G1< : ! C .
5 ; - 2A -
. ; C !GN
! > 5 ) !8 jHV >
3 (
' 3 ?A : C 3 ?A . -
.2A 9 3 ?A ?& W1< > ;> 5
D - !HN = ! !Q ! 2 i W I 2 - :H- 2
!
&
;
6&7 89 M N & B
10
±
# . . -
3 45 &J ;
() * + K L)
D
1 .2 - 89 . '
3 45 ;
!
#
"
!s U 2 !
D H< !8
!
" # . -
Initialization
Do loop while terminating iterations
Calculate all possible candidates’ probabilities
Choose the best action and add operator error
Apply the action to the mathematical model
Calculate reward
Update action values, decrease learning rate, temperature End loop
Figure 1. Flowchart of the investigated algorithm during learning.
) !8 4 (
. . . 2. . ℎ
;D ! ;(!& E bC: ) H<
!s !1 ! b 0
6 / 0 ;2
;( ) !s y <
!s J -
;( ) (!& E z% ) W E -
/ ' ℎ
( ) !s 2 f !1 ! -
. - 2&HM _ j1< B - !1 _ j1< ;!C) 8
2 / 1
!:H- W1< .2 h & ;
2 9 J - ;.&< B -
!s
&7 6 89 1 !
89 J .2 !Q % ;6 &( 7 ;
1
1< !s !8 ) -
5 ! % ! (
89 !%& _ j1< j1< _ L &( ! G L
?&Z I . - W1< !
$ ( @ ' & B
A
! !8 " # : -
) !8 5 (
. . 1 . 2. . ℎ
;D ! M N 6&7 89
10 i ! $ f .2 - ! A J 7 #
.& ;!s J - ! 2 J -
) WI N ) M N J - (
I (
J - J ; &J A V . 3& h ! !s + 1 N .
-
' D ! g 89 ; -
) !8 6 ! - !H L ( !A 0
. -
) !8 6 (
! " ! ∗ $%
;D ! !GN - &) A = 89
; &J V
! .
; 89 .& 5 N
" .
89 f N &) $%
&) T '# .& < ! 2 A = <
h &J ;3 45 . .
5 # ± 10 - # ±
! 1 .2 - &) A =
1<
-
5 # ± 10 # ±
) W:- 2 D U ( .2 -
Figure 2. Random numbers produced and applied for & ' &() *+ percent.
&' # (
) *
:& &
ICSS
1985 ? B
!&H- T
C D Q J
! I ! - !& % &| J &S
!&H- kG + :&) & B -
. - 2 ? ! T .
D Q "f C ! ' J &S
! WN !V
) Manz & Schaalje,
(1992
;3 45 . .
j&'G .
.2A J I ' &J
!&H- ! & ! ; -
- 9 AICSS
!s J - !Q $ f " H L $ % Q 9
& j1<
r+U &9i - ;j1< ' .
!H L W1< > 2 > 5 . -
+, )
;3 45 . '
E1R1
!:H-
! i 8 &( 3- ;
V 5 / 2830 W:-) 2
3 .& b&- .(
00012 / 0 0012 / 0 k y < ;
!G# A 1003 C
5 / 1 !8 .
!G# A &( . 9 WL 5
/ 2830 T C
H Q b&- .2
! & 1 " #
1V:1.5H
.2 - N V
Figure 3. The schematic view of E1R1 canal.
!C) 8 2&A ^ M N 74
/ 2
bC:
H1 ! !Q ; .2 !& E
kG + " C) 8 D Q D ! D Q ; -
2 / 1 .2 !& E bC:
;3 45 .
.&
5 / 0 4 / 1 bC:
$ .2 ! A J I ;?&
&J > 1<
# R 5 89 89 no
10 !s J - ! #
+ 1< ; - C ! 2 i ! $ f . -
&J W&1: : 650
:
_ L . - .&&C 89 D ! -
! j1<
?& _ j1< &J Q 9 D <
2 Q ;_ j1< j1< _ L # ' 1< 89 D ?& !
2 - ; -
!8 ) 7 .(
) !8 7 ( 100
*
arg y
y d t et
;D ! # b N _ L d
. -
- $) ./
.&&C &J ;3 45 . () ! &
89 Q
&J A V .2 - &
&5 3 ?A ! & W1< >
!
) !8 ! !Q ! ; 9 M N 2
j1< J ;(
T - _ j1<
T ? -
. . -
! ; N U "
t &I ! -
! 1 . T < ! 5 " &&| 1
W:- &J A V &J w ) 4 (1 ( & ! 2 - D U (
: 600 . .2 : &J 1< ; N
500 : 2 - $ % 500
! C
! 6 &J J - 1< V
!s
> - U WHI : !
3&
I ' ; -
~< ! &J
-
! R ) > 3 ?A D (Q
1 N ;
D +
3&
J - T 2 -
! D <
3& ! & J - 1 N .
. - +
Figure 4. Temperature and learning rate curves during the learning.
) Q
1
) > (
(Q
J - j1< j1< _ L B ! &
# R 5 89 2) N _ 10
#
M N WI N .2 - !c
+ W1< >
! - b&
850 / 0
993 / 0 # R 5 89 2) N % ! -
.& j1< 89 ! 2
/ 0 8 / 3
# .2 -
89 10 > M N WI N ; & #
+ W1<
! - b&
666 / 0 935 / 0
j1< 89 ! % ! - 5
/ 2 # 5 / 10
j1< 89 ;! H) .2 - # 10
#
D ! T ? 6
40 I #
2 A @ ' !
R . . -
D U " # &J !
&J ! I ?& &J V D ! 89 1<
! & ()
$ % . - -
; &J ! - r+U ; Q 9 R 89 1<
9 G#
1 * 9 ! & W1< ;2
& 89 .&( & h ! !Q . &J
) W:- 1 '# 89 .&( & ! ( . ; -
W1< ; 8 - ! 2A J !%& D ! &
: C Z 9 &5 + j&I R ; -
!
. 9 2
Table 1. The final action values and average water depth deviations from target value.
Operator Error (%)
±5% ±10%
flow (m3/s)
Action Value
Deviation (%)
Action Value
Deviation (%)
0.5 0.981 0.5 0.935 2.5
0.6 0.979 2.2 0.935 2.5
0.7 0.980 0.3 0.935 2.5
0.8 0.989 0.3 0.935 2.5
0.9 0.986 0.2 0.778 5.9
1.0 0.987 0.6 0.705 8.1
1.1 0.993 2.4 0.848 2.8
1.2 0.850 3.8 0.767 9.1
1.3 0.935 2.5 0.874 7.8
1.4 0.935 2.5 0.666 10.5
max 0.993 3.8 0.935 10.5
min 0.850 0.2 0.666 2.5
&J A V > " &&| 1
# R 5 89 ) W:-
5 !c (
D 1 .2 - U ! V
" &&| -
; !1 t H +1 " &&|
: C (1
600 U . -
W1< C 3&
+ D - . -
W1< . C &J B no 6
-
: : ? 600
6 N ;
10 1 N W1<
3&
: C - 600
T t H ;
3& ; W1<
3& + 1 N . .
>
+ W1< 2 . -
2 / 1
> 5 ;!& E bC:
U ?& ' -
z W1< !1 Q C ! ! . -
Z : C
500 1 U ' > 5 D 1 . -
. ! V
W:- U &J ; -
T N ;
2) N . .2 U !Q 2 :- ?&
89 10 U ?& & # 2 Q ! ! -
W:- C 3 .2 U !c %
Flow=0.5 l/s Flow=0.6 l/s
Flow=0.7 l/s Flow=0.8 l/s
Flow=0.9 l/s Flow=1.0 l/s
Flow=1.1 l/s Flow=1.2 l/s
Flow=1.4 l/s Flow=1.3 l/s
Figure 5. Action values for different flows for the operator’s error of 5%.
D 1 U ! V -
4 / 1 - 2 / 1
6 g 3 5 ;bC:
2 C . ! -
W1< M ! 1< +
H b -
> 5 2A (!1 Q) '
. . ; N
. R .2 - W# N (1 3 45
2&1 ;
'
>
!C < = >
D
Q N ! -
! ?& &J V 89 1<
! & ;
&5 .
0 $)
" # &J ;3 45 . K L) 2A J I & 89 D
89 ; : R 5
10
! & # V
! J Q
U+ !C) 8 . - 1< ! kG +
!&H- 2 I - D
T& &
G1< + 1 N . - $ % ICSS
z W1< ( - > D ?& h ! -
W1< &J A V > ;
3&
3& W1< 2 - > .
! + ! & W1< D <
! D U R . -
89 2) N R 5
& # Q 9 89 ;
2 / 0
8 / 3 ; # 89
10
& # Q 9 89
5 / 2 5 / 10 # D 1 . -
(1 ;
M N . U U ?& 2 :- T N - : C 650
. .
6 : N
600 . &5 ! & W1<
3 45 GHI
- > ?J A R 2 - $ % () * + . A J 7 () * + ;3 45 . ;2 R - K L) ?& & :G1< - 89
jA U&5 ;. . - W# N ?&
. -
?& & 89 - ' ! $ I K L)
. . () &J C T ; N
U&5 . &J $ % G 3 45 ! -
#1 2' )
1. Reinforcement Learning 2. Action Value
34 5 6 +%
€&
Q D J B A y C ! J .
3 5
1. Anwar, A. A., Bhatti, M. T., & de Vries, T. T.
(2016). Canal operations planner. I:
Maximizing delivery performance ratio.
Journal of Irrigation and Drainage Engineering, 142(12), 04016057.
2. Clemmens, A. J., Kacerek, T. F., Grawitz, B.,
& Schuurmans, W. (1998). Test cases for canal control algorithms. Journal of Irrigation and Drainage Engineering, 124(1), 23-30.
3. Fatemeh, O., Hesam, G., & Shahverdi, K.
(2020). Comparing Fuzzy SARSA Learning (FSL) and Ant Colony Optimization (ACO) Algorithms in Water Delivery Scheduling under Water Shortage Conditions. Irrigation and Drainage Engineering.
4. Liu, Y., Yang, T., Zhao, R.-H., Li, Y.-B., Zhao, W.-J., & Ma, X.-Y. (2018). Irrigation Canal System Delivery Scheduling Based on a Particle Swarm Optimization Algorithm.
Water, 10(9), 1281.
5. Manz, D. H., & Schaalje, M. (1992).
Development and application of the irrigation conveyance system simulation model. Proc., International Seminar on the Application of the Irrigation Mathematical Modeling for the Improvement of Irrigation Canal Operation.
6. Molden, D.J., & Gates, T.K. (1990).
Performance measures for evaluation of irrigation-water-delivery systems. Journal of Irrigation and Drainage Engineering, 116(6), 804-823.
7. Savari, H., Monem, M., & Shahverdi, K.
(2016). Comparing the Performance of FSL and Traditional Operation Methods for On- Request Water Delivery in the Aghili Network, Iran. Journal of Irrigation and Drainage Engineering, 142(11), 04016055.
8. Savari, H., & Monem, M. J. (2022). Optimal operational instructions for on-request delivery using hybrid genetic algorithm and artificial neural network, considering unsteady flow.
Irrigation and Drainage,
https://doi.org/10.1002/ird.2670.
9. Shahverdi, K. (2022). Evaluating utilization of structures' settings of one reach in the others.
Water and Irrigation Management, 11(4), 769- 779 (In Persian).
10. Shahverdi, K., & Javad Monem, M. (2022).
Irrigation canal control using enhanced fuzzy SARSA learning. Irrigation and Drainage,
https://doi.org/10.1002/ird.2684.
11. Shahverdi, K., Maestre, J., Alamiyan-Harandi, F., & Tian, X. (2020). Generalizing Fuzzy SARSA Learning for Real-Time Operation of Irrigation Canals. Water, 12(9), 2407.
12. Shahverdi, K., Monem, M. J., & Nili, M. (2016).
Fuzzy SARSA learning of operational instructions to schedule water distribution and delivery.
Irrigation and Drainage, 65(3), 276-284.
13. Sothea, H., Pierre-Olivier, M., Gilles, B., & Cyril, D. (2013). Optimization of water distribution for open-channel irrigation networks.
