Research on Intelligent Grading Evaluation of Water Conservancy Project Safety Risks Based on Deep Learning

Abstract

With the rise of artificial intelligence and big data technologies, it is increasingly significant to apply these emerging technologies to scientific decision-making in water conservancy project construction management in the face of many problems in the process of water conservancy project construction. Different from using traditional assessment methods for risk classification of water conservancy construction hazards, this paper integrates a priori attention and constructs a transformer risk prediction model based on a sliding window, which deeply explores the data value of water conservancy construction hazards information, further predicts the risk level of water conservancy construction hazards and realizes efficient and intelligent management of water conservancy project construction hazard identification management.

1. Introduction

With the booming development of China’s water conservancy industry, the construction of water conservancy projects has been accelerated.

The focus of engineering can be a system project with more complex procedures, extensive scope, many participating units, and interlocking with the construction process [1]. According to the dam failure statistics of the International Commission on Large Dams (ICOLD) and the Chinese Institution of Dam Engineering (ChinCOLD), a total of 2068 dam failures were recorded in 57 countries worldwide (excluding China) by the end of 2020, and a total of 99 dam failures in China from 2000–2021 [2,3,4]. Ge [5] based on domestic and international research and the actual application of the project, believes that the consequences of water conservancy project accident risk broadly include four aspects, including loss of life, economic loss (direct economic loss such as housing and agriculture due to inundation and indirect economic loss such as affecting transportation and normal production of factories and mining enterprises), social impact, and environmental impact (changes in river morphology and human landscape, major pollution, etc.). Sheng [6] studied and analyzed the causes of dam failure accidents from multiple dimensions, such as dam type, project scale, age, and geographical area, concluding that dam type is not the decisive factor leading to dam failure and that strengthening supervision and ensuring that all management systems for project operation are put into practice is the key to ensuring the long-term stability of project construction.

According to the above study, project risk assessment management is the key to project construction. The development of regulatory measures and emergency response systems based on the risk level of hazard sources can effectively reduce the consequences of accident risk in water conservancy projects. The purpose of this paper is to study the evaluation of safety risk level of water conservancy projects based on deep learning. The traditional risk evaluation cannot be applied to the practical application of large projects, while the transformer model based on the global self-attention mechanism can extract the contextual semantic features in the sentences, and the risk level prediction is more accurate, but it also leads to a large amount of network computation as a result. The main contributions of this research are as follows:

• The set of task scenarios of water conservancy projects was collated, and the sample data were organized in chunks by task scenarios to construct a two-by-two linearly independent task scenario vector.
• To address the problem of missing features in short texts, this paper constructs a vector of hazard source information representation based on a priori knowledge as auxiliary information, and weighted fusion of sample data in terms of task scenarios through an a priori attention mechanism, which makes the model have a more similar learning style to that of humans.
• In response to the defects of the transformer network model with large computation, integrating project characteristics task scenes and sliding windows, proposed an improved water engineering safety risk evaluation model based on transformer and built a task scene judgment gate to restrict the attention mechanism to a sliding window with task scenario, which reduces the network computation and improves the model operation efficiency.

2. Literature Review

Engineering safety risk evaluation refers to the analysis and acquisition of possible risk elements in the process of engineering construction with the help of relevant working principles and methods, and the prediction of the probability of occurrence of risks and the severity of consequences, on the basis of which effective risk prevention and control measures are established through quantitative and qualitative analysis [7]. The US Department of Defense actively advocated the implementation of risk management as early as the 1970s [8,9], outlining how the project risk management process can be modified to promote an uncertainty management perspective [10]. Bing L and Tiong RLK [11] based on the characteristics of water conservancy and hydropower projects, conducted an overall evaluation of safety evaluation factors that could not be analyzed quantitatively, using an expert survey method and hierarchical analysis [12], and computed the relative weights among the indicators [13]. They introduced a simple and practical approach to identify, assess, monitor, and manage risks in an informed and structured manner. Hreinsson [14] introduced the Monte Carlo simulation method into the risk evaluation system of water engineering for hydropower plant engineering expansion projects. While domestic research scholars continue to introduce advanced concepts from abroad, they carry out risk evaluation applications on mega-projects such as the Three Gorges Project, making the whole system of theoretical research and practical application of risk evaluation in China gradually more advanced. Yang [15] adopted hierarchical analysis and fuzzy theory in risk evaluation of water conservancy projects [16]. The combination of this method and the fuzzy theory is used to evaluate the risk of large water conservancy projects by decomposition analysis, aiming at the many factors affecting the risk of water conservancy projects. Zhang [17] proposed to use the safety checklist method to inspect the derricks and bases of offshore drilling and repairing rigs, and on this basis, the accident tree analysis method was used to identify the types of hidden faults.

Currently, traditional machine learning methods like Bayesian classifiers [18], support vector machines [19], and deep learning models, such as convolutional neural networks [20,21], recurrent neural networks [22,23], and long and short-term memory neural networks [24], have been widely used in long text classification tasks. Liu proposed a moderated deep learning model [25], the BERT model was used to generate the dynamic feature vector of the character set of the accident text, and the Bi-LSTM model was used to mine the semantic features of the accident report text, deeply analyzed the causes of water conservancy project construction accidents. In order to detect cracks on the dam surface and reduce labor costs, Lin [26] used the deep learning YOLOv3 model for prediction, which enhanced the detection of small cracks. Liu [27] used the improved YOLOv3-DN algorithm to identify the dangerous source elements of the construction site, and fed back in real time on the building information model platform to realize the intelligence of water conservancy information.

Although the traditional risk evaluation methods can solve the needs of some application scenarios to a certain extent, the performance is poor and cannot deal with such problems efficiently. In the safety risk evaluation of water conservancy construction, the information on hazard sources collected manually is mostly recorded in the form of text tables in the risk hazard database. They are described in the form of short text, and there is semantic ambiguity in the context of sparse features [28]. If deep learning models such as BERT are directly applied to short text classification, it will cause problems, like poor classification results and poor model performance.

The transformer model based on a multi-headed self-attentive mechanism [29,30,31] can fully extract the sentence context semantics, which can solve the short text type’s defect of feature sparsity. Since the model is based on global attention, the network computation is large; therefore, this paper proposes an improved water engineering safety risk evaluation model based on transformer, which reduces the computational complexity of the network model by building a task scene judgment gate and restricting the attention mechanism to the inside of a sliding window with the task scene as the unit. At the same time, the attention mechanism is used to weigh the hazard source information representation vector that incorporates a priori knowledge so that the model has a learning mode more similar to that of the human brain and improves the accuracy of model risk level prediction [32,33,34].

3. Materials and Methods

3.1. Study Area and Data Source

The project in Xinbei District of the Xinmeng River Extension and Dredging Project involves 36 villages in five towns from north to south, including Menghe Town, Xixiashu Town, Luoxi Town, Penniu Town, and Chunjiang Town (abandoned area), with a total length of about 25.29 km: 21.81 km (3.3 km newly opened) in the section north of the canal; 1.4 km in the section of Penniu Junction; and 2.06 km in the newly opened section of the southern extension. Jiepai of New Meng River Extension Dredging Project: The water conservancy pivot project is an important part of the extension and dredging project of the New Meng River, located in the town of Jiepai, Danyang City, Zhenjiang, at the mouth of the river in the northward extension section of the New Meng River, consisting of a ship lock, a restraint lock, and a pumping station, with buildings arranged in sequence from west to east.

The Xinmeng River Extension and Dredging Project is a backbone project with comprehensive benefits of flood prevention, drainage, water resources allocation, water ecology improvement and navigation, which is also listed in the 172 national major water conservation and water supply projects and the key project of the construction of Yangtze River Economic Belt, according to the “Overall Plan for the Comprehensive Management of Water Environment in Taihu Basin”, “Taihu Basin Flood Control Plan”, and “Taihu Basin Water Resources Comprehensive Plan” [35,36]. The overall layout is from the right bank of the Yangtze River in the north. The overall layout starts from the right bank of the Yangtze River in the north to the Dajie River, and the Jiepai Water Conservancy Hub will be built at the mouth of the diversion river and dredged along the old Xinmeng River to the Beijing-Hangzhou Canal, with the main function of improving the water environment of Taihu Lake and the western part of the lake and raising the flood control and drainage standards of the basin and the region.

The scale of the project is large, involving a large number of construction workers and adjacent to residential areas. Therefore, it is important to carry out an intelligent risk evaluation of the project’s risk sources, explore the potential risk sources, predict the risk level, reduce the possibility of accidents in the project, and ensure the long-term safety of the project.

3.2. Task Scenarios

The water conservancy project is a more complex procedure, the tasks to be completed in different construction tasks are very different, then the potential sources of hazards in the construction process will also have a large difference. For example, in the earthwork open excavation task scenario, close attention must be paid to the operation site because the surrounding steep slopes and mountains hold the possibility of landslides, runoff, and other major disasters. While in the earthwork blasting task scenario, the need to supervise the construction tasks such as transportation of blasting equipment, manual handling of blasting equipment, and blasting operations, there is the risk of blasting injuries to construction personnel.

Therefore, this paper proposes to take the construction task scenario as the unit and chunk the text data for pre-processing and model training. Take the first-level task scenario “earthwork” in the risk and hazard database of the Xinmeng River Extension and Dredging Project as an example, as shown in

Table 1. Task scenario information.

First-Class Mission Scenes	Secondary Mission Scenes
Earthwork	Basic regulations
	Open earth excavation
	Concealed earth excavation
	Open stone excavation
	Concealed stone excavation
	Stonework blasting operation
	Construction and safety support
	earth-rock filling

.

${\mathrm{T}}_{\mathrm{i}}$ vector representation of the task scenarios in the project, a linearly unrelated vector group can be constructed using the linearly related vector group [37]. Therefore, the vectors of task scenes are transformed as follows so that there is no direct relationship between the task scene representation vectors, where ${\mathrm{c}}_{\mathrm{i}},\mathrm{i}$ are non-zero integers.

$${\mathrm{T}}_{\mathrm{i}}=\left[{\mathrm{T}}_{\mathrm{i}},0\cdots {\mathrm{c}}_{\mathrm{i}}\cdots 0\right]$$

(1)

$${\mathrm{c}}_{\mathrm{i}}=\mathrm{i} \left(\mathrm{i}=1,2\dots \mathrm{n}\right),$$

(2)

According to Formula (1), the task scenario vector set is linearly independent and can be obtained using:

$$\left[{\mathrm{T}}_{\mathrm{i}}\cdot {\mathrm{T}}_{\mathrm{j}}\right]=\left\{\begin{array}{l}0,\hspace{1em}\hspace{1em}\mathrm{i}\ne \mathrm{j}\\ \left|{\mathrm{T}}_{\mathrm{i}}\right|,\hspace{1em}\mathrm{i}=\mathrm{j}\end{array}\right.,$$

(3)

3.3. A Priori Attention

In a multi-classification task, we exclude some classes to which the sample cannot belong and classify the sample on the remaining classes, which is equivalent to giving this sample certain a priori knowledge [38]. Combining the a priori knowledge with the deep learning network model allows the model to have a learning style more similar to that of the human brain, thus improving the risk level prediction accuracy of the network model. In the actual construction process of hydraulic engineering projects, the collected samples of hazard sources are limited, too redundant, and too complicated, and there are sparse features and text semantic ambiguity after data pre-processing. Therefore, an a priori knowledge-based hazard information source representation vector (hereinafter referred to as an a priori vector) can be used as auxiliary information. The text vector incorporating an a priori knowledge makes up for the problem of sparse sample features to a certain extent and provides some feasibility for improving the risk prediction accuracy.

If the a priori vector is simply spliced and fused with the text vector, it cannot reflect the importance of a word to the text data of different task scenarios, and the importance of each word that constitutes the text to the text can directly affect the final model effect. Therefore, this paper uses a priori attention, introduces the attention mechanism on the basis of a priori knowledge, calculates the importance weight coefficients of a priori vectors for text data of different task scenes, and realizes the optimal use of information carried by a priori knowledge.

The expertise in the field of water conservancy engineering has some guidance for the model based on the “Guidelines for Identification of Hazardous Sources and Risk Evaluation of Water Conservancy and Hydropower Projects (for Trial Implementation)” and “Classification Standards for Enterprise Employee Casualty Accidents” formulated by the Ministry of Water Resources, etc., and combined with the risk and hazard database of the New Meng River Extension Dredging Project can be inscribed for each hazard source record with two information representations of the category of hazard sources and the category of accidents that may result. The specific division is shown in

Table 2. Classification of hazard source information representation.

Hazardous Source Category	Types of Accidents That May Result
Equipment, facilities, tools, accessories defects	Object strikes
Management Factor Deficiency	Vehicle Injuries
Harsh climate and environment	Mechanical damage
Behavioral hazards	Lifting Injuries
Construction operations do not meet the specifications	Electrocution
Toxic and harmful gases	Falling from a height
Toxic chemical spill	Collapse
Behavioral hazards	Explosion, fire
Poor working site environment	Poisoning, asphyxiation
Fire Safety	Other Injuries

.

The two a priori knowledge elements, the category of hazards and the category of accidents that may result, are obtained from the hazard source description text by expert experience judgment in the field of the water resources industry. According to the hazard source information representation in Table 1, the information representation of two risk factors is divided: 0 means that the hazard description text does not contain this kind of elements, 1 means that the hazard description text contains this kind of elements, can be constructed based on an a priori vector P(d).

The structure of the a priori attention model is shown in Figure 1; a priori attention is the introduction of an attention mechanism based on a priori knowledge, and the results obtained from the calculation of the attention mechanism are passed through the SoftMax function, which represents the importance of the a priori knowledge representation vector for each task scenario, as shown in Equation (4), where w is the importance coefficient. The obtained importance coefficients are weighted and fused with the text blocks in terms of task scenes to achieve more optimal utilization of the information carried by the prior knowledge.

$$\mathrm{w}=\mathrm{SelfAttention}\left(\mathrm{Q},\mathrm{K},\mathrm{V}\right)=\mathrm{softmax}\left(\frac{\mathrm{Q}\cdot {\mathrm{K}}^{\mathrm{T}}}{\sqrt{{\mathrm{d}}_{\mathrm{k}}}}\right)\cdot \mathrm{V},$$

(4)

3.4. Improved the Water Engineering Safety Risk Evaluation Model Based on Transformer

In recent years, the transformer network model proposed by Google in 2017 [26] has been widely used in the field of natural language processing. The transformer model essentially follows the classical encoder-decoder structure of sequence information modeling, which is a model composed of a multi-head attention mechanism and a feedforward neural network. The transformer encoder consists of a self-attention layer and a feedforward network layer, which perform residual connection and layer normalization operations, respectively. A multi-head attention mechanism is introduced in the self-attention layer to obtain contextual semantic information and process all words in parallel. It can both achieve parallel computing and capture global semantic information. The model structure is shown in Figure 2.

The traditional transformer structure avoids repetition and convolution in neural networks by global self-attention but also because global attention computation can lead to a large network computation. Therefore, this paper proposes a self-attention mechanism within the task scene, which is implemented by the task scene judgment gate to limit the global attention to the task scene window and reduce the network computation of the self-attention mechanism. The model structure is shown in Figure 3.

As discussed above, the source of danger in the construction process of hydraulic engineering is discussed in the specific task scenario; therefore, the safety risk rating of hydraulic engineering should depend on the task scenario. In the process of model training, the training weight matrix should also depend on the task scenario to which it belongs; the risk prediction of the hazard source in task scenario A only focuses on the risk of the hazard source in task scenario A itself and has little correlation with the risk source in other task scenarios. Therefore, the training matrix ${\mathrm{w}}_{\mathrm{q}}$, ${\mathrm{w}}_{\mathrm{k}}$, ${\mathrm{w}}_{\mathrm{v}}$ of the improved transformer model proposed in this paper should design to meet the conditions mentioned above so that the danger source only pays attention to the training matrix in the sliding window of its task scene, as shown in Formulas (5)–(7). Among them, ${\mathrm{w}}_{\mathrm{q}}$, ${\mathrm{w}}_{\mathrm{k}}$, ${\mathrm{w}}_{\mathrm{v}}$ are the weight matrix in the sliding window of the task scene, which is trained by numerous project data, and ${\mathrm{T}}_{\mathrm{i} }$ is the task scene vector.

$${\mathrm{w}}_{\mathrm{q}}={\mathrm{w}}_{\mathrm{q}1}\times {\mathrm{T}}_1+{\mathrm{w}}_{\mathrm{q}2}\times {\mathrm{T}}_2+\cdots +{\mathrm{w}}_{\mathrm{qn}}\times {\mathrm{T}}_{\mathrm{n}}={\sum }_{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{w}}_{\mathrm{qi}}\times {\mathrm{T}}_{\mathrm{i}}$$

(5)

$${\mathrm{w}}_{\mathrm{k}}={\mathrm{w}}_{\mathrm{k}1}\times {\mathrm{T}}_1+{\mathrm{w}}_{\mathrm{k}2}\times {\mathrm{T}}_2+\cdots +{\mathrm{w}}_{\mathrm{kn}}\times {\mathrm{T}}_{\mathrm{n}}=\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{w}}_{\mathrm{ki}}\times {\mathrm{T}}_{\mathrm{i}},$$

(6)

$${\mathrm{w}}_{\mathrm{v}}={\mathrm{w}}_{\mathrm{v}1}\times {\mathrm{T}}_1+{\mathrm{w}}_{\mathrm{v}2}\times {\mathrm{T}}_2+\cdots +{\mathrm{w}}_{\mathrm{vn}}\times {\mathrm{T}}_{\mathrm{n}}=\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{w}}_{\mathrm{vi}}\times {\mathrm{T}}_{\mathrm{i}},$$

(7)

The task scene judgment gate structure is shown in Figure 4. The task scene vector ${\mathrm{T}}_{\mathrm{i}}$, to which this text vector belongs, is embedded in the input of the internal model of the sliding window, which is to perform the self-attention calculation within the task scene. Therefore, the input of the model is shown in Equation (8), where X is the text vector to be input to the model, and${\mathrm{T}}_{\mathrm{i}}$ is the task scene vector to which it belongs.

$$\mathrm{Input}={\mathrm{T}}_{\mathrm{i}}\cdot \mathrm{X},$$

(8)

According to the linear irrelevance of the task scenario vector representation group in Section 3.2 of this paper, it is possible to prevent other task scenarios from influencing the risk prediction results of this text ($\left[{\mathrm{T}}_{\mathrm{i}}\cdot {\mathrm{T}}_{\mathrm{j}}\right]=0, \mathrm{i}\ne \mathrm{j}$). There is a task scene judgment gate in the internal model, and it enters into this sliding window for model training and prediction when the conditions are met. Because of this, each text vector can only enter into the sliding window of the task scene it belongs to and cannot be influenced by other task scenes.

It is because the task scene vectors are embedded in the input and training matrices of the model that the final model output vectors are expanded by $\left|{\mathrm{T}}_{\mathrm{j}}\right|$ times. In order not to affect the final risk level prediction, the model output is scaled as shown in Equations (9)–(11), where ${\mathrm{Q}}_{\mathrm{i}}$, ${\mathrm{K}}_{\mathrm{i}}$, ${\mathrm{V}}_{\mathrm{i}}$ are the query matrix, key matrix, and value matrix of the model before scaling, respectively, and ${\mathrm{T}}_{\mathrm{j}}$ is the sliding window, ${\mathrm{Task}}_{\mathrm{j}}$ the task scene vector to which it belongs.

$${\mathrm{Q}}_{\mathrm{i}}=\frac{{\mathrm{Q}}_{\mathrm{i}}}{\left|{\mathrm{T}}_{\mathrm{j}}\right|},$$

(9)

$${\mathrm{K}}_{\mathrm{i}}=\frac{{\mathrm{K}}_{\mathrm{i}}}{\left|{\mathrm{T}}_{\mathrm{j}}\right|},$$

(10)

$${\mathrm{V}}_{\mathrm{i}}=\frac{{\mathrm{V}}_{\mathrm{i}}}{\left|{\mathrm{T}}_{\mathrm{j}}\right|},$$

(11)

Taking the query matrix ${\mathrm{Q}}_{\mathrm{i}}$ as an example, the text is entered within the sliding window model shown in Figure 2, and the derivation formula is found below, and from the final result, it is clear that ${\mathrm{Q}}_{\mathrm{i}}$ only correlates with the training matrix within the sliding window of its task scene ${\mathrm{w}}_{\mathrm{qi}}$ correlated with self-attentiveness within the task scene is achieved.

$$\begin{array}{ll}{\mathrm{Q}}_{\mathrm{i}}& =\frac{{\mathrm{w}}_{\mathrm{q}}\cdot \mathrm{Input}}{\left|{\mathrm{T}}_{\mathrm{j}}\right|}=\frac{({\mathrm{w}}_{\mathrm{q}1}\times {\mathrm{T}}_1+{\mathrm{w}}_{\mathrm{q}2}\times {\mathrm{T}}_2+\cdots +{\mathrm{w}}_{\mathrm{qn}}\times {\mathrm{T}}_{\mathrm{n}})\cdot {\mathrm{T}}_{\mathrm{i}}\cdot \mathrm{X}}{\left|{\mathrm{T}}_{\mathrm{j}}\right|}\\ & =\frac{{\mathrm{w}}_{\mathrm{qi}}\cdot {\mathrm{T}}_1\cdot {\mathrm{T}}_{\mathrm{i}}\cdot \mathrm{X}}{\left|{\mathrm{T}}_{\mathrm{j}}\right|}+\dots +\frac{{\mathrm{w}}_{\mathrm{qi}}\cdot {\mathrm{T}}_{\mathrm{i}}\cdot {\mathrm{T}}_{\mathrm{i}}\cdot \mathrm{X}}{\left|{\mathrm{T}}_{\mathrm{j}}\right|}+\dots +\frac{{\mathrm{w}}_{\mathrm{qi}}\cdot {\mathrm{T}}_{\mathrm{n}}\cdot {\mathrm{T}}_{\mathrm{i}}\cdot \mathrm{X}}{\left|{\mathrm{T}}_{\mathrm{j}}\right|}\\ & \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}=\frac{{\mathrm{w}}_{\mathrm{qi}}\cdot {\mathrm{T}}_{\mathrm{i}}\cdot {\mathrm{T}}_{\mathrm{i}}\cdot \mathrm{X}}{\left|{\mathrm{T}}_{\mathrm{j}}\right|}={\mathrm{w}}_{\mathrm{qi}}\cdot \mathrm{X}\end{array}$$

(12)

The core code of the improved water engineering safety risk evaluation model based on transformer is as follows (Algorithm 1).

Algorithm 1: Intelligent hierarchical evaluation model of water conservancy project safety risk based on transformer algorithm
Input	Text vector based on task scene ${\mathrm{X}}_1$
	Priori knowledge vector ${\mathrm{X}}_2$
	Task scene vector T
Output	The risk prediction level corresponding to the text vector Y
1	Function Linearity-Agnostic(T):
2	for (i = 0; i < T.length; i++)
3	Temp = new Array(T.length).fill(0)
4	temp[i] = i
5	T[i] = concat(T[i],temp)
6	Return T
7	End function
8	Function a priori-Attention(${\mathrm{X}}_1$, ${\mathrm{X}}_2$)
9	Q = ${\mathrm{W}}^{\mathrm{Q}}\cdot {\mathrm{X}}_1$, K = ${\mathrm{W}}^{\mathrm{K}}\cdot {\mathrm{X}}_1$, V = ${\mathrm{W}}^{\mathrm{V}}\cdot {\mathrm{X}}_1$
10	R = Scale(Q $\cdot { \mathrm{K}}^{\mathrm{T}}$) = $\frac{\mathrm{Q}\cdot {\mathrm{K}}^{\mathrm{T}}}{\sqrt{{\mathrm{d}}_{\mathrm{k}}}}$
11	Z = SoftMax(R)$\cdot$V
12	Return Z + ${\mathrm{X}}_2$
13	End function
14	Function Slide_Window_i(X, T):
15	$if \mathrm{T}\left[\mathrm{i}\right]\cdot \mathrm{T}\left[\mathrm{j}\right]!=0$
16	${\mathrm{Q}}^{\mathrm{i}}={\mathrm{W}}_{\mathrm{qi}}\cdot \mathrm{X},{\mathrm{K}}^{\mathrm{i}}={\mathrm{W}}_{\mathrm{ki}}\cdot \mathrm{X}$, ${\mathrm{V}}^{\mathrm{i}}={\mathrm{W}}_{\mathrm{vi}}\cdot \mathrm{X}$
17	${\mathrm{Q}}_{\mathrm{i}}=\frac{{\mathrm{Q}}_{\mathrm{i}}}{\left|{\mathrm{T}}_{\mathrm{i}}\right|}$, ${\mathrm{K}}_{\mathrm{i}}=\frac{{\mathrm{K}}_{\mathrm{i}}}{\left|{\mathrm{T}}_{\mathrm{i}}\right|}$, ${\mathrm{V}}_{\mathrm{i}}=\frac{{\mathrm{V}}_{\mathrm{i}}}{\left|{\mathrm{T}}_{\mathrm{i}}\right|}$
18	R = Scale(${\mathrm{Q}}_{\mathrm{i}}$ $\cdot { \mathrm{K}}_{\mathrm{i}}{}^{\mathrm{T}}$) = $\frac{{\mathrm{Q}}_{\mathrm{i}} \cdot {\mathrm{K}}_{\mathrm{i}}{}^{\mathrm{T}}}{\sqrt{{\mathrm{d}}_{\mathrm{k}}}}$
19	Y = SoftMax(R) $\cdot$V
20	Else continue
21	Return Y
22	End function
23	T = Linearity-Agnostic(T)
24	X = a priori-Attention(${\mathrm{X}}_1,{\mathrm{X}}_2$)
25	Y = Slide_Window_i(X, T)

4. Results

4.1. Experimental Preparation

In this chapter, the risk and hazard database of the New Meng River Extension Dredging Project is selected as the experimental object, and the data of 7500 hazard source records in the construction risk database of the second and third bidding sections of the New Meng River Extension Dredging Project located in Jintan District are selected for the experiments in this chapter. Among them, the risk classification of hazard sources belonged to four different risk categories. The number of samples, category labels, and examples of each category are shown in

Table 3. Risk level classification and examples.

Risk Level	Quantity	Example Sentences of Experimental Corpus
Level I	1830	Structure support material does not meet the requirements
Level II	1877	The mud discharge line needs to pass through the bridge hole and pile group, did not check the mud discharge pipe fixed measures
Level III	2748	When connecting and dismantling the mud pipe in windy and rough waters, the operator did not fasten the safety rope
Level IV	1045	The setting of steel escalators does not meet the safety requirements

.

The overall experimental idea is to divide the original hazard source text corpus into training and test sets in the ratio of 7:3 after pre-processing. To verify the effectiveness of the risk prediction model proposed in this chapter, the same dataset is used, and the classical machine learning method and the deep learning method are selected as the baseline for comparison experiments.

• In the a priori knowledge validity experiments, the same dataset and prediction model are used, and the only difference is whether or not a priori knowledge is introduced and a priori knowledge is used as auxiliary information to compensate for the feature defects in the short text for comparison experiments to verify the validity of a priori knowledge.
• In the model prediction correctness experiments, multiple network models (SVM, CNN, GAT, RCNN, transformer) are selected for the risk level prediction correctness comparison experiments on the project dataset. The SVM model parameters are as follows: the penalty parameter c was set as the default value 1, the kernel function was selected as the Gaussian radial basis function, and the function parameter g was set as 0.25; other parameters of the neural network model are as follows: embedding-dim is 256, max-length is 100, batch-size is 16, and the learning rate is 1 × 10⁻⁵.
• In the model efficiency experiments, the transformer model and the improved model are selected for comparative analysis in terms of running time to verify the efficiency of the improved model.

Using accuracy as an evaluation indicator, it can reflect the proportion of samples for which the prediction model can accurately identify the risk level of the hazard source so that the model prediction accuracy can be determined, and the accuracy indicator is calculated as shown in Equation (13).

$$\mathrm{Accuracy}=\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}$$

(13)

The number of samples belonging to this risk level that is predicted to be in this level is recorded as TP, the number of samples not belonging to this risk level that is predicted to be in this level is recorded as FP, the number of samples belonging to this risk level that is predicted to be in other levels is recorded as FN, and the number of samples not belonging to this risk level that is predicted to be in other levels is recorded as TN.

4.2. A Priori Attentional Validity Experiment

In this paper, we construct a vector of hazard source information representation based on a priori knowledge as auxiliary information and introduce a self-attentive mechanism to weigh the fused text vector to compensate for the sparse sample features. This subsection verifies the effectiveness of fusing a priori knowledge to construct a network for risk prediction through two sets of experiments.

Both groups of experiments use the transformer model based on risk level prediction. The first group of experiments uses the classical transformer model for risk level prediction, the second group of experiments uses the transformer model with simple spliced a priori knowledge vectors for risk level prediction, and the third group of experiments uses the transformer model with the weighted fusion of a priori knowledge vectors by attention mechanism for risk level prediction and compares the accuracy of the prediction of the three groups of experiments.

From the experimental results in Figure 5, we can obtain that the accuracy value of risk level prediction of the traditional transformer model is 0.548, the accuracy value of risk level prediction of the transformer model with simple splicing of a priori knowledge vectors is 0.617, and the accuracy value of risk level prediction of transformer model with the weighted fusion of a priori knowledge vectors by attention mechanism is 0.736. Through the comparative analysis of the first and second groups of experiments, the use of a priori knowledge vectors as model auxiliary information is effective in improving the accuracy of risk level prediction; through the comparative analysis of the second and third groups of experiments, the use of attention mechanism weighted fusion of a priori knowledge is effective in improving the accuracy of risk level prediction.

4.3. Experiment on the Correctness of Model Prediction

In this paper, we propose the sliding window transformer risk prediction model based on fused a priori attention. To verify the effectiveness of this model in water conservancy construction risk prediction, classical deep learning network models (SVM, CNN, GAT, RCNN, transformer) are selected for comparison experiments, and the accuracy results obtained are shown in Figure 6.

• SVM: this method uses TF-IDF to construct a hazard feature vector for hazard source text and input it into the SVM model for training to realize hazard source prediction;
• CNN: this method uses CNN to extract the text feature information of hazard sources and then uses softmax as the classifier;
• RCNN: this method is a new model constructed by combining CNN and recurrent neural network (RNN), which can combine the advantages of the two neural networks and improve the performance of the model;
• GAT: the heterogeneous text graph is constructed by using the hazard information representation vector based on prior knowledge as input. This method uses the attention mechanism on the basis of graph convolutional network modeling, which can achieve good results.
• Transformer: The model is a neural network model based on a self-attention mechanism, which is widely used in the field of natural language processing, such as machine translation, language understanding, and generation.

From the analysis of the experimental results in Figure 6, it can be seen that the risk level prediction accuracy of other deep learning network models is not much different, and they are all less than 0.6. The risk level prediction accuracy of the improved model is 0.736, which is about 25% higher than the traditional transformer model. The evaluation index data of the improved model are shown in

Table 4. Improved model evaluation index based on transformer.

		Prediction Level				Precision	Recall	F1	Accuracy
		Grade I	Grade II	Grade III	Grade IV
True level	Grade I	1424	361	35	10	0.759	0.778	0.768	0.778
	Grade II	275	1309	254	39	0.619	0.697	0.656	0.697
	Grade III	146	362	2033	207	0.814	0.740	0.775	0.740
	Grade IV	32	82	177	754	0.747	0.722	0.714	0.722

.

4.4. Model Efficiency Experiments

In this paper, we propose a transformer model based on a sliding window according to the actual project situation, which restricts the attention calculation to the inside of the sliding window in terms of task scenarios and reduces the amount of network computation of the attention mechanism. In order to verify the efficiency of the improved model, the running time of the traditional transformer model and the improved model proposed in this paper are compared and analyzed on the same project dataset, and the obtained experimental results are shown in Figure 7.

From the analysis of the experimental results in Figure 7, it can be seen that the running time of the traditional transformer model is 596 s, and the running time of the improved model is 435 s. Compared with the traditional transformer model, the running time of the model is reduced by about 27%, which proves that the improved model can reduce the amount of network computation, improve the running speed of the model, and verify the efficiency of the model.

5. Discussion

The improved water engineering safety risk evaluation model based on transformer proposed in this paper integrates project feature task scenario and sliding windows, builds task scene judgment gates, and restricts the attention mechanism to a sliding window with task scene as the unit. Analysis from the theoretical point of view: the traditional transformer model calculates the global attention, and the algorithm time complexity is $\mathsf{O}\left(n^2\right)$, where n is the sum of the text lengths of all task scenarios. The improved transformer model proposed in this paper calculates the a priori attention within the task scenario, and the algorithm time complexity is $\mathsf{O}\left(m^2\right)$, where m is the maximum text length of the task scenario ($m\ll n$). Therefore, when the sample data volume is large, there will be $m\ll n$, so the time complexity of the improved transformer model is much smaller than that of the traditional transformer model. Analysis from the perspective of example data: from Figure 7, it can be seen that the improved transformer network model reduces the running time by 27% compared with the traditional transformer model, which improves the running speed of the model.

In this paper, three experiments are conducted in the a priori attention validity experiment. In the comparison experiments of the first and second groups, using whether or not to introduce a priori knowledge as a control variable, it is argued that it is obtained that a priori knowledge can indeed compensate for the deficiency of insufficient characteristics of sample data and can improve the accuracy of model risk level prediction; in the experiments of the second and third groups, using the way of a priori knowledge fusion as a control variable, it is argued that it is obtained that introducing a self-attentive mechanism to weight fused a priori knowledge is more effective than simply splicing the accuracy of model prediction is higher. The effectiveness of a priori attention on the data of this project is argued by comparing the two groups of experiments.

6. Conclusions

This paper proposes an improved water engineering safety risk evaluation model based on transformer, which builds a task scene judgment gate and restricts the attention mechanism to a sliding window with the task scenario as the unit, so this model reduces the computation of the network to a greater extent compared with the traditional model. At the same time, this paper introduces the attention mechanism on the basis of a priori knowledge, calculates the importance weight coefficients through the attention mechanism, and weighted fusion of a priori vectors and text vectors, which makes the model have a learning mode more similar to the human brain, compensates for the deficiency of insufficient sample features and improves the accuracy of model risk level prediction. Experiments were conducted on the data of the Xinmeng River Extension and Dredging Project, which proved the effectiveness of a priori attention for improving the accuracy of the model and also demonstrated that the sliding window in terms of task scenarios reduced the computational effort of the network model, and the research in this paper was successfully applied on real data.