An IoT Data Clustering Algorithm for Efficient and Cost-Effective Human Resource Assessment

Literally, the focus of human resources assessment is on the assessment. However, human resource teams across multiple industries are leveraging the IoT to improve employee health, safety, productivity, and comfort. In human resource management, the human resource manager supervises and enhances the relationship between employees, from hiring to firing. They must hire the right person for their enterprises to close the skills gap between employees, ensure the safety and comfort of their employees, and provide them with the necessary services and benefits [14-16]. Human resource managers must constantly monitor and track employee work, behavior, activities, and comfort. Traditionally, human resource managers monitor employees by observing them, using feedback forms, or integrating monitoring software into their computers. However, this monitoring method needs to provide accurate and objective assessments, which can lead to better communication and decreased employee engagement [17]. Therefore, such consequences can be avoided by using the IoT in human resource management.

IoT in human resource assessment can help human resource managers perform almost all recruiting, monitoring, or paying duties [18-20]. Only in a few cases does the human resource manager get honest feedback. Even so, it was achieved through much hard work. Getting honest feedback is vital for human resource leaders to take steps to improve the overall workplace and workforce. IoT devices can reduce the burden on human resource specialists while collecting feedback.

IoT nodes need to meet the characteristics of low latency after collecting data, which cannot be satisfied by traditional cloud computing task scheduling [21-22]. In [23], the authors used the Markov decision process to propose a one-dimensional search algorithm to solve the power-limited latency minimization problem, which was an optimal search algorithm. However, this method needs to obtain accurate information, such as the channel quality of the cluster and the task arrival rate in advance, which has certain limitations in practical application. In [24], the authors used the optimized results of the sequential quadratic programming method to train the deep neural network to reduce latency and energy consumption. However, this method is also limited to the single-edge server scenario. In [25], the authors proposed a multi-platform resource offloading algorithm, which used the k-nearest neighbor algorithm to select cloud computing, edge computing, or local computing and then optimize resource allocation through reinforcement learning. However, it focuses more on latency optimization and ignores device energy consumption. Additionally, some kinds of literature use heuristic algorithms to solve the edge computing offloading problem. In [26], the authors proposed a novel network architecture and used Lyapunov stochastic optimization tool to solve the problems of latency and reliability constraints of calculation and transmission power minimization. In [27], the authors use heuristics such as first-fit, online-first, and linear programming to optimize energy and latency problems. When using the heuristic algorithm to solve the problem of large-scale task offloading, due to the high dimensionality of the problem, it takes too long to generate offloading decisions, so it cannot meet the needs of real-time task processing.

Currently, most edge computing offloading strategies are based on a priori distribution or historical data. Since task offloading in MEC is transmitted wirelessly, available resources change dynamically with factors such as channel quality, task arrival rate, and energy. Therefore, a priori distribution or historical data predicting algorithms are difficult to apply in actual scenarios [28-30]. Based on the shortcomings of the above research, the offloading scenario is defined as a multi-IoT nodes multi-edge server scenario. Considering the latency, energy consumption, and cost factors simultaneously, a multi-IoT nodes cooperative relationship is proposed. On the premise of ensuring the maximum latency, the energy consumption and cost are reduced, and the resource utilization rate of the edge server cluster is improved. This study introduces deep reinforcement learning, combines the decision-making ability of reinforcement learning with the perception ability of deep learning, and relies on the representation learning of robust deep neural networks to solve the high-latitude dynamic task offloading problem [31].

The soft subspace clustering algorithm obtains the importance of each data feature by weighting the individual features in each data cluster to find the subspace where the features with large weights are located [32]. Compared with the hard subspace clustering algorithm, people pay more and more attention to the soft subspace clustering algorithm because of its better flexibility and adaptability to data processing. The soft subspace clustering algorithm is integral to the cluster analysis algorithm. It distinguishes the importance of features by assigning weighting coefficients to different data features and realizes flexible control of the clustering results. The key factors that affect the performance of the soft subspace clustering algorithm include the number of data clusters and the initial clustering center [33]. An excellent clustering algorithm should be able to converge to a reasonable number of clusters, and the initial selection of clustering centers has little influence on the clustering results. Currently, popular clustering methods include data cluster evaluation criteria, cluster visualization, and fuzzy clustering methods. Although these methods can obtain a reasonable number of clusters, considering various data features must be more comprehensive.

The model built in this study is the multi-edge server model of the IoT nodes, which is the task offloading scenario of edge computing. As shown in Fig. 1, the model includes N edge servers (ES) and M smart devices (SD), where ES is deployed on the base station, and SD transmits data through a wireless link. Tasks on SD can be executed locally or on ES based on the latency and power consumption requirements.

Fig. 1. Network model diagram.

Download Original Figure

Given task granularity [34], this study adopts fine-grained task partitioning regarding resource utilization and computational complexity. Each task on the IoT nodes can be divided into k subtasks, and different subtasks can be executed on different ES or locally, thus reducing the computing latency. The task set is defined as $R,R_{ij}\triangleq \left(D_{ij}^{in},C_{ij},D_{ij}^{out},{\omega }_{ij}^{max}\right)$, where i ≤ M, j ≤ k. $D_{ij}^{in}$ represents the data amount entered by the jth subtask on the ith IoT node, including codes and input parameters. C_ij represents the computing resources required to execute the ijth subtask. Here, the computing resource refers to the CPU. It is assumed that the computing resource required to execute the task on the CPU of the local IoT node is the same as that required to execute the task on the CPU of the ES. $D_{ij}^{\text{out}}$ represents the result data obtained after completing the ijth subtask computing, and ${\omega }_{ij}^{\text{max}}$ represents the maximum latency constraint for executing the ijth subtask.

If x ∈ {0,1,2,⋯,p,⋯,N}, x = 0 indicates that the subtask is executed locally. If x = 1, the subtask will be offloaded to ES whose number is 1. Similarly, x = p indicates that the subtask will be offloaded to ES whose number is p.

Assuming that the latency of the local processing task is generated only by the CPU, which is defined as follows.

where C_ij is the computing resource required by R_ij. Specifically, it is the number of CPU clock cycles required to execute R_ij, and $f_i^{SD}$ represents the CPU clock speed of IoT node i.

Suppose R_ij needs to be offloaded to the ES for execution. In that case, it is necessary to analyze the latency on the ES from three aspects: sending latency, transmission latency, and processing latency, defined as follows.

where $v_i^{\text{SD}-\text{up}}$ is the upload speed of IoT node i, $v_p^{\text{ES}-\text{up}}$ is the upload speed of edge server p, v^tra is the transmission speed on the channel, l^SD−ES is the length of the channel between the IoT node and the edge server, $f_p^{\text{ES}}$ is the CPU frequency of the edge server p, and (2l^SD−ES/_v^tra is the round-trip transmission latency.

When there is the dependency between subtasks, the total latency is the sum of the latency of subtasks executed locally plus the sum of subtasks executed on the edge server.

When subtasks are independent, the total latency is the maximum latency of all subtasks on the ith IoT node.

Assuming that the energy consumption generated by tasks executed locally is only related to the CPU, the energy consumption model is defined as follows.

where $p_i^{SD}$ is the CPU power of IoT node i.

When offloading, the energy consumed by the jth subtask on SD_i at the IoT node side is defined as follows.

where $p_i^{\text{SD}-\text{up}}$ is the upload power of the CPU of IoT node i, and $p_i^{\text{SD}-\text{free}}$ is the discharge power of the CPU of IoT node i when it is idle.

The cost refers to the cost paid to the operator by the user for using the computing resources provided by the operator. Therefore, the cost is only incurred after offloading and does not involve the local execution part. In this study, the cost model based on the remaining computing resources is used; that is, the more computing resources are left, the lower the cost paid by the user, which is defined as follows.

where u^ES represents the price per unit time of the edge server, $R_p^{ES}$ represents the utilization rate of computing resources of server p, and $To{t}_p^{ES}$ represents the total computing resources of server p.

IoT nodes’ edge computing task offloading problem is an optimization problem. The goal of this study is to minimize energy consumption and cost under the premise of guaranteeing latency and improving the resource utilization rate of the MEC system while extending the battery life of IoT nodes. The objective function is defined as follows.

where β and ε are the weights of energy consumption and cost, respectively.

This study proposes a partially observable and fully cooperative edge computing task offloading algorithm based on deep reinforcement learning to solve the above model. In fact, due to the instability of the underlying physical link, the state of the edge server cannot be observed entirely, which is called partially observable [35]. In the reinforcement learning of IoT nodes, each IoT node is autonomous and interactive, which independently receives the status information of the edge server and decides the current offloading scheme according to its policy function. At the same time, IoT nodes communicate with each other to jointly optimize the target value, and a comprehensive standard judges the overall offloading effect.

Markov decision process (MDP) generally contains state space (S), action space (A), and reward function (R) [36]. The state space S is defined as follows: resource surplus vector U, decision matrix X, and objective function ℒ.

where U = [u₁, u₂,⋯u_p,⋯,u_N], and u_p represents the computing resources available to the ES with number p.

The action space A represents all actions that can be taken in each state, which is defined as follows.

where λ is the offloading scheme of the jth subtask on the IoT node i, and λ ∈ {0,1,2,⋯,p,⋯,N}.

The reward function R represents the reward obtained by the IoT nodes according to the environmental feedback after performing actions and state transitions, which is defined as follows.

where ℒ_t is the objective function value in period t, ℒ_t+1 is the objective function value in period t+1 after the IoT node makes the response action A_t in state S_t, and ℒ^SD is the objective function value when the task is executed locally. The above three objective function values are calculated using equation (9). It should be noted that when executed locally, the objective function does not have a cost part (equal to zero), only an energy consumption part and latency constraint. When ℒ_t+1 > ℒ_t, that is, after the action A_t is executed, ℒ becomes large; that is, the energy consumption and cost increase, which develops in the undesired direction and gives a negative reward. Conversely, a positive reward is given.

Deep recurrent Q-network (DRQN) is an improvement to deep Q-network (DQN), which uses a recurrent neural network (RNN) to solve partially observable MDP (POMDP) to improve DQN performance. In the MEC application scenario, due to the instability of the underlying physical link and the limitations of the IoT node’s perception, it is difficult for the IoT node to receive accurate state information of the environment in the current period, leading to the POMDP problem [37]. DRQN retains the previous state information, supplements the current state information, and solves the problem of missing values. The specific implementation process is to replace the first fully connected layer after the convolutional layer in DQN with a long short-term memory network and RNN. The DRQN architecture is shown in Fig. 2.

Fig. 2. DRQN architecture diagram.

Download Original Figure

The state information returned by the edge server ES is used as input. The convolutional layer performs convolution operations on the state information to obtain a feature map, which is transferred to the LSTM layer. LSTM supplements the state information according to the previous memory, updates the memory, and finally outputs the Q value through a fully connected layer. Here, h_t is the return value of the network at the previous time step, that is, h_t = LSTM (h_t™1, a_t), which is used as the input for the next time step.

DRQN uses a neural network with a weight of θ to approximate the Q function, that is, Q(h_t,a_t;θ)≈Q*(s_t,a_t). The loss function is the mean square error between the target value and the predicted value, which is defined as follows.

where r is the reward, γ is the discount factor, between 0 and 1, determining the importance of immediate and future rewards. When γ close to 0, immediate rewards are more important. When γ close to 1, future rewards are more important. The neural network updates θ by gradient descent to minimize the loss. It should be noted that the parameter of the target Q is θ′, which is differs the parameter θ of the prediction Q. DRQN uses an independent network called the target network to calculate the target value.

The IoT node executes an action a from the state s to the state s′, and gets a reward r. < s, a, s′, r > is an experience of the IoT node, which is stored in the experience cache. DRQN randomly selects a batch of training samples from the experience cache to reduce the correlation between experiences and improve the algorithm’s generalization.

This study compares DRQN with DQN and random offloading in large-scale heterogeneous clusters in terms of energy consumption, cost, and latency to evaluate the advantages and disadvantages of DRQN in offloading IoT nodes’ human resources assessment tasks [38]. The cluster simulated by the simulation includes 50 edge servers and different numbers of IoT nodes. Among them, the edge servers are evenly distributed in nine areas with the same area, and the IoT nodes are randomly distributed within 150 m from the base station. Assuming that the bandwidth is 20 MHz, the upload speed of the IoT node is 5.3 Mb/s, the upload power is 1,500 mW, the idle discharge power is 200 mW, the limit CPU clock speed $f_{\text{max}}^{SD}$ is 2 GHz/s, and the limit CPU frequency $f_{\text{max}}^{ES}$ of the edge server is 50 GHz/s. Moreover, the data amount (Kbit) of R follows the uniform distribution of (200, 2000), and the required computing resources follow the uniform distribution of (1200, 3000). When γ is set to 0.7 and 0.9, the energy consumption and cost under different tasks are compared. As shown in Fig. 3, the performance of energy consumption and cost when γ = 0.9 is better than that of 0.7, which means that future reward plays an essential role in edge computing, so γ is set to 0.9 in this study.

Fig. 3. Comparison of energy consumption and cost of DRQN with different discount factors.

Download Original Figure

To meet the maximum latency, this study takes the energy and cost reduction ratio (ECRR) of the difference between DRQN, DQN, random offloading, and local execution as one of the evaluation metrics of the algorithm. As shown in Table 1, the ECRR of DQN and DRQN reaches more than 50%, and DRQN is nearly two percentage points higher than DQN.

Next, the number of tasks ranges from [20,100], and the step size is 10. The comparison between local execution, random offloading, DQN, and DRQN in energy consumption, cost, and latency is shown in Fig. 4. It can be seen from Fig. 4 that with the increasing number of tasks, the offloading strategies generated by each algorithm generally show an upward trend in terms of energy consumption and cost. DRQN performs better than random offloading and DQN regarding energy consumption and cost. Local offloading consumes the most energy, which is caused by the fact that the computing power of IoT nodes is much smaller than edge servers. Additionally, it can be seen from Fig. 4 that the random offloading has inevitable fluctuations caused by the algorithm’s randomness. The above can prove that the data collected by the IoT nodes can meet the needs of human resources assessment after real-time processing.

Fig. 4. Comparison of simulation results.

Download Original Figure

To intuitively observe the clustering effect of human resources assessment data, three types of Gaussian distribution human resources assessment data are generated in two-dimensional space as the algorithm processes the dataset to verify whether the algorithm can achieve reliable clustering performance. The clustering centers of the three types of human resources assessment data are (20, 60), (40, 40), and (60, 20), respectively. Each class includes 100 data points, and the covariance matrices of the three types are $\left[\begin{array}{cc}5& 0\\ 0& 15\end{array}\right]$, $\left[\begin{array}{cc}8& 0\\ 0& 8\end{array}\right]$, and $\left[\begin{array}{cc}15& 0\\ 0& 5\end{array}\right]$, respectively.

Suppose the initial number of clustering centers is 15, the initial value of learning factor τ₀ is 0.7, the time constants t₀ and ξ are 10 and 15, the merging threshold parameter η is 0.75, and the fuzzy weighting exponent m is 3. The data points, the true cluster centers and the initial cluster centers are shown in Fig. 5.

Fig. 5. Data points and distribution of clustering center.

Download Original Figure

After several iterations, the clustering results are shown in Fig. 6. Among them, Fig. 6(a) shows that after two iterations, 10 of the 15 initial clustering centers remain; Fig. 6(b) shows that after five iterations, there are 6 cluster centers left; Fig. 6(c) shows that after nine iterations, there are 5 cluster centers left; Fig. 6(d) shows that after 12 iterations, there are 3 cluster centers left, and the number of cluster centers does not change in subsequent iterations. Therefore, a good clustering effect can distinguish dense and sparse areas in the object space, which is helpful for human resource assessment.

Fig. 6. Clustering results.

Download Original Figure

Five different groups of real cluster centers were set for the experiment. Each data group is generated the same way as in the previous test, by specifying the corresponding cluster centers and generating random data samples based on different covariance matrices. The mutual information index is used to evaluate the accuracy of clustering. The mutual information index marks the accuracy of correct classification by calculating the average mutual information size after matching the clustering results with the actual classification, defined as follows.

where n_pq is the number of data whose real class is p and is divided into clustering result q, n_p is the number of data whose clustering result is p, n_q is the number of data whose true class is q, N is the total number of data points. The NMI of different cluster centers is shown in Table 2.

Table 2 shows that the clustering algorithm has high accuracy for data clustering. The real clustering center does not affect the clustering results, so the algorithm can automatically cluster data samples in various situations.

The different initial numbers of cluster centers are selected for operation, and the change in the number of cluster centers in the iteration is shown in Fig. 7. Fig. 7 shows that although the initial number of cluster centers selected is different, the number of cluster centers is decreasing in the iteration process. After several iterations, the number of cluster centers is stable at three. Therefore, the clustering algorithm is not sensitive to the initial number of cluster centers change and can reliably converge to a reasonable number of cluster centers.

Fig. 7. Number of clustering center in operation process.

Download Original Figure