A Deep Reinforcement Learning Approach for Efficient Image Processing Task Offloading in Edge-Cloud Collaborative Environments

In the current digital era, remarkable developments in image-based applications have been observed. As the proliferation of smartphones, intelligent devices, and IoT gadgets has been documented, a heightened intricacy in real-time image processing has emerged [1, 2]. Often, traditional cloud computing architectures were identified as insufficient to address the evolving needs of these complex image processing tasks, faced with challenges such as increased latency and limited computational capacity.

The impetus for the present research was derived from the pressing need to facilitate real-time image processing across various sectors, spanning healthcare, entertainment, autonomous vehicles, and industrial automation. A mounting urgency has been discerned for the formulation of an adept system proficient in offloading extensive image processing tasks for myriad users spread across diverse geographic terrains.

1.1 Problem definition and contextualization

At the heart of the study lies the investigation of task offloading within the framework of edge-cloud collaboration. Interactions between edge devices, the central cloud data center, and neighboring edge devices were examined, focusing on enhancing image processing efficiency. The intricacy of this paradigm can be exemplified in a multi-user environment, characterized by concurrent image processing tasks (e.g., multiple colored users in various regions as illustrated in Figure 1). If tasks were processed locally, notable delays could be experienced, attributable to the limited computational capacities of individual devices. Although offloading to an edge node or cloud data center might reduce processing time, subsequent challenges, including transmission delay and queuing time, could be encountered.

1.png

Figure 1. A motivation example

1.2 Proposed framework: I-CTOS-DRL

In response to these challenges, the I-CTOS-DRL framework has been proposed. The salient contributions of this study include:

· A keen understanding of image processing requirements, with the offloading problem articulated to optimize total costs, encompassing delays and energy expenditure. Collaboration among edge servers was underscored to facilitate efficient real-time image processing.

· The introduction of the I-CTOS-DRL framework, an innovative method utilizing DRL and a heuristic algorithm to reduce inherent complexities. An innovative task updating mechanism for image processing was incorporated, promoting fluid collaboration among edge servers.

· A tailored simulator was devised to rigorously appraise the I-CTOS-DRL methodology. Evaluations corroborated its prowess in managing real-time image processing tasks across distinct sectors.

1.3 Applications and future directions

The I-CTOS-DRL exhibits considerable potential for applications in domains like augmented/virtual reality, digital medical intervention, autonomous driving, and advanced industrial control. A shift in the understanding and application of image processing within the realm of contemporary digital technology is anticipated.

This study augments the existing body of literature by offering a robust solution to the challenges posed by real-time image processing via edge computing. Fresh perspectives in the image processing domain have been unearthed, paving the way for future innovations.

1.4 Conclusion

This research illuminates a significant stride in image processing via edge computing, thereby enhancing the technological panorama. By delineating and addressing the intrinsic challenges of real-time image processing, a prospective era where digital interactions become more intuitive, prompt, and significant is envisioned.

3.1 The edge-cloud system architecture

In this research, an overarching topology of the edge-cloud framework, with an emphasis on image processing tasks, is delineated by the notation G={C, M, U}.

· Cloud Data Center (C): Situated distally from the user group and acting as the pinnacle of the system's hierarchy, the cloud data center, designated by C, is recognized as the central hub for expansive image processing undertakings.

· Edge Nodes and Base Stations: It is observed that the user cluster interacts with edge nodes via wireless channels and base stations. These edge layer entities, with their heterogeneous and autonomous stances, span across diverse regions. The notational representation, $\boldsymbol{M}=\left\{m_k\right\}$, has been adopted to capture the spread of edge nodes facilitated by various operators. Every individual edge node, given by $m_k$, is noted to be connected to its respective base station with a predetermined computing potential reserved for image data tasks. The computing capacity of each $m_k$ is subsequently denoted by $f_k$.

· Adjacent Edge Servers: These are depicted by $\boldsymbol{A}\left(m_k\right)$. It is inferred that the selection of neighboring edge servers for cooperation can substantially enhance the effectiveness of image processing and dissemination.

· User Layer: Within this structure, the set of users interfacing with the edge node $m_k$ is represented as $\boldsymbol{U}^{m_k}=\left\{u_i^k\right\}$. Users are found to establish connections with the closest edge node, which in turn augments the system's capability to swiftly address image processing demands.

· End Devices: Such devices, embodying users within this schema, range from computers and smartphones to smart wristbands. A continuous stream of atomic tasks is generated by these entities. These tasks, ranging from image recognition to filtering and enhancement, are intrinsic to image processing. The computational capacity of $u_i^k$, enveloping the essentialities of image processing, is indicated by $l_i^k$.

For a comprehensive understanding, Table 1 has been curated to enlist pivotal notations, with a special focus on those crafted to elucidate the specifics of image processing tasks.

Table 1. Symbol definition

3.2 Transmission model for image processing tasks

In this segment, a specialized transmission model has been formulated, concentrating predominantly on the intricacies of managing image processing tasks. Emphasis is placed on the offloading of tasks by users to edge servers. The pronounced need for extensive bandwidth and dependable transmission arises from the nature of high-resolution image data. Under this framework, the collaborative essence of edge nodes is highlighted, ascertaining proficient offloading to either a directly connected edge node or its collaborative counterpart.

Based on the Rayleigh fading channel model [20], the transmission rate $r_{u_i^k, m_k}$ between user $u_i^k$ and the associated edge node $m_k$ is articulated as:

$r_{u_i^k, m_k}=b_{i, k} \cdot \log _2\left(1+\frac{\gamma_{i, k} h_{i, k}}{p\left(u_i^k, m_k\right)^{\omega_i} {N_i}}\right)$                    (1)

where, $b_{i, k}$ and $h_{i, k}$ are established to signify the channel gain and transmission bandwidth between user $u_i^k$ and $m_k$, respectively. Given the data-heavy constitution of images, these parameters become central to the process. Concurrently, $p\left(u_i^k, m_k\right)$ defines the spatial relation between $u_i^k$ to $m_k$, while $\gamma_{i, k}$ represents the transmission power. Parameters like Gaussian noise and path loss exponent, represented by $N_i$ and $\omega_i$ respectively, are identified as cardinal for maintaining transmitted image data's authenticity.

In parallel, the transmission rate $r_{m_k, m_h}$ between edge nodes $m_k$ and $m_h$ is framed as:

$r_{m_k, m_h}=B_{k, h} \cdot \log _2\left(1+\frac{\Upsilon_{k, h} H_{k, h}}{p\left(m_k, m_h\right)^{\omega_i} {N_i}}\right)$              (2)

The above Eq. (2) elucidates the channel gain $B_{k, h}$ and transmission bandwidth $H_{k, h}$ between the said edge nodes. By facilitating collaborations in image processing, an uptick in efficiency and a dip in latency are witnessed. The spatial dynamics between $m_k$ and $m_h$ are mapped by $p\left(m_k, m_h\right)$, and $\Upsilon_{k, h}$ documents the transmission power. The Gaussian noise and path loss exponent for the edge node $m_k$ are symbolized by $N_k$ and $\omega_k$, respectively.

In the presented model, meticulous considerations are made to cater to the unique transmission requirements of images. Factors such as bandwidth, channel gain, and inter-node collaboration are seamlessly integrated to form a comprehensive framework. This model, thus, offers insights tailored to fortify reliable image processing tasks, cognizant of the nuanced challenges that high-fidelity image data transmission incurs.

3.3 Execution model for image processing

The concept of task offloading, particularly for image processing tasks, is examined within a three-tier edge-cloud framework. Image processing tasks possess unique attributes like high computational requirements and significant data volume, necessitating a distinct approach when considering offloading. Potential offloading locations for these tasks could encompass the local device, the directly connected edge node, collaboratively connected edge nodes, or the cloud. Various scenarios have been identified and the execution models are outlined as follows:

(1) Execution on Local Device

The offloading issue is initially envisioned for multiple users choosing to conduct image processing tasks locally. These tasks could encompass image enhancement, filtering, or object detection, all of which demand considerable computational resources. The execution delay for a user, denoted by $D_l\left(u_i^k\right)$, is formulated as follows:

$D_l\left(u_i^k\right)=\frac{w_i^k}{l_i^k}$            (3)

In this context, $w_i^k$ represents the workload, and $l_i^k$ signifies the CPU frequency of the user $u_i^k$. The energy consumption for task $\chi_i^k$ of $u_i^k$ is then defined:

$E_l\left(u_i^k\right)=\kappa_i^k \cdot w_i^k \cdot\left(l_i^k\right)^2$                 (4)

where, $\kappa_i^k$ is the coefficient factor of chip architecture for user $u_i^k$, reflecting the specific requirements of image-related computations.

(2) Execution on Connected Edge Node

Subsequently, the scenario is studied wherein the image processing tasks are executed on the directly connected edge node. This strategy proves to be advantageous for real-time image analytics. The transmission delay and the execution delay between $u_i^k$ and $m_k$ are respectively defined as follows:

$D_m^t\left(u_i^k\right)=\frac{d_i^k}{r_{u_i^k, m_k}}$            (5)

where, $d_i^k$ is the data size of $u_i^k$, reflecting significant volume of image data. The execution delay on edge node $m_k$ is

$D_m^e\left(u_i^k\right)=\frac{w_i^k}{f_k}$           (6)

Thus, the total delay for image processing on the connected edge node is expressed as:

$D_m\left(u_i^k\right)=D_m^t\left(u_i^k\right)+D_m^e\left(u_i^k\right)$              (7)

And the energy consumption is given by:

$E_m\left(u_i^k\right)=\gamma_{i, k} \cdot D_m^t\left(u_i^k\right)$              (8)

where, $\gamma_{i, k}$ is the transmission power from user  $u_i^k$ to $m_k$.

(3) Execution on Collaborative Edge Node

For more intricate image processing tasks, an analysis is conducted where execution on the collaborative edge node is contemplated. The transmission delay in this case is composed of two parts:

$D_z^t\left(u_i^k\right)=\frac{d_i^k}{r_{u_i^k, m_k}}+\frac{d_i^k}{r_{m_k, m_h}}$                (9)

where, $\frac{d_i^k}{r_{u_i^k, m_k}}$ is the transmission delay from user $u_i^k$ to edge node $m_k . \frac{d_i^k}{r_{m_k, m_h}}$ is the transmission delay from edge nodes $m_k$ to $m_h$.

Since the task produced by user $u_i^k$ processes on edge node $m_h$, the execution delay $D_z^e\left(u_i^k\right)$ is defined as:

$D_z^e\left(u_i^k\right)=\frac{w_i^k}{f_h}$                (10)

where, $f_h$ is the computing capacity of $m_h$. So the total delay for executing image tasks on the collaborative edge node is:

$D_z\left(u_i^k\right)=D_z^t\left(u_i^k\right)+D_z^e\left(u_i^k\right)$              (11)

which is the sum of the transmission delay and the execution delay. Similar to the scenario of execution on the connected edge node, the energy consumption of execution on the collaborative edge node is defined as:

$E_z\left(u_i^k\right)=\gamma_{i, k} \cdot \frac{d_i^k}{r_{u_i^k, m_k}}+\Upsilon_{k, h} \cdot \frac{d_i^k}{r_{m_k, m_h}}$                (12)

where, $\Upsilon_{k, h}$ is the transmission power from $m_k$ to $m_h$.

(4) Execution on Cloud

Lastly, the prospect of executing image processing tasks on the cloud is scrutinized, particularly for tasks that necessitate significant computational resources. The transmission delay and energy consumption in this case are defined as follows:

$D_c^t\left(u_i^k\right)=\frac{d_i^k}{r_{u_i^k, \mathrm{C}}}$                (13)

where, $r_{u_i^k, C}$ is the transmission rate between user $u_i^k$ and the cloud C. Based on that, the energy consumption of execution on the cloud is defined as:

$E_c\left(u_i^k\right)=\gamma_{i, k} \cdot \frac{d_i^k}{r_{u_i^k, \mathbf{C}}}$                 (14)

This section has conducted a thorough exploration of diverse execution models within an edge-cloud framework, with a specific focus on image processing tasks. Explicit expressions for delays and energy consumption across different scenarios have been provided, incorporating the unique attributes of image processing into a cohesive framework. This investigation aids in understanding the specific requirements and potential advantages of different offloading strategies, ensuring the efficient use of computational resources and effective management of the unique aspects of image data. The models proposed here can further guide the design and optimization of edge-cloud architectures for real-time image analytics.

This section introduces a DRL-based Collaborative Task Offloading and Image Processing Framework (CTOS-DRL-IP). This framework extends the CTOS-DRL, with a focused objective of optimizing image processing in edge network environments. The key goal of CTOS-DRL-IP is the efficient management of image processing tasks by crafting a practical collaborative set during the decision-making process. This is accomplished using deep Q-learning. The subsequent sections provide an in-depth discussion of the components and functions of the CTOS-DRL-IP.

5.1 DRL formulation for image processing

In this subsection, the CTOS-DRL-IP framework, inspired by deep Q-learning and specifically tailored for image processing applications, is introduced. This includes the representation of the total cost and the agent's knowledge of image processing applications within the state space to accurately model the edge network environment.

Definition 1 (State):

The state, denoted as $s_t$, is defined as a vector consisting of $s_t=\left[T_i^k, U_i / \widehat{U}_i, I_i^p\right]_t$. Here $\boldsymbol{U}_i /\left|\boldsymbol{\widehat{U}}_i\right|$ represents the task generated by users that requires processing. $\boldsymbol{T}_i^k=\sum_{k=1}^{\left|\hat{\boldsymbol{U}}_i\right|} T_i^k$ signifies the cumulative cost of the scheduled completed tasks $\widehat{\boldsymbol{U}}_i$, while the last part of the vector symbolizes the attributes of the image processing task, such as resolution, type, and processing requirements.

Definition 2 (Action):

The action space $a_t$, embodying the adjusting action, is represented by the vector $\left[\zeta_i^l, \zeta_i^c, \zeta_i\right]_t$. Within this, $\zeta_i^l$ or $\zeta_i^c$ indicates whether the destination location of the adjustment resides on a local device or in the cloud. The vector $\zeta_i$, denoting the set of feasible edge nodes $\mathbb{M}_i$ is characterized by $\zeta_i=\left(\left.\zeta_i^m(k)\right|_{m_k \in \mathbb{M}_i}\right)$. Additionally, depicts the processing action specific to the image task, such as filtering, sharpening, or scaling.

Definition 3 (Reward):

The immediate reward is expressed by $R\left(s_t, a_t\right)=\left(\bar{E}_i-E\right) / \bar{E}-\alpha \cdot Q_i^p$, where $\overline{\boldsymbol{E}}_i$ is the aggregate cost of multiple tasks for users, $\bar{\boldsymbol{E}}$ is a fundamental cost that is being selectively offloaded, and $\alpha \cdot Q_i^p$ is a quality factor pertinent to image processing. This factor ensures a balance between cost efficiency and quality of image processing.

In this context, the agent is prepared by selecting a destination for various users and defining particular image processing tasks. Training the agent, as a result, is anticipated to facilitate the simultaneous accomplishment of offloading and image processing tasks. The action $a_t$ is designed for each time slot t, and the agent is rewarded $R\left(s_t, a_t\right)$ in a given state $s_t$. Efforts are made to minimize the total cost, comprised of energy consumption and delay, without undermining the quality of image processing. This aligns with the reinforcement learning objective of maximizing long-term reward.

In conclusion, the CTOS-DRL-IP framework presents an innovative method for collaborative image processing and task offloading in edge networks, utilizing DRL. Through the provision of tailored state, action, and reward definitions, the framework provides a robust solution for managing complex image processing tasks. This results in the achievement of optimal energy efficiency and a reduction in delay without a compromise in image quality. The insights and findings from this study extend the current understanding of the field, opening new avenues for exploration and application in real-world scenarios.

5.2 Feasible collaborative set construction with image processing consideration (FCSC-IP)

The goal of the agent operating within the CTOS-DRL-IP framework is to achieve effective task offloading for multiple users, aiming to minimize the overall cost, with special attention given to image processing requirements. The nature of these decisions depends on the observation of the environment and the overall architecture, which, given the constraint of edge server capacities, influences the number of tasks that can be offloaded to individual regions. When an edge node reaches its capacity, collaboration with neighboring nodes becomes necessary, particularly when managing complex image processing tasks that require stringent adherence to processing, quality, and time parameters. The complexity and high dimensionality of this collaborative set construction process are addressed through the introduction of the FCSC-IP method.

5.2.1 Image processing integration into collaborative decisions

In an edge network comprising seven interconnected edge nodes, where each node has the ability to choose collaboration partners for task completion, decisions are not made solely based on proximity and availability. A novel method, FCSC-IP, is proposed, taking into account specific image processing capabilities such as resolution handling, filtering, and transformation. Users serving at edge nodes $m_1$ that connect with $m_2$ to $m_5$ will have collaboration options, with consideration given to the complexity introduced by the adjacency matrix.

Definition 4 (Collaborative Influence Factor with Image Processing Consideration):

The collaborative influence factor with image processing consideration, denoted as $\varphi\left(m_k\right)$, is formulated to assess the collaborative capacity of edge node $m_k \in \mathbf{A}\left(m_k\right)$ for user $m_k$, reflecting the image processing requirements:

$\phi\left(m_k\right)=\frac{f_h}{\left(p\left(m_k, m_h\right) \cdot\left|U^{\left(m_k\right)}\right|\right)} \times \beta\left(I_i^p\right)$

where, $\beta\left(I_i^p\right)$ serves as a weight parameter, factoring in image processing attributes such as type, resolution, and complexity.

Definition 5 (Feasible Collaborative Set with Image Processing Consideration):

The feasible collaborative set with image processing consideration, denoted as $\mathbb{Z}\left(m_k\right)=\left\{m_h\right\}$, is derived from the edge nodes in set $\boldsymbol{A}\left(m_k\right)$, with the collaborative influence factor under the boundary parameter $\Gamma_{m_k}$, such that $\varphi\left(m_h\right) \leq \Gamma_{m_k}$.

5.2.2 Algorithm for FCSC-IP

The procedure for feasible collaborative set construction is outlined in Algorithm 1, with explicit attention to the requirements of image processing. The FCSC-IP algorithm focuses on image processing needs, aiming to reduce complexity while improving efficiency in the formation of a feasible collaborative set. The time complexity of FCSC-IP is $O\left(|\boldsymbol{M}| \cdot\left|\boldsymbol{U}^h\right|\right)$, taking into account the attributes of image processing.

In conclusion, the FCSC-IP presents an innovative method that enables edge nodes to participate in collaborative decisions in task offloading, specifically reflecting the complexity inherent in image processing tasks. By integrating image processing prerequisites into both collaborative influence factors and feasible set construction, FCSC-IP offers a balanced and efficient solution, harmonizing cost, efficiency, and quality in managing complex image processing tasks within edge networks. This method not only enhances existing offloading strategies but also provides a nuanced understanding of the interaction between task distribution and image processing, making a significant contribution to the field.

5.3 CTOS-DRL for image processing tasks

In this subsection, a task offloading algorithm specifically designed to meet the demands of image processing tasks is introduced, grounded in DRL. The fundamental principle of CTOS-DRL, summarized in Algorithm 2, involves using a DRL agent to facilitate dynamic offloading of image processing tasks from multiple users, with the goal of minimizing the total cost.

The set of image processing tasks, denoted by χ, comes from various users in set U and forms the input. The output, denoted as the offloading strategy X, might include numerous image manipulations such as filtering, object detection, segmentation, and enhancements that require substantial computational resources.

The approach is summarized as follows:

(1) Initialization: Basic settings are initialized (Line 1), including the capacity N of replay memory Θ, the target action-value function $\hat{Q}$ with weights $\bar{\theta}:=\theta$, and the random weights θ of action-value function Q.

(2) Environment Familiarization: Over a series of kappa episodes, the agent becomes familiar with the environment, tailored to the complex nature of image processing tasks (Lines 2 to 19).

(3) Sequence Initialization: The sequence for multiple users is initialized (Line 3), considering the unique characteristics inherent to image processing tasks.

(4) Collaborative Set Construction: A feasible collaborative set Z is constructed (Line 4) based on Algorithm 1, with specific attention to the constraints and requirements of image processing.

(5) Training Process: Spanning lines 5 to 18, the agent's decision-making process encompasses action selection (lines 6-7), state and action setting (line 8), storing transitions in replay memory (line 9), and applying gradient descent to minimize total cost (lines 14 to 16).

(6) Final Offloading Strategy: Representing the optimized solution for handling image processing tasks, the offloading strategy X is returned (Line 20).

(7) Time Complexity: The time complexity of CTOS-DRL is denoted by $O\left(\kappa \cdot|\boldsymbol{M}| \cdot\left|\boldsymbol{U}^h\right| \cdot\left|\tilde{\boldsymbol{U}}_i\right|\right)$, where κ represents the number of episodes, and U ˜_i refers to the scale of sub-tasks pending scheduling on edge server mi, expressed as $\left|\tilde{\boldsymbol{U}}_i\right|=\left|\boldsymbol{U}_i / \hat{\boldsymbol{U}}_i\right|$.

The CTOS-DRL algorithm provides a robust and efficient method for managing the offloading of image processing tasks. By recognizing and accommodating the specific requirements and complexities associated with image processing, it offers a flexible and optimized solution that can adapt to the dynamic nature of contemporary applications involving image manipulation.