48 lines
14 KiB
TeX
48 lines
14 KiB
TeX
% Copyright (c) 2014,2016,2021 Casper Ti. Vector
|
|
% Public domain.
|
|
|
|
\begin{cabstract}
|
|
% 近些年随着卫星互联网、手机直连卫星以及第六代无线通信技术的发展，卫星通信的需求有了巨大的发展。目前，卫星多采用多波束以及频率复用等技术来应对巨大的通信需求
|
|
% 但受限于功率、频率等资源，卫星通信系统的资源分配问题变得愈发复杂。除此之外，卫星通信的业务存在着空间分布和时间分布不均匀，服务质量要求多样等特点，这要求寻找到一种有效的动态资源分配算法，提升卫星通信系统整体的资源利用率。
|
|
|
|
% 本文针对卫星通信系统中资源分配进行了研究，主要针对两个方面的问题，一个是考虑时空关联性的动态信道分配问题，另一个考虑卫星跳波束的动态规划问题。本文在研究方法主要与目前广受关注的深度强化学习技术相结合，在信道分配问题上利用神经网络发掘时空关联性，在跳波束问题上利用多智能体等技术解决跳波束问题信道与波束点亮联合分配的问题。
|
|
|
|
% 首先，本文概述了多波束卫星通信系统的基本架构、卫星的信道传输特性以及马尔科夫决策过程和强化学习等技术的基本知识。然后，建立起基于深度强化学习的信道分配问题的基本场景与用户通信模型。为了保证用户质量通信质量，设计一用户通信的阻塞率作为优化目标，设计了基于深度强化学习的动态信道分配算法。该算法以将用户的地理位置通过状态转换使用多通道的图片描述。通过基于卷积神经网络和深度Q网络的求解得到实时的信道分配策略。仿真结果表明，基于深度强化学习的信道分配算法相比其他算法，能够有效保证系统的接通率，提高频谱效率。
|
|
|
|
% 最后，针对跳波束问题，本文概述了跳波束卫星通信系统的前向链路模型，以优化通信吞吐量和时延的公平性为优化目标函数，建立起基于深度强化学习的跳波束与信道联合分配算法。该算法包含两个独立的深度Q网络，分别对波束点亮方案和信道分配方式进行优化，通过多头输出的方式解决跳波束分配问题中空间复杂度过高的问题。仿真结果表明，与传统算法相比，该方法能有效提高系统的吞吐量，并且能够保证系统的时延公平性，同时相比遗传算法等启发式算法能够有效减少系统的决策时间，能够应对动态变化的系统环境。
|
|
|
|
近年来随着卫星互联网、手机直连卫星快速发展，国际标准组织3GPP、ITU等机构已将卫星通信纳入第六代无线通信的重要组成部分，卫星通信将步入飞速发展的新阶段，星通信将会得到更深入的融合应用，卫星通信业务及用户数量等将迎来快速增长。传统频率复用、多波束技术等有效提升了卫星资源使用效率，但随着用户数量的增加和服务质量需求的多样化，业务在时间和空间上的分布不均问题变得更加突出，这对系统灵活高效地分配资源提出了更高的挑战。论文针对卫星通信中资源分配面临的新问题，结合目前广受关注的深度强化学习技术，在信道分配问题上利用神经网络发掘时空关联性，在跳波束问题上利用多智能体等技术解决跳波束问题中信道与波束点亮联合分配的问题。
|
|
|
|
本文首先介绍了多波束卫星通信系统的基本架构、卫星的信道传输特性以及马尔科夫决策过程和强化学习等技术，建立了基于深度强化学习的信道分配问题的基本场景与用户通信模型。为了保证用户的通信质量，以用户通信的阻塞率作为优化目标，设计了基于深度强化学习的动态信道分配算法。该算法通过状态转换将用户的地理位置描述为多通道图像，并基于卷积神经网络和深度Q网络进行求解，从而获得实时的信道分配策略。仿真结果表明，基于深度强化学习的信道分配算法相比其他算法，能够有效保证系统的接通率，提高频谱效率。
|
|
|
|
进一步针对跳波束应用下的信道分配，以优化通信吞吐量和时延的公平性为优化目标函数，建立起基于深度强化学习的跳波束与信道联合分配算法。该算法包含两个独立的深度Q网络，分别对波束点亮方案和信道分配方式进行优化，采用多头输出的方法解决了跳波束分配问题中空间复杂度过高的问题。仿真结果表明，与传统算法相比，该方法能有效提高系统的吞吐量，并且能够保证系统的时延公平性，同时相比遗传算法等启发式算法能够有效减少系统的决策时间，能够应对动态变化的通信环境。
|
|
|
|
\end{cabstract}
|
|
|
|
\ifblind\begin{beabstract}\else\begin{eabstract}\fi
|
|
|
|
% In recent years, with the development of satellite internet, direct satellite connections for mobile phones, and sixth-generation wireless communication technologies, the demand for satellite communication has experienced significant growth. Currently, satellites often employ multiple techniques such as multi-beam and frequency reuse to meet the vast communication needs.
|
|
|
|
% However, limited by resources such as power and frequency, the resource allocation problem in satellite communication systems has become increasingly complex. In addition, the services in satellite communication have characteristics like uneven spatial and temporal distribution and diverse quality of service requirements. This demands an effective dynamic resource allocation algorithm to enhance the overall resource utilization of satellite communication systems.
|
|
|
|
% This paper focuses on the resource allocation in satellite communication systems, mainly addressing two issues: one is the dynamic channel allocation problem considering spatiotemporal correlations, and the other is the dynamic planning problem for satellite beam hopping. The study primarily combines the currently popular deep reinforcement learning techniques, using neural networks to explore spatiotemporal correlations in channel allocation and multi-agent technologies to solve the joint allocation problem of channel and beam lighting in the beam hopping issue.
|
|
|
|
% First, this paper outlines the basic architecture of multi-beam satellite communication systems, the channel transmission characteristics of satellites, and the fundamental knowledge of Markov decision processes and reinforcement learning. Then, it establishes the basic scenario and user communication model for the channel allocation problem based on deep reinforcement learning. To ensure the quality of user communication, the user communication blocking rate is designed as the optimization objective, and a dynamic channel allocation algorithm based on deep reinforcement learning is developed. This algorithm describes the user's geographic location through state transitions using multi-channel images. The real-time channel allocation strategy is obtained through solutions based on convolutional neural networks and deep Q networks. Simulation results show that the channel allocation algorithm based on deep reinforcement learning can effectively ensure the system's connection rate and improve spectral efficiency compared to other algorithms.
|
|
|
|
% Finally, for the beam hopping issue, this paper outlines the forward link model of the beam hopping satellite communication system and establishes the joint allocation algorithm for beam hopping and channels based on deep reinforcement learning, with the optimization of communication throughput and fairness of delay as the objective function. This algorithm contains two independent deep Q networks, optimizing the beam lighting scheme and channel allocation method, respectively, and solves the problem of high spatial complexity in beam hopping allocation using multi-head output. Simulation results indicate that compared to traditional algorithms, this method can effectively improve the system's throughput and ensure the fairness of system delay. Compared to heuristic algorithms like genetic algorithms, it can significantly reduce the system's decision time, enabling it to cope with dynamically changing environments.
|
|
|
|
% In recent years, with the rapid development of satellite internet and direct satellite connections via mobile phones, international standard organizations such as 3GPP and ITU have incorporated satellite communications into the sixth generation of wireless communications as a key component. Satellite communications are entering a phase of rapid development and will see deeper integration and application, leading to rapid growth in satellite communication services and users. Traditional frequency reuse and multi-beam technologies have effectively improved the utilization efficiency of satellite resources. However, as the number of users increases and service quality demands become more diverse, with prominent temporal and spatial distribution imbalances, the need for flexible and efficient resource allocation systems has become more challenging. This paper addresses the new issues faced in resource allocation in satellite communications, utilizing the widely discussed deep reinforcement learning technology. Neural networks are used to explore spatiotemporal correlations in channel allocation, and multi-agent techniques are employed to address joint channel and beam allocation in the beam-hopping problem.
|
|
|
|
% Firstly, the paper introduces the basic architecture of multi-beam satellite communication systems, the channel transmission characteristics of satellites, as well as Markov decision processes and reinforcement learning techniques. It establishes the basic scenarios and user communication models for the channel allocation problem based on deep reinforcement learning. To ensure the quality of user communication, the blocking rate of user communication is used as the optimization target, and a dynamic channel allocation algorithm based on deep reinforcement learning is designed. This algorithm uses the geographic location of users described through state transitions using multi-channel images, and real-time channel allocation strategies are obtained through convolutional neural networks and deep Q-networks. Simulation results show that the channel allocation algorithm based on deep reinforcement learning can effectively ensure system connectivity and improve spectral efficiency compared to other algorithms.
|
|
|
|
% Furthermore, for channel allocation in beam-hopping applications, the optimization of communication throughput and delay fairness is used as the objective function. A joint allocation algorithm of beam-hopping and channels based on deep reinforcement learning is established. This algorithm includes two independent deep Q-networks that optimize the beam-lighting scheme and channel allocation method, respectively. It employs a multi-head output method to address the high spatial complexity in beam-hopping allocation. Simulation results indicate that compared to traditional algorithms, this method can effectively increase system throughput, ensure delay fairness, and significantly reduce decision-making time compared to heuristic algorithms like genetic algorithms, making it capable of handling dynamically changing communication environments.
|
|
|
|
In recent years, with the rapid development of satellite internet and direct-to-mobile satellite communications, international standard organizations such as 3GPP and ITU have incorporated satellite communications as an essential component of the sixth generation of wireless communications. Satellite communications are entering a new phase of rapid development. There will be a deeper integration and application of satellite communications, and the satellite communications industry, along with the number of users, will experience rapid growth. Traditional frequency reuse and multi-beam technologies have effectively improved the efficiency of satellite resource utilization. However, with the increase in the number of users and the diversification of service quality demands, the uneven distribution of services in time and space has become more prominent. This poses higher challenges for the system to allocate resources flexibly and efficiently. This paper addresses the new challenges in resource allocation in satellite communications, combining the widely focused deep reinforcement learning technology. It uses neural networks to explore spatio-temporal correlations in channel allocation problems and employs multi-agent technologies to solve the joint allocation of channels and beam illumination in hop-beam problems.
|
|
|
|
This paper first introduces the basic architecture of multi-beam satellite communication systems, the channel transmission characteristics of satellites, and technologies such as the Markov decision process and reinforcement learning. It establishes the basic scenario and user communication model for the channel allocation problem based on deep reinforcement learning. To ensure the quality of user communications, the user communication blockage rate is used as the optimization target. A dynamic channel allocation algorithm based on deep reinforcement learning is designed. This algorithm uses state transitions to describe the user's geographic location as a multi-channel image and solves it based on convolutional neural networks and deep Q-networks to obtain real-time channel allocation strategies. Simulation results show that the channel allocation algorithm based on deep reinforcement learning can effectively ensure the system's connectivity rate and improve spectral efficiency compared to other algorithms.
|
|
|
|
Further, for channel allocation under hop-beam applications, with the optimization of communication throughput and the fairness of delay as the optimization objective function, a hop-beam and channel joint allocation algorithm based on deep reinforcement learning is established. This algorithm contains two independent deep Q-networks, which optimize the beam illumination scheme and channel allocation method, respectively. It uses a multi-head output method to solve the problem of high spatial complexity in hop-beam allocation. Simulation results show that compared to traditional algorithms, this method can effectively improve the system's throughput and ensure the system's delay fairness. Compared to heuristic algorithms like genetic algorithms, it can significantly reduce the system's decision-making time and cope with dynamically changing communication environments.
|
|
\ifblind\end{beabstract}\else\end{eabstract}\fi
|
|
|
|
% vim:ts=4:sw=4
|