环境

  • Ubuntu 18.04

因为Ubuntu中包含了很多软件包,所以更新量较大,18.04 amd64的容量大概120G;其过程比centos还简单;

apt-mirror

Ubuntu、Debian使用一个叫apt-mirror的工具进行同步,全程自动化,中断续传;然后使用一个web服务器就可以对外发布了

开始安装

sudo apt install apt-mirror -y

编辑配置文件

sudo vim /etc/apt/mirror.list

############# config ##################
#
# 下载的路径
set base_path    /var/spool/apt-mirror
#
# 镜像路径(非ISO镜像)
# set mirror_path  $base_path/mirror
# 临时索引目录
# set skel_path    $base_path/skel
# 日志目录
# set var_path     $base_path/var
# 清理脚本
# set cleanscript $var_path/clean.sh
# 系统架构
set defaultarch amd64
# set postmirror_script $var_path/postmirror.sh
# set run_postmirror 0
# 最大线程数
set nthreads     20
set _tilde 0
#
############# end config ##############
# 如果想下载多个版本,则都写在这,但是每个版本的mirror总量都超过100G,注意磁盘容量

# 16.04 Xenial Xerus
#deb http://mirrors.aliyun.com/ubuntu/ xenial main
#deb http://mirrors.aliyun.com/ubuntu/ xenial-updates main
#deb http://mirrors.aliyun.com/ubuntu/ xenial universe
#deb http://mirrors.aliyun.com/ubuntu/ xenial-updates universe
#deb http://mirrors.aliyun.com/ubuntu/ xenial-security main
#deb http://mirrors.aliyun.com/ubuntu/ xenial-security universe
#
# #deb-src http://mirrors.aliyun.com/ubuntu/ xenial main
# #deb-src http://mirrors.aliyun.com/ubuntu/ xenial-updates main
# #deb-src http://mirrors.aliyun.com/ubuntu/ xenial main
# #deb-src http://mirrors.aliyun.com/ubuntu/ xenial universe
# #deb-src http://mirrors.aliyun.com/ubuntu/ xenial-updates universe
# #deb-src http://mirrors.aliyun.com/ubuntu/ xenial-security main
# #deb-src http://mirrors.aliyun.com/ubuntu/ xenial-security universe
#
# # 18.04 Bionic Beaver
deb http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse
#
# #deb-src http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
# #deb-src http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
# #deb-src http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
# #deb-src http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
# #deb-src http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse

clean http://mirrors.aliyun.com/ubuntu

如果想下载32位和64位的mirror,则需要变更一下

# 注释掉
#set defaultarch amd64
# 然后在每个更新地址前面添加系统架构
# 18.04 32位架构
deb-i386 http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb-i386 http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb-i386 http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb-i386 http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb-i386 http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse
# 18.04 64位架构
deb-amd64 http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb-amd64 http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb-amd64 http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb-amd64 http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb-amd64 http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse

对外发布

使用Nginx或者Apache等web服务器,把目录指向下载的路径,如Nginx的配置

server {
    listen 80 default_server;
    listen [::]:80 default_server;
    root /var/spool/apt-mirror/mirror/mirrors.aliyun.com/ubuntu/;
    autoindex on;
    autoindex_exact_size off;
    autoindex_localtime on;
    server_name _;
    location = /favicon.ico {
        log_not_found off;
        access_log off;
    }
}

没有设置index,可能需要root用户来运行Nginx,不然会提示403;

apt-mirror适合单独一台服务器提供资源使用,还有一个工具通过缓存的方法来实现本地镜像(mirror);

客户端设置

deb http://192.168.0.111/ bionic main restricted universe multiverse
deb http://192.168.0.111/ bionic-security main restricted universe multiverse
deb http://192.168.0.111/ bionic-updates main restricted universe multiverse
deb http://192.168.0.111/ bionic-proposed main restricted universe multiverse
deb http://192.168.0.111/ bionic-backports main restricted universe multiverse

添加定时任务

每天晚上九点执行

sudo crontab -e

0 21 * * * apt-mirror

# 重启定时任务
sudo systemctl restart cron

apt-cacher

apt-cacher不同于apt-mirror的地方是,它不会完整的同步全部仓库内容,只是简单的缓存客户端请求的包;就像本身名字一样,cacher,就像CDN或者varnish的缓存一样,处于客户端和服务端之间的角色;

每次有客户端需要安装软件包还是更新包的时候,它都会从外网仓库里面获取这个包,自己存一份,发给客户端一份,这样再有其他客户端有同样请求的时候,cacher直接给出结果;

这种方法即节省带宽又节省空间。

开始安装

sudo apt install apt-cacher -y

图1

编辑配置文件;除非需要改变值,否则默认即可

sudo vim /etc/apt-cacher/apt-cacher.conf

#################################################################################
# This is the config file for apt-cacher. On most Debian systems you can safely #
# leave the defaults alone.                                                     #
#                                                                               #
# Commented defaults or examples are given. They can be changed here, or        #
# overridden using a fragment placed in ./conf.d/                               #
#################################################################################

### GENERAL ###

# The location of the local cache/working directory. This can become quite
# large, so make sure it is somewhere with plenty of space.
# 缓存路径
cache_dir = /var/cache/apt-cacher

# The directory to use for apt-cacher access and error logs.
# The access log records every request in the format:
#
#  date-time|PID|client IP address|HIT/HEAD/MISS/EXPIRED/NOTMOD|object size|object name
#
# The error log is slightly more free-form, and is also used for debug messages
# if debug mode is turned on.
# 日志路径
log_dir = /var/log/apt-cacher

# The email address of the administrator is displayed in the info page and
# traffic reports.
# 显示在信息页面和流量报告中
admin_email = root@localhost

# Daemon port setting, only useful in stand-alone mode. You need to run the
# daemon as root to use privileged ports (<1024).
#
# For standalone daemon auto startup settings please edit the file
# /etc/default/apt-cacher.
# 进程端口号
daemon_port = 3142

# Optional settings, user and group to run the daemon as. Make sure they have
# sufficient permissions within the cache and log directories. Comment the
# settings to run apt-cacher as the invoking user.
# 和web服务器的运行用户一致
group = www-data
user = www-data

# optional setting, binds the listening daemon to specified IP(s).
# 访问地址
daemon_addr = localhost

# Apt-cacher can be used in offline mode which just uses files already cached,
# but doesn't make any new outgoing connections by setting this to 1.
# 离线模式,只使用本地已有的,不会再去请求新的
#offline_mode = 1

# To enable data checksumming, install libberkeleydb-perl and set this option to
# 1. Then wait until the Packages/Sources files have been refreshed once (and so
# the database has been built up). You can also delete them from the cache to
# trigger the database update.
# 启用校验和,需要安装一个软件包,也会加大系统负载
#checksum = 1

# Importing checksums from new index files into the checksum database can cause
# high CPU usage on slower systems. This option sets a limit to the number of
# index files that are imported simultaneously, thereby limiting CPU load
# average, but, possibly, taking longer. Set to 0 for no limit.
# 将新的索引校验和导入到数据库的时候,会占用大量CPU资源,使用参数来限制,但是较低的数值会带来更长的时间,
#concurrent_import_limit = 1

# CGI mode is deprecated.
#
# Send a 410 (Gone) HTTP message with the specified text when accessed via
# CGI. Useful to tell users to adapt their sources.list files when the
# apt-cacher server is being relocated (via apt-get's error messages while
# running "update")
#
#cgi_advise_to_use = Please use http://cacheserver:3142/ as apt-cacher access URL
#cgi_advise_to_use = Server relocated. To change sources.list, run 
# perl -pe "s,/apt-cacher??,:3142," -i /etc/apt/sources.list
#
# To further facilitate migration from CGI to daemon mode this setting will
# automatically redirect incoming CGI requests to the specified daemon URL.
#
#cgi_redirect = http://localhost:3142/

### UPSTREAM PROXY ###

# Apt-cacher can pass all its requests to an external HTTP proxy like Squid,
# which could be very useful if you are using an ISP that blocks port 80 and
# requires all web traffic to go through its proxy. The format is
# 'http://[user[:password]@]hostname:port', eg: 'http://proxy.example.com:8080'.
# 使用反代
#http_proxy = proxy.example.com:8080

# This sets the interface to use for the upstream connection.
# Specify an interface name, an IP address or a host name.
# If unset, the default route is used.
# 指定网卡
#interface = eth0

# Rate limiting sets the maximum bandwidth in bytes per second to use for
# fetching packages.  Use 0 value for no rate limiting.
# 限制最大带宽
#limit = 0

### ACCESS and SECURITY ###

# Server mapping - this allows mapping virtual paths that appear in the access
# URL to real server names. The syntax is the part of the beginning of the URL
# to replace (the key), followed by a list of mirror URLs, all space
# separated. Multiple mappings are separated by semicolons or commas, as
# usual. Note that you need to specify all keys (or use the '%PATH_MAP%'
# shorthand) in the allowed_locations option, if you make use of it. Also note
# that the paths should not overlap each other.
#
# The keys are also used to separate the caching of multiple distributions
# within a single apt-cacher instance if distinct_namespaces is also set.
#
#path_map = debian ftp.uni-kl.de/pub/linux/debian ftp2.de.debian.org/debian ; 
# ubuntu archive.ubuntu.com/ubuntu ; 
# security security.debian.org/debian-security ftp2.de.debian.org/debian-security
#
# There are 2 default internal path_map settings for the Debian and Ubuntu
# changelog servers which will be merged with this option.
#
#   debian-changelogs packages.debian.org metadata.ftp-master.debian.org
#   ubuntu-changelogs changelogs.ubuntu.com
#
# These can be overridden by specifying an alternative mirror for that key, or
# deleted by just specifying the key with no mirror.
#
#path_map = debian-changelogs

# From version 1.7.0 there is support for caching multiple distibutions (eg
# Debian and Ubuntu) within the same apt-cacher instance. Enable this by setting
# distinct_namespaces to 1. Distribution package files are cached in separate
# directories whose names are derived from the relevant path_map key. So
# generally there will be a path_map key => server(s) setting for each
# distribution that is cached. Having enabled distinct_namespaces, existing
# packages can be imported into the correct directory by running (as root)
#
#  /usr/share/apt-cacher/apt-cacher-import.pl -u {cache_dir}/packages
#
#distinct_namespaces = 0

# If the apt-cacher machine is directly exposed to the Internet and you are
# worried about unauthorised machines fetching packages through it, you can
# specify a list of IP addresses which are allowed to use it and another list of
# IP addresses which are prohibited.
#
# Localhost (127.0.0.1/8, ::ffff:127.0.0.1/8 and ::1) are always allowed. Other
# addresses must be matched by allowed_hosts and not by denied_hosts to be
# permitted to use the cache.  Setting allowed_hosts to "*" means "allow all"
# (which was the default before version 1.7.0). The default is now ''.
#
# The format is a comma-separated list containing addresses, optionally with
# masks (like 10.0.0.0/24 or 10.0.0.0/255.255.255.0), or ranges of addresses
# (two addresses separated by a hyphen with no masks, specifying a valid subnet,
# like '192.168.0.0-63' or '192.168.0.0 - 192.168.0.63') or a DNS resolvable
# hostname.  The corresponding IPv6 options allowed_hosts_6 and denied_hosts_6
# are deprecated (but will still be honoured, if set). IPv6 addresses can now be
# added directly to allowed_hosts and denied_hosts along with IPv4 addresses.
# 允许访问的主机,星号代表所有
#allowed_hosts = *
# 拒绝的主机列表
#denied_hosts = 

# Only allow HTTPS/SSL proxy CONNECT to hosts or IPs which match an item in this
# list.
# 允许ssl的地址
#allowed_ssl_locations =

# Only allow HTTPS/SSL proxy CONNECT to ports which match an item in this list.
# Adding further items to this option can pose a significant security risk.  DO
# NOT do it unless you understand the full implications.
# ssl的端口
#allowed_ssl_ports = 443

# Optional setting to limit access to upstream mirrors based on server names in
# the URLs. This is matched before any path_map settings are expanded. If
# '%PATH_MAP%' in included in this option, it will be expanded to the keys of
# the path_map setting. Note these items are strings, not regexps.
#
#allowed_locations = ftp.uni-kl.de, ftp.nerim.net, debian.tu-bs.de/debian
#allowed_locations = ftp.debian.org, %PATH_MAP%
#allowed_locations = %PATH_MAP%

# List of Architectures that is used to expand %VALID_ARCHS% in *_files_regexps
# (see below).
# 支持的架构
#supported_archs = i386, amd64
#supported_archs = avr32, amd64, alpha, arm, arm64, armel, armhf, hppa, hurd-i386, i386, ia64, kfreebsd-amd64, kfreebsd-i386, m32r, m68k, mips, mipsel, netbsd-alpha, netbsd-i386, powerpc, powerpcspe, ppc64, s390, s390x, sh4, sparc, sparc64, x32

# List of Ubuntu release names used to expand %VALID_UBUNTU_RELEASE_NAMES% in
# *_files_regexp (see below). This is required to allow the Ubuntu installer to
# fetch upgrade information. As the naming scheme is unpredictable, new release
# names need to be added to this list.
# 支持的Ubuntu的版本号
#ubuntu_release_names = dapper, edgy, feisty, gutsy, hardy, intrepid, jaunty, karmic, lucid, maverick, natty, oneiric, precise, quantal, raring, saucy, trusty, utopic, vivid, wily, xenial, yakkety, zesty

### HOUSEKEEPING ###

# Apt-cacher can generate usage reports every 24 hours if you set this directive
# to 1. You can view the reports in a web browser by pointing to your cache
# machine with 'report' on the end, like this:
#
#      http://yourcache.example.com:3142/report
#
# Generating reports is very fast even with many thousands of logfile lines, so
# you can safely turn this on without creating much additional system load.
# 开启生成报告
#generate_reports = 1

# Apt-cacher can clean up its cache directory every 24 hours if you set this
# directive to 1. Cleaning the cache can take some time to run (generally in the
# order of a few minutes) and removes all package files that are not mentioned
# in any existing 'Packages' lists. This has the effect of deleting packages
# that have been superseded by an updated 'Packages' list.
# 如果值为1,则24小时清理一次
#clean_cache = 1

### INTERNALS ###

# Debug mode makes apt-cacher write a lot of extra debug information to the
# error log (whose location is defined with the 'log_dir' directive).  Leave
# this off unless you need it, or your error log will get very big. Acceptable
# values are 0 or an integer up to 7. See man apt-cacher (1) for further
# details.
#
#debug = 0

# You shouldn't need to change anything below here. If you do, ensure you
# understand the full implications of doing so.

# As a convenience the following strings are expanded within the *_files_regexp
# settings at runtime:
#  %VALID_UBUNTU_RELEASE_NAMES% --> A regexp derived from ubuntu_release_names
#  %VALID_ARCHS% --> A regexp derived from supported_archs
#  %VALID_PACKAGE_NAME% --> A regexp matching valid package names
#  %VALID_VERSION% --> A regexp matching valid package versions

# Permitted package files -- this is a perl regular expression which matches all
# package-type files (files that are uniquely identified by their filename).
#
#package_files_regexp = (?:(?:^|/)%VALID_PACKAGE_NAME%_%VALID_VERSION%(?:_%VALID_ARCHS%\.(?:u|d)?deb|\.dsc|\.tar\.(?:gz|bz2|xz|lzma)(?:\.asc)?|\.diff\.gz)|\.rpm|index\.db-.+\.gz|\.jigdo|\.template)$

# Permitted APT pdiff files -- this is a perl regular expression which matches
# APT pdiff files which are ed(1) scripts used to patch index files rather than
# redownloading the whole file afresh.
#
#pdiff_files_regexp = (?:^|/)2\d{3}-\d{2}-\d{2}-\d{4}\.\d{2}\.gz$

# Permitted Index files -- this is the perl regular expression which matches all
# index-type files (files that are uniquely identified by their full path and
# need to be checked for freshness).
#
#index_files_regexp = (?:^|/)(?:Index(?:\.bz2)?|(?:Sources|Packages|release)(?:\.(?:x|g)z|\.bz2)?|Release(?:\.gpg)?|InRelease|Contents-(?:[a-z]+-)?[a-zA-Z0-9]+\.gz|(?:srclist|pkglist)\.[a-z-]+\.bz2|Components-%VALID_ARCHS%\.yml\.(?:x|g)z|icons-(64|128)x\g{-1}\.tar\.(?:x|g)z|Translation-[a-z]{2,3}(?:_[A-Z]{2}(?:\.[a-zA-Z0-9-]+)?)?(?:\.gz|\.bz2|\.xz|\.lzma)?)$

# Permitted installer files -- this is the perl regular expression which matches
# all installer-type files (files that are uniquely identified by their full
# path but don’t need to be checked for freshness). These are typically files
# used by Debian/Ubuntu Installer, Debian Live and apt.
#
#installer_files_regexp = (?:^|/)(?:vmlinuz|linux|initrd\.gz|(?:%VALID_PACKAGE_NAME%_%VALID_VERSION%[_\.])?changelog|NEWS\.Debian|%VALID_UBUNTU_RELEASE_NAMES%\.tar\.gz(?:\.gpg)?|(?:by-hash/(?i:MD5SUM/[0-9a-f]{32}|SHA1/[0-9a-f]{40}|SHA256/[0-9a-f]{64}))|(?:Devel|EOL)?ReleaseAnnouncement(?:\.html)?|meta-release(?:-lts)?(?:-(?:development|proposed))?)$

# Perl regular expression which matches Index files from which to read checksums
# if checksum is enabled.
#
#checksum_files_regexp = (?:^|/)(?:(?:Sources|Packages)(?:\.(?:x|g)z|\.bz2)?|(?:In)?Release|Index(?:\.bz2)?)$

# Perl regular expression which matches files for which checksum validation is
# not performed. NB files matched by installer_files_regexp are skipped
# automatically and do not need to be added here as well.
#
#skip_checksum_files_regexp = (?:^|/)(?:(?:In)?Release|Release\.gpg)$

# Perl regular expression which matches URLs to be permitted for Debian bugs
# SOAP requests as made by apt-listbugs(1).
#
#soap_url_regexp = ^(?:http://)?bugs\.debian\.org(?::80)?/cgi-bin/soap\.cgi$

未完待续……

参考资料

https://blog.fleeto.us/post/build-ubuntu-repository-with-apt-mirror-and-apt-cacher/

https://www.centos.bz/2017/07/apt-mirror-deploy-ubuntu-local-repositry/

https://blog.programster.org/set-up-a-local-ubuntu-mirror-with-apt-mirror

Setting up an ‘Apt-Cache’ Server Using ‘Apt-Cacher-NG’ in Ubuntu 14.04 Server

https://help.ubuntu.com/community/Apt-Cacher-Server

发表评论

电子邮件地址不会被公开。 必填项已用*标注

1 × 3 =