From 25b6a4a5fa620555322d90eb4531911b0c51c837 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E5=BC=A0=E5=AE=88=E6=B0=B8?= Date: Thu, 7 Dec 2017 22:49:53 +0800 Subject: [PATCH 1/3] Update 20171207 How to use cron in Linux.md --- .../tech/20171207 How to use cron in Linux.md | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/sources/tech/20171207 How to use cron in Linux.md b/sources/tech/20171207 How to use cron in Linux.md index 3165aa8139..3df9c1b402 100644 --- a/sources/tech/20171207 How to use cron in Linux.md +++ b/sources/tech/20171207 How to use cron in Linux.md @@ -1,28 +1,27 @@ translating by yongshouzhang -How to use cron in Linux +如何在linux中使用cron ============================================================ -### No time for commands? Scheduling tasks with cron means programs can run but you don't have to stay up late. - +### 没有时间键入命令？ 使用 cron 调度任务意味着你不必熬夜守着程序，就可以让它运行。 [![](https://opensource.com/sites/default/files/styles/byline_thumbnail/public/david-crop.jpg?itok=Wnz6HdS0)][10] 06 Nov 2017 [David Both][11] [Feed][12] 27[up][13] [9 comments][14] -![How to use cron in Linux](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/linux-penguins.png?itok=yKOpaJM_) +![如何在 linux 中使用cron](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/linux-penguins.png?itok=yKOpaJM_) Image by : [Internet Archive Book Images][15]. Modified by Opensource.com. [CC BY-SA 4.0][16] -One of the challenges (among the many advantages) of being a sysadmin is running tasks when you'd rather be sleeping. For example, some tasks (including regularly recurring tasks) need to run overnight or on weekends, when no one is expected to be using computer resources. I have no time to spare in the evenings to run commands and scripts that have to operate during off-hours. And I don't want to have to get up at oh-dark-hundred to start a backup or major update. +作为系统管理员的一个挑战（也是众多优势之一）就是在你想睡觉时如何让任务运行。例如，一些任务（包括定期循环的作业）需要整夜或每逢周末运行，当没人想占用计算机资源时。晚上我没有空闲时间去运行在非高峰时间必须运行的命令和脚本。我也不想摸黑起床，进行备份和主更新。 -Instead, I use two service utilities that allow me to run commands, programs, and tasks at predetermined times. The [cron][17] and at services enable sysadmins to schedule tasks to run at a specific time in the future. The at service specifies a one-time task that runs at a certain time. The cron service can schedule tasks on a repetitive basis, such as daily, weekly, or monthly. +与之代替的是，我用了两个能够在既定时间运行命令，程序，任务的服务实用程序。[cron][17] 和 at 服务能够让系统管理雨在未来特定时间内运行计划任务。at 服务指定一个在特定时间运行的一次性任务。cron服务可在重复的基础上调度任务，如每天，每周或每月。 -In this article, I'll introduce the cron service and how to use it. +在本文中我将会介绍 cron 服务以及如何使用。 -### Common (and uncommon) cron uses +### cron 的常见（和不常见）用法 I use the cron service to schedule obvious things, such as regular backups that occur daily at 2 a.m. I also use it for less obvious things. From c7df54d7f2fad2f82148c1f8055aac6e262611ef Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E5=BC=A0=E5=AE=88=E6=B0=B8?= Date: Fri, 8 Dec 2017 20:12:40 +0800 Subject: [PATCH 2/3] =?UTF-8?q?=E9=80=89=E9=A2=98=207=20tools=20for=20anal?= =?UTF-8?q?yzing=20performance=20in=20Linux?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 选题文章名 7 tools for analyzing performance in Linux with bcc/BPF --- ...lyzing performance in Linux with bccBPF.md | 402 ++++++++++++++++++ 1 file changed, 402 insertions(+) create mode 100644 sources/tech/20171207 7 tools for analyzing performance in Linux with bccBPF.md diff --git a/sources/tech/20171207 7 tools for analyzing performance in Linux with bccBPF.md b/sources/tech/20171207 7 tools for analyzing performance in Linux with bccBPF.md new file mode 100644 index 0000000000..e6fd19e212 --- /dev/null +++ b/sources/tech/20171207 7 tools for analyzing performance in Linux with bccBPF.md @@ -0,0 +1,402 @@ +translating by yongshouzhang + +7 tools for analyzing performance in Linux with bcc/BPF +============================================================ + +### Look deeply into your Linux code with these Berkeley Packet Filter (BPF) Compiler Collection (bcc) tools. + + [![](https://opensource.com/sites/default/files/styles/byline_thumbnail/public/pictures/brendan_face2017_620d.jpg?itok=xZzBQNcY)][7] 21 Nov 2017 [Brendan Gregg][8] [Feed][9] + +43[up][10] + + [4 comments][11] +![7 superpowers for Fedora bcc/BPF performance analysis](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/penguins%20in%20space_0.jpg?itok=umpCTAul) + +Image by : + +opensource.com + +A new technology has arrived in Linux that can provide sysadmins and developers with a large number of new tools and dashboards for performance analysis and troubleshooting. It's called the enhanced Berkeley Packet Filter (eBPF, or just BPF), although these enhancements weren't developed in Berkeley, they operate on much more than just packets, and they do much more than just filtering. I'll discuss one way to use BPF on the Fedora and Red Hat family of Linux distributions, demonstrating on Fedora 26. + +BPF can run user-defined sandboxed programs in the kernel to add new custom capabilities instantly. It's like adding superpowers to Linux, on demand. Examples of what you can use it for include: + +* Advanced performance tracing tools: programmatic low-overhead instrumentation of filesystem operations, TCP events, user-level events, etc. + +* Network performance: dropping packets early on to improve DDOS resilience, or redirecting packets in-kernel to improve performance + +* Security monitoring: 24x7 custom monitoring and logging of suspicious kernel and userspace events + +BPF programs must pass an in-kernel verifier to ensure they are safe to run, making it a safer option, where possible, than writing custom kernel modules. I suspect most people won't write BPF programs themselves, but will use other people's. I've published many on GitHub as open source in the [BPF Compiler Collection (bcc)][12] project. bcc provides different frontends for BPF development, including Python and Lua, and is currently the most active project for BPF tooling. + +### 7 useful new bcc/BPF tools + +To understand the bcc/BPF tools and what they instrument, I created the following diagram and added it to the bcc project: + +### [bcc_tracing_tools.png][13] + +![Linux bcc/BPF tracing tools diagram](https://opensource.com/sites/default/files/u128651/bcc_tracing_tools.png) + +Brendan Gregg, [CC BY-SA 4.0][14] + +These are command-line interface (CLI) tools you can use over SSH (secure shell). Much analysis nowadays, including at my employer, is conducted using GUIs and dashboards. SSH is a last resort. But these CLI tools are still a good way to preview BPF capabilities, even if you ultimately intend to use them only through a GUI when available. I've began adding BPF capabilities to an open source GUI, but that's a topic for another article. Right now I'd like to share the CLI tools, which you can use today. + +### 1\. execsnoop + +Where to start? How about watching new processes. These can consume system resources, but be so short-lived they don't show up in top(1) or other tools. They can be instrumented (or, using the industry jargon for this, they can be traced) using [execsnoop][15]. While tracing, I'll log in over SSH in another window: + +``` +# /usr/share/bcc/tools/execsnoop +PCOMM PID PPID RET ARGS +sshd 12234 727 0 /usr/sbin/sshd -D -R +unix_chkpwd 12236 12234 0 /usr/sbin/unix_chkpwd root nonull +unix_chkpwd 12237 12234 0 /usr/sbin/unix_chkpwd root chkexpiry +bash 12239 12238 0 /bin/bash +id 12241 12240 0 /usr/bin/id -un +hostname 12243 12242 0 /usr/bin/hostname +pkg-config 12245 12244 0 /usr/bin/pkg-config --variable=completionsdir bash-completion +grepconf.sh 12246 12239 0 /usr/libexec/grepconf.sh -c +grep 12247 12246 0 /usr/bin/grep -qsi ^COLOR.*none /etc/GREP_COLORS +tty 12249 12248 0 /usr/bin/tty -s +tput 12250 12248 0 /usr/bin/tput colors +dircolors 12252 12251 0 /usr/bin/dircolors --sh /etc/DIR_COLORS +grep 12253 12239 0 /usr/bin/grep -qi ^COLOR.*none /etc/DIR_COLORS +grepconf.sh 12254 12239 0 /usr/libexec/grepconf.sh -c +grep 12255 12254 0 /usr/bin/grep -qsi ^COLOR.*none /etc/GREP_COLORS +grepconf.sh 12256 12239 0 /usr/libexec/grepconf.sh -c +grep 12257 12256 0 /usr/bin/grep -qsi ^COLOR.*none /etc/GREP_COLORS +``` + +Welcome to the fun of system tracing. You can learn a lot about how the system is really working (or not working, as the case may be) and discover some easy optimizations along the way. execsnoop works by tracing the exec() system call, which is usually used to load different program code in new processes. + +### 2\. opensnoop + +Continuing from above, so, grepconf.sh is likely a shell script, right? I'll run file(1) to check, and also use the [opensnoop][16] bcc tool to see what file is opening: + +``` +# /usr/share/bcc/tools/opensnoop +PID COMM FD ERR PATH +12420 file 3 0 /etc/ld.so.cache +12420 file 3 0 /lib64/libmagic.so.1 +12420 file 3 0 /lib64/libz.so.1 +12420 file 3 0 /lib64/libc.so.6 +12420 file 3 0 /usr/lib/locale/locale-archive +12420 file -1 2 /etc/magic.mgc +12420 file 3 0 /etc/magic +12420 file 3 0 /usr/share/misc/magic.mgc +12420 file 3 0 /usr/lib64/gconv/gconv-modules.cache +12420 file 3 0 /usr/libexec/grepconf.sh +1 systemd 16 0 /proc/565/cgroup +1 systemd 16 0 /proc/536/cgroup +``` + +``` +# file /usr/share/misc/magic.mgc /etc/magic +/usr/share/misc/magic.mgc: magic binary file for file(1) cmd (version 14) (little endian) +/etc/magic: magic text file for file(1) cmd, ASCII text +``` + +### 3\. xfsslower + +bcc/BPF can analyze much more than just syscalls. The [xfsslower][17] tool traces common XFS filesystem operations that have a latency of greater than 1 millisecond (the argument): + +``` +# /usr/share/bcc/tools/xfsslower 1 +Tracing XFS operations slower than 1 ms +TIME COMM PID T BYTES OFF_KB LAT(ms) FILENAME +14:17:34 systemd-journa 530 S 0 0 1.69 system.journal +14:17:35 auditd 651 S 0 0 2.43 audit.log +14:17:42 cksum 4167 R 52976 0 1.04 at +14:17:45 cksum 4168 R 53264 0 1.62 [ +14:17:45 cksum 4168 R 65536 0 1.01 certutil +14:17:45 cksum 4168 R 65536 0 1.01 dir +14:17:45 cksum 4168 R 65536 0 1.17 dirmngr-client +14:17:46 cksum 4168 R 65536 0 1.06 grub2-file +14:17:46 cksum 4168 R 65536 128 1.01 grub2-fstest +[...] +``` + +This is a useful tool and an important example of BPF tracing. Traditional analysis of filesystem performance focuses on block I/O statistics—what you commonly see printed by the iostat(1) tool and plotted by many performance-monitoring GUIs. Those statistics show how the disks are performing, but not really the filesystem. Often you care more about the filesystem's performance than the disks, since it's the filesystem that applications make requests to and wait for. And the performance of filesystems can be quite different from that of disks! Filesystems may serve reads entirely from memory cache and also populate that cache via a read-ahead algorithm and for write-back caching. xfsslower shows filesystem performance—what the applications directly experience. This is often useful for exonerating the entire storage subsystem; if there is really no filesystem latency, then performance issues are likely to be elsewhere. + +### 4\. biolatency + +Although filesystem performance is important to study for understanding application performance, studying disk performance has merit as well. Poor disk performance will affect the application eventually, when various caching tricks can no longer hide its latency. Disk performance is also a target of study for capacity planning. + +The iostat(1) tool shows the average disk I/O latency, but averages can be misleading. It can be useful to study the distribution of I/O latency as a histogram, which can be done using [biolatency][18]: + +``` +# /usr/share/bcc/tools/biolatency +Tracing block device I/O... Hit Ctrl-C to end. +^C + usecs : count distribution + 0 -> 1 : 0 | | + 2 -> 3 : 0 | | + 4 -> 7 : 0 | | + 8 -> 15 : 0 | | + 16 -> 31 : 0 | | + 32 -> 63 : 1 | | + 64 -> 127 : 63 |**** | + 128 -> 255 : 121 |********* | + 256 -> 511 : 483 |************************************ | + 512 -> 1023 : 532 |****************************************| + 1024 -> 2047 : 117 |******** | + 2048 -> 4095 : 8 | | +``` + +It's worth noting that many of these tools support CLI options and arguments as shown by their USAGE message: + +``` +# /usr/share/bcc/tools/biolatency -h +usage: biolatency [-h] [-T] [-Q] [-m] [-D] [interval] [count] + +Summarize block device I/O latency as a histogram + +positional arguments: + interval output interval, in seconds + count number of outputs + +optional arguments: + -h, --help show this help message and exit + -T, --timestamp include timestamp on output + -Q, --queued include OS queued time in I/O time + -m, --milliseconds millisecond histogram + -D, --disks print a histogram per disk device + +examples: + ./biolatency # summarize block I/O latency as a histogram + ./biolatency 1 10 # print 1 second summaries, 10 times + ./biolatency -mT 1 # 1s summaries, milliseconds, and timestamps + ./biolatency -Q # include OS queued time in I/O time + ./biolatency -D # show each disk device separately +``` + +### 5\. tcplife + +Another useful tool and example, this time showing lifespan and throughput statistics of TCP sessions, is [tcplife][19]: + +``` +# /usr/share/bcc/tools/tcplife +PID COMM LADDR LPORT RADDR RPORT TX_KB RX_KB MS +12759 sshd 192.168.56.101 22 192.168.56.1 60639 2 3 1863.82 +12783 sshd 192.168.56.101 22 192.168.56.1 60640 3 3 9174.53 +12844 wget 10.0.2.15 34250 54.204.39.132 443 11 1870 5712.26 +12851 curl 10.0.2.15 34252 54.204.39.132 443 0 74 505.90 +``` + +### 6\. gethostlatency + +Every previous example involves kernel tracing, so I need at least one user-level tracing example. Here is [gethostlatency][20], which instruments gethostbyname(3) and related library calls for name resolution: + +``` +# /usr/share/bcc/tools/gethostlatency +TIME PID COMM LATms HOST +06:43:33 12903 curl 188.98 opensource.com +06:43:36 12905 curl 8.45 opensource.com +06:43:40 12907 curl 6.55 opensource.com +06:43:44 12911 curl 9.67 opensource.com +06:45:02 12948 curl 19.66 opensource.cats +06:45:06 12950 curl 18.37 opensource.cats +06:45:07 12952 curl 13.64 opensource.cats +06:45:19 13139 curl 13.10 opensource.cats +``` + +### 7\. trace + +Okay, one more example. The [trace][21] tool was contributed by Sasha Goldshtein and provides some basic printf(1) functionality with custom probes. For example: + +``` +# /usr/share/bcc/tools/trace 'pam:pam_start "%s: %s", arg1, arg2' +PID TID COMM FUNC - +13266 13266 sshd pam_start sshd: root +``` + +### Install bcc via packages + +The best way to install bcc is from an iovisor repository, following the instructions from the bcc [INSTALL.md][22]. [IO Visor][23] is the Linux Foundation project that includes bcc. The BPF enhancements these tools use were added in the 4.x series Linux kernels, up to 4.9\. This means that Fedora 25, with its 4.8 kernel, can run most of these tools; and Fedora 26, with its 4.11 kernel, can run them all (at least currently). + +If you are on Fedora 25 (or Fedora 26, and this post was published many months ago—hello from the distant past!), then this package approach should just work. If you are on Fedora 26, then skip to the [Install via Source][24] section, which avoids a [known][25] and [fixed][26] bug. That bug fix hasn't made its way into the Fedora 26 package dependencies at the moment. The system I'm using is: + +``` +# uname -a +Linux localhost.localdomain 4.11.8-300.fc26.x86_64 #1 SMP Thu Jun 29 20:09:48 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux +# cat /etc/fedora-release +Fedora release 26 (Twenty Six) +``` + +``` +# echo -e '[iovisor]\nbaseurl=https://repo.iovisor.org/yum/nightly/f25/$basearch\nenabled=1\ngpgcheck=0' | sudo tee /etc/yum.repos.d/iovisor.repo +# dnf install bcc-tools +[...] +Total download size: 37 M +Installed size: 143 M +Is this ok [y/N]: y +``` + +``` +# ls /usr/share/bcc/tools/ +argdist dcsnoop killsnoop softirqs trace +bashreadline dcstat llcstat solisten ttysnoop +[...] +``` + +``` +# /usr/share/bcc/tools/opensnoop +chdir(/lib/modules/4.11.8-300.fc26.x86_64/build): No such file or directory +Traceback (most recent call last): + File "/usr/share/bcc/tools/opensnoop", line 126, in + b = BPF(text=bpf_text) + File "/usr/lib/python3.6/site-packages/bcc/__init__.py", line 284, in __init__ + raise Exception("Failed to compile BPF module %s" % src_file) +Exception: Failed to compile BPF module +``` + +``` +# dnf install kernel-devel-4.11.8-300.fc26.x86_64 +[...] +Total download size: 20 M +Installed size: 63 M +Is this ok [y/N]: y +[...] +``` + +``` +# /usr/share/bcc/tools/opensnoop +PID COMM FD ERR PATH +11792 ls 3 0 /etc/ld.so.cache +11792 ls 3 0 /lib64/libselinux.so.1 +11792 ls 3 0 /lib64/libcap.so.2 +11792 ls 3 0 /lib64/libc.so.6 +[...] +``` + +### Install via source + +If you need to install from source, you can also find documentation and updated instructions in [INSTALL.md][27]. I did the following on Fedora 26: + +``` +sudo dnf install -y bison cmake ethtool flex git iperf libstdc++-static \ + python-netaddr python-pip gcc gcc-c++ make zlib-devel \ + elfutils-libelf-devel +sudo dnf install -y luajit luajit-devel # for Lua support +sudo dnf install -y \ + http://pkgs.repoforge.org/netperf/netperf-2.6.0-1.el6.rf.x86_64.rpm +sudo pip install pyroute2 +sudo dnf install -y clang clang-devel llvm llvm-devel llvm-static ncurses-devel +``` + +``` +Curl error (28): Timeout was reached for http://pkgs.repoforge.org/netperf/netperf-2.6.0-1.el6.rf.x86_64.rpm [Connection timed out after 120002 milliseconds] +``` + +Here are the remaining bcc compilation and install steps: + +``` +git clone https://github.com/iovisor/bcc.git +mkdir bcc/build; cd bcc/build +cmake .. -DCMAKE_INSTALL_PREFIX=/usr +make +sudo make install +``` + +``` +# /usr/share/bcc/tools/opensnoop +PID COMM FD ERR PATH +4131 date 3 0 /etc/ld.so.cache +4131 date 3 0 /lib64/libc.so.6 +4131 date 3 0 /usr/lib/locale/locale-archive +4131 date 3 0 /etc/localtime +[...] +``` + +More Linux resources + +* [What is Linux?][1] + +* [What are Linux containers?][2] + +* [Download Now: Linux commands cheat sheet][3] + +* [Advanced Linux commands cheat sheet][4] + +* [Our latest Linux articles][5] + +This was a quick tour of the new BPF performance analysis superpowers that you can use on the Fedora and Red Hat family of operating systems. I demonstrated the popular + + [bcc][28] + +frontend to BPF and included install instructions for Fedora. bcc comes with more than 60 new tools for performance analysis, which will help you get the most out of your Linux systems. Perhaps you will use these tools directly over SSH, or perhaps you will use the same functionality via monitoring GUIs once they support BPF. + +Also, bcc is not the only frontend in development. There are [ply][29] and [bpftrace][30], which aim to provide higher-level language for quickly writing custom tools. In addition, [SystemTap][31] just released [version 3.2][32], including an early, experimental eBPF backend. Should this continue to be developed, it will provide a production-safe and efficient engine for running the many SystemTap scripts and tapsets (libraries) that have been developed over the years. (Using SystemTap with eBPF would be good topic for another post.) + +If you need to develop custom tools, you can do that with bcc as well, although the language is currently much more verbose than SystemTap, ply, or bpftrace. My bcc tools can serve as code examples, plus I contributed a [tutorial][33] for developing bcc tools in Python. I'd recommend learning the bcc multi-tools first, as you may get a lot of mileage from them before needing to write new tools. You can study the multi-tools from their example files in the bcc repository: [funccount][34], [funclatency][35], [funcslower][36], [stackcount][37], [trace][38], and [argdist][39]. + +Thanks to [Opensource.com][40] for edits. + +### Topics + + [Linux][41][SysAdmin][42] + +### About the author + + [![](https://opensource.com/sites/default/files/styles/profile_pictures/public/pictures/brendan_face2017_620d.jpg?itok=LIwTJjL9)][43] Brendan Gregg + +- + + Brendan Gregg is a senior performance architect at Netflix, where he does large scale computer performance design, analysis, and tuning.[More about me][44] + +* [Learn how you can contribute][6] + +-------------------------------------------------------------------------------- + +via:https://opensource.com/article/17/11/bccbpf-performance + +作者：[Brendan Gregg ][a] +译者：[yongshouzhang](https://github.com/yongshouzhang) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: +[1]:https://opensource.com/resources/what-is-linux?intcmp=70160000000h1jYAAQ&utm_source=intcallout&utm_campaign=linuxcontent +[2]:https://opensource.com/resources/what-are-linux-containers?intcmp=70160000000h1jYAAQ&utm_source=intcallout&utm_campaign=linuxcontent +[3]:https://developers.redhat.com/promotions/linux-cheatsheet/?intcmp=70160000000h1jYAAQ&utm_source=intcallout&utm_campaign=linuxcontent +[4]:https://developers.redhat.com/cheat-sheet/advanced-linux-commands-cheatsheet?intcmp=70160000000h1jYAAQ&utm_source=intcallout&utm_campaign=linuxcontent +[5]:https://opensource.com/tags/linux?intcmp=70160000000h1jYAAQ&utm_source=intcallout&utm_campaign=linuxcontent +[6]:https://opensource.com/participate +[7]:https://opensource.com/users/brendang +[8]:https://opensource.com/users/brendang +[9]:https://opensource.com/user/77626/feed +[10]:https://opensource.com/article/17/11/bccbpf-performance?rate=r9hnbg3mvjFUC9FiBk9eL_ZLkioSC21SvICoaoJjaSM +[11]:https://opensource.com/article/17/11/bccbpf-performance#comments +[12]:https://github.com/iovisor/bcc +[13]:https://opensource.com/file/376856 +[14]:https://opensource.com/usr/share/bcc/tools/trace +[15]:https://github.com/brendangregg/perf-tools/blob/master/execsnoop +[16]:https://github.com/brendangregg/perf-tools/blob/master/opensnoop +[17]:https://github.com/iovisor/bcc/blob/master/tools/xfsslower.py +[18]:https://github.com/iovisor/bcc/blob/master/tools/biolatency.py +[19]:https://github.com/iovisor/bcc/blob/master/tools/tcplife.py +[20]:https://github.com/iovisor/bcc/blob/master/tools/gethostlatency.py +[21]:https://github.com/iovisor/bcc/blob/master/tools/trace.py +[22]:https://github.com/iovisor/bcc/blob/master/INSTALL.md#fedora---binary +[23]:https://www.iovisor.org/ +[24]:https://opensource.com/article/17/11/bccbpf-performance#InstallViaSource +[25]:https://github.com/iovisor/bcc/issues/1221 +[26]:https://reviews.llvm.org/rL302055 +[27]:https://github.com/iovisor/bcc/blob/master/INSTALL.md#fedora---source +[28]:https://github.com/iovisor/bcc +[29]:https://github.com/iovisor/ply +[30]:https://github.com/ajor/bpftrace +[31]:https://sourceware.org/systemtap/ +[32]:https://sourceware.org/ml/systemtap/2017-q4/msg00096.html +[33]:https://github.com/iovisor/bcc/blob/master/docs/tutorial_bcc_python_developer.md +[34]:https://github.com/iovisor/bcc/blob/master/tools/funccount_example.txt +[35]:https://github.com/iovisor/bcc/blob/master/tools/funclatency_example.txt +[36]:https://github.com/iovisor/bcc/blob/master/tools/funcslower_example.txt +[37]:https://github.com/iovisor/bcc/blob/master/tools/stackcount_example.txt +[38]:https://github.com/iovisor/bcc/blob/master/tools/trace_example.txt +[39]:https://github.com/iovisor/bcc/blob/master/tools/argdist_example.txt +[40]:http://opensource.com/ +[41]:https://opensource.com/tags/linux +[42]:https://opensource.com/tags/sysadmin +[43]:https://opensource.com/users/brendang +[44]:https://opensource.com/users/brendang From 97a0c97bd5089cded76d22ad63575ae28cfec558 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E5=BC=A0=E5=AE=88=E6=B0=B8?= Date: Fri, 8 Dec 2017 20:33:04 +0800 Subject: [PATCH 3/3] Delete 20171207 How to use cron in Linux.md --- .../20171207 How to use cron in Linux.md | 287 ------------------ 1 file changed, 287 deletions(-) delete mode 100644 sources/tech/20171207 How to use cron in Linux.md diff --git a/sources/tech/20171207 How to use cron in Linux.md b/sources/tech/20171207 How to use cron in Linux.md deleted file mode 100644 index 3df9c1b402..0000000000 --- a/sources/tech/20171207 How to use cron in Linux.md +++ /dev/null @@ -1,287 +0,0 @@ -translating by yongshouzhang - -如何在linux中使用cron -============================================================ - -### 没有时间键入命令？ 使用 cron 调度任务意味着你不必熬夜守着程序，就可以让它运行。 - [![](https://opensource.com/sites/default/files/styles/byline_thumbnail/public/david-crop.jpg?itok=Wnz6HdS0)][10] 06 Nov 2017 [David Both][11] [Feed][12] - -27[up][13] - - [9 comments][14] -![如何在 linux 中使用cron](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/linux-penguins.png?itok=yKOpaJM_) - -Image by : - -[Internet Archive Book Images][15]. Modified by Opensource.com. [CC BY-SA 4.0][16] - -作为系统管理员的一个挑战（也是众多优势之一）就是在你想睡觉时如何让任务运行。例如，一些任务（包括定期循环的作业）需要整夜或每逢周末运行，当没人想占用计算机资源时。晚上我没有空闲时间去运行在非高峰时间必须运行的命令和脚本。我也不想摸黑起床，进行备份和主更新。 - -与之代替的是，我用了两个能够在既定时间运行命令，程序，任务的服务实用程序。[cron][17] 和 at 服务能够让系统管理雨在未来特定时间内运行计划任务。at 服务指定一个在特定时间运行的一次性任务。cron服务可在重复的基础上调度任务，如每天，每周或每月。 - -在本文中我将会介绍 cron 服务以及如何使用。 - -### cron 的常见（和不常见）用法 - -I use the cron service to schedule obvious things, such as regular backups that occur daily at 2 a.m. I also use it for less obvious things. - -* The system times (i.e., the operating system time) on my many computers are set using the Network Time Protocol (NTP). While NTP sets the system time, it does not set the hardware time, which can drift. I use cron to set the hardware time based on the system time. - -* I also have a Bash program I run early every morning that creates a new "message of the day" (MOTD) on each computer. It contains information, such as disk usage, that should be current in order to be useful. - -* Many system processes and services, like [Logwatch][1], [logrotate][2], and [Rootkit Hunter][3], use the cron service to schedule tasks and run programs every day. - -The crond daemon is the background service that enables cron functionality. - -The cron service checks for files in the /var/spool/cron and /etc/cron.d directories and the /etc/anacrontab file. The contents of these files define cron jobs that are to be run at various intervals. The individual user cron files are located in /var/spool/cron, and system services and applications generally add cron job files in the /etc/cron.ddirectory. The /etc/anacrontab is a special case that will be covered later in this article. - -### Using crontab - -The cron utility runs based on commands specified in a cron table (crontab). Each user, including root, can have a cron file. These files don't exist by default, but can be created in the /var/spool/cron directory using the crontab -e command that's also used to edit a cron file (see the script below). I strongly recommend that you not use a standard editor (such as Vi, Vim, Emacs, Nano, or any of the many other editors that are available). Using the crontab command not only allows you to edit the command, it also restarts the crond daemon when you save and exit the editor. The crontabcommand uses Vi as its underlying editor, because Vi is always present (on even the most basic of installations). - -New cron files are empty, so commands must be added from scratch. I added the job definition example below to my own cron files, just as a quick reference, so I know what the various parts of a command mean. Feel free to copy it for your own use. - -``` -# crontab -e -SHELL=/bin/bash -MAILTO=root@example.com -PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin - -# For details see man 4 crontabs - -# Example of job definition: -# .---------------- minute (0 - 59) -# | .------------- hour (0 - 23) -# | | .---------- day of month (1 - 31) -# | | | .------- month (1 - 12) OR jan,feb,mar,apr ... -# | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat -# | | | | | -# * * * * * user-name command to be executed - -# backup using the rsbu program to the internal 4TB HDD and then 4TB external -01 01 * * * /usr/local/bin/rsbu -vbd1 ; /usr/local/bin/rsbu -vbd2 - -# Set the hardware clock to keep it in sync with the more accurate system clock -03 05 * * * /sbin/hwclock --systohc - -# Perform monthly updates on the first of the month -# 25 04 1 * * /usr/bin/dnf -y update -``` - -The first three lines in the code above set up a default environment. The environment must be set to whatever is necessary for a given user because cron does not provide an environment of any kind. The SHELL variable specifies the shell to use when commands are executed. This example specifies the Bash shell. The MAILTO variable sets the email address where cron job results will be sent. These emails can provide the status of the cron job (backups, updates, etc.) and consist of the output you would see if you ran the program manually from the command line. The third line sets up the PATH for the environment. Even though the path is set here, I always prepend the fully qualified path to each executable. - -There are several comment lines in the example above that detail the syntax required to define a cron job. I'll break those commands down, then add a few more to show you some more advanced capabilities of crontab files. - -``` -01 01 * * * /usr/local/bin/rsbu -vbd1 ; /usr/local/bin/rsbu -vbd2 -``` - -This line runs my self-written Bash shell script, rsbu, that backs up all my systems. This job kicks off at 1:01 a.m. (01 01) every day. The asterisks (*) in positions three, four, and five of the time specification are like file globs, or wildcards, for other time divisions; they specify "every day of the month," "every month," and "every day of the week." This line runs my backups twice; one backs up to an internal dedicated backup hard drive, and the other backs up to an external USB drive that I can take to the safe deposit box. - -The following line sets the hardware clock on the computer using the system clock as the source of an accurate time. This line is set to run at 5:03 a.m. (03 05) every day. - -``` -03 05 * * * /sbin/hwclock --systohc -``` - -I was using the third and final cron job (commented out) to perform a dnf or yumupdate at 04:25 a.m. on the first day of each month, but I commented it out so it no longer runs. - -``` -# 25 04 1 * * /usr/bin/dnf -y update -``` - -### Other scheduling tricks - -Now let's do some things that are a little more interesting than these basics. Suppose you want to run a particular job every Thursday at 3 p.m.: - -``` -00 15 * * Thu /usr/local/bin/mycronjob.sh -``` - -Or, maybe you need to run quarterly reports after the end of each quarter. The cron service has no option for "The last day of the month," so instead you can use the first day of the following month, as shown below. (This assumes that the data needed for the reports will be ready when the job is set to run.) - -``` -02 03 1 1,4,7,10 * /usr/local/bin/reports.sh -``` - -The following shows a job that runs one minute past every hour between 9:01 a.m. and 5:01 p.m. - -``` -01 09-17 * * * /usr/local/bin/hourlyreminder.sh -``` - -I have encountered situations where I need to run a job every two, three, or four hours. That can be accomplished by dividing the hours by the desired interval, such as */3 for every three hours, or 6-18/3 to run every three hours between 6 a.m. and 6 p.m. Other intervals can be divided similarly; for example, the expression */15 in the minutes position means "run the job every 15 minutes." - -``` -*/5 08-18/2 * * * /usr/local/bin/mycronjob.sh -``` - -One thing to note: The division expressions must result in a remainder of zero for the job to run. That's why, in this example, the job is set to run every five minutes (08:05, 08:10, 08:15, etc.) during even-numbered hours from 8 a.m. to 6 p.m., but not during any odd-numbered hours. For example, the job will not run at all from 9 p.m. to 9:59 a.m. - -I am sure you can come up with many other possibilities based on these examples. - -### Limiting cron access - -More Linux resources - -* [What is Linux?][4] - -* [What are Linux containers?][5] - -* [Download Now: Linux commands cheat sheet][6] - -* [Advanced Linux commands cheat sheet][7] - -* [Our latest Linux articles][8] - -Regular users with cron access could make mistakes that, for example, might cause system resources (such as memory and CPU time) to be swamped. To prevent possible misuse, the sysadmin can limit user access by creating a - -**/etc/cron.allow** - - file that contains a list of all users with permission to create cron jobs. The root user cannot be prevented from using cron. - -By preventing non-root users from creating their own cron jobs, it may be necessary for root to add their cron jobs to the root crontab. "But wait!" you say. "Doesn't that run those jobs as root?" Not necessarily. In the first example in this article, the username field shown in the comments can be used to specify the user ID a job is to have when it runs. This prevents the specified non-root user's jobs from running as root. The following example shows a job definition that runs a job as the user "student": - -``` -04 07 * * * student /usr/local/bin/mycronjob.sh -``` - -### cron.d - -The directory /etc/cron.d is where some applications, such as [SpamAssassin][18] and [sysstat][19], install cron files. Because there is no spamassassin or sysstat user, these programs need a place to locate cron files, so they are placed in /etc/cron.d. - -The /etc/cron.d/sysstat file below contains cron jobs that relate to system activity reporting (SAR). These cron files have the same format as a user cron file. - -``` -# Run system activity accounting tool every 10 minutes -*/10 * * * * root /usr/lib64/sa/sa1 1 1 -# Generate a daily summary of process accounting at 23:53 -53 23 * * * root /usr/lib64/sa/sa2 -A -``` - -The sysstat cron file has two lines that perform tasks. The first line runs the sa1program every 10 minutes to collect data stored in special binary files in the /var/log/sadirectory. Then, every night at 23:53, the sa2 program runs to create a daily summary. - -### Scheduling tips - -Some of the times I set in the crontab files seem rather random—and to some extent they are. Trying to schedule cron jobs can be challenging, especially as the number of jobs increases. I usually have only a few tasks to schedule on each of my computers, which is simpler than in some of the production and lab environments where I have worked. - -One system I administered had around a dozen cron jobs that ran every night and an additional three or four that ran on weekends or the first of the month. That was a challenge, because if too many jobs ran at the same time—especially the backups and compiles—the system would run out of RAM and nearly fill the swap file, which resulted in system thrashing while performance tanked, so nothing got done. We added more memory and improved how we scheduled tasks. We also removed a task that was very poorly written and used large amounts of memory. - -The crond service assumes that the host computer runs all the time. That means that if the computer is turned off during a period when cron jobs were scheduled to run, they will not run until the next time they are scheduled. This might cause problems if they are critical cron jobs. Fortunately, there is another option for running jobs at regular intervals: anacron. - -### anacron - -The [anacron][20] program performs the same function as crond, but it adds the ability to run jobs that were skipped, such as if the computer was off or otherwise unable to run the job for one or more cycles. This is very useful for laptops and other computers that are turned off or put into sleep mode. - -As soon as the computer is turned on and booted, anacron checks to see whether configured jobs missed their last scheduled run. If they have, those jobs run immediately, but only once (no matter how many cycles have been missed). For example, if a weekly job was not run for three weeks because the system was shut down while you were on vacation, it would be run soon after you turn the computer on, but only once, not three times. - -The anacron program provides some easy options for running regularly scheduled tasks. Just install your scripts in the /etc/cron.[hourly|daily|weekly|monthly]directories, depending how frequently they need to be run. - -How does this work? The sequence is simpler than it first appears. - -1. The crond service runs the cron job specified in /etc/cron.d/0hourly. - -``` -# Run the hourly jobs -SHELL=/bin/bash -PATH=/sbin:/bin:/usr/sbin:/usr/bin -MAILTO=root -01 * * * * root run-parts /etc/cron.hourly -``` - -1. The cron job specified in /etc/cron.d/0hourly runs the run-parts program once per hour. - -2. The run-parts program runs all the scripts located in the /etc/cron.hourlydirectory. - -3. The /etc/cron.hourly directory contains the 0anacron script, which runs the anacron program using the /etdc/anacrontab configuration file shown here. - -``` -# /etc/anacrontab: configuration file for anacron - -# See anacron(8) and anacrontab(5) for details. - -SHELL=/bin/sh -PATH=/sbin:/bin:/usr/sbin:/usr/bin -MAILTO=root -# the maximal random delay added to the base delay of the jobs -RANDOM_DELAY=45 -# the jobs will be started during the following hours only -START_HOURS_RANGE=3-22 - -#period in days delay in minutes job-identifier command -1 5 cron.daily nice run-parts /etc/cron.daily -7 25 cron.weekly nice run-parts /etc/cron.weekly -@monthly 45 cron.monthly nice run-parts /etc/cron.monthly -``` - -1. The anacron program runs the programs located in /etc/cron.daily once per day; it runs the jobs located in /etc/cron.weekly once per week, and the jobs in cron.monthly once per month. Note the specified delay times in each line that help prevent these jobs from overlapping themselves and other cron jobs. - -Instead of placing complete Bash programs in the cron.X directories, I install them in the /usr/local/bin directory, which allows me to run them easily from the command line. Then I add a symlink in the appropriate cron directory, such as /etc/cron.daily. - -The anacron program is not designed to run programs at specific times. Rather, it is intended to run programs at intervals that begin at the specified times, such as 3 a.m. (see the START_HOURS_RANGE line in the script just above) of each day, on Sunday (to begin the week), and on the first day of the month. If any one or more cycles are missed, anacron will run the missed jobs once, as soon as possible. - -### More on setting limits - -I use most of these methods for scheduling tasks to run on my computers. All those tasks are ones that need to run with root privileges. It's rare in my experience that regular users really need a cron job. One case was a developer user who needed a cron job to kick off a daily compile in a development lab. - -It is important to restrict access to cron functions by non-root users. However, there are circumstances when a user needs to set a task to run at pre-specified times, and cron can allow them to do that. Many users do not understand how to properly configure these tasks using cron and they make mistakes. Those mistakes may be harmless, but, more often than not, they can cause problems. By setting functional policies that cause users to interact with the sysadmin, individual cron jobs are much less likely to interfere with other users and other system functions. - -It is possible to set limits on the total resources that can be allocated to individual users or groups, but that is an article for another time. - -For more information, the man pages for [cron][21], [crontab][22], [anacron][23], [anacrontab][24], and [run-parts][25] all have excellent information and descriptions of how the cron system works. - -### Topics - - [Linux][26][SysAdmin][27] - -### About the author - - [![](https://opensource.com/sites/default/files/styles/profile_pictures/public/david-crop.jpg?itok=oePpOpyV)][28] David Both - -- - - David Both is a Linux and Open Source advocate who resides in Raleigh, North Carolina. He has been in the IT industry for over forty years and taught OS/2 for IBM where he worked for over 20 years. While at IBM, he wrote the first training course for the original IBM PC in 1981\. He has taught RHCE classes for Red Hat and has worked at MCI Worldcom, Cisco, and the State of North Carolina. He has been working with Linux and Open Source Software for almost 20 years. David has written articles for... [more about David Both][29][More about me][30] - -* [Learn how you can contribute][9] - --------------------------------------------------------------------------------- - -via: https://opensource.com/article/17/11/how-use-cron-linux - -作者：[David Both ][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]: -[1]:https://sourceforge.net/projects/logwatch/files/ -[2]:https://github.com/logrotate/logrotate -[3]:http://rkhunter.sourceforge.net/ -[4]:https://opensource.com/resources/what-is-linux?intcmp=70160000000h1jYAAQ&utm_source=intcallout&utm_campaign=linuxcontent -[5]:https://opensource.com/resources/what-are-linux-containers?intcmp=70160000000h1jYAAQ&utm_source=intcallout&utm_campaign=linuxcontent -[6]:https://developers.redhat.com/promotions/linux-cheatsheet/?intcmp=70160000000h1jYAAQ&utm_source=intcallout&utm_campaign=linuxcontent -[7]:https://developers.redhat.com/cheat-sheet/advanced-linux-commands-cheatsheet?intcmp=70160000000h1jYAAQ&utm_source=intcallout&utm_campaign=linuxcontent -[8]:https://opensource.com/tags/linux?intcmp=70160000000h1jYAAQ&utm_source=intcallout&utm_campaign=linuxcontent -[9]:https://opensource.com/participate -[10]:https://opensource.com/users/dboth -[11]:https://opensource.com/users/dboth -[12]:https://opensource.com/user/14106/feed -[13]:https://opensource.com/article/17/11/how-use-cron-linux?rate=9R7lrdQXsne44wxIh0Wu91ytYaxxi86zT1-uHo1a1IU -[14]:https://opensource.com/article/17/11/how-use-cron-linux#comments -[15]:https://www.flickr.com/photos/internetarchivebookimages/20570945848/in/photolist-xkMtw9-xA5zGL-tEQLWZ-wFwzFM-aNwxgn-aFdWBj-uyFKYv-7ZCCBU-obY1yX-UAPafA-otBzDF-ovdDo6-7doxUH-obYkeH-9XbHKV-8Zk4qi-apz7Ky-apz8Qu-8ZoaWG-orziEy-aNwxC6-od8NTv-apwpMr-8Zk4vn-UAP9Sb-otVa3R-apz6Cb-9EMPj6-eKfyEL-cv5mwu-otTtHk-7YjK1J-ovhxf6-otCg2K-8ZoaJf-UAPakL-8Zo8j7-8Zk74v-otp4Ls-8Zo8h7-i7xvpR-otSosT-9EMPja-8Zk6Zi-XHpSDB-hLkuF3-of24Gf-ouN1Gv-fJzkJS-icfbY9 -[16]:https://creativecommons.org/licenses/by-sa/4.0/ -[17]:https://en.wikipedia.org/wiki/Cron -[18]:http://spamassassin.apache.org/ -[19]:https://github.com/sysstat/sysstat -[20]:https://en.wikipedia.org/wiki/Anacron -[21]:http://man7.org/linux/man-pages/man8/cron.8.html -[22]:http://man7.org/linux/man-pages/man5/crontab.5.html -[23]:http://man7.org/linux/man-pages/man8/anacron.8.html -[24]:http://man7.org/linux/man-pages/man5/anacrontab.5.html -[25]:http://manpages.ubuntu.com/manpages/zesty/man8/run-parts.8.html -[26]:https://opensource.com/tags/linux -[27]:https://opensource.com/tags/sysadmin -[28]:https://opensource.com/users/dboth -[29]:https://opensource.com/users/dboth -[30]:https://opensource.com/users/dboth