diff --git a/published/20060430 Linux Find Out Last System Reboot Time and Date Command.md b/published/20060430 Linux Find Out Last System Reboot Time and Date Command.md new file mode 100644 index 0000000000..9569c998f4 --- /dev/null +++ b/published/20060430 Linux Find Out Last System Reboot Time and Date Command.md @@ -0,0 +1,199 @@ +如何在 Linux 系统查询机器最近重启时间 +====== + +在你的 Linux 或类 UNIX 系统中，你是如何查询系统上次重新启动的日期和时间？怎样显示系统关机的日期和时间？ `last` 命令不仅可以按照时间从近到远的顺序列出该会话的特定用户、终端和主机名，而且还可以列出指定日期和时间登录的用户。输出到终端的每一行都包括用户名、会话终端、主机名、会话开始和结束的时间、会话持续的时间。要查看 Linux 或类 UNIX 系统重启和关机的时间和日期，可以使用下面的命令。 + +- `last` 命令 +- `who` 命令 + + +### 使用 who 命令来查看系统重新启动的时间/日期 + +你需要在终端使用 [who][1] 命令来打印有哪些人登录了系统，`who` 命令同时也会显示上次系统启动的时间。使用 `last` 命令来查看系统重启和关机的日期和时间，运行： + +``` +$ who -b +``` + +示例输出： + +``` +system boot 2017-06-20 17:41 +``` + +使用 `last` 命令来查询最近登录到系统的用户和系统重启的时间和日期。输入： + +``` +$ last reboot | less +``` + +示例输出： + +[![Fig.01: last command in action][2]][2] + +或者，尝试输入： + +``` +$ last reboot | head -1 +``` + +示例输出： + +``` +reboot system boot 4.9.0-3-amd64 Sat Jul 15 19:19 still running +``` + +`last` 命令通过查看文件 `/var/log/wtmp` 来显示自 wtmp 文件被创建时的所有登录（和登出）的用户。每当系统重新启动时，这个伪用户 `reboot` 就会登录。因此，`last reboot` 命令将会显示自该日志文件被创建以来的所有重启信息。 + +### 查看系统上次关机的时间和日期 + +可以使用下面的命令来显示上次关机的日期和时间： + +``` +$ last -x|grep shutdown | head -1 +``` + +示例输出： + +``` +shutdown system down 2.6.15.4 Sun Apr 30 13:31 - 15:08 (01:37) +``` + +命令中， + +* `-x`：显示系统关机和运行等级改变信息 + + +这里是 `last` 命令的其它的一些选项： + +``` +$ last +$ last -x +$ last -x reboot +$ last -x shutdown +``` + +示例输出： + +![Fig.01: How to view last Linux System Reboot Date/Time ][3] + +### 查看系统正常的运行时间 + +评论区的读者建议的另一个命令如下： + +``` +$ uptime -s +``` + +示例输出： + +``` +2017-06-20 17:41:51 +``` + +### OS X/Unix/FreeBSD 查看最近重启和关机时间的命令示例 + +在终端输入下面的命令： + +``` +$ last reboot +``` + +在 OS X 示例输出结果如下： + +``` +reboot ~ Fri Dec 18 23:58 +reboot ~ Mon Dec 14 09:54 +reboot ~ Wed Dec 9 23:21 +reboot ~ Tue Nov 17 21:52 +reboot ~ Tue Nov 17 06:01 +reboot ~ Wed Nov 11 12:14 +reboot ~ Sat Oct 31 13:40 +reboot ~ Wed Oct 28 15:56 +reboot ~ Wed Oct 28 11:35 +reboot ~ Tue Oct 27 00:00 +reboot ~ Sun Oct 18 17:28 +reboot ~ Sun Oct 18 17:11 +reboot ~ Mon Oct 5 09:35 +reboot ~ Sat Oct 3 18:57 + + +wtmp begins Sat Oct 3 18:57 +``` + +查看关机日期和时间，输入： + +``` +$ last shutdown +``` + +示例输出： + +``` +shutdown ~ Fri Dec 18 23:57 +shutdown ~ Mon Dec 14 09:53 +shutdown ~ Wed Dec 9 23:20 +shutdown ~ Tue Nov 17 14:24 +shutdown ~ Mon Nov 16 21:15 +shutdown ~ Tue Nov 10 13:15 +shutdown ~ Sat Oct 31 13:40 +shutdown ~ Wed Oct 28 03:10 +shutdown ~ Sun Oct 18 17:27 +shutdown ~ Mon Oct 5 09:23 + + +wtmp begins Sat Oct 3 18:57 +``` + +### 如何查看是谁重启和关闭机器？ + +你需要[启用 psacct 服务然后运行下面的命令][4]来查看执行过的命令（包括用户名），在终端输入 [lastcomm][5] 命令查看信息 + +``` +# lastcomm userNameHere +# lastcomm commandNameHere +# lastcomm | more +# lastcomm reboot +# lastcomm shutdown +### 或者查看重启和关机时间 +# lastcomm | egrep 'reboot|shutdown' +``` + +示例输出： + +``` +reboot S X root pts/0 0.00 secs Sun Dec 27 23:49 +shutdown S root pts/1 0.00 secs Sun Dec 27 23:45 +``` + +我们可以看到 root 用户在当地时间 12 月 27 日星期二 23:49 在 pts/0 重新启动了机器。 + +### 参见 + +* 更多信息可以查看 man 手册（`man last`）和参考文章 [如何在 Linux 服务器上使用 tuptime 命令查看历史和统计的正常的运行时间][6]。 + +### 关于作者 + +作者是 nixCraft 的创立者，同时也是一名经验丰富的系统管理员，也是 Linux，类 Unix 操作系统 shell 脚本的培训师。他曾与全球各行各业的客户工作过，包括 IT，教育，国防和空间研究以及非营利部门等等。你可以在 [Twitter][7]、[Facebook][8]、[Google+][9] 关注他。 + + +-------------------------------------------------------------------------------- + +via: https://www.cyberciti.biz/tips/linux-last-reboot-time-and-date-find-out.html + +作者：[Vivek Gite][a] +译者：[amwps290](https://github.com/amwps290) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.cyberciti.biz/ +[1]:https://www.cyberciti.biz/faq/unix-linux-who-command-examples-syntax-usage/ "See Linux/Unix who command examples for more info" +[2]:https://www.cyberciti.biz/tips/wp-content/uploads/2006/04/last-reboot.jpg +[3]:https://www.cyberciti.biz/media/new/tips/2006/04/check-last-time-system-was-rebooted.jpg +[4]:https://www.cyberciti.biz/tips/howto-log-user-activity-using-process-accounting.html +[5]:https://www.cyberciti.biz/faq/linux-unix-lastcomm-command-examples-usage-syntax/ "See Linux/Unix lastcomm command examples for more info" +[6]:https://www.cyberciti.biz/hardware/howto-see-historical-statistical-uptime-on-linux-server/ +[7]:https://twitter.com/nixcraft +[8]:https://facebook.com/nixcraft +[9]:https://plus.google.com/+CybercitiBiz diff --git a/published/20070129 How To Debug a Bash Shell Script Under Linux or UNIX.md b/published/20070129 How To Debug a Bash Shell Script Under Linux or UNIX.md new file mode 100644 index 0000000000..1517fc151d --- /dev/null +++ b/published/20070129 How To Debug a Bash Shell Script Under Linux or UNIX.md @@ -0,0 +1,293 @@ +如何在 Linux 或者 UNIX 下调试 Bash Shell 脚本 +====== + +来自我的邮箱： + +> 我写了一个 hello world 小脚本。我如何能调试运行在 Linux 或者类 UNIX 的系统上的 bash shell 脚本呢？ + +这是 Linux / Unix 系统管理员或新用户最常问的问题。shell 脚本调试可能是一项繁琐的工作（不容易阅读）。调试 shell 脚本有多种方法。 + +您需要传递 `-x` 或 `-v` 参数，以在 bash shell 中浏览每行代码。 + +[![如何在 Linux 或者 UNIX 下调试 Bash Shell 脚本][1]][1] + +让我们看看如何使用各种方法调试 Linux 和 UNIX 上运行的脚本。 + +### -x 选项来调试脚本 + +用 `-x` 选项来运行脚本： + +``` +$ bash -x script-name +$ bash -x domains.sh +``` +### 使用 set 内置命令 + +bash shell 提供调试选项，可以打开或关闭使用 [set 命令][2]： + +* `set -x` : 显示命令及其执行时的参数。 +* `set -v` : 显示 shell 输入行作为它们读取的 + +可以在 shell 脚本本身中使用上面的两个命令： + +``` +#!/bin/bash +clear + +# turn on debug mode +set -x +for f in * +do + file $f +done +# turn OFF debug mode +set +x +ls +# more commands +``` + +你可以代替 [标准释伴][3] 行： + +``` +#!/bin/bash +``` + +用以下代码（用于调试）： + +``` +#!/bin/bash -xv +``` + +### 使用智能调试功能 + +首先添加一个叫做 `_DEBUG` 的特殊变量。当你需要调试脚本的时候，设置 `_DEBUG` 为 `on`： + +``` +_DEBUG="on" +``` + +在脚本的开头放置以下函数： + +``` +function DEBUG() +{ + [ "$_DEBUG" == "on" ] && $@ +} +``` + +现在，只要你需要调试，只需使用 `DEBUG` 函数如下： + +``` +DEBUG echo "File is $filename" +``` + +或者： + +``` +DEBUG set -x +Cmd1 +Cmd2 +DEBUG set +x +``` + +当调试完（在移动你的脚本到生产环境之前）设置 `_DEBUG` 为 `off`。不需要删除调试行。 + +``` +_DEBUG="off" # 设置为非 'on' 的任何字符 +``` + +示例脚本： + +``` +#!/bin/bash +_DEBUG="on" +function DEBUG() +{ + [ "$_DEBUG" == "on" ] && $@ +} + +DEBUG echo 'Reading files' +for i in * +do + grep 'something' $i > /dev/null + [ $? -eq 0 ] && echo "Found in $i file" +done +DEBUG set -x +a=2 +b=3 +c=$(( $a + $b )) +DEBUG set +x +echo "$a + $b = $c" +``` + +保存并关闭文件。运行脚本如下： + +``` +$ ./script.sh +``` + +输出： + +``` +Reading files +Found in xyz.txt file ++ a=2 ++ b=3 ++ c=5 ++ DEBUG set +x ++ '[' on == on ']' ++ set +x +2 + 3 = 5 +``` + +现在设置 `_DEBUG` 为 `off`（你需要编辑该文件）： + +``` +_DEBUG="off" +``` + +运行脚本： + +``` +$ ./script.sh +``` + +输出： + +``` +Found in xyz.txt file +2 + 3 = 5 +``` + +以上是一个简单但非常有效的技术。还可以尝试使用 `DEBUG` 作为别名而不是函数。 + +### 调试 Bash Shell 的常见错误 + +Bash 或者 sh 或者 ksh 在屏幕上给出各种错误信息，在很多情况下，错误信息可能不提供详细的信息。 + +#### 跳过在文件上应用执行权限 + +当你 [编写你的第一个 hello world 脚本][4],您可能会得到一个错误，如下所示： + +``` +bash: ./hello.sh: Permission denied +``` + +设置权限使用 `chmod` 命令: + +``` +$ chmod +x hello.sh +$ ./hello.sh +$ bash hello.sh +``` + +#### 文件结束时发生意外的错误 + +如果您收到文件结束意外错误消息，请打开脚本文件，并确保它有打开和关闭引号。在这个例子中，`echo` 语句有一个开头引号，但没有结束引号： + +``` +#!/bin/bash + +... +.... + +echo 'Error: File not found + ^^^^^^^ + missing quote +``` + +还要确保你检查缺少的括号和大括号 `{}`： + +``` +#!/bin/bash +..... +[ ! -d $DIRNAME ] && { echo "Error: Chroot dir not found"; exit 1; + ^^^^^^^^^^^^^ + missing brace } +... +``` + +#### 丢失像 fi，esac，;; 等关键字。 + +如果你缺少了结尾的关键字，如 `fi` 或 `;;` 你会得到一个错误，如 “XXX 意外”。因此，确保所有嵌套的 `if` 和 `case` 语句以适当的关键字结束。有关语法要求的页面。在本例中，缺少 `fi`： + +``` +#!/bin/bash +echo "Starting..." +.... +if [ $1 -eq 10 ] +then + if [ $2 -eq 100 ] + then + echo "Do something" +fi + +for f in $files +do + echo $f +done + +# 注意 fi 丢失了 +``` + +#### 在 Windows 或 UNIX 框中移动或编辑 shell 脚本 + +不要在 Linux 上创建脚本并移动到 Windows。另一个问题是编辑 Windows 10上的 shell 脚本并将其移动到 UNIX 服务器上。这将由于换行符不同而导致命令没有发现的错误。你可以使用下列命令 [将 DOS 换行转换为 CR-LF 的Unix/Linux 格式][5] ： + +``` +dos2unix my-script.sh +``` + +### 技巧 + +#### 技巧 1 - 发送调试信息输出到标准错误 + +[标准错误] 是默认错误输出设备，用于写所有系统错误信息。因此，将消息发送到默认的错误设备是个好主意： + +``` +# 写错误到标准输出 +echo "Error: $1 file not found" +# +# 写错误到标准错误（注意 1>&2 在 echo 命令末尾） +# +echo "Error: $1 file not found" 1>&2 +``` + +#### 技巧 2 - 在使用 vim 文本编辑器时，打开语法高亮 + +大多数现代文本编辑器允许设置语法高亮选项。这对于检测语法和防止常见错误如打开或关闭引号非常有用。你可以在不同的颜色中看到。这个特性简化了 shell 脚本结构中的编写，语法错误在视觉上截然不同。高亮不影响文本本身的意义，它只为你提示而已。在这个例子中，我的脚本使用了 vim 语法高亮： + +[!如何调试 Bash Shell 脚本，在 Linux 或者 UNIX 使用 Vim 语法高亮特性][7]][7] + +#### 技巧 3 - 使用 shellcheck 检查脚本 + +[shellcheck 是一个用于静态分析 shell 脚本的工具][8]。可以使用它来查找 shell 脚本中的错误。这是用 Haskell 编写的。您可以使用这个工具找到警告和建议。你可以看看如何在 Linux 或 类UNIX 系统上安装和使用 shellcheck 来改善你的 shell 脚本，避免错误和高效。 + +作者：Vivek Gite + +作者是 nixCraft 创造者，一个经验丰富的系统管理员和一个练习 Linux 操作系统/ UNIX shell 脚本的教练。他曾与全球客户和各种行业，包括 IT，教育，国防和空间研究，以及非营利部门。关注他的 [推特][9]，[脸谱网][10]，[谷歌+ ][11]。 + +-------------------------------------------------------------------------------- + +via: https://www.cyberciti.biz/tips/debugging-shell-script.html + +作者：[Vivek Gite][a] +译者：[zjon](https://github.com/zjon) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.cyberciti.biz +[1]:https://www.cyberciti.biz/tips/wp-content/uploads/2007/01/How-to-debug-a-bash-shell-script-on-Linux-or-Unix.jpg +[2]:https://bash.cyberciti.biz/guide/Set_command +[3]:https://bash.cyberciti.biz/guide/Shebang +[4]:https://www.cyberciti.biz/faq/hello-world-bash-shell-script/ +[5]:https://www.cyberciti.biz/faq/howto-unix-linux-convert-dos-newlines-cr-lf-unix-text-format/ +[6]:https://bash.cyberciti.biz/guide/Standard_error +[7]:https://www.cyberciti.biz/media/new/tips/2007/01/bash-vim-debug-syntax-highlighting.png +[8]:https://www.cyberciti.biz/programming/improve-your-bashsh-shell-script-with-shellcheck-lint-script-analysis-tool/ +[9]:https://twitter.com/nixcraft +[10]:https://facebook.com/nixcraft +[11]:https://plus.google.com/+CybercitiBiz + + diff --git a/published/20071007 Linux Check IDE - SATA SSD Hard Disk Transfer Speed.md b/published/20071007 Linux Check IDE - SATA SSD Hard Disk Transfer Speed.md new file mode 100644 index 0000000000..c7aab5ed81 --- /dev/null +++ b/published/20071007 Linux Check IDE - SATA SSD Hard Disk Transfer Speed.md @@ -0,0 +1,158 @@ +在 Linux 上检测 IDE/SATA SSD 硬盘的传输速度 +====== + +你知道你的硬盘在 Linux 下传输有多快吗？不打开电脑的机箱或者机柜，你知道它运行在 SATA I (150 MB/s) 、 SATA II (300 MB/s) 还是 SATA III (6.0Gb/s) 呢？ + +你能够使用 `hdparm` 和 `dd` 命令来检测你的硬盘速度。它为各种硬盘的 ioctls 提供了命令行界面，这是由 Linux 系统的 ATA / IDE / SATA 设备驱动程序子系统所支持的。有些选项只能用最新的内核才能正常工作（请确保安装了最新的内核）。我也推荐使用最新的内核源代码的包含头文件来编译 `hdparm` 命令。 + +### 如何使用 hdparm 命令来检测硬盘的传输速度 + +以 root 管理员权限登录并执行命令： + +``` +$ sudo hdparm -tT /dev/sda +``` + +或者， + +``` +$ sudo hdparm -tT /dev/hda +``` + +输出： + +``` +/dev/sda: + Timing cached reads: 7864 MB in 2.00 seconds = 3935.41 MB/sec + Timing buffered disk reads: 204 MB in 3.00 seconds = 67.98 MB/sec +``` + +为了检测更精准，这个操作应该**重复2-3次** 。这显示了无需访问磁盘，直接从 Linux 缓冲区缓存中读取的速度。这个测量实际上是被测系统的处理器、高速缓存和存储器的吞吐量的指标。这是一个 [for 循环的例子][1]，连续运行测试 3 次： + +``` +for i in 1 2 3; do hdparm -tT /dev/hda; done +``` + +这里， + + * `-t` ：执行设备读取时序 + * `-T` ：执行缓存读取时间 + * `/dev/sda` ：硬盘设备文件 + + +要 [找出 SATA 硬盘的连接速度][2] ，请输入： + +``` +sudo hdparm -I /dev/sda | grep -i speed +``` + +输出： + +``` + * Gen1 signaling speed (1.5Gb/s) + * Gen2 signaling speed (3.0Gb/s) + * Gen3 signaling speed (6.0Gb/s) + +``` + +以上输出表明我的硬盘可以使用 1.5Gb/s、3.0Gb/s 或 6.0Gb/s 的速度。请注意，您的 BIOS/主板必须支持 SATA-II/III 才行： + +``` +$ dmesg | grep -i sata | grep 'link up' +``` + +[![Linux Check IDE SATA SSD Hard Disk Transfer Speed][3]][3] + +### dd 命令 + +你使用 `dd` 命令也可以获取到相应的速度信息： + +``` +dd if=/dev/zero of=/tmp/output.img bs=8k count=256k +rm /tmp/output.img +``` + +输出： + +``` +262144+0 records in +262144+0 records out +2147483648 bytes (2.1 GB) copied, 23.6472 seconds, `90.8 MB/s` +``` + +下面是 [推荐的 dd 命令参数][4]： + +``` +dd if=/dev/input.file of=/path/to/output.file bs=block-size count=number-of-blocks oflag=dsync + +## GNU dd syntax ## +dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=dsync + +## OR alternate syntax for GNU/dd ## +dd if=/dev/zero of=/tmp/testALT.img bs=1G count=1 conv=fdatasync +``` + +这是上面命令的第三个命令的输出结果： + +``` +1+0 records in +1+0 records out +1073741824 bytes (1.1 GB, 1.0 GiB) copied, 4.23889 s, 253 MB/s +``` + +### “磁盘与存储” - GUI 工具 + +您还可以使用位于“系统>管理>磁盘实用程序”菜单中的磁盘实用程序。请注意，在最新版本的 Gnome 中，它简称为“磁盘”。 + +#### 如何使用 Linux 上的“磁盘”测试我的硬盘的性能？ + +要测试硬盘的速度： + + 1. 从“活动概览”中打开“磁盘”（按键盘上的 super 键并键入“disks”） + 2. 从“左侧窗格”的列表中选择“磁盘” + 3. 选择菜单按钮并从菜单中选择“测试磁盘性能……” + 4. 单击“开始性能测试……”并根据需要调整传输速率和访问时间参数。 + 5. 选择“开始性能测试”来测试从磁盘读取数据的速度。需要管理权限请输入密码。 + +以上操作的快速视频演示： + +https://www.cyberciti.biz/tips/wp-content/uploads/2007/10/disks-performance.mp4 + +#### 只读 Benchmark (安全模式下) + +然后，选择 > 只读: + +![Fig.01: Linux Benchmarking Hard Disk Read Only Test Speed][5] + +上述选项不会销毁任何数据。 + +#### 读写的 Benchmark（所有数据将丢失，所以要小心） + +访问“系统>管理>磁盘实用程序菜单>单击性能测试>单击开始读/写性能测试按钮： + +![Fig.02:Linux Measuring read rate, write rate and access time][6] + +### 作者 + +作者是 nixCraft 的创造者，是经验丰富的系统管理员，也是 Linux 操作系统/ Unix shell 脚本的培训师。他曾与全球客户以及 IT，教育，国防和空间研究以及非营利部门等多个行业合作。在Twitter，Facebook和Google+上关注他。 + +-------------------------------------------------------------------------------- + +via: https://www.cyberciti.biz/tips/how-fast-is-linux-sata-hard-disk.html + +作者：[Vivek Gite][a] +译者：[MonkeyDEcho](https://github.com/MonkeyDEcho) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.cyberciti.biz/ +[1]:https://www.cyberciti.biz/faq/bash-for-loop/ +[2]:https://www.cyberciti.biz/faq/linux-command-to-find-sata-harddisk-link-speed/ +[3]:https://www.cyberciti.biz/tips/wp-content/uploads/2007/10/Linux-Check-IDE-SATA-SSD-Hard-Disk-Transfer-Speed.jpg +[4]:https://www.cyberciti.biz/faq/howto-linux-unix-test-disk-performance-with-dd-command/ +[5]:https://www.cyberciti.biz/media/new/tips/2007/10/Linux-Hard-Disk-Speed-Benchmark.png (Linux Benchmark Hard Disk Speed) +[6]:https://www.cyberciti.biz/media/new/tips/2007/10/Linux-Hard-Disk-Read-Write-Benchmark.png (Linux Hard Disk Benchmark Read / Write Rate and Access Time) +[7]:https://twitter.com/nixcraft +[8]:https://facebook.com/nixcraft +[9]:https://plus.google.com/+CybercitiBiz diff --git a/published/20141029 What does an idle CPU do.md b/published/20141029 What does an idle CPU do.md new file mode 100644 index 0000000000..d5259385c5 --- /dev/null +++ b/published/20141029 What does an idle CPU do.md @@ -0,0 +1,89 @@ +当 CPU 空闲时它都在做什么？ +============================================================ + +在 [上篇文章中][2] 我说了操作系统行为的基本原理是，*在任何一个给定的时刻*，在一个 CPU 上**有且只有一个任务是活动的**。但是，如果 CPU 无事可做的时候，又会是什么样的呢？ + +事实证明，这种情况是非常普遍的，对于绝大多数的个人电脑来说，这确实是一种常态：大量的睡眠进程，它们都在等待某种情况下被唤醒，差不多在 100% 的 CPU 时间中，都处于虚构的“空闲任务”中。事实上，如果一个普通用户的 CPU 处于持续的繁忙中，它可能意味着有一个错误、bug、或者运行了恶意软件。 + +因为我们不能违反我们的原理，*一些任务需要在一个 CPU 上激活*。首先是因为，这是一个良好的设计：持续很长时间去遍历内核，检查是否*有*一个活动任务，这种特殊情况是不明智的做法。最好的设计是*没有任何例外的情况*。无论何时，你写一个 `if` 语句，Nyan Cat 就会喵喵喵。其次，我们需要使用空闲的 CPU 去做*一些事情*，让它们充满活力，你懂得，就是创建天网计划呗。 + +因此，保持这种设计的连续性，并领先于那些邪恶计划一步，操作系统开发者创建了一个**空闲任务**，当没有其它任务可做时就调度它去运行。我们可以在 Linux 的 [引导过程][3] 中看到，这个空闲任务就是进程 0，它是由计算机打开电源时运行的第一个指令直接派生出来的。它在 [rest_init][4] 中初始化，在 [init_idle_bootup_task][5] 中初始化空闲调度类scheduling class。 + +简而言之，Linux 支持像实时进程、普通用户进程等等的不同调度类。当选择一个进程变成活动任务时，这些类按优先级进行查询。通过这种方式，核反应堆的控制代码总是优先于 web 浏览器运行。尽管在通常情况下，这些类返回 `NULL`，意味着它们没有合适的任务需要去运行 —— 它们总是处于睡眠状态。但是空闲调度类，它是持续运行的，从不会失败：它总是返回空闲任务。 + +好吧，我们来看一下这个空闲任务*到底做了些什么*。下面是 [cpu_idle_loop][6]，感谢开源能让我们看到它的代码： + +``` +while (1) { + while(!need_resched()) { + cpuidle_idle_call(); + } + + /* + [Note: Switch to a different task. We will return to this loop when the idle task is again selected to run.] + */ + schedule_preempt_disabled(); +} +``` + +*cpu_idle_loop* + +我省略了很多的细节，稍后我们将去了解任务切换，但是，如果你阅读了这些源代码，你就会找到它的要点：由于这里不需要重新调度（即改变活动任务），它一直处于空闲状态。以所经历的时间来计算，这个循环和其它操作系统中它的“堂兄弟们”相比，在计算的历史上它是运行的最多的代码片段。对于 Intel 处理器来说，处于空闲状态意味着运行着一个 [halt][7] 指令： + +``` +static inline void native_halt(void) + { + asm volatile("hlt": : :"memory"); + } +``` + +*native_halt* + +`hlt` 指令停止处理器中的代码执行，并将它置于 `halt` 的状态。奇怪的是，全世界各地数以百万计的 Intel 类的 CPU 们花费大量的时间让它们处于 `halt` 的状态，甚至它们在通电的时候也是如此。这并不是高效、节能的做法，这促使芯片制造商们去开发处理器的深度睡眠状态，以带来着更少的功耗和更长休眠时间。内核的 [cpuidle 子系统][8] 是这些节能模式能够产生好处的原因。 + +现在，一旦我们告诉 CPU 去 `halt`（睡眠）之后，我们需要以某种方式让它醒来。如果你读过 [上篇文章《你的操作系统什么时候运行？》][9] ，你可能会猜到*中断*会参与其中，而事实确实如此。中断促使 CPU 离开 `halt` 状态返回到激活状态。因此，将这些拼到一起，下图是当你阅读一个完全呈现的 web 网页时，你的系统主要做的事情： + +![](https://manybutfinite.com/img/os/idle.png) + +除定时器中断外的其它中断也会使处理器再次发生变化。如果你再次点击一个 web 页面就会产生这种变化，例如：你的鼠标发出一个中断，它的驱动会处理它，并且因为它产生了一个新的输入，突然进程就可运行了。在那个时刻， `need_resched()` 返回 `true`，然后空闲任务因你的浏览器而被踢出而终止运行。 + +如果我们呆呆地看着这篇文章，而不做任何事情。那么随着时间的推移，这个空闲循环就像下图一样： + +![](https://manybutfinite.com/img/os/idleCycles.png) + +在这个示例中，由内核计划的定时器中断会每 4 毫秒发生一次。这就是滴答tick周期。也就是说每秒钟将有 250 个滴答，因此，这个*滴答速率（频率）*是 250 Hz。这是运行在 Intel 处理器上的 Linux 的典型值，而其它操作系统喜欢使用 100 Hz。这是由你构建内核时在 `CONFIG_HZ` 选项中定义的。 + +对于一个*空闲 CPU* 来说，它看起来似乎是个无意义的工作。如果外部世界没有新的输入，在你的笔记本电脑的电池耗尽之前，CPU 将始终处于这种每秒钟被唤醒 250 次的地狱般折磨的小憩中。如果它运行在一个虚拟机中，那我们正在消耗着宿主机 CPU 的性能和宝贵的时钟周期。 + +在这里的解决方案是 [动态滴答][10]，当 CPU 处于空闲状态时，定时器中断被 [暂停或重计划][11]，直到内核*知道*将有事情要做时（例如，一个进程的定时器可能要在 5 秒内过期，因此，我们不能再继续睡眠了），定时器中断才会重新发出。这也被称为*无滴答模式*。 + +最后，假设在一个系统中你有一个*活动进程*，例如，一个长时间运行的 CPU 密集型任务。那样几乎就和一个空闲系统是相同的：这些示意图仍然是相同的，只是将空闲任务替换为这个进程，并且相应的描述也是准确的。在那种情况下，每 4 毫秒去中断一次任务仍然是无意义的：它只是操作系统的性能抖动，甚至会使你的工作变得更慢而已。Linux 也可以在这种单一进程的场景中停止这种固定速率的滴答，这被称为 [自适应滴答][12] 模式。最终，这种固定速率的滴答可能会 [完全消失][13]。 + +对于阅读一篇文章来说，CPU 基本是无事可做的。内核的这种空闲行为是操作系统难题的一个重要部分，并且它与我们看到的其它情况非常相似，因此，这将帮助我们理解一个运行中的内核。更多的内容将发布在下周的 [RSS][14] 和 [Twitter][15] 上。 + +-------------------------------------------------------------------------------- + +via: https://manybutfinite.com/post/what-does-an-idle-cpu-do/ + +作者：[Gustavo Duarte][a] +译者：[qhwdw](https://github.com/qhwdw) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://duartes.org/gustavo/blog/about/ +[1]:https://manybutfinite.com/post/what-does-an-idle-cpu-do/ +[2]:https://linux.cn/article-9095-1.html +[3]:https://manybutfinite.com/post/kernel-boot-process +[4]:https://github.com/torvalds/linux/blob/v3.17/init/main.c#L393 +[5]:https://github.com/torvalds/linux/blob/v3.17/kernel/sched/core.c#L4538 +[6]:https://github.com/torvalds/linux/blob/v3.17/kernel/sched/idle.c#L183 +[7]:https://github.com/torvalds/linux/blob/v3.17/arch/x86/include/asm/irqflags.h#L52 +[8]:http://lwn.net/Articles/384146/ +[9]:https://linux.cn/article-9095-1.html +[10]:https://github.com/torvalds/linux/blob/v3.17/Documentation/timers/NO_HZ.txt#L17 +[11]:https://github.com/torvalds/linux/blob/v3.17/Documentation/timers/highres.txt#L215 +[12]:https://github.com/torvalds/linux/blob/v3.17/Documentation/timers/NO_HZ.txt#L100 +[13]:http://lwn.net/Articles/549580/ +[14]:https://manybutfinite.com/feed.xml +[15]:http://twitter.com/manybutfinite \ No newline at end of file diff --git a/translated/tech/20170707 Lessons from my first year of live coding on Twitch.md b/published/20170707 Lessons from my first year of live coding on Twitch.md similarity index 86% rename from translated/tech/20170707 Lessons from my first year of live coding on Twitch.md rename to published/20170707 Lessons from my first year of live coding on Twitch.md index 7a2f0aa98a..042603760b 100644 --- a/translated/tech/20170707 Lessons from my first year of live coding on Twitch.md +++ b/published/20170707 Lessons from my first year of live coding on Twitch.md @@ -1,10 +1,11 @@ -我在 Twitch 平台直播编程的第一年 +我在 Twitch 平台直播编程的经验 ============================================================ -去年 7 月我进行了第一次直播。不像大多数人那样在 Twitch 上进行游戏直播，我想直播的内容是我利用个人时间进行的开源工作。我对 NodeJS 硬件库有一定的研究（其中大部分是靠我自学的）。考虑到我已经在 Twitch 上有了一个直播间，为什么不再建一个更小更专业的直播间，比如使用 JavaScript 驱动硬件JavaScript powered hardware 来建立直播间 :) 我注册了 [我自己的频道][1] ，从那以后我就开始定期直播。 + +去年 7 月我进行了第一次直播。不像大多数人那样在 Twitch 上进行游戏直播，我想直播的内容是我利用个人时间进行的开源工作。我对 NodeJS 硬件库有一定的研究（其中大部分是靠我自学的）。考虑到我已经在 Twitch 上有了一个直播间，为什么不再建一个更小更专业的直播间，比如 由 JavaScript 驱动的硬件JavaScript powered hardware ;) 我注册了 [我自己的频道][1] ，从那以后我就开始定期直播。 我当然不是第一个这么做的人。[Handmade Hero][2] 是我最早看到的几个在线直播编程的程序员之一。很快这种直播方式被 Vlambeer 发扬光大，他在 Twitch 的 [Nuclear Throne live][3] 直播间进行直播。我对 Vlambeer 尤其着迷。 -我的朋友 [Nolan Lawson][4] 让我 _真正开始做_ 这件事，而不只是单纯地 _想要做_ 。我看了他 [在周末直播开源工作][5] ，做得棒极了。他解释了他当时做的每一件事。每一件事。回复 GitHub 上的 问题issues ，鉴别 bug ，在 分支branches 中调试程序，你知道的。这令我着迷，因为 Nolan 使他的开源库得到了广泛的使用。他的开源生活和我的完全不一样。 +我的朋友 [Nolan Lawson][4] 让我 _真正开始做_ 这件事，而不只是单纯地 _想要做_ 。我看了他 [在周末直播开源工作][5] ，做得棒极了。他解释了他当时做的每一件事。是的，每一件事，包括回复 GitHub 上的 问题issues ，鉴别 bug ，在 分支branches 中调试程序，你知道的。这令我着迷，因为 Nolan 使他的开源库得到了广泛的使用。他的开源生活和我的完全不一样。 你甚至可以看到我在他视频下的评论： @@ -14,27 +15,27 @@ 那个星期六我极少的几个听众给了我很大的鼓舞，因此我坚持了下去。现在我有了超过一千个听众，他们中的一些人形成了一个可爱的小团体，他们会定期观看我的直播，我称呼他们为 “noopkat 家庭” 。 -我们很开心。我想称呼这个即时编程部分为“多玩家在线组队编程”。我真的被他们每个人的热情和才能触动了。一次，一个团体成员指出我的 Arduino 开发板没有连接上软件，因为板子上的芯片丢了。这真是最有趣的时刻之一。 +我们很开心。我想称呼这个即时编程部分为“多玩家在线组队编程”。我真的被他们每个人的热情和才能触动了。一次，一个团体成员指出我的 Arduino 开发板不能随同我的软件工作，因为板子上的芯片丢了。这真是最有趣的时刻之一。 -我经常暂停直播，检查我的收件箱，看看有没有人对我提过的，不再有时间完成的工作发起 拉取请求pull request 。感谢我 Twitch 社区对我的帮助和鼓励。 +我经常暂停直播，检查我的收件箱，看看有没有人对我提及过但没有时间完成的工作发起 拉取请求pull request 。感谢我 Twitch 社区对我的帮助和鼓励。 -我很想聊聊 Twitch 直播给我带来的好处，但它的内容太多了，我应该会在我下一个博客里介绍。我在这里想要分享的，是我学习的关于如何自己实现直播编程的课程。最近几个开发者问我怎么开始自己的直播，因此我在这里想大家展示我给他们的建议！ +我很想聊聊 Twitch 直播给我带来的好处，但它的内容太多了，我应该会在我下一篇博客里介绍。我在这里想要分享的，是我学习的关于如何自己实现直播编程的课程。最近几个开发者问我怎么开始自己的直播，因此我在这里想大家展示我给他们的建议！ 首先，我在这里贴出一个给过我很大帮助的教程 [“Streaming and Finding Success on Twitch”][7] 。它专注于 Twitch 与游戏直播，但也有很多和我们要做的东西相关的部分。我建议首先阅读这个教程，然后再考虑一些建立直播频道的细节（比如如何选择设备和软件）。 -下面我列出我自己的配置。这些配置是从我多次的错误经验中总结出来的，其中要感谢我的直播同行的智慧与建议（对，你们知道就是你们！）。 +下面我列出我自己的配置。这些配置是从我多次的错误经验中总结出来的，其中要感谢我的直播同行的智慧与建议。（对，你们知道就是你们！） ### 软件 -有很多免费的直播软件。我用的是 [Open Broadcaster Software (OBS)][8] 。它适用于大多数的平台。我觉得它十分直观且易于入门，但掌握其他的进阶功能则需要一段时间的学习。学好它你会获得很多好处！这是今天我直播时 OBS 的桌面截图（点击查看大图）： +有很多免费的直播软件。我用的是 [Open Broadcaster Software (OBS)][8] 。它适用于大多数的平台。我觉得它十分直观且易于入门，但掌握其他的进阶功能则需要一段时间的学习。学好它你会获得很多好处！这是今天我直播时 OBS 的桌面截图： ![](https://cdn-images-1.medium.com/max/1600/0*s4wyeYuaiThV52q5.png) -你直播时需要在不用的“场景”中进行切换。一个“场景”是多个“素材”通过堆叠和组合产生的集合。一个“素材”可以是照相机，麦克风，你的桌面，网页，动态文本，图片等等。 OBS 是一个很强大的软件。 +你直播时需要在不用的“场景scenes”中进行切换。一个“场景”是多个“素材sources”通过堆叠和组合产生的集合。一个“素材”可以是照相机、麦克风、你的桌面、网页、动态文本、图片等等。 OBS 是一个很强大的软件。 最上方的桌面场景是我编程的环境，我直播的时候主要停留在这里。我使用 iTerm 和 vim ，同时打开一个可以切换的浏览器窗口来查阅文献或在 GitHub 上分类检索资料。 -底部的黑色长方形是我的网络摄像头，人们可以通过这种个人化的连接方式来观看我工作。 +底部的黑色长方形是我的网络摄像头，人们可以通过这种更个人化的连接方式来观看我工作。 我的场景中有一些“标签”，很多都与状态或者顶栏信息有关。顶栏只是添加了个性化信息，它在直播时是一个很好的连续性素材。这是我在 [GIMP][9] 里制作的图片，在你的场景里它会作为一个素材来加载。一些标签是从文本文件里添加的动态内容（例如最新粉丝）。另一个标签是一个 [custom one I made][10] ，它可以展示我直播的房间的动态温度与湿度。 @@ -62,7 +63,7 @@ ### 硬件 -我从使用便宜的器材开始，当我意识到我会长期坚持直播之后，才将他们逐渐换成更好的。开始的时候尽量使用你现有的器材，即使是只用电脑内置的摄像头与麦克风。 +我从使用便宜的器材开始，当我意识到我会长期坚持直播之后，才将它们逐渐换成更好的。开始的时候尽量使用你现有的器材，即使是只用电脑内置的摄像头与麦克风。 现在我使用 Logitech Pro C920 网络摄像头，和一个固定有支架的 Blue Yeti 麦克风。花费是值得的。我直播的质量完全不同了。 @@ -116,7 +117,7 @@ 当你即将开始的时候，你会感觉很奇怪，不适应。你会在人们看着你写代码的时候感到紧张。这很正常！尽管我之前有过公共演说的经历，我一开始的时候还是感到陌生而不适应。我感觉我无处可藏，这令我害怕。我想：“大家可能都觉得我的代码很糟糕，我是一个糟糕的开发者。”这是一个困扰了我 _整个职业生涯_ 的想法，对我来说不新鲜了。我知道带着这些想法，我不能在发布到 GitHub 之前仔细地再检查一遍代码，而这样做更有利于我保持我作为开发者的声誉。 -我从 Twitch 直播中发现了很多关于我代码风格的东西。我知道我的风格绝对是“先让它跑起来，然后再考虑可读性，然后再考虑运行速度”。我不再在前一天晚上提前排练好直播的内容（一开始的三四次直播我都是这么做的），所以我在 Twitch 上写的代码是相当粗糙的，我还得保证它们运行起来没问题。当我不看别人的聊天和讨论的时候，我可以写出我最好的代码，这样是没问题的。但我总会忘记我使用过无数遍的方法的名字，而且每次直播的时候都会犯“愚蠢的”错误。一般来说，这不是一个让你能达到你最好状态的生产环境。 +我从 Twitch 直播中发现了很多关于我代码风格的东西。我知道我的风格绝对是“先让它跑起来，然后再考虑可读性，然后再考虑运行速度”。我不再在前一天晚上提前排练好直播的内容（一开始的三、四次直播我都是这么做的），所以我在 Twitch 上写的代码是相当粗糙的，我还得保证它们运行起来没问题。当我不看别人的聊天和讨论的时候，我可以写出我最好的代码，这样是没问题的。但我总会忘记我使用过无数遍的方法的名字，而且每次直播的时候都会犯“愚蠢的”错误。一般来说，这不是一个让你能达到你最好状态的生产环境。 我的 Twitch 社区从来不会因为这个苛求我，反而是他们帮了我很多。他们理解我正同时做着几件事，而且真的给了很多务实的意见和建议。有时是他们帮我找到了解决方法，有时是我要向他们解释为什么他们的建议不适合解决这个问题。这真的很像一般意义的组队编程！ @@ -128,7 +129,7 @@ 如果你周日想要加入我的直播，你可以 [订阅我的 Twitch 频道][13] :) -最后我想说一下，我个人十分感谢 [Mattias Johansson][14] 在我早期开始直播的时候给我的建议和鼓励。他的 [FunFunFunction YouTube channel][15] 也是一个令人激动的定期直播频道。 +最后我想说一下，我自己十分感谢 [Mattias Johansson][14] 在我早期开始直播的时候给我的建议和鼓励。他的 [FunFunFunction YouTube channel][15] 也是一个令人激动的定期直播频道。 另：许多人问过我的键盘和其他工作设备是什么样的， [这是我使用的器材的完整列表][16] 。感谢关注！ @@ -136,9 +137,9 @@ via: https://medium.freecodecamp.org/lessons-from-my-first-year-of-live-coding-on-twitch-41a32e2f41c1 -作者：[ Suz Hinton][a] +作者：[Suz Hinton][a] 译者：[lonaparte](https://github.com/lonaparte) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 diff --git a/published/20170915 How To Install And Setup Vagrant.md b/published/20170915 How To Install And Setup Vagrant.md new file mode 100644 index 0000000000..92d56ab7b8 --- /dev/null +++ b/published/20170915 How To Install And Setup Vagrant.md @@ -0,0 +1,269 @@ +如何安装并设置 Vagrant +============================= + +Vagrant 对于虚拟机来说是一个强大的工具，在这里我们将研究如何在 Ubuntu 上设置和使用 Virtualbox 和 Vagrant 来提供可复制的虚拟机。 + +### 虚拟机，并不复杂 + +多年来，开发人员一直使用虚拟机作为其工作流程的一部分，允许他们交换和更改运行软件的环境，这通常是为了防止项目之间的冲突，例如需要 php 5.3 的项目 A 和需要 php 5.4 的项目 B。 + +并且使用虚拟机意味着你只需要你正在使用的计算机就行，而不需要专用硬件来镜像你的生产环境。 + +当多个开发人员在一个项目上工作时，它也很方便，他们都可以运行一个包含所有需求的环境，但是维护多台机器并确保所有的需求都具有相同的版本是非常困难的，这时 Vagrant 就能派上用场了。 + +#### 使用虚拟机的好处 + +- 你的虚拟机与主机环境是分开的 +- 你可以根据你代码的要求裁剪一个定制虚拟机 +- 不会影响其他虚拟机 +- 可以运行在你的主机上无法运行的程序，例如在 Ubuntu 中运行一些只能在 Windows 运行的软件 + +### 什么是 Vagrant + +简而言之，这是一个与虚拟机一起工作的工具，可以让你自动创建和删除虚拟机。 + +它围绕一个名为 `VagrantFile` 的配置文件而工作，这个配置文件告诉 Vagrant 你想要安装的操作系统，以及一些其他选项，如 IP 和目录同步。 你还可以在虚拟机上添加一个命令的配置脚本。 + +通过共享这个 `VagrantFile`，项目的所有开发人员全可以使用完全相同的虚拟机。 + +### 安装要求 + +#### 安装 VirtualBox + +VirtualBox 是运行虚拟机的程序，它可以从 Ubuntu 仓库中安装。 + +``` +sudo apt-get install virtualbox +``` + +#### 安装 Vagrant + +对于 Vagrant 本身，你要前往 [https://www.vagrantup.com/downloads.html ](https://www.vagrantup.com/downloads.html) 查看适用于你的操作系统的安装软件包。 + +#### 安装增强功能 + +如果你打算与虚拟机共享任何文件夹，则需要安装以下插件。 + +``` +vagrant plugin install vagrant-vbguest +``` + +### 配置 Vagrant + +首先我们需要为 Vagrant 创建一个文件夹。 + +``` +mkdir ~/Vagrant/test-vm +cd ~/Vagrant/test-vm +``` + +创建 VagrantFile： + +``` +vagrant init +``` + +开启虚拟机： + +``` +vagrant up +``` + +登录机器： + +``` +vagrant-ssh +``` + +此时，你将拥有一个基本的 vagrant 机器，以及一个名为 `VagrantFile` 的文件。 + +### 定制 + +在上面的步骤中创建的 `VagrantFile` 看起来类似于以下内容 + +VagrantFile： + +``` +# -*- mode: ruby -*- +# vi: set ft=ruby : +# All Vagrant configuration is done below. The "2" in Vagrant.configure +# configures the configuration version (we support older styles for +# backwards compatibility). Please don't change it unless you know what +# you're doing. +Vagrant.configure("2") do |config| + # The most common configuration options are documented and commented below. + # For a complete reference, please see the online documentation at + # https://docs.vagrantup.com. + + # Every Vagrant development environment requires a box. You can search for + # boxes at https://vagrantcloud.com/search. + config.vm.box = "base" + + # Disable automatic box update checking. If you disable this, then + # boxes will only be checked for updates when the user runs + # `vagrant box outdated`. This is not recommended. + # config.vm.box_check_update = false + + # Create a forwarded port mapping which allows access to a specific port + # within the machine from a port on the host machine. In the example below, + # accessing "localhost:8080" will access port 80 on the guest machine. + # NOTE: This will enable public access to the opened port + # config.vm.network "forwarded_port", guest: 80, host: 8080 + + # Create a forwarded port mapping which allows access to a specific port + # within the machine from a port on the host machine and only allow access + # via 127.0.0.1 to disable public access + # config.vm.network "forwarded_port", guest: 80, host: 8080, host_ip: "127.0.0.1" + + # Create a private network, which allows host-only access to the machine + # using a specific IP. + # config.vm.network "private_network", ip: "192.168.33.10" + + # Create a public network, which generally matched to bridged network. + # Bridged networks make the machine appear as another physical device on + # your network. + # config.vm.network "public_network" + + # Share an additional folder to the guest VM. The first argument is + # the path on the host to the actual folder. The second argument is + # the path on the guest to mount the folder. And the optional third + # argument is a set of non-required options. + # config.vm.synced_folder "../data", "/vagrant_data" + + # Provider-specific configuration so you can fine-tune various + # backing providers for Vagrant. These expose provider-specific options. + # Example for VirtualBox: + # + # config.vm.provider "virtualbox" do |vb| + # # Display the VirtualBox GUI when booting the machine + # vb.gui = true + # + # # Customize the amount of memory on the VM: + # vb.memory = "1024" + # end + # + # View the documentation for the provider you are using for more + # information on available options. + + # Enable provisioning with a shell script. Additional provisioners such as + # Puppet, Chef, Ansible, Salt, and Docker are also available. Please see the + # documentation for more information about their specific syntax and use. + # config.vm.provision "shell", inline: <<-SHELL + # apt-get update + # apt-get install -y apache2 + # SHELL +end +``` + +现在这个 `VagrantFile` 将创建基本的虚拟机。但 Vagrant 背后的理念是让虚拟机为我们的特定任务而配置，所以我们删除注释和调整配置。 + +VagrantFile： + +``` +# -*- mode: ruby -*- +# vi: set ft=ruby : + +Vagrant.configure("2") do |config| + # Set the Linux Version to Debian Jessie + config.vm.box = "debian/jessie64" + # Set the IP of the Box + config.vm.network "private_network", ip: "192.168.33.10" + # Sync Our Projects Directory with the WWW directory + config.vm.synced_folder "~/Projects", "/var/www/" + # Run the following to Provision + config.vm.provision "shell", path: "install.sh" +end +``` + +现在我们有一个简单的 `VagrantFile`，它将 Linux 版本设置为 debian jessie，设置一个 IP 给我们使用，同步我们感兴趣的文件夹，并最后运行 `install.sh`，这是我们可以运行 shell 命令的地方。 + +install.sh： + +``` +#! /usr/bin/env bash +# Variables +DBHOST=localhost +DBNAME=dbname +DBUSER=dbuser +DBPASSWD=test123 + +echo "[ Provisioning machine ]" +echo "1) Update APT..." +apt-get -qq update + +echo "1) Install Utilities..." +apt-get install -y tidy pdftk curl xpdf imagemagick openssl vim git + +echo "2) Installing Apache..." +apt-get install -y apache2 + +echo "3) Installing PHP and packages..." +apt-get install -y php5 libapache2-mod-php5 libssh2-php php-pear php5-cli php5-common php5-curl php5-dev php5-gd php5-imagick php5-imap php5-intl php5-mcrypt php5-memcached php5-mysql php5-pspell php5-xdebug php5-xmlrpc +#php5-suhosin-extension, php5-mysqlnd + +echo "4) Installing MySQL..." +debconf-set-selections <<< "mysql-server mysql-server/root_password password secret" +debconf-set-selections <<< "mysql-server mysql-server/root_password_again password secret" +apt-get install -y mysql-server +mysql -uroot -p$DBPASSWD -e "CREATE DATABASE $DBNAME" +mysql -uroot -p$DBPASSWD -e "grant all privileges on $DBNAME.* to '$DBUSER'@'localhost' identified by '$DBPASSWD'" + +echo "5) Generating self signed certificate..." +mkdir -p /etc/ssl/localcerts +openssl req -new -x509 -days 365 -nodes -subj "/C=US/ST=Denial/L=Springfield/O=Dis/CN=www.example.com" -out /etc/ssl/localcerts/apache.pem -keyout /etc/ssl/localcerts/apache.key +chmod 600 /etc/ssl/localcerts/apache* + +echo "6) Setup Apache..." +a2enmod rewrite +> /etc/apache2/sites-enabled/000-default.conf +echo " + + ServerAdmin webmaster@localhost + DocumentRoot /var/www/ + ErrorLog ${APACHE_LOG_DIR}/error.log + CustomLog ${APACHE_LOG_DIR}/access.log combined + + +" >> /etc/apache2/sites-enabled/000-default.conf +service apache2 restart + +echo "7) Composer Install..." +curl --silent https://getcomposer.org/installer | php +mv composer.phar /usr/local/bin/composer + +echo "8) Install NodeJS..." +curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash - +apt-get -qq update +apt-get -y install nodejs + +echo "9) Install NPM Packages..." +npm install -g gulp gulp-cli + +echo "Provisioning Completed" +``` + +通过上面的步骤，在你的目录中会有 `VagrantFile` 和 `install.sh`，运行 vagrant 会做下面的事情： + +- 采用 Debian Jessie 来创建虚拟机 +- 将机器的 IP 设置为 192.168.33.10 +- 同步 `~/Projects` 和 `/var/www/` 目录 +- 安装并设置 Apache、Mysql、PHP、Git、Vim +- 安装并运行 Composer +- 安装 Nodejs 和 gulp +- 创建一个 MySQL 数据库 +- 创建自签名证书 + +通过与其他人共享 `VagrantFile` 和 `install.sh`，你可以在两台不同的机器上使用完全相同的环境。 + +-------------------------------------------------------------------------------- + +via: https://www.chris-shaw.com/blog/how-to-install-and-setup-vagrant + +作者：[Christopher Shaw][a] +译者：[MjSeven](https://github.com/MjSeven) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.chris-shaw.com +[1]:/cdn-cgi/l/email-protection diff --git a/published/20171005 Reasons Kubernetes is cool.md b/published/20171005 Reasons Kubernetes is cool.md new file mode 100644 index 0000000000..15f17d827f --- /dev/null +++ b/published/20171005 Reasons Kubernetes is cool.md @@ -0,0 +1,134 @@ +为什么 Kubernetes 很酷 +============================================================ + +在我刚开始学习 Kubernetes（大约是一年半以前吧？）时，我真的不明白为什么应该去关注它。 + +在我使用 Kubernetes 全职工作了三个多月后，我才逐渐明白了为什么我应该使用它。（我距离成为一个 Kubernetes 专家还很远！）希望这篇文章对你理解 Kubernetes 能做什么会有帮助！ + +我将尝试去解释我对 Kubernetes 感兴趣的一些原因，而不去使用 “原生云cloud native”、“编排系统orchestration”、“容器container”，或者任何 Kubernetes 专用的术语 :)。我去解释的这些观点主要来自一位 Kubernetes 操作者/基础设施工程师，因为，我现在的工作就是去配置 Kubernetes 和让它工作的更好。 + +我不会去尝试解决一些如 “你应该在你的生产系统中使用 Kubernetes 吗？”这样的问题。那是非常复杂的问题。（不仅是因为“生产系统”根据你的用途而总是有不同的要求） + +### Kubernetes 可以让你无需设置一台新的服务器即可在生产系统中运行代码 + +我首次被说教使用 Kubernetes 是与我的伙伴 Kamal 的下面的谈话： + +大致是这样的： + +* Kamal： 使用 Kubernetes 你可以通过一条命令就能设置一台新的服务器。 +* Julia： 我觉得不太可能吧。 +* Kamal： 像这样，你写一个配置文件，然后应用它，这时候，你就在生产系统中运行了一个 HTTP 服务。 +* Julia： 但是，现在我需要去创建一个新的 AWS 实例，明确地写一个 Puppet 清单，设置服务发现，配置负载均衡，配置我们的部署软件，并且确保 DNS 正常工作，如果没有什么问题的话，至少在 4 小时后才能投入使用。 +* Kamal: 是的，使用 Kubernetes 你不需要做那么多事情，你可以在 5 分钟内设置一台新的 HTTP 服务，并且它将自动运行。只要你的集群中有空闲的资源它就能正常工作！ +* Julia: 这儿一定是一个“坑”。 + +这里有一种陷阱，设置一个生产用 Kubernetes 集群（在我的经险中）确实并不容易。（查看 [Kubernetes 艰难之旅][3] 中去开始使用时有哪些复杂的东西）但是，我们现在并不深入讨论它。 + +因此，Kubernetes 第一个很酷的事情是，它可能使那些想在生产系统中部署新开发的软件的方式变得更容易。那是很酷的事，而且它真的是这样，因此，一旦你使用一个运作中的 Kubernetes 集群，你真的可以仅使用一个配置文件就在生产系统中设置一台 HTTP 服务（在 5 分钟内运行这个应用程序，设置一个负载均衡，给它一个 DNS 名字，等等）。看起来真的很有趣。 + +### 对于运行在生产系统中的代码，Kubernetes 可以提供更好的可见性和可管理性 + +在我看来，在理解 etcd 之前，你可能不会理解 Kubernetes 的。因此，让我们先讨论 etcd！ + +想像一下，如果现在我这样问你，“告诉我你运行在生产系统中的每个应用程序，它运行在哪台主机上？它是否状态很好？是否为它分配了一个 DNS 名字？”我并不知道这些，但是，我可能需要到很多不同的地方去查询来回答这些问题，并且，我需要花很长的时间才能搞定。我现在可以很确定地说不需要查询，仅一个 API 就可以搞定它们。 + +在 Kubernetes 中，你的集群的所有状态 – 运行中的应用程序 (“pod”)、节点、DNS 名字、 cron 任务、 等等 —— 都保存在一个单一的数据库中（etcd）。每个 Kubernetes 组件是无状态的，并且基本是通过下列方式工作的： + +* 从 etcd 中读取状态（比如，“分配给节点 1 的 pod 列表”） +* 产生变化（比如，“在节点 1 上运行 pod A”） +* 更新 etcd 中的状态（比如，“设置 pod A 的状态为 ‘running’”） + +这意味着，如果你想去回答诸如 “在那个可用区中有多少台运行着 nginx 的 pod？” 这样的问题时，你可以通过查询一个统一的 API（Kubernetes API）去回答它。并且，你可以在每个其它 Kubernetes 组件上运行那个 API 去进行同样的访问。 + +这也意味着，你可以很容易地去管理每个运行在 Kubernetes 中的任何东西。比如说，如果你想要： + +* 部署实现一个复杂的定制的部署策略（部署一个东西，等待 2 分钟，部署 5 个以上，等待 3.7 分钟，等等） +* 每当推送到 github 上一个分支，自动化 [启动一个新的 web 服务器][1] +* 监视所有你的运行的应用程序，确保它们有一个合理的内存使用限制。 + +这些你只需要写一个程序与 Kubernetes API（“controller”）通讯就可以了。 + +另一个关于 Kubernetes API 的令人激动的事情是，你不会局限于 Kubernetes 所提供的现有功能！如果对于你要部署/创建/监视的软件有你自己的方案，那么，你可以使用 Kubernetes API 去写一些代码去达到你的目的！它可以让你做到你想做的任何事情。 + +### 即便每个 Kubernetes 组件都“挂了”，你的代码将仍然保持运行 + +关于 Kubernetes 我（在各种博客文章中 :)）承诺的一件事情是，“如果 Kubernetes API 服务和其它组件‘挂了’也没事，你的代码将一直保持运行状态”。我认为理论上这听起来很酷，但是我不确定它是否真是这样的。 + +到目前为止，这似乎是真的！ + +我已经断开了一些正在运行的 etcd，发生了这些情况： + +1. 所有的代码继续保持运行状态 +2. 不能做 _新的_ 事情（你不能部署新的代码或者生成变更，cron 作业将停止工作） +3. 当它恢复时，集群将赶上这期间它错过的内容 + +这样做意味着如果 etcd 宕掉，并且你的应用程序的其中之一崩溃或者发生其它事情，在 etcd 恢复之前，它不能够恢复。 + +### Kubernetes 的设计对 bug 很有弹性 + +与任何软件一样，Kubernetes 也会有 bug。例如，到目前为止，我们的集群控制管理器有内存泄漏，并且，调度器经常崩溃。bug 当然不好，但是，我发现 Kubernetes 的设计可以帮助减轻它的许多核心组件中的错误的影响。 + +如果你重启动任何组件，将会发生： + +* 从 etcd 中读取所有的与它相关的状态 +* 基于那些状态（调度 pod、回收完成的 pod、调度 cron 作业、按需部署等等），它会去做那些它认为必须要做的事情 + +因为，所有的组件并不会在内存中保持状态，你在任何时候都可以重启它们，这可以帮助你减轻各种 bug 的影响。 + +例如，如果在你的控制管理器中有内存泄露。因为，控制管理器是无状态的，你可以每小时定期去重启它，或者，在感觉到可能导致任何不一致的问题发生时重启它。又或者，在调度器中遇到了一个 bug，它有时忘记了某个 pod，从来不去调度它们。你可以每隔 10 分钟来重启调度器来缓减这种情况。（我们并不会这么做，而是去修复这个 bug，但是，你_可以这样做_ :）） + +因此，我觉得即使在它的核心组件中有 bug，我仍然可以信任 Kubernetes 的设计可以让我确保集群状态的一致性。并且，总在来说，随着时间的推移软件质量会提高。唯一你必须去操作的有状态的东西就是 etcd。 + +不用过多地讨论“状态”这个东西 —— 而我认为在 Kubernetes 中很酷的一件事情是，唯一需要去做备份/恢复计划的东西是 etcd （除非为你的 pod 使用了持久化存储的卷）。我认为这样可以使 Kubernetes 运维比你想的更容易一些。 + +### 在 Kubernetes 之上实现新的分布式系统是非常容易的 + +假设你想去实现一个分布式 cron 作业调度系统！从零开始做工作量非常大。但是，在 Kubernetes 里面实现一个分布式 cron 作业调度系统是非常容易的！（仍然没那么简单，毕竟它是一个分布式系统） + +我第一次读到 Kubernetes 的 cron 作业控制器的代码时，我对它是如此的简单感到由衷高兴。去读读看，其主要的逻辑大约是 400 行的 Go 代码。去读它吧！ => [cronjob_controller.go][4] <= + +cron 作业控制器基本上做的是： + +* 每 10 秒钟： + * 列出所有已存在的 cron 作业 + * 检查是否有需要现在去运行的任务 + * 如果有，创建一个新的作业对象去调度，并通过其它的 Kubernetes 控制器实际运行它 + * 清理已完成的作业 + * 重复以上工作 + +Kubernetes 模型是很受限制的（它有定义在 etcd 中的资源模式，控制器读取这个资源并更新 etcd），我认为这种相关的固有的/受限制的模型，可以使它更容易地在 Kubernetes 框架中开发你自己的分布式系统。 + +Kamal 给我说的是 “Kubernetes 是一个写你自己的分布式系统的很好的平台” ，而不是“ Kubernetes 是一个你可以使用的分布式系统”，并且，我觉得它真的很有意思。他做了一个 [为你推送到 GitHub 的每个分支运行一个 HTTP 服务的系统][5] 的原型。这花了他一个周末的时间，大约 800 行 Go 代码，我认为它真不可思议！ + +### Kubernetes 可以使你做一些非常神奇的事情（但并不容易） + +我一开始就说 “kubernetes 可以让你做一些很神奇的事情，你可以用一个配置文件来做这么多的基础设施，它太神奇了”。这是真的！ + +为什么说 “Kubernetes 并不容易”呢？是因为 Kubernetes 有很多部分，学习怎么去成功地运营一个高可用的 Kubernetes 集群要做很多的工作。就像我发现它给我了许多抽象的东西，我需要去理解这些抽象的东西才能调试问题和正确地配置它们。我喜欢学习新东西，因此，它并不会使我发狂或者生气，但是我认为了解这一点很重要 :） + +对于 “我不能仅依靠抽象概念” 的一个具体的例子是，我努力学习了许多 [Linux 上网络是如何工作的][6]，才让我对设置 Kubernetes 网络稍有信心，这比我以前学过的关于网络的知识要多很多。这种方式很有意思但是非常费时间。在以后的某个时间，我或许写更多的关于设置 Kubernetes 网络的困难/有趣的事情。 + +或者，为了成功设置我的 Kubernetes CA，我写了一篇 [2000 字的博客文章][7]，述及了我不得不学习 Kubernetes 不同方式的 CA 的各种细节。 + +我觉得，像 GKE （Google 的 Kubernetes 产品) 这样的一些监管的 Kubernetes 的系统可能更简单，因为，他们为你做了许多的决定，但是，我没有尝试过它们。 + +-------------------------------------------------------------------------------- + +via: https://jvns.ca/blog/2017/10/05/reasons-kubernetes-is-cool/ + +作者：[Julia Evans][a] +译者：[qhwdw](https://github.com/qhwdw) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://jvns.ca/about +[1]:https://github.com/kamalmarhubi/kubereview +[2]:https://jvns.ca/categories/kubernetes +[3]:https://github.com/kelseyhightower/kubernetes-the-hard-way +[4]:https://github.com/kubernetes/kubernetes/blob/e4551d50e57c089aab6f67333412d3ca64bc09ae/pkg/controller/cronjob/cronjob_controller.go +[5]:https://github.com/kamalmarhubi/kubereview +[6]:https://jvns.ca/blog/2016/12/22/container-networking/ +[7]:https://jvns.ca/blog/2017/08/05/how-kubernetes-certificates-work/ + + diff --git a/published/20171103 How To Fully Update And Upgrade Offline Debian-based Systems.md b/published/20171103 How To Fully Update And Upgrade Offline Debian-based Systems.md new file mode 100644 index 0000000000..e5d9adc1e6 --- /dev/null +++ b/published/20171103 How To Fully Update And Upgrade Offline Debian-based Systems.md @@ -0,0 +1,127 @@ +怎样完整地离线更新并升级基于 Debian 的操作系统 +====== + +![](https://www.ostechnix.com/wp-content/uploads/2017/11/Upgrade-Offline-Debian-based-Systems-2-720x340.png) + +不久之前我已经向你展示了如何在任意离线的 [Ubuntu][1] 和 [Arch Linux][2] 操作系统上安装软件。 今天，我们将会看看如何完整地离线更新并升级基于 Debian 的操作系统。 和之前所述方法的不同之处在于，这次我们将会升级整个操作系统，而不是单个的软件包。这个方法在你没有网络链接或拥有的网络速度很慢的时候十分有用。 + +### 完整地离线更新并升级基于 Debian 的操作系统 + +首先假设，你在单位拥有正在运行并配置有高速互联网链接的系统（Windows 或者 Linux），而在家有一个没有网络链接或网络很慢（例如拨号网络）的 Debian 或其衍生的操作系统。现在如果你想要离线更新你家里的操作系统怎么办？购买一个更加高速的网络链接？不，根本不需要！你仍然可以通过互联网离线更新升级你的操作系统。这正是 **Apt-Offline**工具可以帮助你做到的。 + +正如其名，apt-offline 是一个为 Debian 及其衍生发行版（诸如 Ubuntu、Linux Mint 这样基于 APT 的操作系统）提供的离线 APT 包管理器。使用 apt-offline，我们可以完整地更新/升级我们的 Debian 系统而不需要网络链接。这个程序是由 Python 编程语言写成的兼具 CLI 和图形界面的跨平台工具。 + +#### 准备工作 + +* 一个已经联网的操作系统（Windows 或者 Linux）。在这份指南中，为了便于理解，我们将之称为在线操作系统。 +* 一个离线操作系统（Debian 及其衍生版本）。我们称之为离线操作系统。 +* 有足够空间容纳所有更新包的 USB 驱动器或者外接硬盘。 + +#### 安装 + +Apt-Offline 可以在 Debian 及其衍生版本的默认仓库中获得。如果你的在线操作系统是运行的 Debian、Ubuntu、Linux Mint，及其它基于 DEB 的操作系统，你可以通过下面的命令安装 Apt-Offline： + +``` +sudo apt-get install apt-offline +``` + +如果你的在线操作系统运行的是非 Debian 类的发行版，使用 `git clone` 获取 Apt-Offline 仓库： + +``` +git clone https://github.com/rickysarraf/apt-offline.git +``` + +切换到克隆的目录下并在此处运行： + +``` +cd apt-offline/ +sudo ./apt-offline +``` + +#### 在离线操作系统（没有联网的操作系统）上的步骤 + +到你的离线操作系统上创建一个你想存储签名文件的目录： + +``` +mkdir ~/tmp +cd ~/tmp/ +``` + +你可以自己选择使用任何目录。接下来，运行下面的命令生成签名文件： + +``` +sudo apt-offline set apt-offline.sig +``` + +示例输出如下： + +``` +Generating database of files that are needed for an update. +Generating database of file that are needed for operation upgrade +``` + +默认条件下，apt-offline 将会生成需要更新和升级的相关文件的数据库。你可以使用 `--update` 或者 `--upgrade` 选项相应创建。 + +拷贝完整的 `tmp` 目录到你的 USB 驱动器或者或者外接硬盘上，然后换到你的在线操作系统（有网络链接的操作系统）。 + +#### 在在线操作系统上的步骤 + +插入你的 USB 驱动器然后进入 `tmp` 文件夹： + +``` +cd tmp/ +``` + +然后，运行如下命令： + +``` +sudo apt-offline get apt-offline.sig --threads 5 --bundle apt-offline-bundle.zip +``` + +在这里的 `-threads 5` 代表着（并发连接的） APT 仓库的数目。如果你想要从更多的仓库下载软件包，你可以增加这里的数值。然后 `-bundle apt-offline-bundle.zip` 选项表示所有的软件包将会打包到一个叫做 `apt-offline-bundle.zip` 的单独存档中。这个存档文件将会被保存在你的当前工作目录中（LCTT 译注：即 `tmp` 目录）。 + +上面的命令将会按照之前在离线操作系统上生成的签名文件下载数据。 + +![][4] + +根据你的网络状况，这个操作将会花费几分钟左右的时间。请记住，apt-offline 是跨平台的，所以你可以在任何操作系统上使用它下载包。 + +一旦下载完成，拷贝 `tmp` 文件夹到你的 USB 或者外接硬盘上并且返回你的离线操作系统（LCTT 译注：此处的复制操作似不必要，因为我们一直在 USB 存储器的 `tmp` 目录中操作）。千万保证你的 USB 驱动器上有足够的空闲空间存储所有的下载文件，因为所有的包都放在 `tmp` 文件夹里了。 + +#### 离线操作系统上的步骤 + +把你的设备插入你的离线操作系统，然后切换到你之前下载了所有包的 `tmp`目录下。 + +``` +cd tmp +``` + +然后，运行下面的命令来安装所有下载好的包。 + +``` +sudo apt-offline install apt-offline-bundle.zip +``` + +这个命令将会更新 APT 数据库，所以 APT 将会在 APT 缓冲里找所有需要的包。 + +**注意事项：** 如果在线和离线操作系统都在同一个局域网中，你可以通过 `scp` 或者其他传输应用程序将 `tmp` 文件传到离线操作系统中。如果两个操作系统在不同的位置（LCTT 译注：意指在不同的局域网），那就使用 USB 设备来拷贝。 + +好了大伙儿，现在就这么多了。 希望这篇指南对你有用。还有更多好东西正在路上。敬请关注！ + +祝你愉快！ + +-------------------------------------------------------------------------------- + +via: https://www.ostechnix.com/fully-update-upgrade-offline-debian-based-systems/ + +作者：[SK][a] +译者：[leemeans](https://github.com/leemeans) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.ostechnix.com/author/sk/ +[1]:https://www.ostechnix.com/install-softwares-offline-ubuntu-16-04/ +[2]:https://www.ostechnix.com/install-packages-offline-arch-linux/ +[3]:data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7 +[4]:http://www.ostechnix.com/wp-content/uploads/2017/11/apt-offline.png diff --git a/translated/tech/20171112 Step by Step guide for creating Master Slave replication in MariaDB.md b/published/20171112 Step by Step guide for creating Master Slave replication in MariaDB.md similarity index 59% rename from translated/tech/20171112 Step by Step guide for creating Master Slave replication in MariaDB.md rename to published/20171112 Step by Step guide for creating Master Slave replication in MariaDB.md index 397843785e..e00a2615a6 100644 --- a/translated/tech/20171112 Step by Step guide for creating Master Slave replication in MariaDB.md +++ b/published/20171112 Step by Step guide for creating Master Slave replication in MariaDB.md @@ -1,27 +1,26 @@ -一步一步学习如何在 MariaDB 中配置主从复制 +循序渐进学习如何在 MariaDB 中配置主从复制 ====== -在我们前面的教程中，我们已经学习了 [**如何安装和配置 MariaDB**][1]，也学习了 [**管理 MariaDB 的一些基础命令**][2]。现在我们来学习，如何在 MariaDB 服务器上配置一个主从复制。 -复制是用于为我们的数据库去创建多个副本，这些副本可以在其它数据库上用于运行查询，像一些非常繁重的查询可能会影响主数据库服务器的性能，或者我们可以使用它来做数据冗余，或者兼具以上两个目的。我们可以将这个过程自动化，即主服务器到从服务器的复制过程自动进行。执行备份而不影响在主服务器上的写操作。 +在我们前面的教程中，我们已经学习了 [如何安装和配置 MariaDB][1]，也学习了 [管理 MariaDB 的一些基础命令][2]。现在我们来学习，如何在 MariaDB 服务器上配置一个主从复制。 + +复制是用于为我们的数据库创 建多个副本，这些副本可以在其它数据库上用于运行查询，像一些非常繁重的查询可能会影响主数据库服务器的性能，或者我们可以使用它来做数据冗余，或者兼具以上两个目的。我们可以将这个过程自动化，即主服务器到从服务器的复制过程自动进行。执行备份而不影响在主服务器上的写操作。 因此，我们现在去配置我们的主-从复制，它需要两台安装了 MariaDB 的机器。它们的 IP 地址如下： - **主服务器 -** 192.168.1.120 **主机名** master.ltechlab.com +- **主服务器 -** 192.168.1.120 **主机名 -** master.ltechlab.com +- **从服务器 -** 192.168.1.130 **主机名 -** slave.ltechlab.com - **从服务器 -** 192.168.1.130 **主机名 -** slave.ltechlab.com +MariaDB 安装到这些机器上之后，我们继续进行本教程。如果你需要安装和配置 MariaDB 的教程，请查看[**这个教程**][1]。 -MariaDB 安装到这些机器上之后，我们继续进行本教程。如果你需要安装和配置 MariaDB 的教程，请查看[ **这个教程**][1]。 +### 第 1 步 - 主服务器配置 - -### **第 1 步 - 主服务器配置** - -我们现在进入到 MariaDB 中的一个命名为 ' **important '** 的数据库，它将被复制到我们的从服务器。为开始这个过程，我们编辑名为 ' **/etc/my.cnf** ' 的文件，它是 MariaDB 的配置文件。 +我们现在进入到 MariaDB 中的一个命名为 `important` 的数据库，它将被复制到我们的从服务器。为开始这个过程，我们编辑名为 `/etc/my.cnf` 的文件，它是 MariaDB 的配置文件。 ``` $ vi /etc/my.cnf ``` -在这个文件中找到 [mysqld] 节，然后输入如下内容： +在这个文件中找到 `[mysqld]` 节，然后输入如下内容： ``` [mysqld] @@ -43,7 +42,7 @@ $ systemctl restart mariadb $ mysql -u root -p ``` -在它上面创建一个命名为 'slaveuser' 的为主从复制使用的新用户，然后运行如下的命令为它分配所需要的权限： +在它上面创建一个命名为 `slaveuser` 的为主从复制使用的新用户，然后运行如下的命令为它分配所需要的权限： ``` STOP SLAVE; @@ -53,19 +52,19 @@ FLUSH TABLES WITH READ LOCK; SHOW MASTER STATUS; ``` -**注意： ** 我们配置主从复制需要 **MASTER_LOG_FILE 和 MASTER_LOG_POS ** 的值，它可以通过 'show master status' 来获得，因此，你一定要确保你记下了它们的值。 +**注意：** 我们配置主从复制需要 `MASTER_LOG_FILE` 和 `MASTER_LOG_POS` 的值，它可以通过 `show master status` 来获得，因此，你一定要确保你记下了它们的值。 -这些命令运行完成之后，输入 'exit' 退出这个会话。 +这些命令运行完成之后，输入 `exit` 退出这个会话。 ### 第 2 步 - 创建一个数据库备份，并将它移动到从服务器上 -现在，我们需要去为我们的数据库 'important' 创建一个备份，可以使用 'mysqldump' 命令去备份。 +现在，我们需要去为我们的数据库 `important` 创建一个备份，可以使用 `mysqldump` 命令去备份。 ``` $ mysqldump -u root -p important > important_backup.sql ``` -备份完成后，我们需要重新登陆到 MariaDB 数据库，并解锁我们的表。 +备份完成后，我们需要重新登录到 MariaDB 数据库，并解锁我们的表。 ``` $ mysql -u root -p @@ -78,7 +77,7 @@ $ UNLOCK TABLES; ### 第 3 步：配置从服务器 -我们再次去编辑 '/etc/my.cnf' 文件，找到配置文件中的 [mysqld] 节，然后输入如下内容： +我们再次去编辑（从服务器上的） `/etc/my.cnf` 文件，找到配置文件中的 `[mysqld]` 节，然后输入如下内容： ``` [mysqld] @@ -93,7 +92,7 @@ replicate-do-db=important $ mysql -u root -p < /data/ important_backup.sql ``` -当这个恢复过程结束之后，我们将通过登入到从服务器上的 MariaDB，为数据库 'important' 上的用户 'slaveuser' 授权。 +当这个恢复过程结束之后，我们将通过登入到从服务器上的 MariaDB，为数据库 `important` 上的用户 'slaveuser' 授权。 ``` $ mysql -u root -p @@ -110,9 +109,9 @@ FLUSH PRIVILEGES; $ systemctl restart mariadb ``` -### **第 4 步：启动复制** +### 第 4 步：启动复制 -记住，我们需要 **MASTER_LOG_FILE 和 MASTER_LOG_POS** 变量的值，它可以通过在主服务器上运行 'SHOW MASTER STATUS' 获得。现在登入到从服务器上的 MariaDB，然后通过运行下列命令，告诉我们的从服务器它应该去哪里找主服务器。 +记住，我们需要 `MASTER_LOG_FILE` 和 `MASTER_LOG_POS` 变量的值，它可以通过在主服务器上运行 `SHOW MASTER STATUS` 获得。现在登入到从服务器上的 MariaDB，然后通过运行下列命令，告诉我们的从服务器它应该去哪里找主服务器。 ``` STOP SLAVE; @@ -131,13 +130,13 @@ SHOW SLAVE STATUS\G; $ mysql -u root -p ``` -选择数据库为 'important'： +选择数据库为 `important`： ``` use important; ``` -在这个数据库上创建一个名为 ‘test’ 的表： +在这个数据库上创建一个名为 `test` 的表： ``` create table test (c int); @@ -175,10 +174,10 @@ via: http://linuxtechlab.com/creating-master-slave-replication-mariadb/ 作者：[Shusain][a] 译者：[qhwdw](https://github.com/qhwdw) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 [a]:http://linuxtechlab.com/author/shsuain/ -[1]:http://linuxtechlab.com/installing-configuring-mariadb-rhelcentos/ -[2]:http://linuxtechlab.com/mariadb-administration-commands-beginners/ +[1]:https://linux.cn/article-8320-1.html +[2]:https://linux.cn/article-9306-1.html diff --git a/published/20171127 Protecting Your Website From Application Layer DOS Attacks With mod.md b/published/20171127 Protecting Your Website From Application Layer DOS Attacks With mod.md new file mode 100644 index 0000000000..869e02b6c7 --- /dev/null +++ b/published/20171127 Protecting Your Website From Application Layer DOS Attacks With mod.md @@ -0,0 +1,212 @@ +用 Apache 服务器模块保护您的网站免受应用层 DOS 攻击 +====== + +有多种可以导致网站下线的攻击方法，比较复杂的方法要涉及数据库和编程方面的技术知识。一个更简单的方法被称为“拒绝服务Denial Of Service”（DOS）攻击。这个攻击方法的名字来源于它的意图：使普通客户或网站访问者的正常服务请求被拒绝。 + +一般来说，有两种形式的 DOS 攻击： + +1. OSI 模型的三、四层，即网络层攻击 +2. OSI 模型的七层，即应用层攻击 + +第一种类型的 DOS 攻击——网络层，发生于当大量的垃圾流量流向网页服务器时。当垃圾流量超过网络的处理能力时，网站就会宕机。 + +第二种类型的 DOS 攻击是在应用层，是利用合法的服务请求，而不是垃圾流量。当页面请求数量超过网页服务器能承受的容量时，即使是合法访问者也将无法使用该网站。 + +本文将着眼于缓解应用层攻击，因为减轻网络层攻击需要大量的可用带宽和上游提供商的合作，这通常不是通过配置网络服务器就可以做到的。 + +通过配置普通的网页服务器，可以保护网页免受应用层攻击，至少是适度的防护。防止这种形式的攻击是非常重要的，因为 [Cloudflare][1] 最近 [报告称][2] 网络层攻击的数量正在减少，而应用层攻击的数量则在增加。 + +本文将介绍如何使用 [zdziarski][4] 开发的 Apache2 的模块 [mod_evasive][3]。 + +另外，mod_evasive 会阻止攻击者通过尝试数百个用户名和密码的组合来进行猜测（即暴力攻击）的企图。 + +mod_evasive 会记录来自每个 IP 地址的请求的数量。当这个数字超过相应 IP 地址的几个阈值之一时，会出现一个错误页面。错误页面所需的资源要比一个能够响应合法访问的在线网站少得多。 + +### 在 Ubuntu 16.04 上安装 mod_evasive + +Ubuntu 16.04 默认的软件库中包含了 mod_evasive，名称为 “libapache2-mod-evasive”。您可以使用 `apt-get` 来完成安装： + +``` +apt-get update +apt-get upgrade +apt-get install libapache2-mod-evasive +``` + +现在我们需要配置 mod_evasive。 + +它的配置文件位于 `/etc/apache2/mods-available/evasive.conf`。默认情况下，所有模块的设置在安装后都会被注释掉。因此，在修改配置文件之前，模块不会干扰到网站流量。 + +``` + + #DOSHashTableSize 3097 + #DOSPageCount 2 + #DOSSiteCount 50 + #DOSPageInterval 1 + #DOSSiteInterval 1 + #DOSBlockingPeriod 10 + + #DOSEmailNotify you@yourdomain.com + #DOSSystemCommand "su - someuser -c '/sbin/... %s ...'" + #DOSLogDir "/var/log/mod_evasive" + +``` + +第一部分的参数的含义如下： + + * `DOSHashTableSize` - 正在访问网站的 IP 地址列表及其请求数的当前列表。 + * `DOSPageCount` - 在一定的时间间隔内，每个页面的请求次数。时间间隔由 DOSPageInterval 定义。 + * `DOSPageInterval` - mod_evasive 统计页面请求次数的时间间隔。 + * `DOSSiteCount` - 与 `DOSPageCount` 相同，但统计的是来自相同 IP 地址对网站内任何页面的请求数量。 + * `DOSSiteInterval` - mod_evasive 统计网站请求次数的时间间隔。 + * `DOSBlockingPeriod` - 某个 IP 地址被加入黑名单的时长（以秒为单位）。 + +如果使用上面显示的默认配置，则在如下情况下，一个 IP 地址会被加入黑名单： + + * 每秒请求同一页面超过两次。 + * 每秒请求 50 个以上不同页面。 + +如果某个 IP 地址超过了这些阈值，则被加入黑名单 10 秒钟。 + +这看起来可能不算久，但是，mod_evasive 将一直监视页面请求，包括在黑名单中的 IP 地址，并重置其加入黑名单的起始时间。只要一个 IP 地址一直尝试使用 DOS 攻击该网站，它将始终在黑名单中。 + +其余的参数是： + + * `DOSEmailNotify` - 用于接收 DOS 攻击信息和 IP 地址黑名单的电子邮件地址。 + * `DOSSystemCommand` - 检测到 DOS 攻击时运行的命令。 + * `DOSLogDir` - 用于存放 mod_evasive 的临时文件的目录。 + +### 配置 mod_evasive + +默认的配置是一个很好的开始，因为它不会阻塞任何合法的用户。取消配置文件中的所有参数（`DOSSystemCommand` 除外）的注释，如下所示： + +``` + + DOSHashTableSize 3097 + DOSPageCount 2 + DOSSiteCount 50 + DOSPageInterval 1 + DOSSiteInterval 1 + DOSBlockingPeriod 10 + + DOSEmailNotify JohnW@example.com + #DOSSystemCommand "su - someuser -c '/sbin/... %s ...'" + DOSLogDir "/var/log/mod_evasive" + +``` + +必须要创建日志目录并且要赋予其与 apache 进程相同的所有者。这里创建的目录是 `/var/log/mod_evasive` ，并且在 Ubuntu 上将该目录的所有者和组设置为 `www-data` ，与 Apache 服务器相同： + +``` +mkdir /var/log/mod_evasive +chown www-data:www-data /var/log/mod_evasive +``` + +在编辑了 Apache 的配置之后，特别是在正在运行的网站上，在重新启动或重新加载之前，最好检查一下语法，因为语法错误将影响 Apache 的启动从而使网站宕机。 + +Apache 包含一个辅助命令，是一个配置语法检查器。只需运行以下命令来检查您的语法： + +``` +apachectl configtest +``` + +如果您的配置是正确的，会得到如下结果： + +``` +Syntax OK +``` + +但是，如果出现问题，您会被告知在哪部分发生了什么错误，例如： + +``` +AH00526: Syntax error on line 6 of /etc/apache2/mods-enabled/evasive.conf: +DOSSiteInterval takes one argument, Set site interval +Action 'configtest' failed. +The Apache error log may have more information. +``` + +如果您的配置通过了 configtest 的测试，那么这个模块可以安全地被启用并且 Apache 可以重新加载： + +``` +a2enmod evasive +systemctl reload apache2.service +``` + +mod_evasive 现在已配置好并正在运行了。 + +### 测试 + +为了测试 mod_evasive，我们只需要向服务器提出足够的网页访问请求，以使其超出阈值，并记录来自 Apache 的响应代码。 + +一个正常并成功的页面请求将收到如下响应： + +``` +HTTP/1.1 200 OK +``` + +但是，被 mod_evasive 拒绝的将返回以下内容： + +``` +HTTP/1.1 403 Forbidden +``` + +以下脚本会尽可能迅速地向本地主机（127.0.0.1，localhost）的 80 端口发送 HTTP 请求，并打印出每个请求的响应代码。 + +你所要做的就是把下面的 bash 脚本复制到一个文件中，例如 `mod_evasive_test.sh`： + +``` +#!/bin/bash +set -e + +for i in {1..50}; do + curl -s -I 127.0.0.1 | head -n 1 +done +``` + +这个脚本的部分含义如下： + + * `curl` - 这是一个发出网络请求的命令。 + * `-s` - 隐藏进度表。 + * `-I` - 仅显示响应头部信息。 + * `head` - 打印文件的第一部分。 + * `-n 1` - 只显示第一行。 + +然后赋予其执行权限： + +``` +chmod 755 mod_evasive_test.sh +``` + +在启用 mod_evasive **之前**，脚本运行时，将会看到 50 行 “HTTP / 1.1 200 OK” 的返回值。 + +但是，启用 mod_evasive 后，您将看到以下内容： + +``` +HTTP/1.1 200 OK +HTTP/1.1 200 OK +HTTP/1.1 403 Forbidden +HTTP/1.1 403 Forbidden +HTTP/1.1 403 Forbidden +HTTP/1.1 403 Forbidden +HTTP/1.1 403 Forbidden +... +``` + +前两个请求被允许，但是在同一秒内第三个请求发出时，mod_evasive 拒绝了任何进一步的请求。您还将收到一封电子邮件（邮件地址在选项 `DOSEmailNotify` 中设置），通知您有 DOS 攻击被检测到。 + +mod_evasive 现在已经在保护您的网站啦！ + +-------------------------------------------------------------------------------- + +via: https://bash-prompt.net/guides/mod_proxy/ + +作者：[Elliot Cooper][a] +译者：[jessie-pang](https://github.com/jessie-pang) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://bash-prompt.net/about/ +[1]:https://www.cloudflare.com +[2]:https://blog.cloudflare.com/the-new-ddos-landscape/ +[3]:https://github.com/jzdziarski/mod_evasive +[4]:https://www.zdziarski.com/blog/ \ No newline at end of file diff --git a/published/20171202 MariaDB administration commands for beginners.md b/published/20171202 MariaDB administration commands for beginners.md new file mode 100644 index 0000000000..4650044649 --- /dev/null +++ b/published/20171202 MariaDB administration commands for beginners.md @@ -0,0 +1,174 @@ +为初学者准备的 MariaDB 管理命令 +====== + +之前我们学过了[在 Centos/RHEL 7 上安装 MariaDB 服务器并保证其安全][1]，使之成为了 RHEL/CentOS 7 的默认数据库。现在我们再来看看一些有用的 MariaDB 管理命令。这些都是使用 MariaDB 最基础的命令，而且它们对 MySQL 也同样适合，因为 Mariadb 就是 MySQL 的一个分支而已。 + +**(推荐阅读：[在 RHEL/CentOS 上安装并配置 MongoDB][2])** + +### MariaDB 管理命令 + +#### 1、查看 MariaDB 安装的版本 + +要查看所安装数据库的当前版本，在终端中输入下面命令： + +``` +$ mysql -version +``` + +该命令会告诉你数据库的当前版本。此外你也可以运行下面命令来查看版本的详细信息： + +``` +$ mysqladmin -u root -p version +``` + +#### 2、登录 MariaDB + +要登录 MariaDB 服务器，运行： + +``` +$ mysql -u root -p +``` + +然后输入密码登录。 + +#### 3、列出所有的数据库 + +要列出 MariaDB 当前拥有的所有数据库，在你登录到 MariaDB 中后运行： + +``` +> show databases; +``` + +（LCTT 译注：`$` 这里代表 shell 的提示符，`>` 这里代表 MariaDB shell 的提示符。） + +#### 4、创建新数据库 + +在 MariaDB 中创建新数据库，登录 MariaDB 后运行： + +``` +> create database dan; +``` + +若想直接在终端创建数据库，则运行： + +``` +$ mysqladmin -u user -p create dan +``` + +这里，`dan` 就是新数据库的名称。 + +#### 5、删除数据库 + +要删除数据库，在已登录的 MariaDB 会话中运行： + +``` +> drop database dan; +``` + +此外你也可以运行， + +``` +$ mysqladmin -u root -p drop dan +``` + +**注意：** 若在运行 `mysqladmin` 命令时提示 “access denied” 错误，这应该是由于我们没有给 root 授权。要对 root 授权，请参照第 7 点方法，只是要将用户改成 root。 + +#### 6、创建新用户 + +为数据库创建新用户，运行： + +``` +> CREATE USER 'dan'@'localhost' IDENTIFIED BY 'password'; +``` + +#### 7、授权用户访问某个数据库 + +授权用户访问某个数据库，运行： + +``` +> GRANT ALL PRIVILEGES ON test.* to 'dan'@'localhost'; +``` + +这会赋予用户 `dan` 对名为 `test` 的数据库完全操作的权限。我们也可以限定为用户只赋予 `SELECT`、`INSERT`、`DELETE` 权限。 + +要赋予访问所有数据库的权限，将 `test` 替换成 `*` 。像这样： + +``` +> GRANT ALL PRIVILEGES ON *.* to 'dan'@'localhost'; +``` + +#### 8、备份/导出数据库 + +要创建单个数据库的备份，在终端窗口中运行下列命令， + +``` +$ mysqldump -u root -p database_name>db_backup.sql +``` + +若要一次性创建多个数据库的备份则运行： + +``` +$ mysqldump -u root -p --databases db1 db2 > db12_backup.sql +``` + +要一次性导出多个数据库，则运行： + +``` +$ mysqldump -u root -p --all-databases > all_dbs.sql +``` + +#### 9、从备份中恢复数据库 + +要从备份中恢复数据库，运行： + +``` +$ mysql -u root -p database_name < db_backup.sql +``` + +但这条命令成功的前提是预先没有存在同名的数据库。如果想要恢复数据库数据到已经存在的数据库中，则需要用到 `mysqlimport` 命令： + +``` +$ mysqlimport -u root -p database_name < db_backup.sql +``` + +#### 10、更改 mariadb 用户的密码 + +本例中我们会修改 `root` 的密码，但修改其他用户的密码也是一样的过程。 + +登录 mariadb 并切换到 'mysql' 数据库： + +``` +$ mysql -u root -p +> use mysql; +``` + +然后运行下面命令： + +``` +> update user set password=PASSWORD('your_new_password_here') where User='root'; +``` + +下一步，重新加载权限： + +``` +> flush privileges; +``` + +然后退出会话。 + +我们的教程至此就结束了，在本教程中我们学习了一些有用的 MariaDB 管理命令。欢迎您的留言。 + + +-------------------------------------------------------------------------------- + +via: http://linuxtechlab.com/mariadb-administration-commands-beginners/ + +作者：[Shusain][a] +译者：[lujun9972](https://github.com/lujun9972) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://linuxtechlab.com/author/shsuain/ +[1]:http://linuxtechlab.com/installing-configuring-mariadb-rhelcentos/ +[2]:http://linuxtechlab.com/mongodb-installation-configuration-rhelcentos/ diff --git a/published/20171212 How To Count The Number Of Files And Folders-Directories In Linux.md b/published/20171212 How To Count The Number Of Files And Folders-Directories In Linux.md new file mode 100644 index 0000000000..3f8b2c6312 --- /dev/null +++ b/published/20171212 How To Count The Number Of Files And Folders-Directories In Linux.md @@ -0,0 +1,201 @@ +如何统计 Linux 中文件和文件夹/目录的数量 +====== + +嗨，伙计们，今天我们再次带来一系列可以多方面帮助到你的复杂的命令。 通过操作命令，可以帮助您计数当前目录中的文件和目录、递归计数，统计特定用户创建的文件列表等。 + +在本教程中，我们将向您展示如何使用多个命令，并使用 `ls`、`egrep`、`wc` 和 `find` 命令执行一些高级操作。 下面的命令将可用在多个方面。 + +为了实验，我打算总共创建 7 个文件和 2 个文件夹（5 个常规文件和 2 个隐藏文件）。 下面的 `tree` 命令的输出清楚的展示了文件和文件夹列表。 + +``` +# tree -a /opt +/opt +├── magi +│   └── 2g +│   ├── test5.txt +│   └── .test6.txt +├── test1.txt +├── test2.txt +├── test3.txt +├── .test4.txt +└── test.txt + +2 directories, 7 files +``` + +### 示例-1 + +统计当前目录的文件（不包括隐藏文件）。 运行以下命令以确定当前目录中有多少个文件，并且不计算点文件（LCTT 译注：点文件即以“.” 开头的文件，它们在 Linux 默认是隐藏的）。 + +``` +# ls -l . | egrep -c '^-' +4 +``` + +**细节：** + + * `ls` ： 列出目录内容 + * `-l` ： 使用长列表格式 + * `.` ： 列出有关文件的信息（默认为当前目录） + * `|` ： 将一个程序的输出发送到另一个程序进行进一步处理的控制操作符 + * `egrep` ： 打印符合模式的行 + * `-c` ： 通用输出控制 + * `'^-'` ： 以“-”开头的行（`ls -l` 列出长列表时，行首的 “-” 代表普通文件） + +### 示例-2 + +统计当前目录包含隐藏文件在内的文件。 包括当前目录中的点文件。 + +``` +# ls -la . | egrep -c '^-' +5 +``` + +### 示例-3 + +运行以下命令来计数当前目录的文件和文件夹。 它会计算所有的文件和目录。 + +``` +# ls -l | wc -l +5 +``` + +**细节：** + + * `ls` ： 列出目录内容 + * `-l` ： 使用长列表格式 + * `|` ： 将一个程序的输出发送到另一个程序进行进一步处理的控制操作符 + * `wc` ： 这是一个统计每个文件的换行符、单词和字节数的命令 + * `-l` ： 输出换行符的数量 + +### 示例-4 + +统计当前目录包含隐藏文件和目录在内的文件和文件夹。 + +``` +# ls -la | wc -l +8 +``` + +### 示例-5 + +递归计算当前目录的文件，包括隐藏文件。 + +``` +# find . -type f | wc -l +7 +``` + +**细节 ：** + + * `find` ： 搜索目录结构中的文件 + * `-type` ： 文件类型 + * `f` ： 常规文件 + * `wc` ： 这是一个统计每个文件的换行符、单词和字节数的命令 + * `-l` ： 输出换行符的数量 + +### 示例-6 + +使用 `tree` 命令输出目录和文件数（不包括隐藏文件）。 + +``` +# tree | tail -1 +2 directories, 5 files +``` + +### 示例-7 + +使用包含隐藏文件的 `tree` 命令输出目录和文件计数。 + +``` +# tree -a | tail -1 +2 directories, 7 files +``` + +### 示例-8 + +运行下面的命令递归计算包含隐藏目录在内的目录数。 + +``` +# find . -type d | wc -l +3 +``` + +### 示例-9 + +根据文件扩展名计数文件数量。 这里我们要计算 `.txt` 文件。 + +``` +# find . -name "*.txt" | wc -l +7 +``` + +### 示例-10 + +组合使用 `echo` 命令和 `wc` 命令统计当前目录中的所有文件。 `4` 表示当前目录中的文件数量。 + +``` +# echo *.* | wc +1 4 39 +``` + +### 示例-11 + +组合使用 `echo` 命令和 `wc` 命令来统计当前目录中的所有目录。 第二个 `1` 表示当前目录中的目录数量。 + +``` +# echo */ | wc +1 1 6 +``` + +### 示例-12 + +组合使用 `echo` 命令和 `wc` 命令来统计当前目录中的所有文件和目录。 `5` 表示当前目录中的目录和文件的数量。 + +``` +# echo * | wc +1 5 44 +``` + +### 示例-13 + +统计系统（整个系统）中的文件数。 + +``` +# find / -type f | wc -l +69769 +``` + +### 示例-14 + +统计系统（整个系统）中的文件夹数。 + +``` +# find / -type d | wc -l +8819 +``` + +### 示例-15 + +运行以下命令来计算系统（整个系统）中的文件、文件夹、硬链接和符号链接数。 + +``` +# find / -type d -exec echo dirs \; -o -type l -exec echo symlinks \; -o -type f -links +1 -exec echo hardlinks \; -o -type f -exec echo files \; | sort | uniq -c + 8779 dirs + 69343 files + 20 hardlinks + 11646 symlinks +``` + +-------------------------------------------------------------------------------- + +via: https://www.2daygeek.com/how-to-count-the-number-of-files-and-folders-directories-in-linux/ + +作者：[Magesh Maruthamuthu][a] +译者：[Flowsnow](https://github.com/Flowsnow) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.2daygeek.com/author/magesh/ +[1]:https://www.2daygeek.com/empty-a-file-delete-contents-lines-from-a-file-remove-matching-string-from-a-file-remove-empty-blank-lines-from-a-file/ diff --git a/published/20171215 Linux Vs Unix.md b/published/20171215 Linux Vs Unix.md new file mode 100644 index 0000000000..953c14610d --- /dev/null +++ b/published/20171215 Linux Vs Unix.md @@ -0,0 +1,70 @@ +Linux 与 Unix 之差异 +============== + +[![Linux vs. Unix](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/unix-vs-linux_orig.jpg)][1] + +在计算机时代，相当一部分的人错误地认为 **Unix** 和 **Linux** 操作系统是一样的。然而，事实恰好相反。让我们仔细看看。 + +### 什么是 Unix？ + +[![what is unix](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/unix_orig.png)][2] + +在 IT 领域，以操作系统而为人所知的 Unix，是 1969 年 AT&T 公司在美国新泽西所开发的（目前它的商标权由国际开放标准组织所拥有）。大多数的操作系统都受到了 Unix 的启发，而 Unix 也受到了未完成的 Multics 系统的启发。Unix 的另一版本是来自贝尔实验室的 Play 9。 + +#### Unix 被用于哪里？ + +作为一个操作系统，Unix 大多被用在服务器、工作站，现在也有用在个人计算机上。它在创建互联网、计算机网络或客户端/服务器模型方面发挥着非常重要的作用。 + +#### Unix 系统的特点 + +* 支持多任务 +* 相比 Multics 操作更加简单 +* 所有数据以纯文本形式存储 +* 采用单一根文件的树状存储 +* 能够同时访问多用户账户 + +#### Unix 操作系统的组成 + +**a)** 单核操作系统，负责低级操作以及由用户发起的操作，内核之间的通信通过系统调用进行。 +**b)** 系统工具 +**c)** 其他应用程序 + +### 什么是 Linux？ + +[![what is linux](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/linux_orig.png)][4] + +这是一个基于 Unix 操作系统原理的开源操作系统。正如开源的含义一样，它是一个可以自由下载的系统。它也可以通过编辑、添加及扩充其源代码而定制该系统。这是它最大的好处之一，而不像今天的其它操作系统（Windows、Mac OS X 等）需要付费。Unix 系统不是创建新系统的唯一模版，另外一个重要的因素是 MINIX 系统，不像 Linus，此版本被其缔造者（Andrew Tanenbaum）用于商业系统。 + +Linux 由 Linus Torvalds 开发于 1991 年，这是一个其作为个人兴趣的操作系统。为什么 Linux 借鉴 Unix 的一个主要原因是因为其简洁性。Linux 第一个官方版本（0.01）发布于 1991 年 9 月 17 日。虽然这个系统并不是很完美和完善，但 Linus 对它产生很大的兴趣，并在几天内，Linus 发出了一些关于 Linux 源代码扩展以及其他想法的电子邮件。 + +#### Linux 的特点 + +Linux 的基石是 Unix 内核，其基于 Unix 的基本特点以及 **POSIX** 和单独的 **UNIX 规范标准**。看起来，该操作系统官方名字取自于 **Linus**，其中其操作系统名称的尾部的 “x” 和 **Unix 系统**相联系。 + +#### 主要功能 + +* 同时运行多任务（多任务） +* 程序可以包含一个或多个进程（多用途系统），且每个进程可能有一个或多个线程。 +* 多用户，因此它可以运行多个用户程序。 +* 个人帐户受适当授权的保护。 +* 因此账户准确地定义了系统控制权。 + +**企鹅 Tux** 的 Logo 作者是 Larry Ewing，他选择这个企鹅作为他的开源 **Linux 操作系统**的吉祥物。**Linux Torvalds** 最初提出这个新的操作系统的名字为 “Freax” ，即为 “自由（free）” + “奇异（freak）” + x（UNIX 系统）的结合字，而不像存放它的首个版本的 FTP 服务器上所起的名字（Linux）。 + + +-------------------------------------------------------------------------------- + +via: http://www.linuxandubuntu.com/home/linux-vs-unix + +作者：[linuxandubuntu][a] +译者：[HardworkFish](https://github.com/HardworkFish) +校对：[imquanquan](https://github.com/imquanquan), [wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://www.linuxandubuntu.com +[1]:http://www.linuxandubuntu.com/home/linux-vs-unix +[2]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/unix_orig.png +[3]:http://www.unix.org/what_is_unix.html +[4]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/linux_orig.png +[5]:https://www.linux.com diff --git a/published/20171218 Internet Chemotherapy.md b/published/20171218 Internet Chemotherapy.md new file mode 100644 index 0000000000..faac86eceb --- /dev/null +++ b/published/20171218 Internet Chemotherapy.md @@ -0,0 +1,76 @@ +互联网化疗 +====== + +> LCTT 译注：本文作者 janit0r 被认为是 BrickerBot 病毒的作者。此病毒会攻击物联网上安全性不足的设备并使其断开和其他网络设备的连接。janit0r 宣称他使用这个病毒的目的是保护互联网的安全，避免这些设备被入侵者用于入侵网络上的其他设备。janit0r 称此项目为“互联网化疗”。janit0r 决定在 2017 年 12 月终止这个项目，并在网络上发表了这篇文章。 + +> —— 12/10 2017 + +### --[ 1 互联网化疗 + +互联网化疗Internet Chemotherapy是在 2016 年 11 月 到 2017 年 12 月之间的一个为期 13 个月的项目。它曾被称为 “BrickerBot”、“错误的固件升级”、“勒索软件”、“大规模网络瘫痪”，甚至 “前所未有的恐怖行为”。最后一个有点伤人了，费尔南德斯（LCTT 译注：委内瑞拉电信公司 CANTV 的光纤网络曾在 2017 年 8 月受到病毒攻击，公司董事长曼努埃尔·费尔南德斯称这次攻击为[“前所未有的恐怖行为”][1]），但我想我大概不能让所有人都满意吧。 + +你可以从 http://91.215.104.140/mod_plaintext.py 下载我的代码模块，它可以基于 http 和 telnet 发送恶意请求（LCTT 译注：这个链接已经失效，不过在 [Github][2] 上有备份）。因为平台的限制，模块里是代码混淆过的单线程 Python 代码，但载荷payload（LCTT 译注：payload，指实质的攻击/利用代码）依然是明文，任何合格的程序员应该都能看得懂。看看这里面有多少载荷、0-day 漏洞和入侵技巧，花点时间让自己接受现实。然后想象一下，如果我是一个黑客，致力于创造出强大的 DDoS 生成器来勒索那些最大的互联网服务提供商（ISP）和公司的话，互联网在 2017 年会受到怎样的打击。我完全可以让他们全部陷入混乱，并同时对整个互联网造成巨大的伤害。 + +我的 ssh 爬虫太危险了，不能发布出来。它包含很多层面的自动化，可以只利用一个被入侵的路由器就能够在设计有缺陷的 ISP 的网络上平行移动并加以入侵。正是因为我可以征用数以万计的 ISP 的路由器，而这些路由器让我知晓网络上发生的事情并给我提供源源不断的节点用来进行入侵行动，我才得以进行我的反物联网僵尸网络项目。我于 2015 年开始了我的非破坏性的 ISP 的网络清理项目，于是当 Mirai 病毒入侵时我已经做好了准备来做出回应。主动破坏其他人的设备仍然是一个困难的决定，但无比危险的 CVE-2016-10372 漏洞让我别无选择。从那时起我就决定一不做二不休。 + +（LCTT 译注：上一段中提到的 Mirai 病毒首次出现在 2016 年 8 月。它可以远程入侵运行 Linux 系统的网络设备并利用这些设备构建僵尸网络。本文作者 janit0r 宣称当 Mirai 入侵时他利用自己的 BrickerBot 病毒强制将数以万计的设备从网络上断开，从而减少 Mirai 病毒可以利用的设备数量。） + +我在此警告你们，我所做的只是权宜之计，它并不足以在未来继续拯救互联网。坏人们正变得更加聪明，潜在存在漏洞的设备数量在持续增加，发生大规模的、能使网络瘫痪的事件只是时间问题。如果你愿意相信，我曾经在一个持续 13 个月的项目中使上千万有漏洞的设备变得无法使用，那么不过分地说，如此严重的事件本已经在 2017 年发生了。 + +__你们应该意识到，只需要再有一两个严重的物联网漏洞，我们的网络就会严重瘫痪。__ 考虑到我们的社会现在是多么依赖数字网络，而计算机安全应急响应组（CERT）、ISP 们和政府们又是多么地忽视这种问题的严重性，这种事件造成的伤害是无法估计的。ISP 在持续地部署暴露了控制端口的设备，而且即使像 Shodan 这样的服务可以轻而易举地发现这些问题，国家 CERT 还是似乎并不在意。而很多国家甚至都没有自己的 CERT 。世界上许多最大的 ISP 都没有雇佣任何熟知计算机安全问题的人，而是在出现问题的时候依赖于外来的专家来解决。我曾见识过大型 ISP 在我的僵尸网络的调节之下连续多个月持续受损，但他们还是不能完全解决漏洞（几个好的例子是 BSNL、Telkom ZA、PLDT、某些时候的 PT Telkom，以及南半球大部分的大型 ISP ）。只要看看 Telkom ZA 解决他们的 Aztech 调制解调器问题的速度有多慢，你就会开始理解现状有多么令人绝望。在 99% 的情况下，要解决这个问题只需要 ISP 部署合理的访问控制列表，并把部署在用户端的设备（CPE）单独分段就行，但是几个月过去之后他们的技术人员依然没有弄明白。如果 ISP 在经历了几周到几个月的针对他们设备的蓄意攻击之后仍然无法解决问题，我们又怎么能期望他们会注意到并解决 Mirai 在他们网络上造成的问题呢？世界上许多最大的 ISP 对这些事情无知得令人发指，而这毫无疑问是最大的危险，但奇怪的是，这应该也是最容易解决的问题。 + +我已经尽自己的责任试着去让互联网多坚持一段时间，但我已经尽力了。接下来要交给你们了。即使很小的行动也是非常重要的。你们能做的事情有： + +* 使用像 Shodan 之类的服务来检查你的 ISP 的安全性，并驱使他们去解决他们网络上开放的 telnet、http、httpd、ssh 和 tr069 等端口。如果需要的话，可以把这篇文章给他们看。从来不存在什么好的理由来让这些端口可以从外界访问。暴露控制端口是业余人士的错误。如果有足够的客户抱怨，他们也许真的会采取行动！ +* 用你的钱包投票！拒绝购买或使用任何“智能”产品，除非制造商保证这个产品能够而且将会收到及时的安全更新。在把你辛苦赚的钱交给提供商之前，先去查看他们的安全记录。为了更好的安全性，可以多花一些钱。 +* 游说你本地的政治家和政府官员，让他们改进法案来规范物联网设备，包括路由器、IP 照相机和各种“智能”设备。不论私有还是公有的公司目前都没有足够的动机去在短期内解决该问题。这件事情和汽车或者通用电器的安全标准一样重要。 +* 考虑给像 GDI 基金会或者 Shadowserver 基金会这种缺少支持的白帽黑客组织贡献你的时间或者其他资源。这些组织和人能产生巨大的影响，并且他们可以很好地发挥你的能力来帮助互联网。 +* 最后，虽然希望不大，但可以考虑通过设立法律先例来让物联网设备成为一种“诱惑性危险品attractive nuisance”（LCTT 译注：attractive nuisance 是美国法律中的一个原则，意思是如果儿童在私人领地上因为某些对儿童有吸引力的危险物品而受伤，领地的主人需要负责，无论受伤的儿童是否是合法进入领地）。如果一个房主可以因为小偷或者侵入者受伤而被追责，我不清楚为什么设备的主人（或者 ISP 和设备制造商）不应该因为他们的危险的设备被远程入侵所造成的伤害而被追责。连带责任原则应该适用于对设备应用层的入侵。如果任何有钱的大型 ISP 不愿意为设立这样的先例而出钱（他们也许的确不会，因为他们害怕这样的先例会反过来让自己吃亏），我们甚至可以在这里还有在欧洲为这个行动而进行众筹。 ISP 们：把你们在用来应对 DDoS 的带宽上省下的可观的成本当做我为这个目标的间接投资，也当做它的好处的证明吧。 + +### --[ 2 时间线 + +下面是这个项目中一些值得纪念的事件： + +* 2016 年 11 月底的德国电信 Mirai 事故。我匆忙写出的最初的 TR069/64 请求只执行了 `route del default`，不过这已经足够引起 ISP 去注意这个问题，而它引发的新闻头条警告了全球的其他 ISP 来注意这个迫近的危机。 +* 大约 1 月 11 日 到 12 日，一些位于华盛顿特区的开放了 6789 控制端口的硬盘录像机被 Mirai 入侵并瘫痪，这上了很多头条新闻。我要给 Vemulapalli 点赞，她居然认为 Mirai 加上 `/dev/urandom` 一定是“非常复杂的勒索软件”（LCTT 译注：Archana Vemulapalli 当时是华盛顿市政府的 CTO）。欧洲的那两个可怜人又怎么样了呢？ +* 2017 年 1 月底发生了第一起真正的大规模 ISP 下线事件。Rogers Canada 的提供商 Hitron 非常粗心地推送了一个在 2323 端口上监听的无验证的 root shell（这可能是一个他们忘记关闭的 debug 接口）。这个惊天的失误很快被 Mirai 的僵尸网络所发现，造成大量设备瘫痪。 +* 在 2017 年 2 月，我注意到 Mirai 在这一年里的第一次扩张，Netcore/Netis 以及 Broadcom 的基于 CLI （命令行接口）的调制解调器都遭受了攻击。BCM CLI 后来成为了 Mirai 在 2017 年的主要战场，黑客们和我自己都在这一年的余下时间里花大量时间寻找无数 ISP 和设备制造商设置的默认密码。前面代码中的“broadcom” 载荷也许看上去有点奇怪，但它们是统计角度上最可能禁用那些大量的有问题的 BCM CLI 固件的序列。 +* 在 2017 年 3 月，我大幅提升了我的僵尸网络的节点数量并开始加入更多的网络载荷。这是为了应对包括 Imeij、Amnesia 和 Persirai 在内的僵尸网络的威胁。大规模地禁用这些被入侵的设备也带来了新的一些问题。比如在 Avtech 和 Wificam 设备所泄露的登录信息当中，有一些用户名和密码非常像是用于机场和其他重要设施的，而英国政府官员大概在 2017 年 4 月 1 日关于针对机场和核设施的“实际存在的网络威胁”做出过警告。哎呀。 +* 这种更加激进的扫描还引起了民间安全研究者的注意，安全公司 Radware 在 2017 年 4 月 6 日发表了一篇关于我的项目的文章。这个公司把它叫做“BrickerBot”。显然，如果我要继续增加我的物联网防御措施的规模，我必须想出更好的网络映射与检测方法来应对蜜罐或者其他有风险的目标。 +* 2017 年 4 月 11 日左右的时候发生了一件非常不寻常的事情。一开始这看上去和许多其他的 ISP 下线事件相似，一个叫 Sierra Tel 的半本地 ISP 在一些 Zyxel 设备上使用了默认的 telnet 用户名密码 supervisor/zyad1234。一个 Mirai 运行器发现了这些有漏洞的设备，我的僵尸网络紧随其后，2017 年精彩绝伦的 BCM CLI 战争又开启了新的一场战斗。这场战斗并没有持续很久。它本来会和 2017 年的其他数百起 ISP 下线事件一样，如果不是在尘埃落定之后发生的那件非常不寻常的事情的话。令人惊奇的是，这家 ISP 并没有试着把这次网络中断掩盖成某种网络故障、电力超额或错误的固件升级。他们完全没有对客户说谎。相反，他们很快发表了新闻公告，说他们的调制解调器有漏洞，这让他们的客户们得以评估自己可能遭受的风险。这家全世界最诚实的 ISP 为他们值得赞扬的公开行为而收获了什么呢？悲哀的是，它得到的只是批评和不好的名声。这依然是我记忆中最令人沮丧的“为什么我们得不到好东西”的例子，这很有可能也是为什么 99% 的安全错误都被掩盖而真正的受害者被蒙在鼓里的最主要原因。太多时候，“有责任心的信息公开”会直接变成“粉饰太平”的委婉说法。 +* 在 2017 年 4 月 14 日，国土安全部关于“BrickerBot 对物联网的威胁”做出了警告，我自己的政府把我作为一个网络威胁这件事让我觉得他们很不公平而且目光短浅。跟我相比，对美国人民威胁最大的难道不应该是那些部署缺乏安全性的网络设备的提供商和贩卖不成熟的安全方案的物联网设备制造商吗？如果没有我，数以百万计的人们可能还在用被入侵的设备和网络来处理银行业务和其他需要保密的交易。如果国土安全部里有人读到这篇文章，我强烈建议你重新考虑一下保护国家和公民究竟是什么意思。 +* 在 2017 年 4 月底，我花了一些时间改进我的 TR069/64 攻击方法，然后在 2017 年 5 月初，一个叫 Wordfence 的公司（现在叫 Defiant）报道称一个曾给 Wordpress 网站造成威胁的基于入侵 TR069 的僵尸网络很明显地衰减了。值得注意的是，同一个僵尸网络在几星期后使用了一个不同的入侵方式暂时回归了（不过这最终也同样被化解了）。 +* 在 2017 年 5 月，主机托管公司 Akamai 在它的 2017 年第一季度互联网现状报告中写道，相比于 2016 年第一季度，大型（超过 100 Gbps）DDoS 攻击数减少了 89%，而总体 DDoS 攻击数减少了 30%。鉴于大型 DDoS 攻击是 Mirai 的主要手段，我觉得这给这些月来我在物联网领域的辛苦劳动提供了实际的支持。 +* 在夏天我持续地改进我的入侵技术军火库，然后在 7 月底我针对亚太互联网络信息中心（APNIC）的 ISP 进行了一些测试。测试结果非常令人吃惊。造成的影响之一是数十万的 BSNL 和 MTNL 调制解调器被禁用，而这次中断事故在印度成为了头条新闻。考虑到当时在印度和中国之间持续升级的地缘政治压力，我觉得这个事故有很大的风险会被归咎于中国所为，于是我很罕见地决定公开承认是我所做。Catalin，我很抱歉你在报道这条新闻之后突然被迫放的“两天的假期”。 +* 在处理过亚太互联网络信息中心（APNIC）和非洲互联网络信息中心（AfriNIC）的之后，在 2017 年 8 月 9 日我又针对拉丁美洲与加勒比地区互联网络信息中心（LACNIC）进行了大规模的清理，给这个大洲的许多提供商造成了问题。在数百万的 Movilnet 的手机用户失去连接之后，这次攻击在委内瑞拉被大幅报道。虽然我个人反对政府监管互联网，委内瑞拉的这次情况值得注意。许多拉美与加勒比地区的提供商与网络曾在我的僵尸网络的持续调节之下连续数个月逐渐衰弱，但委内瑞拉的提供商很快加强了他们的网络防护并确保了他们的网络设施的安全。我认为这是由于委内瑞拉相比于该地区的其他国家来说进行了更具侵入性的深度包检测。值得思考一下。 +* F5 实验室在 2017 年 8 月发布了一个题为“狩猎物联网：僵尸物联网的崛起”的报告，研究者们在其中对近期 telnet 活动的平静表达了困惑。研究者们猜测这种活动的减少也许证实了一个或多个非常大型的网络武器正在成型（我想这大概确实是真的）。这篇报告是在我印象中对我的项目的规模最准确的评估，但神奇的是，这些研究者们什么都推断不出来，尽管他们把所有相关的线索都集中到了一页纸上。 +* 2017 年 8 月，Akamai 的 2017 年第二季度互联网现状报告宣布这是三年以来首个该提供商没有发现任何大规模（超过 100 Gbps）攻击的季度，而且 DDoS 攻击总数相比 2017 年第一季度减少了 28%。这看上去给我的清理工作提供了更多的支持。这个出奇的好消息被主流媒体所完全忽视了，这些媒体有着“流血的才是好新闻”的心态，即使是在信息安全领域。这是我们为什么不能得到好东西的又一个原因。 +* 在 CVE-2017-7921 和 7923 于 2017 年 9 月公布之后，我决定更密切地关注海康威视公司的设备，然后我惊恐地发现有一个黑客们还没有发现的方法可以用来入侵有漏洞的固件。于是我在 9 月中旬开启了一个全球范围的清理行动。超过一百万台硬盘录像机和摄像机（主要是海康威视和大华出品）在三周内被禁用，然后包括 IPVM.com 在内的媒体为这些攻击写了多篇报道。大华和海康威视在新闻公告中提到或暗示了这些攻击。大量的设备总算得到了固件升级。看到这次清理活动造成的困惑，我决定给这些闭路电视制造商写一篇[简短的总结][3]（请原谅在这个粘贴板网站上的不和谐的语言）。这令人震惊的有漏洞而且在关键的安全补丁发布之后仍然在线的设备数量应该能够唤醒所有人，让他们知道现今的物联网补丁升级过程有多么无力。 +* 2017 年 9 月 28 日左右，Verisign 发表了报告称 2017 年第二季度的 DDoS 攻击数相比第一季度减少了 55%，而且攻击峰值大幅减少了 81%。 +* 2017 年 11 月 23 日，CDN 供应商 Cloudflare 报道称“近几个月来，Cloudflare 看到试图用垃圾流量挤满我们的网络的简单进攻尝试有了大幅减少”。Cloudflare 推测这可能和他们的政策变化有一定关系，但这些减少也和物联网清理行动有着明显的重合。 +* 2017 年 11 月底，Akamai 的 2017 年第三季度互联网现状报告称 DDoS 攻击数较前一季度小幅增加了 8%。虽然这相比 2016 年的第三季度已经减少了很多，但这次小幅上涨提醒我们危险仍然存在。 +* 作为潜在危险的更进一步的提醒，一个叫做“Satori”的新的 Mirai 变种于 2017 年 11 月至 12 月开始冒头。这个僵尸网络仅仅通过一个 0-day 漏洞而达成的增长速度非常值得注意。这起事件凸显了互联网的危险现状，以及我们为什么距离大规模的事故只差一两起物联网入侵。当下一次威胁发生而且没人阻止的时候，会发生什么？Sinkholing 和其他的白帽或“合法”的缓解措施在 2018 年不会有效，就像它们在 2016 年也不曾有效一样。也许未来各国政府可以合作创建一个国际范围的反黑客特别部队来应对特别严重的会影响互联网存续的威胁，但我并不抱太大期望。 +* 在年末出现了一些危言耸听的新闻报道，有关被一个被称作“Reaper”和“IoTroop”的新的僵尸网络。我知道你们中有些人最终会去嘲笑那些把它的规模估算为一两百万的人，但你们应该理解这些网络安全研究者们对网络上发生的事情以及不由他们掌控的硬件的事情都是一知半解。实际来说，研究者们不可能知道或甚至猜测到大部分有漏洞的设备在僵尸网络出现时已经被禁用了。给“Reaper”一两个新的未经处理的 0-day 漏洞的话，它就会变得和我们最担心的事情一样可怕。 + +### --[ 3 临别赠言 + +我很抱歉把你们留在这种境况当中，但我的人身安全受到的威胁已经不允许我再继续下去。我树了很多敌人。如果你想要帮忙，请看前面列举的该做的事情。祝你好运。 + +也会有人批评我，说我不负责任，但这完全找错了重点。真正的重点是如果一个像我一样没有黑客背景的人可以做到我所做到的事情，那么一个比我厉害的人可以在 2017 年对互联网做比这要可怕太多的事情。我并不是问题本身，我也不是来遵循任何人制定的规则的。我只是报信的人。你越早意识到这点越好。 + +-Dr Cyborkian 又名 janit0r，“病入膏肓”的设备的调节者。 + +-------------------------------------------------------------------------------- + +via：https://ghostbin.com/paste/q2vq2 + +作者：janit0r +译者：[yixunx](https://github.com/yixunx) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译， +[Linux中国](https://linux.cn/) 荣誉推出 + +[1]:https://www.telecompaper.com/news/venezuelan-operators-hit-by-unprecedented-cyberattack--1208384 +[2]:https://github.com/JeremyNGalloway/mod_plaintext.py +[3]:http://depastedihrn3jtw.onion.link/show.php?md5=62d1d87f67a8bf485d43a05ec32b1e6f \ No newline at end of file diff --git a/published/20090127 Anatomy of a Program in Memory.md b/published/201801/20090127 Anatomy of a Program in Memory.md similarity index 100% rename from published/20090127 Anatomy of a Program in Memory.md rename to published/201801/20090127 Anatomy of a Program in Memory.md diff --git a/published/20090718 Vmware Linux Guest Add a New Hard Disk Without Rebooting Guest.md b/published/201801/20090718 Vmware Linux Guest Add a New Hard Disk Without Rebooting Guest.md similarity index 100% rename from published/20090718 Vmware Linux Guest Add a New Hard Disk Without Rebooting Guest.md rename to published/201801/20090718 Vmware Linux Guest Add a New Hard Disk Without Rebooting Guest.md diff --git a/published/201801/20111124 How to find hidden processes and ports on Linux-Unix-Windows.md b/published/201801/20111124 How to find hidden processes and ports on Linux-Unix-Windows.md new file mode 100644 index 0000000000..4bdf16485c --- /dev/null +++ b/published/201801/20111124 How to find hidden processes and ports on Linux-Unix-Windows.md @@ -0,0 +1,243 @@ +如何在 Linux/Unix/Windows 中发现隐藏的进程和端口 +============== + +`unhide` 是一个小巧的网络取证工具，能够发现那些借助 rootkit、LKM 及其它技术隐藏的进程和 TCP/UDP 端口。这个工具在 Linux、UNIX 类、MS-Windows 等操作系统下都可以工作。根据其 man 页面的说明： + +> Unhide 通过下述三项技术来发现隐藏的进程。 +> 1. 进程相关的技术，包括将 `/proc` 目录与 [/bin/ps][1] 命令的输出进行比较。 +> 2. 系统相关的技术，包括将 [/bin/ps][1] 命令的输出结果同从系统调用方面得到的信息进行比较。 +> 3. 穷举法相关的技术，包括对所有的进程 ID 进行暴力求解，该技术仅限于在基于 Linux2.6 内核的系统中使用。 + +绝大多数的 Rootkit 工具或者恶意软件借助内核来实现进程隐藏，这些进程只在内核内部可见。你可以使用 `unhide` 或者诸如 [rkhunter 等工具，扫描 rootkit 程序 、后门程序以及一些可能存在的本地漏洞][2]。 + +![本文讲解如何在多个操作系统下安装和使用unhide][3] + +这篇文章描述了如何安装 unhide 并搜索隐藏的进程和 TCP/UDP 端口。 + +### 如何安装 unhide + +首先建议你在只读介质上运行这个工具。如果使用的是 Ubuntu 或者 Debian 发行版，输入下述的 [apt-get][4]/[apt][5] 命令以安装 Unhide： + +``` +$ sudo apt-get install unhide +``` + +一切顺利的话你的命令行会输出以下内容： + +``` +[sudo] password for vivek: +Reading package lists... Done +Building dependency tree +Reading state information... Done +Suggested packages: + rkhunter +The following NEW packages will be installed: + unhide +0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. +Need to get 46.6 kB of archives. +After this operation, 136 kB of additional disk space will be used. +Get:1 http://in.archive.ubuntu.com/ubuntu artful/universe amd64 unhide amd64 20130526-1 [46.6 kB] +Fetched 46.6 kB in 0s (49.0 kB/s) +Selecting previously unselected package unhide. +(Reading database ... 205367 files and directories currently installed.) +Preparing to unpack .../unhide_20130526-1_amd64.deb ... +Unpacking unhide (20130526-1) ... +Setting up unhide (20130526-1) ... +Processing triggers for man-db (2.7.6.1-2) ... +``` + +### 如何在 RHEL/CentOS/Oracle/Scientific/Fedora 上安装 unhide + +输入下列 yum Type the following yum command (first turn on EPLE repo on a CentOS/RHEL version 6.x or version 7.x): + +输入以下的 [yum][6] 命令（CentOS/RHEL [6.x][7] 或 [7.x][8] 上首先打开 EPEL 仓库）： + +``` +$ sudo yum install unhide +``` + +在 Fedora 上则使用以下 dnf 命令： + +``` +$ sudo dnf install unhide +``` + +### 如何在 Arch 上安装 unhide + +键入以下 pacman 命令安装： + +``` +$ sudo pacman -S unhide +``` + +### 如何在 FreeBSD 上安装 unhide + +可以通过以下的命令使用 port 来安装 unhide： + +``` +# cd /usr/ports/security/unhide/ +# make install clean +``` + +或者可以通过二进制文件安装 hide，使用 pkg 命令安装： + +``` +# pkg install unhide +``` + +### 如何使用 unhide 工具？ + +unhide 的语法是： + +``` +unhide [options] test_list +``` + +`test_list` 参数可以是以下测试列表中的一个或者多个标准测试： + + +1. brute +2. proc +3. procall +4. procfs +5. quick +6. reverse +7. sys + +或基本测试： + +1. checkbrute +2. checkchdir +3. checkgetaffinity +4. checkgetparam +5. checkgetpgid +6. checkgetprio +7. checkRRgetinterval +8. checkgetsched +9. checkgetsid +10. checkkill +11. checknoprocps +12. checkopendir +13. checkproc +14. checkquick +15. checkreaddir +16. checkreverse +17. checksysinfo +18. checksysinfo2 +19. checksysinfo3 + +你可以通过以下示例命令使用 `unhide`： + +``` +# unhide proc +# unhide sys +# unhide quick +``` + +示例输出： + +``` +Unhide 20130526 +Copyright © 2013 Yago Jesus & Patrick Gouin +License GPLv3+ : GNU GPL version 3 or later +http://www.unhide-forensics.info + +NOTE : This version of unhide is for systems using Linux >= 2.6 + +Used options: +[*]Searching for Hidden processes through comparison of results of system calls, proc, dir and ps +``` + +### 如何使用 unhide-tcp 工具辨明 TCP/UDP 端口的身份 + +以下是来自 man 页面的介绍： + +> `unhide-tcp` 取证工具通过对所有可用的 TCP/IP 端口进行暴力求解的方式，辨别所有正在监听，却没有列入 [/bin/netstat][9] 或者 [/bin/ss][10] 命令输出的 TCP/IP 端口身份。 + +> 注一：对于 FreeBSD、OpenBSD系统，一般使用 netstat 命令取代在这些操作系统上不存在的 iproute2，此外，sockstat 命令也用于替代 fuser。 + +> 注二：如果操作系统不支持 iproute2 命令，在使用 `unhide` 时需要在命令上加上 `-n` 或者 `-s` 选项。 + +``` +# unhide-tcp +``` + +示例输出： + +``` +Unhide 20100201 +http://www.security-projects.com/?Unhide + +Starting TCP checking + +Starting UDP checking +``` + +上述操作中，没有发现隐藏的端口。 + +但在下述示例中，我展示了一些有趣的事。 + +``` +# unhide-tcp +``` + +示例输出： + +``` +Unhide 20100201 +http://www.security-projects.com/?Unhide + + +Starting TCP checking + +Found Hidden port that not appears in netstat: 1048 +Found Hidden port that not appears in netstat: 1049 +Found Hidden port that not appears in netstat: 1050 +Starting UDP checking +``` + +可以看到 `netstat -tulpn` 和 `ss` 命令确实没有反映出这三个隐藏的端口： + +``` +# netstat -tulpn | grep 1048 +# ss -lp +# ss -l | grep 1048 +``` + +通过下述的 man 命令可以更多地了解 `unhide`： + +``` +$ man unhide +$ man unhide-tcp +``` + +### Windows 用户如何安装使用 unhide + +你可以通过这个[页面][13]获取 Windows 版本的 unhide。 + +---- + +via: https://www.cyberciti.biz/tips/linux-unix-windows-find-hidden-processes-tcp-udp-ports.html + +作者：[Vivek Gite][a] +译者：[ljgibbslf](https://github.com/ljgibbslf) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.cyberciti.biz +[1]:https://www.cyberciti.biz/faq/show-all-running-processes-in-linux/ (Linux / Unix ps command) +[2]:https://www.cyberciti.biz/faq/howto-check-linux-rootkist-with-detectors-software/ +[3]:https://www.cyberciti.biz/tips/wp-content/uploads/2011/11/Linux-FreeBSD-Unix-Windows-Find-Hidden-Process-Ports.jpg +[4]:https://www.cyberciti.biz/tips/linux-debian-package-management-cheat-sheet.html (See Linux/Unix apt-get command examples for more info) +[5]://www.cyberciti.biz/faq/ubuntu-lts-debian-linux-apt-command-examples/ (See Linux/Unix apt command examples for more info) +[6]:https://www.cyberciti.biz/faq/rhel-centos-fedora-linux-yum-command-howto/ (See Linux/Unix yum command examples for more info) +[7]:https://www.cyberciti.biz/faq/fedora-sl-centos-redhat6-enable-epel-repo/ +[8]:https://www.cyberciti.biz/faq/installing-rhel-epel-repo-on-centos-redhat-7-x/ +[9]:https://www.cyberciti.biz/tips/linux-display-open-ports-owner.html (Linux netstat command) +[10]:https://www.cyberciti.biz/tips/linux-investigate-sockets-network-connections.html +[11]:https://www.cyberciti.biz/tips/netstat-command-tutorial-examples.html +[12]:https://www.cyberciti.biz/tips/linux-investigate-sockets-network-connections.html +[13]:http://www.unhide-forensics.info/?Windows:Download diff --git a/published/20120611 30 Handy Bash Shell Aliases For Linux - Unix - Mac OS X.md b/published/201801/20120611 30 Handy Bash Shell Aliases For Linux - Unix - Mac OS X.md similarity index 100% rename from published/20120611 30 Handy Bash Shell Aliases For Linux - Unix - Mac OS X.md rename to published/201801/20120611 30 Handy Bash Shell Aliases For Linux - Unix - Mac OS X.md diff --git a/translated/tech/20121211 Python Nmon Analyzer- moving away from excel macros.md b/published/201801/20121211 Python Nmon Analyzer- moving away from excel macros.md similarity index 51% rename from translated/tech/20121211 Python Nmon Analyzer- moving away from excel macros.md rename to published/201801/20121211 Python Nmon Analyzer- moving away from excel macros.md index c772ceff73..351c9c1f75 100644 --- a/translated/tech/20121211 Python Nmon Analyzer- moving away from excel macros.md +++ b/published/201801/20121211 Python Nmon Analyzer- moving away from excel macros.md @@ -1,36 +1,40 @@ Python 版的 Nmon 分析器：让你远离 excel 宏 ====== -[Nigel's monitor][1]，也叫做 "Nmon"，是一个很好的监控，记录和分析 Linux/*nix 系统性能随时间变化的工具。Nmon 最初由 IBM 开发并于 2009 年夏天开源。时至今日 Nmon 已经在所有 linux 平台和架构上都可用了。它提供了大量的实时工具来可视化当前系统统计信息，这些统计信息包括 CPU，RAM，网络和磁盘 I/O。然而，Nmon 最棒的特性是可以随着时间的推移记录系统性能快照。 -比如：`nmon -f -s 1`。 -![nmon CPU and Disk utilization][2] -会创建一个日志文件，该日志文件最开头是一些系统的元数据 T( 章节 AAA - BBBV)，后面是定时抓取的监控系统属性的快照，比如 CPU 和内存的使用情况。这个文件很难直接由电子表格应用来处理，因此诞生了 [Nmon_Analyzer][3] excel 宏。如果你用的是 Windows/Mac 并安装了 Microsoft Office，那么这个工具非常不错。如果没有这个环境那也可以使用 Nmon2rrd 工具，这个工具能将日志文件转换 RRD 输入文件，进而生成图形。这个过程很死板而且有点麻烦。现在出现了一个更灵活的工具，像你们介绍一下 pyNmonAnalyzer，它一个可定制化的解决方案来生成结构化的 CSV 文件和基于 [matplotlib][4] 生成图片的简单 HTML 报告。 -### 入门介绍： +[Nigel's monitor][1]，也叫做 “Nmon”，是一个很好的监控、记录和分析 Linux/*nix 系统性能随时间变化的工具。Nmon 最初由 IBM 开发并于 2009 年夏天开源。时至今日 Nmon 已经在所有 Linux 平台和架构上都可用了。它提供了很棒的当前系统统计信息的基于命令行的实时可视化报告，这些统计信息包括 CPU、RAM、网络和磁盘 I/O。然而，Nmon 最棒的特性是可以随着时间的推移记录系统性能快照。 + +比如：`nmon -f -s 1`。 + +![nmon CPU and Disk utilization][2] + +会创建一个日志文件，该日志文件最开头是一些系统的元数据（AAA - BBBV 部分），后面是所监控的系统属性的定时快照，比如 CPU 和内存的使用情况。这个输出的文件很难直接由电子表格应用来处理，因此诞生了 [Nmon_Analyzer][3] excel 宏。如果你用的是 Windows/Mac 并安装了 Microsoft Office，那么这个工具非常不错。如果没有这个环境那也可以使用 Nmon2rrd 工具，这个工具能将日志文件转换 RRD 输入文件，进而生成图形。这个过程很死板而且有点麻烦。现在出现了一个更灵活的工具，我向你们介绍一下 pyNmonAnalyzer，它提供了一个可定制化的解决方案来生成结构化的 CSV 文件和带有用 [matplotlib][4] 生成的图片的简单 HTML 报告。 + +### 入门介绍 系统需求： + 从名字中就能看出我们需要有 python。此外 pyNmonAnalyzer 还依赖于 matplotlib 和 numpy。若你使用的是 debian 衍生的系统，则你需要先安装这些包： -``` -$> sudo apt-get install python-numpy python-matplotlib ``` - -##### 获取 pyNmonAnalyzer： - -你可页克隆 git 仓库： -``` -$> git clone git@github.com:madmaze/pyNmonAnalyzer.git - +$ sudo apt-get install python-numpy python-matplotlib ``` -或者 +#### 获取 pyNmonAnalyzer： -直接从这里下载：[pyNmonAnalyzer-0.1.zip][5] +你可以克隆 git 仓库： -接下来我们需要一个 Nmon 文件，如果没有的话，可以使用发行版中提供的实例或者自己录制一个样本：`nmon -F test.nmon -s 1 -c 120`，会录制每个 1 秒录制一次，供录制 120 个快照道 test.nmon 文件中 .nmon。 +``` +$ git clone git@github.com:madmaze/pyNmonAnalyzer.git +``` + +或者，直接从这里下载：[pyNmonAnalyzer-0.1.zip][5] 。 + +接下来我们需要一个 Nmon 文件，如果没有的话，可以使用发行版中提供的实例或者自己录制一个样本：`nmon -F test.nmon -s 1 -c 120`，会录制 120 个快照，每秒一个，存储到 test.nmon 文件中。 让我们来看看基本的帮助信息： + ``` -$> ./pyNmonAnalyzer.py -h +$ ./pyNmonAnalyzer.py -h usage: pyNmonAnalyzer.py [-h] [-x] [-d] [-o OUTDIR] [-c] [-b] [-r CONFFNAME] input_file @@ -53,30 +57,29 @@ optional arguments: -r CONFFNAME, --reportConfig CONFFNAME Report config file, if none exists: we will write the default config file out (Default: ./report.config) - ``` 该工具有两个主要的选项 - 1。将 nmon 文件传唤成一系列独立的 CSV 文件 - 2。使用 matplotlib 生成带图形的 HTML 报告 - +1. 将 nmon 文件传唤成一系列独立的 CSV 文件 +2. 使用 matplotlib 生成带图形的 HTML 报告 下面命令既会生成 CSV 文件，也会生成 HTML 报告： -``` -$> ./pyNmonAnalyzer.py -c -b test.nmon +``` +$ ./pyNmonAnalyzer.py -c -b test.nmon ``` -这会常见一个 `。/data` 目录，其中有一个存放 CSV 文件的目录 ("。/data/csv/")，一个存放 PNG 图片的目录 ("。/data/img/") 以及一个 HTML 报告 ("。/data/report.html")。 +这会创建一个 `./data` 目录，其中有一个存放 CSV 文件的目录 (`./data/csv/`)，一个存放 PNG 图片的目录 (`./data/img/`) 以及一个 HTML 报告 (`./data/report.html`)。 -默认情况下，HTML 报告中会用图片展示 CPU，磁盘繁忙度，内存使用情况和网络传输情况。所有这些都定义在一个自解释的配置文件中 ("report.config")。目前这个工具 h 那不是特别的灵活，因为 CPU 和 MEM 除了 on 和 off 外，无法做其他的配置。不过下一步将会改进作图的方法并允许用户灵活地指定针对哪些数据使用哪种作图方法。 +默认情况下，HTML 报告中会用图片展示 CPU、磁盘繁忙程度、内存使用情况和网络传输情况。所有这些都定义在一个不言自明的配置文件中 (`report.config`)。目前这个工具还不是特别的灵活，因为 CPU 和 MEM 除了 `on` 和 `off` 外，无法做其他的配置。不过下一步将会改进作图的方法并允许用户灵活地指定针对哪些数据使用哪种作图方法。 -### 报告的例子： +### 报告的例子 -[![pyNmonAnalyzer Graph output][6] -**Click to see the full Report**][7] +![pyNmonAnalyzer Graph output][6] + +[点击查看完整报告][7] 目前这些报告还十分的枯燥而且只能打印出基本的几种标记图表，不过它的功能还在不断的完善中。目前在开发的是一个向导来让配置调整变得更容易。如果有任何建议，找到任何 bug 或者有任何功能需求，欢迎与我交流。 @@ -86,7 +89,7 @@ via: https://matthiaslee.com/python-nmon-analyzer-moving-away-from-excel-macros/ 作者：[Matthias Lee][a] 译者：[lujun9972](https://github.com/lujun9972) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 diff --git a/published/201801/20140210 Three steps to learning GDB.md b/published/201801/20140210 Three steps to learning GDB.md new file mode 100644 index 0000000000..01232ea184 --- /dev/null +++ b/published/201801/20140210 Three steps to learning GDB.md @@ -0,0 +1,113 @@ +三步上手 GDB +=============== + +调试 C 程序，曾让我很困扰。然而当我之前在写我的[操作系统][2]时，我有很多的 Bug 需要调试。我很幸运的使用上了 qemu 模拟器，它允许我将调试器附加到我的操作系统。这个调试器就是 `gdb`。 + +我得解释一下，你可以使用 `gdb` 先做一些小事情，因为我发现初学它的时候真的很混乱。我们接下来会在一个小程序中，设置断点，查看内存。. + +### 1、 设断点 + +如果你曾经使用过调试器，那你可能已经会设置断点了。 + +下面是一个我们要调试的程序（虽然没有任何 Bug）: + +``` +#include +void do_thing() { + printf("Hi!\n"); +} +int main() { + do_thing(); +} +``` + +另存为 `hello.c`. 我们可以使用 `dbg` 调试它，像这样： + +``` +bork@kiwi ~> gcc -g hello.c -o hello +bork@kiwi ~> gdb ./hello +``` + +以上是带调试信息编译 `hello.c`（为了 `gdb` 可以更好工作），并且它会给我们醒目的提示符，就像这样： + +``` +(gdb) +``` + +我们可以使用 `break` 命令设置断点，然后使用 `run` 开始调试程序。 + +``` +(gdb) break do_thing +Breakpoint 1 at 0x4004f8 +(gdb) run +Starting program: /home/bork/hello + +Breakpoint 1, 0x00000000004004f8 in do_thing () +``` + +程序暂停在了 `do_thing` 开始的地方。 + +我们可以通过 `where` 查看我们所在的调用栈。 + +``` +(gdb) where +#0 do_thing () at hello.c:3 +#1 0x08050cdb in main () at hello.c:6 +(gdb) +``` + +### 2、 阅读汇编代码 + +使用 `disassemble` 命令，我们可以看到这个函数的汇编代码。棒级了，这是 x86 汇编代码。虽然我不是很懂它，但是 `callq` 这一行是 `printf` 函数调用。 + +``` +(gdb) disassemble do_thing +Dump of assembler code for function do_thing: + 0x00000000004004f4 <+0>: push %rbp + 0x00000000004004f5 <+1>: mov %rsp,%rbp +=> 0x00000000004004f8 <+4>: mov $0x40060c,%edi + 0x00000000004004fd <+9>: callq 0x4003f0 + 0x0000000000400502 <+14>: pop %rbp + 0x0000000000400503 <+15>: retq +``` + +你也可以使用 `disassemble` 的缩写 `disas`。 + +### 3、 查看内存 + +当调试我的内核时，我使用 `gdb` 的主要原因是，以确保内存布局是如我所想的那样。检查内存的命令是 `examine`，或者使用缩写 `x`。我们将使用`x`。 + +通过阅读上面的汇编代码，似乎 `0x40060c` 可能是我们所要打印的字符串地址。我们来试一下。 + +``` +(gdb) x/s 0x40060c +0x40060c: "Hi!" +``` + +的确是这样。`x/s` 中 `/s` 部分，意思是“把它作为字符串展示”。我也可以“展示 10 个字符”，像这样： + +``` +(gdb) x/10c 0x40060c +0x40060c: 72 'H' 105 'i' 33 '!' 0 '\000' 1 '\001' 27 '\033' 3 '\003' 59 ';' +0x400614: 52 '4' 0 '\000' +``` + +你可以看到前四个字符是 `H`、`i`、`!` 和 `\0`，并且它们之后的是一些不相关的东西。 + +我知道 `gdb` 很多其他的东西，但是我仍然不是很了解它，其中 `x` 和 `break` 让我获得很多。你还可以阅读 [do umentation for examining memory][4]。 + +-------------------------------------------------------------------------------- + +via: https://jvns.ca/blog/2014/02/10/three-steps-to-learning-gdb/ + +作者：[Julia Evans][a] +译者：[Torival](https://github.com/Torival) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://jvns.ca +[1]:https://jvns.ca/categories/spytools +[2]:https://jvns.ca/blog/categories/kernel +[3]:https://twitter.com/mgedmin +[4]:https://ftp.gnu.org/old-gnu/Manuals/gdb-5.1.1/html_chapter/gdb_9.html#SEC56 diff --git a/published/20160117 How to use curl command with proxy username-password on Linux- Unix.md b/published/201801/20160117 How to use curl command with proxy username-password on Linux- Unix.md similarity index 100% rename from published/20160117 How to use curl command with proxy username-password on Linux- Unix.md rename to published/201801/20160117 How to use curl command with proxy username-password on Linux- Unix.md diff --git a/published/20160625 Trying out LXD containers on our Ubuntu.md b/published/201801/20160625 Trying out LXD containers on our Ubuntu.md similarity index 100% rename from published/20160625 Trying out LXD containers on our Ubuntu.md rename to published/201801/20160625 Trying out LXD containers on our Ubuntu.md diff --git a/translated/tech/20160808 Top 10 Command Line Games For Linux.md b/published/201801/20160808 Top 10 Command Line Games For Linux.md similarity index 61% rename from translated/tech/20160808 Top 10 Command Line Games For Linux.md rename to published/201801/20160808 Top 10 Command Line Games For Linux.md index 86d5e6fcf7..0368635e73 100644 --- a/translated/tech/20160808 Top 10 Command Line Games For Linux.md +++ b/published/201801/20160808 Top 10 Command Line Games For Linux.md @@ -1,178 +1,196 @@ -Linux 命令行游戏 Top 10 +十大 Linux 命令行游戏 ====== -概要： 本文列举了 **Linux 中最好的命令行游戏**。 -Linux 从来都不是游戏的首选操作系统。尽管近日来 [Linux 的游戏][1] 提供了很多。你可以在 [下载 Linux 游戏][2] 得到许多资源。 +概要： 本文列举了 Linux 中最好的命令行游戏。 -这有专门的 [游戏版 Linux][3]。它确实存在。但是今天，我们并不是要欣赏游戏版 Linux。 +Linux 从来都不是游戏的首选操作系统，尽管近日来 [Linux 的游戏][1]提供了很多，你也可以从许多资源[下载到 Linux 游戏][2]。 + +也有专门的 [游戏版 Linux][3]。没错，确实有。但是今天，我们并不是要欣赏游戏版 Linux。 Linux 有一个超过 Windows 的优势。它拥有一个强大的 Linux 终端。在 Linux 终端上，你可以做很多事情，包括玩 **命令行游戏**。 -当然，毕竟是 Linux 终端的核心爱好者、拥护者。终端游戏轻便，快速，有地狱般的魔力。而这最有意思的事情是，你可以在 Linux 终端上重温大量经典游戏。 - -[推荐阅读：Linux 上游戏，你所需要了解的全部][20] +当然，我们都是 Linux 终端的骨灰粉。终端游戏轻便、快速、有地狱般的魔力。而这最有意思的事情是，你可以在 Linux 终端上重温大量经典游戏。 ### 最好的 Linux 终端游戏 来揭秘这张榜单，找出 Linux 终端最好的游戏。 -### 1. Bastet +#### 1. Bastet -谁还没花上几个小时玩 [俄罗斯方块][4] ？简单而且容易上瘾。 Bastet 就是 Linux 版的俄罗斯方块。 +谁还没花上几个小时玩[俄罗斯方块][4]？它简单而且容易上瘾。 Bastet 就是 Linux 版的俄罗斯方块。 ![Linux 终端游戏 Bastet][5] -使用下面的命令获取 Bastet: +使用下面的命令获取 Bastet： + ``` sudo apt install bastet ``` -运行下列命令，在终端上开始这个游戏: +运行下列命令，在终端上开始这个游戏： + ``` bastet ``` -使用空格键旋转方块，方向键控制方块移动 +使用空格键旋转方块，方向键控制方块移动。 -### 2. Ninvaders +#### 2. Ninvaders -Space Invaders（太空侵略者）。我任记得这个游戏里，和我弟弟（哥哥）在高分之路上扭打。这是最好的街机游戏之一。 +Space Invaders（太空侵略者）。我仍记得这个游戏里，和我兄弟为了最高分而比拼。这是最好的街机游戏之一。 ![Linux 终端游戏 nInvaders][6] 复制粘贴这段代码安装 Ninvaders。 + ``` sudo apt-get install ninvaders ``` -使用下面的命令开始游戏: +使用下面的命令开始游戏： + ``` ninvaders ``` -方向键移动太空飞船。空格键设计外星人。 +方向键移动太空飞船。空格键射击外星人。 [推荐阅读：2016 你可以开始的 Linux 游戏 Top 10][21] -### 3. Pacman4console +#### 3. Pacman4console -是的，这个就是街机之王。Pacman4console 是最受欢迎的街机游戏 Pacman（吃豆豆）终端版。 +是的，这个就是街机之王。Pacman4console 是最受欢迎的街机游戏 Pacman（吃豆人）的终端版。 ![Linux 命令行吃豆豆游戏 Pacman4console][7] 使用以下命令获取 pacman4console： + ``` sudo apt-get install pacman4console ``` -打开终端，建议使用最大的终端界面（29x32）。键入以下命令启动游戏： +打开终端，建议使用最大的终端界面。键入以下命令启动游戏： + ``` pacman4console ``` 使用方向键控制移动。 -### 4. nSnake +#### 4. nSnake 记得在老式诺基亚手机里玩的贪吃蛇游戏吗？ -这个游戏让我保持对手机着迷很长时间。我曾经设计过各种姿态去获得更长的蛇身。 +这个游戏让我在很长时间内着迷于手机。我曾经设计过各种姿态去获得更长的蛇身。 ![nsnake : Linux 终端上的贪吃蛇游戏][8] 我们拥有 [Linux 终端上的贪吃蛇游戏][9] 得感谢 [nSnake][9]。使用下面的命令安装它： + ``` sudo apt-get install nsnake ``` 键入下面的命令开始游戏： + ``` nsnake ``` -使用方向键控制蛇身，获取豆豆。 +使用方向键控制蛇身并喂它。 -### 5. Greed +#### 5. Greed -Greed 有点像精简调加速和肾上腺素的 Tron（类似贪吃蛇的进化版）。 +Greed 有点像 Tron（类似贪吃蛇的进化版），但是减少了速度，也没那么刺激。 -你当前的位置由‘@’表示。你被数字包围了，你可以在四个方向任意移动。你选择的移动方向上标识的数字，就是你能移动的步数。走过的路不能再走，如果你无路可走，游戏结束。 +你当前的位置由闪烁的 ‘@’ 表示。你被数字所环绕，你可以在四个方向任意移动。 -听起来，似乎我让它变得更复杂了。 +你选择的移动方向上标识的数字，就是你能移动的步数。你将重复这个步骤。走过的路不能再走，如果你无路可走，游戏结束。 + +似乎我让它听起来变得更复杂了。 ![Greed : 命令行上的 Tron][10] 通过下列命令获取 Greed： + ``` sudo apt-get install greed ``` 通过下列命令启动游戏，使用方向键控制游戏。 + ``` greed ``` -### 6. Air Traffic Controller +#### 6. Air Traffic Controller -还有什么比做飞行员更有意思的？空中交通管制员。在你的终端中，你可以模拟一个空中要塞。说实话，在终端里管理空中交通蛮有意思的。 +还有什么比做飞行员更有意思的？那就是空中交通管制员。在你的终端中，你可以模拟一个空中交通系统。说实话，在终端里管理空中交通蛮有意思的。 ![Linux 空中交通管理员][11] 使用下列命令安装游戏： + ``` sudo apt-get install bsdgames ``` 键入下列命令启动游戏： + ``` atc ``` ATC 不是孩子玩的游戏。建议查看官方文档。 -### 7. Backgammon（双陆棋） +#### 7. Backgammon（双陆棋） 无论之前你有没有玩过 [双陆棋][12]，你都应该看看这个。 它的说明书和控制手册都非常友好。如果你喜欢，可以挑战你的电脑或者你的朋友。 ![Linux 终端上的双陆棋][13] 使用下列命令安装双陆棋： + ``` sudo apt-get install bsdgames ``` 键入下列命令启动游戏: + ``` backgammon ``` -当你需要提示游戏规则时，回复 ‘y’。 +当你提示游戏规则时，回复 ‘y’ 即可。 -### 8. Moon Buggy +#### 8. Moon Buggy -跳跃。疯狂。欢乐时光不必多言。 +跳跃、开火。欢乐时光不必多言。 ![Moon buggy][14] 使用下列命令安装游戏： + ``` sudo apt-get install moon-buggy ``` 使用下列命令启动游戏： + ``` moon-buggy ``` -空格跳跃，‘a’或者‘l’射击。尽情享受吧。 +空格跳跃，‘a’ 或者 ‘l’射击。尽情享受吧。 -### 9. 2048 +#### 9. 2048 2048 可以活跃你的大脑。[2048][15] 是一个策咯游戏，很容易上瘾。以获取 2048 分为目标。 ![Linux 终端上的 2048][16] 复制粘贴下面的命令安装游戏： + ``` wget https://raw.githubusercontent.com/mevdschee/2048.c/master/2048.c @@ -180,28 +198,28 @@ gcc -o 2048 2048.c ``` 键入下列命令启动游戏： + ``` ./2048 ``` -### 10. Tron +#### 10. Tron 没有动作类游戏，这张榜单怎么可能结束？ ![Linux 终端游戏 Tron][17] -是的，Linux 终端可以实现这种精力充沛的游戏 Tron。为接下来迅捷的反应做准备吧。无需被下载和安装困扰。一个命令即可启动游戏，你只需要一个网络连接 +是的，Linux 终端可以实现这种精力充沛的游戏 Tron。为接下来迅捷的反应做准备吧。无需被下载和安装困扰。一个命令即可启动游戏，你只需要一个网络连接： + ``` ssh sshtron.zachlatta.com ``` -如果由别的在线游戏者，你可以多人游戏。了解更多：[Linux 终端游戏 Tron][18]. +如果有别的在线游戏者，你可以多人游戏。了解更多：[Linux 终端游戏 Tron][18]。 ### 你看上了哪一款? -朋友，Linux 终端游戏 Top 10，都分享给你了。我猜你现在正准备键入 ctrl+alt+T（终端快捷键） 了。榜单中，那个是你最喜欢的游戏？或者为终端提供其他的有趣的事物？尽情分享吧！ - -在 [Abhishek Prakash][19] 回复。 +伙计，十大 Linux 终端游戏都分享给你了。我猜你现在正准备键入 `ctrl+alt+T`（终端快捷键） 了。榜单中那个是你最喜欢的游戏？或者你有其它的终端游戏么？尽情分享吧！ -------------------------------------------------------------------------------- @@ -209,12 +227,12 @@ via: https://itsfoss.com/best-command-line-games-linux/ 作者：[Aquil Roshan][a] 译者：[CYLeft](https://github.com/CYleft) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 [a]:https://itsfoss.com/author/aquil/ -[1]:https://itsfoss.com/linux-gaming-guide/ +[1]:https://linux.cn/article-7316-1.html [2]:https://itsfoss.com/download-linux-games/ [3]:https://itsfoss.com/manjaro-gaming-linux/ [4]:https://en.wikipedia.org/wiki/Tetris diff --git a/published/20161004 What happens when you start a process on Linux.md b/published/201801/20161004 What happens when you start a process on Linux.md similarity index 100% rename from published/20161004 What happens when you start a process on Linux.md rename to published/201801/20161004 What happens when you start a process on Linux.md diff --git a/published/20170131 Book review Ours to Hack and to Own.md b/published/201801/20170131 Book review Ours to Hack and to Own.md similarity index 100% rename from published/20170131 Book review Ours to Hack and to Own.md rename to published/201801/20170131 Book review Ours to Hack and to Own.md diff --git a/published/20170209 INTRODUCING DOCKER SECRETS MANAGEMENT.md b/published/201801/20170209 INTRODUCING DOCKER SECRETS MANAGEMENT.md similarity index 100% rename from published/20170209 INTRODUCING DOCKER SECRETS MANAGEMENT.md rename to published/201801/20170209 INTRODUCING DOCKER SECRETS MANAGEMENT.md diff --git a/translated/tech/20170319 ftrace trace your kernel functions.md b/published/201801/20170319 ftrace trace your kernel functions.md similarity index 79% rename from translated/tech/20170319 ftrace trace your kernel functions.md rename to published/201801/20170319 ftrace trace your kernel functions.md index ccb5b76256..c166a3c513 100644 --- a/translated/tech/20170319 ftrace trace your kernel functions.md +++ b/published/201801/20170319 ftrace trace your kernel functions.md @@ -3,43 +3,42 @@ ftrace：跟踪你的内核函数！ 大家好！今天我们将去讨论一个调试工具：ftrace，之前我的博客上还没有讨论过它。还有什么能比一个新的调试工具更让人激动呢？ -这个非常棒的 ftrace 并不是个新的工具！它大约在 Linux 的 2.6 内核版本中就有了，时间大约是在 2008 年。[这里是我用谷歌能找到的一些文档][10]。因此，如果你是一个调试系统的“老手”，可能早就已经使用它了！ +这个非常棒的 ftrace 并不是个新的工具！它大约在 Linux 的 2.6 内核版本中就有了，时间大约是在 2008 年。[这一篇是我用谷歌能找到的最早的文档][10]。因此，如果你是一个调试系统的“老手”，可能早就已经使用它了！ -我知道，ftrace 已经存在了大约 2.5 年了，但是还没有真正的去学习它。假设我明天要召开一个专题研究会，那么，关于 ftrace 应该讨论些什么？因此，今天是时间去讨论一下它了！ +我知道，ftrace 已经存在了大约 2.5 年了（LCTT 译注：距本文初次写作时），但是还没有真正的去学习它。假设我明天要召开一个专题研究会，那么，关于 ftrace 应该讨论些什么？因此，今天是时间去讨论一下它了！ ### 什么是 ftrace？ ftrace 是一个 Linux 内核特性，它可以让你去跟踪 Linux 内核的函数调用。为什么要这么做呢？好吧，假设你调试一个奇怪的问题，而你已经得到了你的内核版本中这个问题在源代码中的开始的位置，而你想知道这里到底发生了什么？ -每次在调试的时候，我并不会经常去读内核源代码，但是，极个别的情况下会去读它！例如，本周在工作中，我有一个程序在内核中卡死了。查看到底是调用了什么函数、哪些系统涉及其中，能够帮我更好的理解在内核中发生了什么！（在我的那个案例中，它是虚拟内存系统） +每次在调试的时候，我并不会经常去读内核源代码，但是，极个别的情况下会去读它！例如，本周在工作中，我有一个程序在内核中卡死了。查看到底是调用了什么函数，能够帮我更好的理解在内核中发生了什么，哪些系统涉及其中！（在我的那个案例中，它是虚拟内存系统）。 -我认为 ftrace 是一个十分好用的工具（它肯定没有 strace 那样广泛被使用，使用难度也低于它），但是它还是值得你去学习。因此，让我们开始吧！ +我认为 ftrace 是一个十分好用的工具（它肯定没有 `strace` 那样使用广泛，也比它难以使用），但是它还是值得你去学习。因此，让我们开始吧！ ### 使用 ftrace 的第一步 -不像 strace 和 perf，ftrace 并不是真正的 **程序** – 你不能只运行 `ftrace my_cool_function`。那样太容易了！ +不像 `strace` 和 `perf`，ftrace 并不是真正的 **程序** – 你不能只运行 `ftrace my_cool_function`。那样太容易了！ -如果你去读 [使用 Ftrace 调试内核][11]，它会告诉你从 `cd /sys/kernel/debug/tracing` 开始，然后做很多文件系统的操作。 +如果你去读 [使用 ftrace 调试内核][11]，它会告诉你从 `cd /sys/kernel/debug/tracing` 开始，然后做很多文件系统的操作。 -对于我来说，这种办法太麻烦 – 使用 ftrace 的一个简单例子应该像这样： +对于我来说，这种办法太麻烦——一个使用 ftrace 的简单例子像是这样： ``` cd /sys/kernel/debug/tracing echo function > current_tracer echo do_page_fault > set_ftrace_filter cat trace - ``` -这个文件系统到跟踪系统的接口（“给这些神奇的文件赋值，然后该发生的事情就会发生”）理论上看起来似乎可用，但是它不是我的首选方式。 +这个文件系统是跟踪系统的接口（“给这些神奇的文件赋值，然后该发生的事情就会发生”）理论上看起来似乎可用，但是它不是我的首选方式。 -幸运的是，ftrace 团队也考虑到这个并不友好的用户界面，因此，它有了一个更易于使用的界面，它就是 **trace-cmd**！！！trace-cmd 是一个带命令行参数的普通程序。我们后面将使用它！我在 LWN 上找到了一个 trace-cmd 的使用介绍：[trace-cmd: Ftrace 的一个前端][12]。 +幸运的是，ftrace 团队也考虑到这个并不友好的用户界面，因此，它有了一个更易于使用的界面，它就是 `trace-cmd`！！！`trace-cmd` 是一个带命令行参数的普通程序。我们后面将使用它！我在 LWN 上找到了一个 `trace-cmd` 的使用介绍：[trace-cmd: Ftrace 的一个前端][12]。 -### 开始使用 trace-cmd：让 trace 仅跟踪一个函数 +### 开始使用 trace-cmd：让我们仅跟踪一个函数 首先，我需要去使用 `sudo apt-get install trace-cmd` 安装 `trace-cmd`，这一步很容易。 -对于第一个 ftrace 的演示，我决定去了解我的内核如何去处理一个页面故障。当 Linux 分配内存时，它经常偷懒，（“你并不是  _真的_  计划去使用内存，对吗？”）。这意味着，当一个应用程序尝试去对分配给它的内存进行写入时，就会发生一个页面故障，而这个时候，内核才会真正的为应用程序去分配物理内存。 +对于第一个 ftrace 的演示，我决定去了解我的内核如何去处理一个页面故障。当 Linux 分配内存时，它经常偷懒，（“你并不是_真的_计划去使用内存，对吗？”）。这意味着，当一个应用程序尝试去对分配给它的内存进行写入时，就会发生一个页面故障，而这个时候，内核才会真正的为应用程序去分配物理内存。 我们开始使用 `trace-cmd` 并让它跟踪 `do_page_fault` 函数！ @@ -47,7 +46,6 @@ cat trace $ sudo trace-cmd record -p function -l do_page_fault plugin 'function' Hit Ctrl^C to stop recording - ``` 我将它运行了几秒钟，然后按下了 `Ctrl+C`。 让我大吃一惊的是，它竟然产生了一个 2.5MB 大小的名为 `trace.dat` 的跟踪文件。我们来看一下这个文件的内容！ @@ -68,7 +66,7 @@ $ sudo trace-cmd report ``` -看起来很整洁 – 它展示了进程名（chrome）、进程 ID （15144）、CPU（000）、以及它跟踪的函数。 +看起来很整洁 – 它展示了进程名（chrome）、进程 ID（15144）、CPU ID（000），以及它跟踪的函数。 通过察看整个文件，（`sudo trace-cmd report | grep chrome`）可以看到，我们跟踪了大约 1.5 秒，在这 1.5 秒的时间段内，Chrome 发生了大约 500 个页面故障。真是太酷了！这就是我们做的第一个 ftrace！ @@ -81,14 +79,13 @@ $ sudo trace-cmd report ``` sudo trace-cmd record --help # I read the help! sudo trace-cmd record -p function -P 25314 # record for PID 25314 - ``` `sudo trace-cmd report` 输出了 18,000 行。如果你对这些感兴趣，你可以看 [这里是所有的 18,000 行的输出][13]。 18,000 行太多了，因此，在这里仅摘录其中几行。 -当系统调用 `clock_gettime` 运行时，都发生了什么。 +当系统调用 `clock_gettime` 运行的时候，都发生了什么： ``` compat_SyS_clock_gettime @@ -99,7 +96,6 @@ sudo trace-cmd record -p function -P 25314 # record for PID 25314 __getnstimeofday64 arch_counter_read __compat_put_timespec - ``` 这是与进程调试相关的一些东西： @@ -128,10 +124,9 @@ sudo trace-cmd record -p function -P 25314 # record for PID 25314 ``` sudo trace-cmd record -p function_graph -P 25314 - ``` -同样，这里只是一个片断（这次来自 futex 代码） +同样，这里只是一个片断（这次来自 futex 代码）： ``` | futex_wake() { @@ -149,7 +144,6 @@ sudo trace-cmd record -p function_graph -P 25314 5.250 us | } 0.583 us | put_page(); + 24.208 us | } - ``` 我们看到在这个示例中，在 `futex_wake` 后面调用了 `get_futex_key`。这是在源代码中真实发生的事情吗？我们可以检查一下！！[这里是在 Linux 4.4 中 futex_wake 的定义][15] (我的内核版本是 4.4）。 @@ -170,7 +164,6 @@ futex_wake(u32 __user *uaddr, unsigned int flags, int nr_wake, u32 bitset) return -EINVAL; ret = get_futex_key(uaddr, flags & FLAGS_SHARED, &key, VERIFY_READ); - ``` 如你所见，在 `futex_wake` 中的第一个函数调用真的是 `get_futex_key`！ 太棒了！相比阅读内核代码，阅读函数跟踪肯定是更容易的找到结果的办法，并且让人高兴的是，还能看到所有的函数用了多长时间。 @@ -183,7 +176,7 @@ futex_wake(u32 __user *uaddr, unsigned int flags, int nr_wake, u32 bitset) 现在，我们已经知道了怎么去跟踪内核中的函数，真是太酷了！ -还有一类我们可以跟踪的东西！有些事件与我们的函数调用并不相符。例如，你可能想去知道当一个程序被调度进入或者离开 CPU 时，都发生了什么事件！你可能想通过“盯着”函数调用计算出来，但是，我告诉你，不可行！ +还有一类我们可以跟踪的东西！有些事件与我们的函数调用并不相符。例如，你可能想知道当一个程序被调度进入或者离开 CPU 时，都发生了什么事件！你可能想通过“盯着”函数调用计算出来，但是，我告诉你，不可行！ 由于函数也为你提供了几种事件，因此，你可以看到当重要的事件发生时，都发生了什么事情。你可以使用 `sudo cat /sys/kernel/debug/tracing/available_events` 来查看这些事件的一个列表。  @@ -193,7 +186,6 @@ futex_wake(u32 __user *uaddr, unsigned int flags, int nr_wake, u32 bitset) sudo cat /sys/kernel/debug/tracing/available_events sudo trace-cmd record -e sched:sched_switch sudo trace-cmd report - ``` 输出如下： @@ -207,23 +199,23 @@ sudo trace-cmd report ``` -现在，可以很清楚地看到这些切换，从 PID 24817 -> 15144 -> kernel -> 24817 -> 1561 -> 15114\。(所有的这些事件都发生在同一个 CPU 上） +现在，可以很清楚地看到这些切换，从 PID 24817 -> 15144 -> kernel -> 24817 -> 1561 -> 15114。(所有的这些事件都发生在同一个 CPU 上）。 ### ftrace 是如何工作的？ -ftrace 是一个动态跟踪系统。当启动 ftracing 去跟踪内核函数时，**函数的代码会被改变**。因此 – 我们假设去跟踪 `do_page_fault` 函数。内核将在那个函数的汇编代码中插入一些额外的指令，以便每次该函数被调用时去提示跟踪系统。内核之所以能够添加额外的指令的原因是，Linux 将额外的几个 NOP 指令编译进每个函数中，因此，当需要的时候，这里有添加跟踪代码的地方。 +ftrace 是一个动态跟踪系统。当我们开始 ftrace 内核函数时，**函数的代码会被改变**。让我们假设去跟踪 `do_page_fault` 函数。内核将在那个函数的汇编代码中插入一些额外的指令，以便每次该函数被调用时去提示跟踪系统。内核之所以能够添加额外的指令的原因是，Linux 将额外的几个 NOP 指令编译进每个函数中，因此，当需要的时候，这里有添加跟踪代码的地方。 这是一个十分复杂的问题，因为，当不需要使用 ftrace 去跟踪我的内核时，它根本就不影响性能。而当我需要跟踪时，跟踪的函数越多，产生的开销就越大。 （或许有些是不对的，但是，我认为的 ftrace 就是这样工作的） -### 更容易地使用 ftrace：brendan gregg 的工具 & kernelshark +### 更容易地使用 ftrace：brendan gregg 的工具及 kernelshark 正如我们在文件中所讨论的，你需要去考虑很多的关于单个的内核函数/事件直接使用 ftrace 都做了些什么。能够做到这一点很酷！但是也需要做大量的工作！ -Brendan Gregg （我们的 linux 调试工具“大神”）有个工具仓库，它使用 ftrace 去提供关于像 I/O 延迟这样的各种事情的信息。这是它在 GitHub 上全部的 [perf-tools][16] 仓库。 +Brendan Gregg （我们的 Linux 调试工具“大神”）有个工具仓库，它使用 ftrace 去提供关于像 I/O 延迟这样的各种事情的信息。这是它在 GitHub 上全部的 [perf-tools][16] 仓库。 -这里有一个权衡（tradeoff），那就是这些工具易于使用，但是被限制仅用于 Brendan Gregg 认可的事情。决定将它做成一个工具，那需要做很多的事情！:) +这里有一个权衡，那就是这些工具易于使用，但是你被限制仅能用于 Brendan Gregg 认可并做到工具里面的方面。它包括了很多方面！:) 另一个工具是将 ftrace 的输出可视化，做的比较好的是 [kernelshark][17]。我还没有用过它，但是看起来似乎很有用。你可以使用 `sudo apt-get install kernelshark` 来安装它。 @@ -236,30 +228,22 @@ Brendan Gregg （我们的 linux 调试工具“大神”）有个工具仓库 最后，这里是我找到的一些 ftrace 方面的文章。它们大部分在 LWN （Linux 新闻周刊）上，它是 Linux 的一个极好的资源（你可以购买一个 [订阅][18]！） * [使用 Ftrace 调试内核 - part 1][1] (Dec 2009, Steven Rostedt) - * [使用 Ftrace 调试内核 - part 2][2] (Dec 2009, Steven Rostedt) - * [Linux 函数跟踪器的秘密][3] (Jan 2010, Steven Rostedt) - * [trace-cmd：Ftrace 的一个前端][4] (Oct 2010, Steven Rostedt) - * [使用 KernelShark 去分析实时调试器][5] (2011, Steven Rostedt) - * [Ftrace: 神秘的开关][6] (2014, Brendan Gregg) - * 内核文档：（它十分有用） [Documentation/ftrace.txt][7] - * 你能跟踪的事件的文档 [Documentation/events.txt][8] - * linux 内核开发上的一些 ftrace 设计文档 （不是有用，而是有趣！) [Documentation/ftrace-design.txt][9] -------------------------------------------------------------------------------- via: https://jvns.ca/blog/2017/03/19/getting-started-with-ftrace/ -作者：[Julia Evans ][a] +作者：[Julia Evans][a] 译者：[qhwdw](https://github.com/qhwdw) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 diff --git a/published/20170426 Important Docker commands for Beginners.md b/published/201801/20170426 Important Docker commands for Beginners.md similarity index 100% rename from published/20170426 Important Docker commands for Beginners.md rename to published/201801/20170426 Important Docker commands for Beginners.md diff --git a/published/20170502 A beginner-s guide to Raspberry Pi 3.md b/published/201801/20170502 A beginner-s guide to Raspberry Pi 3.md similarity index 100% rename from published/20170502 A beginner-s guide to Raspberry Pi 3.md rename to published/201801/20170502 A beginner-s guide to Raspberry Pi 3.md diff --git a/published/20170512 Which Official Ubuntu Flavor Is Best for You.md b/published/201801/20170512 Which Official Ubuntu Flavor Is Best for You.md similarity index 100% rename from published/20170512 Which Official Ubuntu Flavor Is Best for You.md rename to published/201801/20170512 Which Official Ubuntu Flavor Is Best for You.md diff --git a/published/20170515 Commands to check System & Hardware Information.md b/published/201801/20170515 Commands to check System & Hardware Information.md similarity index 100% rename from published/20170515 Commands to check System & Hardware Information.md rename to published/201801/20170515 Commands to check System & Hardware Information.md diff --git a/published/20170524 View Counting at Reddit.md b/published/201801/20170524 View Counting at Reddit.md similarity index 100% rename from published/20170524 View Counting at Reddit.md rename to published/201801/20170524 View Counting at Reddit.md diff --git a/translated/tech/20170524 Working with Vi-Vim Editor - Advanced concepts.md b/published/201801/20170524 Working with Vi-Vim Editor - Advanced concepts.md similarity index 63% rename from translated/tech/20170524 Working with Vi-Vim Editor - Advanced concepts.md rename to published/201801/20170524 Working with Vi-Vim Editor - Advanced concepts.md index d31527b055..a9fbe555c3 100644 --- a/translated/tech/20170524 Working with Vi-Vim Editor - Advanced concepts.md +++ b/published/201801/20170524 Working with Vi-Vim Editor - Advanced concepts.md @@ -1,10 +1,11 @@ 使用 Vi/Vim 编辑器：高级概念 ====== + 早些时候我们已经讨论了一些关于 VI/VIM 编辑器的基础知识，但是 VI 和 VIM 都是非常强大的编辑器，还有很多其他的功能可以和编辑器一起使用。在本教程中，我们将学习 VI/VIM 编辑器的一些高级用法。 （**推荐阅读**：[使用 VI 编辑器：基础知识] [1]） -## 使用 VI/VIM 编辑器打开多个文件 +### 使用 VI/VIM 编辑器打开多个文件 要打开多个文件，命令将与打开单个文件相同。我们只要添加第二个文件的名称。 @@ -12,59 +13,65 @@ $ vi file1 file2 file 3 ``` -要浏览到下一个文件，我们可以使用 +要浏览到下一个文件，我们可以（在 vim 命令模式中）使用： ``` -$ :n +:n ``` 或者我们也可以使用 ``` -$ :e filename +:e filename ``` -## 在编辑器中运行外部命令 +### 在编辑器中运行外部命令 -我们可以在 vi 编辑器内部运行外部的 Linux/Unix 命令，也就是说不需要退出编辑器。要在编辑器中运行命令，如果在插入模式下，先返回到命令模式，我们使用 BANG 也就是 “!” 接着是需要使用的命令。运行命令的语法是： +我们可以在 vi 编辑器内部运行外部的 Linux/Unix 命令，也就是说不需要退出编辑器。要在编辑器中运行命令，如果在插入模式下，先返回到命令模式，我们使用 BANG 也就是 `!` 接着是需要使用的命令。运行命令的语法是： ``` -$ :! command +:! command ``` -这是一个例子 +这是一个例子： ``` -$ :! df -H +:! df -H ``` -## 根据模板搜索 +### 根据模板搜索 要在文本文件中搜索一个单词或模板，我们在命令模式下使用以下两个命令： - * 命令 “/” 代表正向搜索模板 - - * 命令 “?” 代表正向搜索模板 - +* 命令 `/` 代表正向搜索模板 +* 命令 `?` 代表正向搜索模板 这两个命令都用于相同的目的，唯一不同的是它们搜索的方向。一个例子是： - `$ :/ search pattern` (如果在文件的开头) - - `$ :? search pattern` (如果在文件末尾) - -## 搜索并替换一个模板 - -我们可能需要搜索和替换我们的文本中的单词或模板。我们不是从整个文本中找到单词的出现的地方并替换它，我们可以在命令模式中使用命令来自动替换单词。使用搜索和替换的语法是： +如果在文件的开头向前搜索， ``` -$ :s/pattern_to_be_found/New_pattern/g +:/ search pattern +``` + +如果在文件末尾向后搜索， + +``` +:? search pattern +``` + +### 搜索并替换一个模式 + +我们可能需要搜索和替换我们的文本中的单词或模式。我们不是从整个文本中找到单词的出现的地方并替换它，我们可以在命令模式中使用命令来自动替换单词。使用搜索和替换的语法是： + +``` +:s/pattern_to_be_found/New_pattern/g ``` 假设我们想要将单词 “alpha” 用单词 “beta” 代替，命令就是这样： ``` -$ :s/alpha/beta/g +:s/alpha/beta/g ``` 如果我们只想替换第一个出现的 “alpha”，那么命令就是： @@ -73,31 +80,35 @@ $ :s/alpha/beta/g $ :s/alpha/beta/ ``` -## 使用 set 命令 +### 使用 set 命令 我们也可以使用 set 命令自定义 vi/vim 编辑器的行为和外观。下面是一些可以使用 set 命令修改 vi/vim 编辑器行为的选项列表： - `$ :set ic ` 在搜索时忽略大小写 +``` +:set ic ' 在搜索时忽略大小写 - `$ :set smartcase ` 搜索强制区分大小写 +:set smartcase ' 搜索强制区分大小写 - `$ :set nu` 在每行开始显示行号 +:set nu ' 在每行开始显示行号 - `$ :set hlsearch ` 高亮显示匹配的单词 +:set hlsearch ' 高亮显示匹配的单词 - `$ : set ro ` 将文件类型更改为只读 +:set ro ' 将文件类型更改为只读 - `$ : set term ` 打印终端类型 +:set term ' 打印终端类型 - `$ : set ai ` 设置自动缩进 +:set ai ' 设置自动缩进 - `$ :set noai ` 取消自动缩进 +:set noai ' 取消自动缩进 +``` 其他一些修改 vi 编辑器的命令是： - `$ :colorscheme ` 用来改变编辑器的配色方案 。（仅适用于 VIM 编辑器） +``` +:colorscheme ' 用来改变编辑器的配色方案 。（仅适用于 VIM 编辑器） - `$ :syntax on ` 为 .xml、.html 等文件打开颜色方案。（仅适用于VIM编辑器） +:syntax on ' 为 .xml、.html 等文件打开颜色方案。（仅适用于VIM编辑器） +``` 这篇结束了本系列教程，请在下面的评论栏中提出你的疑问/问题或建议。 @@ -108,7 +119,7 @@ via: http://linuxtechlab.com/working-vivim-editor-advanced-concepts/ 作者：[Shusain][a] 译者：[geekpi](https://github.com/geekpi) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 diff --git a/published/201801/20170526 Creating a YUM repository from ISO - Online repo.md b/published/201801/20170526 Creating a YUM repository from ISO - Online repo.md new file mode 100644 index 0000000000..2e11780ac4 --- /dev/null +++ b/published/201801/20170526 Creating a YUM repository from ISO - Online repo.md @@ -0,0 +1,116 @@ +从 ISO 和在线仓库创建一个 Yum 仓库 +====== + +Yum 是 Centos/RHEL/Fedora 中最重要的工具之一。尽管在 Fedora 的最新版本中，它已经被 DNF 所取代，但这并不意味着它自生自灭了。它仍然被广泛用于安装 rpm 包，我们已经在前面的教程（[**在这里阅读**] [1]）中用示例讨论了 Yum。 + +在本教程中，我们将学习创建一个本地 Yum 仓库，首先使用系统的 ISO 镜像，然后创建一个在线 Yum 仓库的镜像。 + +### 用 DVD ISO 创建 Yum + +我们在本教程中使用 Centos 7 dvd，同样的过程也应该可以用在 RHEL 7 上。 + +首先在根文件夹中创建一个名为 Yum 的目录 + +``` +$ mkdir /YUM +``` + +然后挂载 Centos 7 ISO： + +``` +$ mount -t iso9660 -o loop /home/dan/Centos-7-x86_x64-DVD.iso /mnt/iso/ +``` + +接下来，从挂载的 ISO 中复制软件包到 `/YUM` 中。当所有的软件包都被复制到系统中后，我们将安装创建 Yum 所需的软件包。打开 `/YUM` 并安装以下 RPM 包： + +``` +$ rpm -ivh deltarpm +$ rpm -ivh python-deltarpm +$ rpm -ivh createrepo +``` + +安装完成后，我们将在 `/etc/yum.repos.d` 中创建一个名 为 `local.repo` 的文件，其中包含所有的 Yum 信息。 + +``` +$ vi /etc/yum.repos.d/local.repo +``` + +``` +LOCAL REPO] +Name=Local YUM +baseurl=file:///YUM +gpgcheck=0 +enabled=1 +``` + +保存并退出文件。接下来，我们将通过运行以下命令来创建仓库数据。 + +``` +$ createrepo -v /YUM +``` + +创建仓库数据需要一些时间。一切完成后，请运行： + +``` +$ yum clean all +``` + +清理缓存，然后运行： + +``` +$ yum repolist +``` + +检查所有仓库列表。你应该在列表中看到 `local.repo`。 + + +### 使用在线仓库创建镜像 Yum 仓库 + +创建在线 Yum 的过程与使用 ISO 镜像创建 Yum 类似，只是我们将从在线仓库而不是 ISO 中获取 rpm 软件包。 + +首先，我们需要找到一个在线仓库来获取最新的软件包。建议你找一个离你位置最近的在线 Yum 仓库，以优化下载速度。我们将使用下面的镜像，你可以从 [CENTOS 镜像列表][2]中选择一个离你最近的镜像。 + +选择镜像之后，我们将使用 `rsync` 将该镜像与我们的系统同步，但在此之前，请确保你服务器上有足够的空间。 + +``` +$ rsync -avz rsync://mirror.fibergrid.in/centos/7.2/os/x86_64/Packages/s/ /YUM +``` + +同步将需要相当长一段时间（也许一个小时），这取决于你互联网的速度。同步完成后，我们将更新我们的仓库数据。 + +``` +$ createrepo - v /YUM +``` + +我们的 Yum 已经可以使用了。我们可以创建一个 cron 任务来根据你的需求每天或每周定时地自动更新仓库数据。 + +要创建一个用于同步仓库的 cron 任务，请运行： + +``` +$ crontab -e +``` + +并添加以下行 + +``` +30 12 * * * rsync -avz http://mirror.centos.org/centos/7/os/x86_64/Packages/ /YUM +``` + +这会在每晚 12:30 同步 Yum。还请记住在 `/etc/yum.repos.d` 中创建仓库配置文件，就像我们上面所做的一样。 + +就是这样，你现在使用你自己的 Yum 仓库了。如果你喜欢它，请分享这篇文章，并在下面的评论栏留下你的意见/疑问。 + + +-------------------------------------------------------------------------------- + +via: http://linuxtechlab.com/creating-yum-repository-iso-online-repo/ + +作者：[Shusain][a] +译者：[geekpi](https://github.com/geekpi) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://linuxtechlab.com/author/shsuain/ +[1]:http://linuxtechlab.com/using-yum-command-examples/ +[2]:http://mirror.centos.org/centos/ diff --git a/translated/tech/20170607 Why Car Companies Are Hiring Computer Security Experts.md b/published/201801/20170607 Why Car Companies Are Hiring Computer Security Experts.md similarity index 67% rename from translated/tech/20170607 Why Car Companies Are Hiring Computer Security Experts.md rename to published/201801/20170607 Why Car Companies Are Hiring Computer Security Experts.md index ba5a9e2c2e..c41aefb701 100644 --- a/translated/tech/20170607 Why Car Companies Are Hiring Computer Security Experts.md +++ b/published/201801/20170607 Why Car Companies Are Hiring Computer Security Experts.md @@ -1,29 +1,29 @@ 为什么车企纷纷招聘计算机安全专家 ============================================================ -Photo ![](https://static01.nyt.com/images/2017/06/08/business/08BITS-GURUS1/08BITS-GURUS1-superJumbo.jpg) -来自 CloudFlare 公司的网络安全专家 Marc Rogers（左）和来自 Lookout 的 Kevin Mahaffey 能够通过直接连接在汽车上的笔记本电脑控制特斯拉的进行许多操作。图为他们在 CloudFlare 的大厅里的的熔岩灯前的合影，这些熔岩灯被用来生成密匙。纽约时报 CreditChristie Hemm Klok 拍摄 -大约在七年前，伊朗的数位顶级核科学家经历过一系列形式类似的暗杀：凶手的摩托车接近他们乘坐的汽车，把磁性炸弹吸附在汽车上，然后逃离并引爆炸弹。 +来自 CloudFlare 公司的网络安全专家 Marc Rogers（左）和来自 Lookout 的 Kevin Mahaffey 能够通过直接连接在汽车上的笔记本电脑控制特斯拉汽车进行许多操作。图为他们在 CloudFlare 的大厅里的的熔岩灯前的合影，这些熔岩灯被用来生成密匙。（纽约时报 CreditChristie Hemm Klok 拍摄） + +大约在七年前，伊朗的几位顶级核科学家经历过一系列形式类似的暗杀：凶手的摩托车接近他们乘坐的汽车，把磁性炸弹吸附在汽车上，然后逃离并引爆炸弹。 安全专家们警告人们，再过 7 年，凶手们不再需要摩托车或磁性炸弹。他们所需要的只是一台笔记本电脑和发送给无人驾驶汽车的一段代码——让汽车坠桥、被货车撞扁或者在高速公路上突然抛锚。 汽车制造商眼中的无人驾驶汽车。在黑客眼中只是一台可以达到时速 100 公里的计算机。 -网络安全公司CloudFlare的首席安全研究员马克·罗杰斯（Marc Rogers）说：“它们已经不再是汽车了。它们是装在车轮上的数据中心。从外界接收的每一条数据都可以作为黑客的攻击载体。“ +网络安全公司 CloudFlare 的首席安全研究员马克·罗杰斯Marc Rogers说：“它们已经不再是汽车了。它们是装在车轮上的数据中心。从外界接收的每一条数据都可以作为黑客的攻击载体。“ -两年前，两名“白帽”黑客——寻找系统漏洞并修复它们的研究员，而不是利用漏洞来犯罪的破坏者（Cracker）——成功地在数里之外用电脑获得了一辆 Jeep Cherokee 的控制权。他们控制汽车撞击一个放置在高速公路中央的假人（在场景设定中是一位紧张的记者），直接终止了假人的一生。 +两年前，两名“白帽”黑客（寻找系统漏洞并修复它们的研究员，而不是利用漏洞来犯罪的破坏者Cracker）成功地在数里之外用电脑获得了一辆 Jeep Cherokee 的控制权。他们控制汽车撞击一个放置在高速公路中央的假人（在场景设定中是一位紧张的记者），直接终止了假人的一生。 黑客 Chris Valasek 和 Charlie Miller（现在是 Uber 和滴滴的安全研究人员）发现了一条 [由 Jeep 娱乐系统通向仪表板的电路][10]。他们利用这条线路控制了车辆转向、刹车和变速——他们在高速公路上撞击假人所需的一切。 -Miller 先生上周日在 Twitter 上写道：“汽车被黑客入侵成为头条新闻，但是人们要清楚，没有人的汽车被坏人入侵过。 这些只是研究人员的测试。” +Miller 先生上周日在 Twitter 上写道：“汽车被黑客入侵成为头条新闻，但是人们要清楚，没有谁的汽车被坏人入侵过。 这些只是研究人员的测试。” -尽管如此，Miller 和 Valasek 的研究使 Jeep 汽车的制造商菲亚特克莱斯勒（Fiat Chrysler）付出了巨大的代价，因为这个安全漏洞，菲亚特克莱斯勒被迫召回了 140 万辆汽车。 +尽管如此，Miller 和 Valasek 的研究使 Jeep 汽车的制造商菲亚特克莱斯勒Fiat Chrysler付出了巨大的代价，因为这个安全漏洞，菲亚特克莱斯勒被迫召回了 140 万辆汽车。 -毫无疑问，后来通用汽车首席执行官玛丽·巴拉（Mary Barra）把网络安全作为公司的首要任务。现在，计算机网络安全领域的人才在汽车制造商和高科技公司推进的无人驾驶汽车项目中的需求量很大。 +毫无疑问，后来通用汽车首席执行官玛丽·巴拉Mary Barra把网络安全作为公司的首要任务。现在，计算机网络安全领域的人才在汽车制造商和高科技公司推进的无人驾驶汽车项目中的需求量很大。 -优步 、[特斯拉][11]、苹果和中国的滴滴一直在积极招聘像 Miller 先生和 Valasek 先生这样的白帽黑客，传统的网络安全公司和学术界也有这样的趋势。 +优步 、特斯拉、苹果和中国的滴滴一直在积极招聘像 Miller 先生和 Valasek 先生这样的白帽黑客，传统的网络安全公司和学术界也有这样的趋势。 去年，特斯拉挖走了苹果 iOS 操作系统的安全经理 Aaron Sigel。优步挖走了 Facebook 的白帽黑客 Chris Gates。Miller 先生在发现 Jeep 的漏洞后就职于优步，然后被滴滴挖走。计算机安全领域已经有数十名优秀的工程师加入无人驾驶汽车项目研究的行列。 @@ -31,19 +31,19 @@ Miller 先生说，他离开了优步的一部分原因是滴滴给了他更自 Miller 星期六在 Twitter 上写道：“汽车制造商对待网络攻击的威胁似乎更加严肃，但我仍然希望有更大的透明度。” -像许多大型科技公司一样，特斯拉和菲亚特克莱斯勒也开始给那些发现并提交漏洞的黑客们提供奖励。通用汽车公司也做了类似的事情，但批评人士认为通用汽车公司的计划与科技公司提供的计划相比诚意不足，迄今为止还收效甚微。 +像许多大型科技公司一样，特斯拉和菲亚特克莱斯勒也开始给那些发现并提交漏洞的黑客们提供奖励。通用汽车公司也做了类似的事情，但批评人士认为通用汽车公司的计划与科技公司们提供的计划相比诚意不足，迄今为止还收效甚微。 在 Miller 和 Valasek 发现 Jeep 漏洞的一年后，他们又向人们演示了所有其他可能危害乘客安全的方式，包括劫持车辆的速度控制系统，猛打方向盘或在高速行驶下拉动手刹——这一切都是由汽车外的电脑操作的。（在测试中使用的汽车最后掉进路边的沟渠，他们只能寻求当地拖车公司的帮助） 虽然他们必须在 Jeep 车上才能做到这一切，但这也证明了入侵的可能性。 -在 Jeep 被入侵之前，华盛顿大学和加利福尼亚大学圣地亚哥分校的安全研究人员第一个通过蓝牙远程控制轿车并控制其刹车。研究人员警告汽车公司：汽车联网程度越高，被入侵的可能性就越大。 +在 Jeep 被入侵之前，华盛顿大学和加利福尼亚大学圣地亚哥分校的[安全研究人员][12]第一个通过蓝牙远程控制轿车并控制其刹车。研究人员警告汽车公司：汽车联网程度越高，被入侵的可能性就越大。 -2015年，安全研究人员们发现了入侵高度软件化的特斯拉 Model S 的途径。Rogers 先生和网络安全公司 Lookout 的首席技术官凯文·马哈菲（Kevin Mahaffey）找到了一种通过直接连接在汽车上的笔记本电脑控制特斯拉汽车的方法。 +2015 年，安全研究人员们发现了入侵高度软件化的特斯拉 Model S 的途径。Rogers 先生和网络安全公司 Lookout 的首席技术官凯文·马哈菲Kevin Mahaffey找到了一种通过直接连接在汽车上的笔记本电脑控制特斯拉汽车的方法。 -一年后，来自中国腾讯的一支团队做了更进一步的尝试。他们入侵了一辆行驶中的特斯拉 Model S 并控制了其刹车器。和 Jeep 不同，特斯拉可以通过远程安装补丁来修复安全漏洞，这使得黑客的远程入侵也变的可能。 +一年后，来自中国腾讯的一支团队做了更进一步的尝试。他们入侵了一辆行驶中的特斯拉 Model S 并控制了其刹车器达12 米远。和 Jeep 不同，特斯拉可以通过远程安装补丁来修复那些可能被黑的安全漏洞。 -以上所有的例子中，入侵者都是无恶意的白帽黑客或者安全研究人员。但是给无人驾驶汽车制造商的教训是惨重的。 +以上所有的例子中，入侵者都是无恶意的白帽黑客或者安全研究人员，但是给无人驾驶汽车制造商的教训是惨重的。 黑客入侵汽车的动机是无穷的。在得知 Rogers 先生和 Mahaffey 先生对特斯拉 Model S 的研究之后，一位中国 app 开发者和他们联系、询问他们是否愿意分享或者出售他们发现的漏洞。（这位 app 开发者正在寻找后门，试图在特斯拉的仪表盘上偷偷安装 app） @@ -51,25 +51,25 @@ Miller 星期六在 Twitter 上写道：“汽车制造商对待网络攻击的 但随着越来越多的无人驾驶和半自动驾驶的汽车驶入公路，它们将成为更有价值的目标。安全专家警告道：无人驾驶汽车面临着更复杂、更多面的入侵风险，每一辆新无人驾驶汽车的加入，都使这个系统变得更复杂，而复杂性不可避免地带来脆弱性。 -20年前，平均每辆汽车有100万行代码，通用汽车公司的2010雪佛兰Volt有大约1000万行代码——比一架F-35战斗机的代码还要多。 +20 年前，平均每辆汽车有 100 万行代码，通用汽车公司的 2010 [雪佛兰 Volt][13] 有大约 1000 万行代码——比一架 [F-35 战斗机][14]的代码还要多。 -如今， 平均每辆汽车至少有1亿行代码。无人驾驶汽车公司预计不久以后它们将有2亿行代码。当你停下来考虑：平均每1000行代码有15到50个缺陷，那么潜在的可利用缺陷就会以很快的速度增加。 +如今， 平均每辆汽车至少有 1 亿行代码。无人驾驶汽车公司预计不久以后它们将有 2 亿行代码。当你停下来考虑：平均每 1000 行代码有 15 到 50 个缺陷，那么潜在的可利用缺陷就会以很快的速度增加。 “计算机最大的安全威胁仅仅是数据被删除，但无人驾驶汽车一旦出现安全事故，失去的却是乘客的生命。”一家致力于解决汽车安全问题的以色列初创公司 Karamba Security 的联合创始人 David Barzilai 说。 安全专家说道：要想真正保障无人驾驶汽车的安全，汽车制造商必须想办法避免所有可能产生的漏洞——即使漏洞不可避免。其中最大的挑战，是汽车制造商和软件开发商们之间的缺乏合作经验。 -网络安全公司 Lookout 的 Mahaffey 先生说：“新的革命已经出现，我们不能固步自封，应该寻求新的思维。我们需要像发明出安全气囊那样的人来解决安全漏洞，但我们现在还没有看到行业内有人做出改变。“ +网络安全公司 Lookout 的 Mahaffey 先生说：“新的革命已经出现，我们不能固步自封，应该寻求新的思维。我们需要像发明出安全气囊那样的人来解决安全漏洞，但我们现在还没有看到行业内有人做出改变。” -Mahaffey 先生说：”在这场无人驾驶汽车的竞争中，那些最注重软件的公司将会成为最后的赢家“ +Mahaffey 先生说：“在这场无人驾驶汽车的竞争中，那些最注重软件的公司将会成为最后的赢家。” -------------------------------------------------------------------------------- via: https://www.nytimes.com/2017/06/07/technology/why-car-companies-are-hiring-computer-security-experts.html -作者：[NICOLE PERLROTH ][a] +作者：[NICOLE PERLROTH][a] 译者：[XiatianSummer](https://github.com/XiatianSummer) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 diff --git a/published/20170802 Creating SWAP partition using FDISK - FALLOCATE commands.md b/published/201801/20170802 Creating SWAP partition using FDISK - FALLOCATE commands.md similarity index 100% rename from published/20170802 Creating SWAP partition using FDISK - FALLOCATE commands.md rename to published/201801/20170802 Creating SWAP partition using FDISK - FALLOCATE commands.md diff --git a/published/20170804 Add speech to your Fedora system.md b/published/201801/20170804 Add speech to your Fedora system.md similarity index 100% rename from published/20170804 Add speech to your Fedora system.md rename to published/201801/20170804 Add speech to your Fedora system.md diff --git a/published/20170820 How To Display Date And Time In History Command.md b/published/201801/20170820 How To Display Date And Time In History Command.md similarity index 100% rename from published/20170820 How To Display Date And Time In History Command.md rename to published/201801/20170820 How To Display Date And Time In History Command.md diff --git a/published/20170828 How To Use YUM History Command To Rollback An Updates In RHEL-CentOS Systems.md b/published/201801/20170828 How To Use YUM History Command To Rollback An Updates In RHEL-CentOS Systems.md similarity index 100% rename from published/20170828 How To Use YUM History Command To Rollback An Updates In RHEL-CentOS Systems.md rename to published/201801/20170828 How To Use YUM History Command To Rollback An Updates In RHEL-CentOS Systems.md diff --git a/published/20170915 12 ip Command Examples for Linux Users.md b/published/201801/20170915 12 ip Command Examples for Linux Users.md similarity index 100% rename from published/20170915 12 ip Command Examples for Linux Users.md rename to published/201801/20170915 12 ip Command Examples for Linux Users.md diff --git a/published/20170915 Fake A Hollywood Hacker Screen in Linux Terminal.md b/published/201801/20170915 Fake A Hollywood Hacker Screen in Linux Terminal.md similarity index 100% rename from published/20170915 Fake A Hollywood Hacker Screen in Linux Terminal.md rename to published/201801/20170915 Fake A Hollywood Hacker Screen in Linux Terminal.md diff --git a/published/20170916 How To Auto Logout Inactive Users After A Period Of Time In Linux.md b/published/201801/20170916 How To Auto Logout Inactive Users After A Period Of Time In Linux.md similarity index 100% rename from published/20170916 How To Auto Logout Inactive Users After A Period Of Time In Linux.md rename to published/201801/20170916 How To Auto Logout Inactive Users After A Period Of Time In Linux.md diff --git a/translated/tech/20170918 3 text editor alternatives to Emacs and Vim.md b/published/201801/20170918 3 text editor alternatives to Emacs and Vim.md similarity index 66% rename from translated/tech/20170918 3 text editor alternatives to Emacs and Vim.md rename to published/201801/20170918 3 text editor alternatives to Emacs and Vim.md index 136214ce33..1fabccbac4 100644 --- a/translated/tech/20170918 3 text editor alternatives to Emacs and Vim.md +++ b/published/201801/20170918 3 text editor alternatives to Emacs and Vim.md @@ -1,52 +1,49 @@ -3 个替代 Emacs 的 Vim 文本编辑器 +3 个可以替代 Emacs 和 Vim 的文本编辑器 ====== ![](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/computer_keyboard_laptop_development_blue.png?itok=IfckxN48) -Emacs 和 Vim 的粉丝们，在你们开始编辑器之争之前，请你们理解，这篇文章并不会把引导放在诸位最喜欢的编辑器上。我是一个 Emacs 爱好者，但是也很喜欢 Vim。 +> Vim 和 Emacs 对于 Linux 桌面上进行文本编辑显然是重量级的，但是它们并非唯一之选。 -就是说，我已经意识到 Emacs 和 Vim 并不适合所有人。也许 [编辑器之争][1] 略显幼稚，让很多人失望了。也许他们只是想要有一个不太苛刻的现代化的编辑器。 +Emacs 和 Vim 的粉丝们，在你们开始编辑器之争之前，请你们理解，这篇文章并不是为了贬低诸位最喜欢的编辑器。我是一个 Emacs 爱好者，但是也很喜欢 Vim。 + +就是说，我认为 Emacs 和 Vim 并不适合所有人。也许 [编辑器之争][1] 略显幼稚，让很多人失望了。也许他们只是想要有一个不太苛刻的现代化的编辑器。 如果你正寻找可以替代 Emacs 或者 Vim 的编辑器，请继续阅读下去。这里有三个可能会让你感兴趣的编辑器。 ### Geany - ![用 Geany 编辑一个 LaTeX 文档][3] +*你可以用 Geany 编辑 LaTeX 文档* -你可以用 Geany 编辑 LaTeX 文档 +[Geany][4] 是一个古老的编辑器，当我还在过时的硬件上运行轻量级 Linux 发行版的时候，[Geany][4] 就是一个优秀的的编辑器。Geany 开始用于我的 [LaTeX][5] 编辑，但是很快就成为我所有的文本编辑的编辑器了。 -[Geany][4] 是一个古老的编辑器，当我还在过时的硬件上运行轻量级 Linux 发行版的时候，[Geany][4] 就是一个优秀的的编辑器。Geany 开始于我的 [LaTeX][5] 编辑，但是很快就成为我所有应用程序的编辑器了。 - -尽管 Geany 号称是轻量且高速的 [IDE][6]（集成开发环境），但是它绝不仅仅是一个技术工具。Geany 轻便快捷，即便是在一个过时的机器或是 [运行 Linux 的 Chromebook][7] 也能轻松运行起来。无论是编辑配置文件维护任务列表、写文章、代码还是脚本，Geany 都能轻松胜任。 +尽管 Geany 号称是轻量且快速的 [IDE][6]（集成开发环境），但是它绝不仅仅是一个技术人的工具。Geany 轻便快捷，即便是在一个过时的机器或是 [运行 Linux 的 Chromebook][7] 也能轻松运行起来。无论是从编辑配置文件到维护任务列表或日志，从写文章或书籍到编写代码和脚本，Geany 都能轻松胜任。 [插件][8] 给 Geany 带来一些额外的魅力。这些插件拓展了 Geany 的功能，让你编码或是处理一些标记语言变得更高效，帮助你处理文本，甚至做拼写检查。 ### Atom - ![使用 Atom 编辑网页][10] - -使用 Atom 编辑网页 +*使用 Atom 编辑网页* 在文本编辑器领域，[Atom][11] 后来居上。很短的时间内，Atom 就获得了一批忠实的追随者。 -Atom 的定制功能让其拥有如此的吸引力。如果有一些技术癖好，你完全可以在这个编辑器上随意设置。如果你不仅仅是忠于技术，Atom 也有 [一些主题][12] ，你可以用来更改编辑器外观。 +Atom 的定制功能让其拥有如此的吸引力。如果有一些技术癖好，你完全可以在这个编辑器上随意设置。如果你不仅仅是沉迷技术，Atom 也有 [一些主题][12] ，你可以用来更改编辑器外观。 -千万不要低估 Atom 数以千计的 [拓展包][13]。它们能在不同功能上拓展 Atom，能根据你的爱好把 Atom 转化成合适的文本编辑器或是开发环境。Atom 不仅为程序员提供服务。它同样适用于 [作家的文本编辑器][14]。 +千万不要低估 Atom 数以千计的 [拓展包][13]。它们能在不同功能上拓展 Atom，能根据你的爱好把 Atom 转化成合适的文本编辑器或是开发环境。Atom 不仅为程序员提供服务。它同样是适用于 [作家的文本编辑器][14]。 ### Xed ![使用 Xed 编辑文章][16] +*使用 Xed 编辑文章* -使用 Xed 编辑文章 +可能对用户体验来说，Atom 和 Geany 略显臃肿。也许你只想要一个轻量级，一个不要太简陋也不要有太多很少使用的特性的编辑器，如此看来，[Xed][17] 正是你所期待的。 -可能对用户体验来说，Atom 和 Geany 略显臃肿。也许你只想要一个轻量级，一个不要太露骨也不要有太多很少使用的特性的编辑器，如此看来，[Xed][17] 正是你所期待的。 - -如果 Xed 你看着眼熟，那是因为它是 MATE 桌面环境中 Pluma 编辑器上的分支。我发现相比于 Pluma，Xed 可能速度更快一点，响应更灵敏一点--不过，因人而异吧。 +如果 Xed 你看着眼熟，那是因为它是 MATE 桌面环境中 Pluma 编辑器上的分支。我发现相比于 Pluma，Xed 可能速度更快一点，响应更灵敏一点——不过，因人而异吧。 虽然 Xed 没有那么多的功能，但也不至于太糟。它有扎实的语法高亮，略强于一般的搜索替换和拼写检查功能以及单窗口编辑多文件的选项卡式界面。 @@ -54,16 +51,14 @@ Atom 的定制功能让其拥有如此的吸引力。如果有一些技术癖好 我不是 KDE 痴，当我工作在 KDE 环境下时， [KDevelop][18] 就已经是我深度工作时的首选了。它很强大而且灵活，又没有过大的体积，很像 Genany。 -虽然我还没感受过爱，但是我发誓我和我了解的几个人都在 [Brackets][19] 感受到了。它很强大，而且不得不承认它的 [拓展][20] 真的很实用。 +虽然我还没感受过对它的喜爱，但是我认识的几个人都用 [Brackets][19] 感受到了。它很强大，而且不得不承认它的 [拓展][20] 真的很实用。 被称为 “开发者的编辑器” 的 [Notepadqq][21] ，总让人联想到 [Notepad++][22]。虽然它的发展仍处于早期阶段，但至少它看起来还是很有前景的。 -对于那些只有简单的文本编辑器需求的人来说，[Gedit][23] 和 [Kate][24] 相比是极好的。它绝不是太过原始的编辑器--它有丰富的功能去完成大型文本编辑。无论是 Gedit 还是 Kate 都缘于速度和易上手而齐名。 +对于那些只有简单的文本编辑器需求的人来说，[Gedit][23] 和 [Kate][24] 相比是极好的。它绝不是太过原始的编辑器——它有足够完成大型文本编辑的功能。无论是 Gedit 还是 Kate 都以速度和易上手而称名。 -你有其他 Emacs 和 Vim 之外的挚爱编辑器么？留言下来，免费分享。 +你有其他 Emacs 和 Vim 之外的挚爱编辑器么？方便的话留言分享。 -### 关于作者 -Scott Nesbitt；我长期使用开源软件；记录各种有趣的事物；利益。做自己力所能及的事，并不把自己当回事。你可以在网络上的这些地方找到我。 -------------------------------------------------------------------------------- @@ -71,7 +66,7 @@ via: https://opensource.com/article/17/9/3-alternatives-emacs-and-vim 作者：[Scott Nesbitt][a] 译者：[CYLeft](https://github.com/CYLeft) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 diff --git a/published/20170918 Linux fmt command - usage and examples.md b/published/201801/20170918 Linux fmt command - usage and examples.md similarity index 100% rename from published/20170918 Linux fmt command - usage and examples.md rename to published/201801/20170918 Linux fmt command - usage and examples.md diff --git a/published/20170919 What Are Bitcoins.md b/published/201801/20170919 What Are Bitcoins.md similarity index 100% rename from published/20170919 What Are Bitcoins.md rename to published/201801/20170919 What Are Bitcoins.md diff --git a/published/201801/20170920 Easy APT Repository - Iain R. Learmonth.md b/published/201801/20170920 Easy APT Repository - Iain R. Learmonth.md new file mode 100644 index 0000000000..ff1d59a757 --- /dev/null +++ b/published/201801/20170920 Easy APT Repository - Iain R. Learmonth.md @@ -0,0 +1,69 @@ +创建一个简易 APT 仓库 +====== + +作为我工作的一部分，我所维护的 [PATHspider][5] 依赖于 [cURL][6] 和 [PycURL][7]中的一些[刚刚][8][被][9]合并或仍在[等待][10]被合并的功能。我需要构建一个包含这些 Debian 包的 Docker 容器，所以我需要快速构建一个 APT 仓库。 + +Debian 仓库本质上可以看作是一个静态的网站，而且内容是经过 GPG 签名的，所以它不一定需要托管在某个可信任的地方（除非可用性对你的程序来说是至关重要的）。我在 [Netlify][11]（一个静态的网站主机）上托管我的博客，我认为它很合适这种情况。他们也[支持开源项目][12]。 + +你可以用下面的命令安装 netlify 的 CLI 工具： + +``` +sudo apt install npm +sudo npm install -g netlify-cli +``` + +设置仓库的基本步骤是： + +``` +mkdir repository +cp /path/to/*.deb repository/ +cd repository +apt-ftparchive packages . > Packages +apt-ftparchive release . > Release +gpg --clearsign -o InRelease Release +netlify deploy +``` + +当你完成这些步骤后，并在 Netlify 上创建了一个新的网站，你也可以通过 Web 界面来管理这个网站。你可能想要做的一些事情是为你的仓库设置自定义域名，或者使用 Let's Encrypt 启用 HTTPS。（如果你打算启用 HTTPS，请确保命令中有 `apt-transport-https`。） + +要将这个仓库添加到你的 apt 源： + +``` +gpg --export -a YOURKEYID | sudo apt-key add - +echo "deb https://SUBDOMAIN.netlify.com/ /" | sudo tee -a /etc/apt/sources.list +sudo apt update +``` + +你会发现这些软件包是可以安装的。注意下 [APT pinnng][13]，因为你可能会发现，根据你的策略，仓库上的较新版本实际上并不是首选版本。 + +**更新**：如果你想要一个更适合平时使用的解决方案，请参考 [repropro][14]。如果你想让最终用户将你的 apt 仓库作为第三方仓库添加到他们的系统中，请查看 [Debian wiki 上的这个页面][15]，其中包含关于如何指导用户使用你的仓库。 + +**更新 2**：有一位评论者指出用 [aptly][16]，它提供了更多的功能，并消除了 repropro 的一些限制。我从来没有用过 aptly，所以不能评论具体细节，但从网站看来，这是一个很好的工具。 + +-------------------------------------------------------------------------------- + +via: https://iain.learmonth.me/blog/2017/2017w383/ + +作者：[Iain R. Learmonth][a] +译者：[geekpi](https://github.com/geekpi) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://iain.learmonth.me +[1]:https://iain.learmonth.me/tags/netlify/ +[2]:https://iain.learmonth.me/tags/debian/ +[3]:https://iain.learmonth.me/tags/apt/ +[4]:https://iain.learmonth.me/tags/foss/ +[5]:https://pathspider.net +[6]:http://curl.haxx.se/ +[7]:http://pycurl.io/ +[8]:https://github.com/pycurl/pycurl/pull/456 +[9]:https://github.com/pycurl/pycurl/pull/458 +[10]:https://github.com/curl/curl/pull/1847 +[11]:http://netlify.com/ +[12]:https://www.netlify.com/open-source/ +[13]:https://wiki.debian.org/AptPreferences +[14]:https://mirrorer.alioth.debian.org/ +[15]:https://wiki.debian.org/DebianRepository/UseThirdParty +[16]:https://www.aptly.info/ diff --git a/translated/tech/20170921 Mastering file searches on Linux.md b/published/201801/20170921 Mastering file searches on Linux.md similarity index 58% rename from translated/tech/20170921 Mastering file searches on Linux.md rename to published/201801/20170921 Mastering file searches on Linux.md index e964a35a64..2740e6a783 100644 --- a/translated/tech/20170921 Mastering file searches on Linux.md +++ b/published/201801/20170921 Mastering file searches on Linux.md @@ -7,52 +7,49 @@ * 快速搜索 * 更复杂的搜索条件 - * 连接条件 + * 组合条件 * 反转条件 * 简单和详细的回应 * 寻找重复的文件 -有很多有用的命令可以搜索文件，**find** 命令可能是其中最有名的，但它不是唯一的命令，也不一定总是找到目标文件的最快方法。 +有很多有用的命令可以搜索文件，`find` 命令可能是其中最有名的，但它不是唯一的命令，也不一定总是找到目标文件的最快方法。 ### 快速搜索命令：which 和 locate -搜索文件的最简单的命令可能就是 **which** 和 **locate** 了，但二者都有一些局限性。**which** 命令只会在系统定义的搜索路径中，查找可执行的文件，通常用于识别命令。如果您对输入 which 时会运行的命令感到好奇，您可以使用命令 which which，它会指向对应的可执行文件。 +搜索文件的最简单的命令可能就是 `which` 和 `locate` 了，但二者都有一些局限性。`which` 命令只会在系统定义的搜索路径中，查找可执行的文件，通常用于识别命令。如果您对输入 `which` 时会运行哪个命令感到好奇，您可以使用命令 `which which`，它会指出对应的可执行文件。 ``` $ which which /usr/bin/which - ``` -**which** 命令会显示它找到的第一个以相应名称命名的可执行文件（也就是使用该命令时将运行的那个文件），然后停止。 +`which` 命令会显示它找到的第一个以相应名称命名的可执行文件（也就是使用该命令时将运行的那个文件），然后停止。 -**locate** 命令更大方一点，它可以查找任意数量的文件，但它也有一个限制：仅当文件名被包含在由 **updatedb** 命令准备的数据库时才有效。该文件可能会存储在某个位置，如 /var/lib/mlocate/mlocate.db，但不能用 locate 以外的任何命令读取。这个文件的更新通常是通过每天通过 cron 运行的 updatedb 进行的。 +`locate` 命令更大方一点，它可以查找任意数量的文件，但它也有一个限制：仅当文件名被包含在由 `updatedb` 命令构建的数据库时才有效。该文件可能会存储在某个位置，如 `/var/lib/mlocate/mlocate.db`，但不能用 `locate` 以外的任何命令读取。这个文件的更新通常是通过每天通过 cron 运行的 `updatedb` 进行的。 -简单的 **find** 命令不需要太多限制，不过它需要搜索的起点和指定搜索条件。最简单的 find 命令：按文件名搜索文件。如下所示： +简单的 `find` 命令没有太多限制，不过它需要指定搜索的起点和搜索条件。最简单的 `find` 命令：按文件名搜索文件。如下所示： ``` $ find . -name runme ./bin/runme - ``` 如上所示，通过文件名搜索文件系统的当前位置将会搜索所有子目录，除非您指定了搜索深度。 ### 不仅仅是文件名 -**find** 命令允许您搜索除文件名以外的多种条件，包括文件所有者、组、权限、大小、修改时间、缺少所有者或组和文件类型等。除了查找文件外，您还可以删除文件、对其进行重命名、更改所有者、更改权限和对文件运行几乎任何命令。 +`find` 命令允许您搜索除文件名以外的多种条件，包括文件所有者、组、权限、大小、修改时间、缺少所有者或组，和文件类型等。除了查找文件外，您还可以删除文件、对其进行重命名、更改所有者、更改权限和对找到的文件运行几乎任何命令。 -下面两条命令会查找：在当前目录中 root 用户拥有的文件，以及非指定用户（在本例中为 shs）拥有的文件。在这个例子中，两个输出是一样的，但并不总是如此。 +下面两条命令会查找：在当前目录中 root 用户拥有的文件，以及不被指定用户（在本例中为 shs）所拥有的文件。在这个例子中，两个输出是一样的，但并不总是如此。 ``` $ find . -user root -ls 396926 0 lrwxrwxrwx 1 root root 21 Sep 21 09:03 ./xyz -> /home/peanut/xyz $ find . ! -user shs -ls 396926 0 lrwxrwxrwx 1 root root 21 Sep 21 09:03 ./xyz -> /home/peanut/xyz - ``` -感叹号“!”字符代表“非”：反转跟随其后的条件。 +感叹号 `!` 字符代表“非”：反转跟随其后的条件。 下面的命令将查找具有特定权限的文件： @@ -61,7 +58,6 @@ $ find . -perm 750 -ls 397176 4 -rwxr-x--- 1 shs shs 115 Sep 14 13:52 ./ll 398209 4 -rwxr-x--- 1 shs shs 117 Sep 21 08:55 ./get-updates 397145 4 drwxr-x--- 2 shs shs 4096 Sep 14 15:42 ./newdir - ``` 接下来的命令显示具有 777 权限的非符号链接文件： @@ -70,19 +66,17 @@ $ find . -perm 750 -ls $ sudo find /home -perm 777 ! -type l -ls 397132 4 -rwxrwxrwx 1 shs shs 18 Sep 15 16:06 /home/shs/bin/runme 396949 4 -rwxrwxrwx 1 root root 558 Sep 21 11:21 /home/oops - ``` -以下命令将查找大小超过千兆字节的文件。请注意，我们找到了一个非常有趣的文件。它在 ELF 核心文件格式中代表该系统的物理内存。 +以下命令将查找大小超过千兆字节的文件。请注意，我们找到了一个非常有趣的文件。它以 ELF core 文件格式表示了该系统的物理内存。 ``` $ sudo find / -size +1G -ls 4026531994 0 -r-------- 1 root root 140737477881856 Sep 21 11:23 /proc/kcore 1444722 15332 -rw-rw-r-- 1 shs shs 1609039872 Sep 13 15:55 /home/shs/Downloads/ubuntu-17.04-desktop-amd64.iso - ``` -只要您知道 find 命令是如何描述文件类型的，就可以通过文件类型来查找文件。 +只要您知道 `find` 命令是如何描述文件类型的，就可以通过文件类型来查找文件。 ``` b = 块设备文件 @@ -93,7 +87,6 @@ f = 常规文件 l = 符号链接 s = 套接字 D = 门（仅限 Solaris） - ``` 在下面的命令中，我们要寻找符号链接和套接字： @@ -103,28 +96,25 @@ $ find . -type l -ls 396926 0 lrwxrwxrwx 1 root root 21 Sep 21 09:03 ./whatever -> /home/peanut/whatever $ find . -type s -ls 395256 0 srwxrwxr-x 1 shs shs 0 Sep 21 08:50 ./.gnupg/S.gpg-agent - ``` -您还可以根据 inode 数字来搜索文件： +您还可以根据 inode 号来搜索文件： ``` $ find . -inum 397132 -ls 397132 4 -rwx------ 1 shs shs 18 Sep 15 16:06 ./bin/runme - ``` -另一种通过 inode 搜索文件的方法是使用 **debugfs** 命令。在大的文件系统上，这个命令可能比 find 快得多，您可能需要安装 icheck。 +另一种通过 inode 搜索文件的方法是使用 `debugfs` 命令。在大的文件系统上，这个命令可能比 `find` 快得多，您可能需要安装 icheck。 ``` $ sudo debugfs -R 'ncheck 397132' /dev/sda1 debugfs 1.42.13 (17-May-2015) Inode Pathname 397132 /home/shs/bin/runme - ``` -在下面的命令中，我们从主目录（〜）开始，限制搜索的深度（是我们将搜索子目录的层数），并只查看在最近一天内创建或修改的文件（mtime 设置）。 +在下面的命令中，我们从主目录（`~`）开始，限制搜索的深度（即我们将搜索子目录的层数），并只查看在最近一天内创建或修改的文件（`mtime` 设置）。 ``` $ find ~ -maxdepth 2 -mtime -1 -ls @@ -132,29 +122,28 @@ $ find ~ -maxdepth 2 -mtime -1 -ls 394006 8 -rw------- 1 shs shs 5909 Sep 21 08:18 /home/shs/.bash_history 399612 4 -rw------- 1 shs shs 53 Sep 21 08:50 /home/shs/.Xauthority 399615 4 drwxr-xr-x 2 shs shs 4096 Sep 21 09:32 /home/shs/Downloads - ``` ### 不仅仅是列出文件 -使用 **-exec** 选项，在您使用 find 命令找到文件后可以以某种方式更改文件。您只需参照 -exec 选项即可运行相应的命令。 +使用 `-exec` 选项，在您使用 `find` 命令找到文件后可以以某种方式更改文件。您只需参照 `-exec` 选项即可运行相应的命令。 ``` $ find . -name runme -exec chmod 700 {} \; $ find . -name runme -ls 397132 4 -rwx------ 1 shs shs 18 Sep 15 16:06 ./bin/runme - ``` -在这条命令中，“{}”代表文件名。此命令将更改当前目录和子目录中任何名为“runme”的文件的权限。 +在这条命令中，`{}` 代表文件名。此命令将更改当前目录和子目录中任何名为 `runme` 的文件的权限。 -把您想运行的任何命令放在 -exec 选项之后，并使用类似于上面命令的语法即可。 +把您想运行的任何命令放在 `-exec` 选项之后，并使用类似于上面命令的语法即可。 ### 其他搜索条件 如上面的例子所示，您还可以通过其他条件进行搜索：文件的修改时间、所有者、权限等。以下是一些示例。 #### 根据用户查找文件 + ``` $ sudo find /home -user peanut /home/peanut @@ -162,23 +151,22 @@ $ sudo find /home -user peanut /home/peanut/.bash_logout /home/peanut/.profile /home/peanut/examples.desktop - ``` -#### 根据权限查找文件 +#### 根据权限查找文件 + ``` $ sudo find /home -perm 777 /home/shs/whatever /home/oops - ``` #### 根据修改时间查找文件 + ``` $ sudo find /home -mtime +100 /home/shs/.mozilla/firefox/krsw3giq.default/gmp-gmpopenh264/1.6/gmpopenh264.info /home/shs/.mozilla/firefox/krsw3giq.default/gmp-gmpopenh264/1.6/libgmpopenh264.so - ``` #### 通过比较修改时间查找文件 @@ -188,12 +176,11 @@ $ sudo find /home -mtime +100 ``` $ sudo find /var/log -newer /var/log/syslog /var/log/auth.log - ``` ### 寻找重复的文件 -如果您正在清理磁盘空间，则可能需要删除较大的重复文件。确定文件是否真正重复的最好方法是使用 **fdupes** 命令。此命令使用 md5 校验和来确定文件是否具有相同的内容。使用 -r（递归）选项，fdupes 将在一个目录下并查找具有相同校验和而被确定为内容相同的文件。 +如果您正在清理磁盘空间，则可能需要删除较大的重复文件。确定文件是否真正重复的最好方法是使用 `fdupes` 命令。此命令使用 md5 校验和来确定文件是否具有相同的内容。使用 `-r`（递归）选项，`fdupes` 将在一个目录下并查找具有相同校验和而被确定为内容相同的文件。 如果以 root 身份运行这样的命令，您可能会发现很多重复的文件，但是很多文件都是创建时被添加到主目录的启动文件。 @@ -209,25 +196,23 @@ $ sudo find /var/log -newer /var/log/syslog /home/tsmith/.bashrc /home/peanut/.bashrc /home/rocket/.bashrc - ``` -同样，您可能会在 /usr 中发现很多重复的但不该删除的配置文件。所以，请谨慎利用 fdupes 的输出。 +同样，您可能会在 `/usr` 中发现很多重复的但不该删除的配置文件。所以，请谨慎利用 `fdupes` 的输出。 -fdupes 命令并不总是很快，但是要记住，它正在对许多文件运行校验和来做比较，你可能会意识到它的有效性。 +`fdupes` 命令并不总是很快，但是要记住，它正在对许多文件运行校验和来做比较，你可能会意识到它是多么有效。 ### 总结 有很多方法可以在 Linux 系统上查找文件。如果您可以描述清楚您正在寻找什么，上面的命令将帮助您找到目标。 - -------------------------------------------------------------------------------- via: https://www.networkworld.com/article/3227075/linux/mastering-file-searches-on-linux.html 作者：[Sandra Henry-Stocker][a] 译者：[jessie-pang](https://github.com/jessie-pang) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 diff --git a/published/20170924 Simulate System Loads.md b/published/201801/20170924 Simulate System Loads.md similarity index 100% rename from published/20170924 Simulate System Loads.md rename to published/201801/20170924 Simulate System Loads.md diff --git a/published/20170925 A Commandline Fuzzy Search Tool For Linux.md b/published/201801/20170925 A Commandline Fuzzy Search Tool For Linux.md similarity index 100% rename from published/20170925 A Commandline Fuzzy Search Tool For Linux.md rename to published/201801/20170925 A Commandline Fuzzy Search Tool For Linux.md diff --git a/published/20170925 Linux Free Command Explained for Beginners (6 Examples).md b/published/201801/20170925 Linux Free Command Explained for Beginners (6 Examples).md similarity index 100% rename from published/20170925 Linux Free Command Explained for Beginners (6 Examples).md rename to published/201801/20170925 Linux Free Command Explained for Beginners (6 Examples).md diff --git a/published/20170927 How To Easily Find Awesome Projects And Resources Hosted In GitHub.md b/published/201801/20170927 How To Easily Find Awesome Projects And Resources Hosted In GitHub.md similarity index 100% rename from published/20170927 How To Easily Find Awesome Projects And Resources Hosted In GitHub.md rename to published/201801/20170927 How To Easily Find Awesome Projects And Resources Hosted In GitHub.md diff --git a/published/20170927 Microservices and containers- 5 pitfalls to avoid.md b/published/201801/20170927 Microservices and containers- 5 pitfalls to avoid.md similarity index 100% rename from published/20170927 Microservices and containers- 5 pitfalls to avoid.md rename to published/201801/20170927 Microservices and containers- 5 pitfalls to avoid.md diff --git a/translated/tech/20171002 Connect To Wifi From The Linux Command Line.md b/published/201801/20171002 Connect To Wifi From The Linux Command Line.md similarity index 92% rename from translated/tech/20171002 Connect To Wifi From The Linux Command Line.md rename to published/201801/20171002 Connect To Wifi From The Linux Command Line.md index 50c25bd839..c866c10590 100644 --- a/translated/tech/20171002 Connect To Wifi From The Linux Command Line.md +++ b/published/201801/20171002 Connect To Wifi From The Linux Command Line.md @@ -28,22 +28,20 @@ wpa_supplicant 可以作为命令行工具来用。使用一个简单的配置 wpa_supplicant 中有一个工具叫做 `wpa_cli`，它提供了一个命令行接口来管理你的 WiFi 连接。事实上你可以用它来设置任何东西，但是设置一个配置文件看起来要更容易一些。 使用 root 权限运行 `wpa_cli`，然后扫描网络。 -``` +``` # wpa_cli > scan - ``` 扫描过程要花上一点时间，并且会显示所在区域的那些网络。记住你想要连接的那个网络。然后输入 `quit` 退出。 ### 生成配置块并且加密你的密码 -还有更方便的工具可以用来设置配置文件。它接受网络名称和密码作为参数，然后生成一个包含该网路配置块(其中的密码被加密处理了)的配置文件。 +还有更方便的工具可以用来设置配置文件。它接受网络名称和密码作为参数，然后生成一个包含该网路配置块（其中的密码被加密处理了）的配置文件。 + ``` - # wpa_passphrase networkname password > /etc/wpa_supplicant/wpa_supplicant.conf - ``` ### 裁剪你的配置 @@ -51,9 +49,9 @@ wpa_supplicant 中有一个工具叫做 `wpa_cli`，它提供了一个命令行 现在你已经有了一个配置文件了，这个配置文件就是 `/etc/wpa_supplicant/wpa_supplicant.conf`。其中的内容并不多，只有一个网络块，其中有网络名称和密码，不过你可以在此基础上对它进行修改。 用喜欢的编辑器打开该文件，首先删掉说明密码的那行注释。然后，将下面行加到配置最上方。 + ``` ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=wheel - ``` 这一行只是让 `wheel` 组中的用户可以管理 wpa_supplicant。这会方便很多。 @@ -61,29 +59,29 @@ ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=wheel 其他的内容则添加到网络块中。 如果你要连接到一个隐藏网络，你可以添加下面行来通知 wpa_supplicant 先扫描该网络。 + ``` scan_ssid=1 - ``` 下一步，设置协议以及密钥管理方面的配置。下面这些是 WPA2 相关的配置。 + ``` proto=RSN key_mgmt=WPA-PSK - ``` -group 和 pairwise 配置告诉 wpa_supplicant 你是否使用了 CCMP，TKIP，或者两者都用到了。为了安全考虑，你应该只用 CCMP。 +`group` 和 `pairwise` 配置告诉 wpa_supplicant 你是否使用了 CCMP、TKIP，或者两者都用到了。为了安全考虑，你应该只用 CCMP。 + ``` group=CCMP pairwise=CCMP - ``` 最后，设置网络优先级。越高的值越会优先连接。 + ``` priority=10 - ``` ![Complete WPA_Supplicant Settings][1] @@ -94,14 +92,13 @@ priority=10 当然，该方法并不是用于即时配置无线网络的最好方法，但对于定期连接的网络来说，这种方法非常有效。 - -------------------------------------------------------------------------------- via: https://linuxconfig.org/connect-to-wifi-from-the-linux-command-line 作者：[Nick Congleton][a] 译者：[lujun9972](https://github.com/lujun9972) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 diff --git a/published/201801/20171004 How To Create A Video From PDF Files In Linux.md b/published/201801/20171004 How To Create A Video From PDF Files In Linux.md new file mode 100644 index 0000000000..ffa9ff915a --- /dev/null +++ b/published/201801/20171004 How To Create A Video From PDF Files In Linux.md @@ -0,0 +1,97 @@ +如何在 Linux 中从 PDF 创建视频 +====== + +![](https://www.ostechnix.com/wp-content/uploads/2017/10/Video-1-720x340.jpg) + +我在我的平板电脑中收集了大量的 PDF 文件，其中主要是 Linux 教程。有时候我懒得在平板电脑上看。我认为如果我能够从 PDF 创建视频，并在大屏幕设备（如电视机或计算机）中观看会更好。虽然我对 [FFMpeg][1] 有一些经验，但我不知道如何使用它来创建视频。经过一番 Google 搜索，我想出了一个很好的解决方案。对于那些想从一组 PDF 文件制作视频文件的人，请继续阅读。这并不困难。 + +### 在 Linux 中从 PDF 创建视频 + +为此，你需要在系统中安装 “FFMpeg” 和 “ImageMagick”。 + +要安装 FFMpeg，请参考以下链接。 + +- [在 Linux 上安装 FFMpeg][2] + +Imagemagick 可在大多数 Linux 发行版的官方仓库中找到。 + +在 Arch Linux 以及 Antergos、Manjaro Linux 等衍生产品上，运行以下命令进行安装。 + +``` +sudo pacman -S imagemagick +``` + +Debian、Ubuntu、Linux Mint： + +``` +sudo apt-get install imagemagick +``` + +Fedora： + +``` +sudo dnf install imagemagick +``` + +RHEL、CentOS、Scientific Linux： + +``` +sudo yum install imagemagick +``` + +SUSE、 openSUSE： + +``` +sudo zypper install imagemagick +``` + +在安装 ffmpeg 和 imagemagick 之后，将你的 PDF 文件转换成图像格式，如 PNG 或 JPG，如下所示。 + +``` +convert -density 400 input.pdf picture.png +``` + +这里，`-density 400` 指定输出图像的水平分辨率。 + +上面的命令会将指定 PDF 的所有页面转换为 PNG 格式。PDF 中的每个页面都将被转换成 PNG 文件，并保存在当前目录中，文件名为： `picture-1.png`、 `picture-2.png` 等。根据选择的 PDF 的页数，这将需要一些时间。 + +将 PDF 中的所有页面转换为 PNG 格式后，运行以下命令以从 PNG 创建视频文件。 + +``` +ffmpeg -r 1/10 -i picture-%01d.png -c:v libx264 -r 30 -pix_fmt yuv420p video.mp4 +``` + +这里： + +* `-r 1/10` ：每张图像显示 10 秒。 +* `-i picture-%01d.png` ：读取以 `picture-` 开头，接着是一位数字（`％01d`），最后以 `.png` 结尾的所有图片。如果图片名称带有 2 位数字（也就是 `picture-10.png`、`picture11.png` 等），在上面的命令中使用（`％02d`）。 +* `-c:v libx264`：输出的视频编码器（即 h264）。 +* `-r 30` ：输出视频的帧率 +* `-pix_fmt yuv420p`：输出的视频分辨率 +* `video.mp4`：以 .mp4 格式输出视频文件。 + +好了，视频文件完成了！你可以在任何支持 .mp4 格式的设备上播放它。接下来，我需要找到一种方法来为我的视频插入一个很酷的音乐。我希望这也不难。 + +如果你想要更高的分辨率，你不必重新开始。只要将输出的视频文件转换为你选择的任何其他更高/更低的分辨率，比如说 720p，如下所示。 + +``` +ffmpeg -i video.mp4 -vf scale=-1:720 video_720p.mp4 +``` + +请注意，使用 ffmpeg 创建视频需要一台配置好的 PC。在转换视频时，ffmpeg 会消耗大量系统资源。我建议在高端系统中这样做。 + +就是这些了。希望你觉得这个有帮助。还会有更好的东西。敬请关注！ + +-------------------------------------------------------------------------------- + +via: https://www.ostechnix.com/create-video-pdf-files-linux/ + +作者：[SK][a] +译者：[geekpi](https://github.com/geekpi) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.ostechnix.com/author/sk/ +[1]:https://www.ostechnix.com/20-ffmpeg-commands-beginners/ +[2]:https://www.ostechnix.com/install-ffmpeg-linux/ \ No newline at end of file diff --git a/translated/tech/20171005 python-hwinfo - Display Summary Of Hardware Information In Linux.md b/published/201801/20171005 python-hwinfo - Display Summary Of Hardware Information In Linux.md similarity index 66% rename from translated/tech/20171005 python-hwinfo - Display Summary Of Hardware Information In Linux.md rename to published/201801/20171005 python-hwinfo - Display Summary Of Hardware Information In Linux.md index e56ce8e292..8593e4f558 100644 --- a/translated/tech/20171005 python-hwinfo - Display Summary Of Hardware Information In Linux.md +++ b/published/201801/20171005 python-hwinfo - Display Summary Of Hardware Information In Linux.md @@ -1,80 +1,81 @@ -# python-hwinfo：使用Linux系统工具展示硬件信息概况 +python-hwinfo：使用 Linux 系统工具展示硬件信息概况 +========== ---- -到目前为止，获取Linux系统硬件信息和配置已经被大部分工具所涵盖，不过也有许多命令可用于相同目的。 +到目前为止，我们已经介绍了大部分获取 Linux 系统硬件信息和配置的工具，不过也有许多命令可用于相同目的。 -而且，一些工具会显示所有硬件组成的详细信息，重置后，只显示特定设备的信息。 +而且，一些工具会显示所有硬件组件的详细信息，或只显示特定设备的信息。 -在这个系列中, 今天我们讨论一下关于[python-hwinfo][1]， 它是一个展示硬件信息概况和整洁配置的工具之一。 +在这个系列中, 今天我们讨论一下关于 [python-hwinfo][1]， 它是一个展示硬件信息概况的工具之一，并且其配置简洁。 -### 什么是python-hwinfo +### 什么是 python-hwinfo -这是一个通过解析系统工具（例如lspci和dmidecode）的输出，来检查硬件和设备的Python库。 +这是一个通过解析系统工具（例如 `lspci` 和 `dmidecode`）的输出，来检查硬件和设备的 Python 库。 -它提供了一个简单的命令行工具，可以用来检查本地，远程和捕获到的主机。用sudo运行命令以获得最大的信息。 +它提供了一个简单的命令行工具，可以用来检查本地、远程的主机和记录的信息。用 `sudo` 运行该命令以获得最大的信息。 -另外，你可以提供服务器IP或者主机名，用户名和密码，在远程的服务器上执行它。当然你也可以使用这个工具查看其它工具捕获的输出（例如demidecode输出的'dmidecode.out'，/proc/cpuinfo输出的'cpuinfo'，lspci -nnm输出的'lspci-nnm.out'）。 +另外，你可以提供服务器 IP 或者主机名、用户名和密码，在远程的服务器上执行它。当然你也可以使用这个工具查看其它工具捕获的输出（例如 `demidecode` 输出的 `dmidecode.out`，`/proc/cpuinfo` 输出的 `cpuinfo`，`lspci -nnm` 输出的 `lspci-nnm.out`）。 -**建议阅读 :** -**(#)** [inxi - A Great Tool to Check Hardware Information on Linux][2] -**(#)** [Dmidecode - Easy Way To Get Linux System Hardware Information][3] -**(#)** [LSHW (Hardware Lister) - A Nifty Tool To Get A Hardware Information On Linux][4] -**(#)** [hwinfo (Hardware Info) - A Nifty Tool To Detect System Hardware Information On Linux][5] -**(#)** [How To Use lspci, lsscsi, lsusb, And lsblk To Get Linux System Devices Information][6] +建议阅读： -### Linux上如何安装python-hwinfo +- [Inxi：一个功能强大的获取 Linux 系统信息的命令行工具][2] +- [Dmidecode：获取 Linux 系统硬件信息的简易方式][3] +- [LSHW (Hardware Lister)： 一个在 Linux 上获取硬件信息的漂亮工具][4] +- [hwinfo (Hardware Info)：一个在 Linux 上检测系统硬件信息的漂亮工具][5] +- [如何使用 lspci、lsscsi、lsusb 和 lsblk 获取 Linux 系统设备信息][6] -在绝大多数Linux发行版，都可以通过pip包安装。为了安装python-hwinfo， 确保你的系统已经有python和python-pip包作为先决条件。 +### Linux 上如何安装 python-hwinfo -pip是Python附带的一个包管理工具，在Linux上安装Python包的推荐工具之一。 +在绝大多数 Linux 发行版，都可以通过 pip 包安装。为了安装 python-hwinfo， 确保你的系统已经有 Python 和python-pip 包作为先决条件。 + +`pip` 是 Python 附带的一个包管理工具，在 Linux 上安装 Python 包的推荐工具之一。 + +在 Debian/Ubuntu 平台，使用 [APT-GET 命令][7] 或者 [APT 命令][8] 安装 `pip`。 -在**`Debian/Ubuntu`**平台，使用[APT-GET 命令][7] 或者 [APT 命令][8] 安装pip。 ``` $ sudo apt install python-pip - ``` -在**`RHEL/CentOS`**平台，使用[YUM 命令][9]安装pip。 +在 RHEL/CentOS 平台，使用 [YUM 命令][9]安装 `pip`。 + ``` $ sudo yum install python-pip python-devel - ``` -在**`Fedora`**平台，使用[DNF 命令][10]安装pip。 +在 Fedora 平台，使用 [DNF 命令][10]安装 `pip`。 + ``` $ sudo dnf install python-pip - ``` -在**`Arch Linux`**平台，使用[Pacman 命令][11]安装pip。 +在 Arch Linux 平台，使用 [Pacman 命令][11]安装 `pip`。 + ``` $ sudo pacman -S python-pip - ``` -在**`openSUSE`**平台，使用[Zypper 命令][12]安装pip。 +在 openSUSE 平台，使用 [Zypper 命令][12]安装 `pip`。 + ``` $ sudo zypper python-pip - ``` -最后，执行下面的pip命令安装python-hwinfo。 +最后，执行下面的 `pip` 命令安装 python-hwinfo。 + ``` $ sudo pip install python-hwinfo - ``` -### 怎么使用python-hwinfo在本地机器 +### 怎么在本地机器使用 python-hwinfo 执行下面的命令，检查本地机器现有的硬件。输出很清楚和整洁，这是我在其他命令中没有看到的。 -它的输出分为了五类。 +它的输出分为了五类： - * **`Bios Info:`** bios供应商名称，系统产品名称， 系统序列号，系统唯一标识符，系统制造商，bios发布日期和bios版本。 - * **`CPU Info:`** 处理器编号，供应商ID，cpu系列代号，型号，制作更新版本，型号名称，cpu主频。 - * **`Ethernet Controller Info:`** 供应商名称，供应商ID，设备名称，设备ID，子供应商名称，子供应商ID，子设备名称，子设备ID。 - * **`Storage Controller Info:`** 供应商名称，供应商ID，设备名称，设备ID，子供应商名称，子供应商ID，子设备名称，子设备ID。 - * **`GPU Info:`** 供应商名称，供应商ID，设备名称，设备ID，子供应商名称，子供应商ID，子设备名称，子设备ID。 +* Bios Info（BIOS 信息）： BIOS 供应商名称、系统产品名称、系统序列号、系统唯一标识符、系统制造商、BIOS 发布日期和BIOS 版本。 +* CPU Info（CPU 信息）：处理器编号、供应商 ID，CPU 系列代号、型号、步进编号、型号名称、CPU 主频。 +* Ethernet Controller Info（网卡信息）： 供应商名称、供应商 ID、设备名称、设备 ID、子供应商名称、子供应商 ID，子设备名称、子设备 ID。 +* Storage Controller Info（存储设备信息）： 供应商名称、供应商 ID、设备名称、设备 ID、子供应商名称，子供应商 ID、子设备名称、子设备 ID。 +* GPU Info（GPU 信息）： 供应商名称、供应商 ID、设备名称、设备 ID、子供应商名称、子供应商 ID、子设备名称、子设备 ID。 ``` @@ -136,20 +137,20 @@ GPU Info: ``` -### 怎么使用python-hwinfo在远处机器上 +### 怎么在远处机器上使用 python-hwinfo + +执行下面的命令检查远程机器现有的硬件，需要远程机器 IP，用户名和密码： -执行下面的命令检查远程机器现有的硬件，需要远程机器IP，用户名和密码 ``` $ hwinfo -m x.x.x.x -u root -p password - ``` -### 如何使用python-hwinfo读取捕获的输出 +### 如何使用 python-hwinfo 读取记录的输出 执行下面的命令，检查本地机器现有的硬件。输出很清楚和整洁，这是我在其他命令中没有看到的。 + ``` $ hwinfo -f [Path to file] - ``` -------------------------------------------------------------------------------- @@ -158,13 +159,13 @@ via: https://www.2daygeek.com/python-hwinfo-check-display-system-hardware-config 作者：[2DAYGEEK][a] 译者：[Torival](https://github.com/Torival) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 [a]:https://www.2daygeek.com/author/2daygeek/ [1]:https://github.com/rdobson/python-hwinfo -[2]:https://www.2daygeek.com/inxi-system-hardware-information-on-linux/ +[2]:https://linux.cn/article-8424-1.html [3]:https://www.2daygeek.com/dmidecode-get-print-display-check-linux-system-hardware-information/ [4]:https://www.2daygeek.com/lshw-find-check-system-hardware-information-details-linux/ [5]:https://www.2daygeek.com/hwinfo-check-display-detect-system-hardware-information-linux/ diff --git a/published/201801/20171008 The most important Firefox command line options.md b/published/201801/20171008 The most important Firefox command line options.md new file mode 100644 index 0000000000..1f9383906c --- /dev/null +++ b/published/201801/20171008 The most important Firefox command line options.md @@ -0,0 +1,55 @@ +最重要的 Firefox 命令行选项 +====== + +Firefox web 浏览器支持很多命令行选项，可以定制它启动的方式。 + +你可能已经接触过一些了，比如 `-P "配置文件名"` 指定浏览器启动加载时的配置文件，`-private` 开启一个私有会话。 + +本指南会列出对 FIrefox 来说比较重要的那些命令行选项。它并不包含所有的可选项，因为很多选项只用于特定的目的，对一般用户来说没什么价值。 + +你可以在 Firefox 开发者网站上看到[完整][1] 的命令行选项列表。需要注意的是，很多命令行选项对其它基于 Mozilla 的产品一样有效，甚至对某些第三方的程序也有效。 + +### 重要的 Firefox 命令行选项 + +![firefox command line][2] + +#### 配置文件相关选项 + + - `-CreateProfile 配置文件名` -- 创建新的用户配置信息，但并不立即使用它。 + - `-CreateProfile "配置文件名 存放配置文件的目录"` -- 跟上面一样，只是指定了存放配置文件的目录。 + - `-ProfileManager`，或 `-P` -- 打开内置的配置文件管理器。 + - `-P "配置文件名"` -- 使用指定的配置文件启动 Firefox。若指定的配置文件不存在则会打开配置文件管理器。只有在没有其他 Firefox 实例运行时才有用。 + - `-no-remote` -- 与 `-P` 连用来创建新的浏览器实例。它允许你在同一时间运行多个配置文件。 + +#### 浏览器相关选项 + + - `-headless` -- 以无头模式（LCTT 译注：无显示界面）启动 Firefox。Linux 上需要 Firefox 55 才支持，Windows 和 Mac OS X 上需要 Firefox 56 才支持。 + - `-new-tab URL` -- 在 Firefox 的新标签页中加载指定 URL。 + - `-new-window URL` -- 在 Firefox 的新窗口中加载指定 URL。 + - `-private` -- 以隐私浏览模式启动 Firefox。可以用来让 Firefox 始终运行在隐私浏览模式下。 + - `-private-window` -- 打开一个隐私窗口。 + - `-private-window URL` -- 在新的隐私窗口中打开 URL。若已经打开了一个隐私浏览窗口，则在那个窗口中打开 URL。 + - `-search 单词` -- 使用 FIrefox 默认的搜索引擎进行搜索。 + - - `url URL` -- 在新的标签页或窗口中加载 URL。可以省略这里的 `-url`，而且支持打开多个 URL，每个 URL 之间用空格分离。 + +#### 其他选项 + + - `-safe-mode` -- 在安全模式下启动 Firefox。在启动 Firefox 时一直按住 Shift 键也能进入安全模式。 + - `-devtools` -- 启动 Firefox，同时加载并打开开发者工具。 + - `-inspector URL` -- 使用 DOM Inspector 查看指定的 URL + - `-jsconsole` -- 启动 Firefox，同时打开浏览器终端。 + - `-tray` -- 启动 Firefox，但保持最小化。 + +-------------------------------------------------------------------------------- + +via: https://www.ghacks.net/2017/10/08/the-most-important-firefox-command-line-options/ + +作者：[Martin Brinkmann][a] +译者：[lujun9972](https://github.com/lujun9972) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.ghacks.net/author/martin/ +[1]:https://developer.mozilla.org/en-US/docs/Mozilla/Command_Line_Options +[2]:https://cdn.ghacks.net/wp-content/uploads/2017/10/firefox-command-line.png \ No newline at end of file diff --git a/published/20171011 What is a firewall.md b/published/201801/20171011 What is a firewall.md similarity index 100% rename from published/20171011 What is a firewall.md rename to published/201801/20171011 What is a firewall.md diff --git a/published/20171012 Install and Use YouTube-DL on Ubuntu 16.04.md b/published/201801/20171012 Install and Use YouTube-DL on Ubuntu 16.04.md similarity index 100% rename from published/20171012 Install and Use YouTube-DL on Ubuntu 16.04.md rename to published/201801/20171012 Install and Use YouTube-DL on Ubuntu 16.04.md diff --git a/translated/tech/20171013 Get Your Weather Forecast From the Linux CLI.md b/published/201801/20171013 Get Your Weather Forecast From the Linux CLI.md similarity index 77% rename from translated/tech/20171013 Get Your Weather Forecast From the Linux CLI.md rename to published/201801/20171013 Get Your Weather Forecast From the Linux CLI.md index 952d87b25a..2e7ee2c400 100644 --- a/translated/tech/20171013 Get Your Weather Forecast From the Linux CLI.md +++ b/published/201801/20171013 Get Your Weather Forecast From the Linux CLI.md @@ -18,13 +18,14 @@ 无需打开网页浏览器就能直接从终端获取最新的天气预报那该多方便啊，对吧？你还能把它写成脚本，或者设置定义定时任务。 -`http：//wttr.in` 是一个允许你搜索世界各地天气预报的网站，而且它的是以 ASCII 字符的形式来显示结果的。通过使用 `cURL` 访问 `http：//wttr.in`，就能直接在终端显示查询结果了。 +`http://wttr.in` 是一个允许你搜索世界各地天气预报的网站，而且它的是以 ASCII 字符的形式来显示结果的。通过使用 `cURL` 访问 `http://wttr.in`，就能直接在终端显示查询结果了。 ### 获取所在地的天气 ![Local weather from wttr.in][1] 要抓取所在地的天气情况非常简单。`wttr.in` 会自动根据 IP 地址来探测你的所在地。除非你用了 VPN，否则它的精度还不错。 + ``` $ curl wttr.in ``` @@ -33,12 +34,14 @@ $ curl wttr.in ![Weather by city from wttr.in][2] -你可以通过在 `wttr.in` 后加上斜杠和城市名称的方式来获得其他城市的天气情况。不过要把名字中的空格替换成`+`。 +你可以通过在 `wttr.in` 后加上斜杠和城市名称的方式来获得其他城市的天气情况。不过要把名字中的空格替换成 `+`。 + ``` $ curl wttr.in/New+York ``` 你也可以以 Unix 时区的形式来填写城市名称。 + ``` $ curl wttr.in/New_York ``` @@ -47,11 +50,10 @@ $ curl wttr.in/New_York ### 获取机场天气 - - ![Weather by airport from wttr.in][3] 若你对地区的三位机场代号很熟悉，你也可以使用机场代号来查询天气。一般来说使用机场要比使用城市更贴近你，而且更精确一些。 + ``` $ curl wttr.in/JFK ``` @@ -61,6 +63,7 @@ $ curl wttr.in/JFK ![Weather by landmark from wttr.in][4] 通过使用 `~` 字符，你可以让 `wttr.in` 通过地标来猜测天气情况。 + ``` $ curl wttr.in/~Statue+Of+Liberty ``` @@ -70,29 +73,28 @@ $ curl wttr.in/~Statue+Of+Liberty ![Weather by domain name from wttr.in][5] 你想不想知道 LinuxConfig 托管地的天气？现在有一个方法可以知道！`wttr.in` 可以通过域名获取天气。是的，这个功能可能不那么实用，但这很有趣啊。 + ``` - $ curl wttr.in/@linuxconfig.org - ``` ### 更改温度单位 ![Change unit system in wttr.in][6] -默认情况下，`wttr.in` 会根据你的实际地址来决定显示哪种温度单位 (C 还是 F)。基本上，在美国，使用的是华氏度，而其他地方显示的是摄氏度。你可以指定显示的温度单位，在 URL 后添加 `？u` 会显示华氏度，而添加 `？m` 会显示摄氏度。 +默认情况下，`wttr.in` 会根据你的实际地址来决定显示哪种温度单位（C 还是 F）。基本上，在美国，使用的是华氏度，而其他地方显示的是摄氏度。你可以指定显示的温度单位，在 URL 后添加 `?u` 会显示华氏度，而添加 `?m` 会显示摄氏度。 + ``` $ curl wttr.in/New_York?m $ curl wttr.in/Toronto?u ``` -在 ZSH 上有一个很奇怪的 bug，会使得这两条语句不能正常工厂，如果你需要更换单位，恐怕需要改成使用 Bash 了。 +在 ZSH 上有一个很奇怪的 bug，会使得这两条语句不能正常工作，如果你需要更换单位，恐怕需要改成使用 Bash 了。 ### 总结 -你可以很方便地在脚本，定时任务，甚至 MOTD(LCTT 注：Message Of The Day 每日消息)中访问 `wttr.in`。当然，你完全没有必要这么做。当你需要查看天气预报的时候只需要访问一下这个超棒的网站就行了。 - +你可以很方便地在脚本，定时任务，甚至 MOTD（LCTT 译注：Message Of The Day - 每日消息）中访问 `wttr.in`。当然，你完全没有必要这么做。当你需要查看天气预报的时候只需要访问一下这个超棒的网站就行了。 -------------------------------------------------------------------------------- @@ -100,7 +102,7 @@ via: https://linuxconfig.org/get-your-weather-forecast-from-the-linux-cli 作者：[Nick Congleton][a] 译者：[lujun9972](https://github.com/lujun9972) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 diff --git a/translated/tech/20171016 Fixing vim in Debian - There and back again.md b/published/201801/20171016 Fixing vim in Debian - There and back again.md similarity index 83% rename from translated/tech/20171016 Fixing vim in Debian - There and back again.md rename to published/201801/20171016 Fixing vim in Debian - There and back again.md index 36dd92d36a..ebe765c4be 100644 --- a/translated/tech/20171016 Fixing vim in Debian - There and back again.md +++ b/published/201801/20171016 Fixing vim in Debian - There and back again.md @@ -1,18 +1,20 @@ -在 Debian 中修复 vim - 去而复得 +修复 Debian 中的 vim 奇怪行为 ====== -I was wondering for quite some time why on my server vim behaves so stupid with respect to the mouse: Jumping around, copy and paste wasn't possible the usual way. All this despite having -我一直在想，为什么我服务器上 vim 为什么在鼠标方面表现得如此愚蠢：不能像平时那样跳转、复制、粘贴。尽管在 `/etc/vim/vimrc.local` 中已经设置了 + +我一直在想，为什么我服务器上 vim 为什么在鼠标方面表现得如此愚蠢：不能像平时那样跳转、复制、粘贴。尽管在 `/etc/vim/vimrc.local` 中已经设置了。 + ``` - set mouse= +set mouse= ``` 最后我终于知道为什么了，多谢 bug [#864074][1] 并且修复了它。 ![][2] -原因是，当没有 `~/.vimrc` 的时候，vim在 `vimrc.local` **之后**加载 `defaults.vim`，从而覆盖了几个设置。 +原因是，当没有 `~/.vimrc` 的时候，vim 在 `vimrc.local` **之后**加载 `defaults.vim`，从而覆盖了几个设置。 在 `/etc/vim/vimrc` 中有一个注释（虽然我没有看到）解释了这一点： + ``` " Vim will load $VIMRUNTIME/defaults.vim if the user does not have a vimrc. " This happens after /etc/vim/vimrc(.local) are loaded, so it will override @@ -22,12 +24,12 @@ I was wondering for quite some time why on my server vim behaves so stupid with " let g:skip_defaults_vim = 1 ``` - 我同意这是在正常安装 vim 后设置 vim 的好方法，但 Debian 包可以做得更好。在错误报告中清楚地说明了这个问题：如果没有 `~/.vimrc`，`/etc/vim/vimrc.local` 中的设置被覆盖。 这在Debian中是违反直觉的 - 而且我也不知道其他包中是否采用类似的方法。 由于 `defaults.vim` 中的设置非常合理，所以我希望使用它，但只修改了一些我不同意的项目，比如鼠标。最后，我在 `/etc/vim/vimrc.local` 中做了以下操作： + ``` if filereadable("/usr/share/vim/vim80/defaults.vim") source /usr/share/vim/vim80/defaults.vim @@ -40,7 +42,6 @@ set mouse= " other override settings go here ``` - 可能有更好的方式来获得一个不依赖于 vim 版本的通用加载语句, 但现在我对此很满意。 -------------------------------------------------------------------------------- @@ -49,7 +50,7 @@ via: https://www.preining.info/blog/2017/10/fixing-vim-in-debian/ 作者：[Norbert Preining][a] 译者：[geekpi](https://github.com/geekpi) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 diff --git a/translated/tech/20171017 check_mk error Cannot fetch deployment URL via curl error.md b/published/201801/20171017 check_mk error Cannot fetch deployment URL via curl error.md similarity index 62% rename from translated/tech/20171017 check_mk error Cannot fetch deployment URL via curl error.md rename to published/201801/20171017 check_mk error Cannot fetch deployment URL via curl error.md index 03952c5750..0cfa74945f 100644 --- a/translated/tech/20171017 check_mk error Cannot fetch deployment URL via curl error.md +++ b/published/201801/20171017 check_mk error Cannot fetch deployment URL via curl error.md @@ -1,12 +1,15 @@ -如何解决 check_mk 出现 "Cannot fetch deployment URL via curl" 的错误 +如何解决 check_mk 出现 “Cannot fetch deployment URL via curl” 的错误 ====== -本文解释了 'ERROR Cannot fetch deployment URL via curl：Couldn't resolve host。The given remote host was not resolved。' 的原因及其解决方案。 + +本文解释了 “ERROR Cannot fetch deployment URL via curl：Couldn't resolve host。The given remote host was not resolved。” 的原因及其解决方案。 ![ERROR Cannot fetch deployment URL via curl：Couldn't resolve host。The given remote host was not resolved。][1] -check_mk 是一个帮你配置 [nagios][2] 监控服务器的工具。然后在配置其中一台机器时，我遇到了下面的错误： +`check_mk` 是一个帮你配置 [nagios][2] 监控服务器的工具。然后在配置其中一台机器时，我遇到了下面的错误： -`ERROR Cannot fetch deployment URL via curl：Couldn't resolve host。The given remote host was not resolved。` +``` +ERROR Cannot fetch deployment URL via curl：Couldn't resolve host。The given remote host was not resolved。 +``` 该错误是在我使用下面命令尝试将该机器注册到监控服务器时发生的： @@ -14,15 +17,16 @@ check_mk 是一个帮你配置 [nagios][2] 监控服务器的工具。然后在 root@kerneltalks# /usr/bin/cmk-update-agent register -s monitor.kerneltalks.com -i master -H `hostname` -p http -U omdadmin -S ASFKWEFUNSHEFKG -v ``` -其中- +其中： -`-s` 指明监控服务器 -`-i` 指定服务器上 Check_MK 站点的名称 -`-H` 指定 agent 所在的主机名 -`-p` 为协议，可以是 http 或 https (默认为 https) -`-U` 允许下载 agent 的用户 ID -`-S` 为密码。用户的自动操作密码(当是自动用户时) -从错误中可以看出，命令无法解析监控服务器的 DNS 名称 `monitor.kerneltalks.com` +- `-s` 指明监控服务器 +- `-i` 指定服务器上 Check_MK 站点的名称 +- `-H` 指定 agent 所在的主机名 +- `-p` 为协议，可以是 http 或 https （默认为 https） +- `-U` 允许下载 agent 的用户 ID +- `-S` 为密码。用户的自动操作密码（当是自动用户时） + +从错误中可以看出，命令无法解析监控服务器的 DNS 名称 `monitor.kerneltalks.com`。 ### 解决方案： @@ -33,7 +37,7 @@ root@kerneltalks# cat /etc/hosts 10.0.10.9 monitor.kerneltalks.com ``` -这就搞定了。你可能成功注册了。 +这就搞定了。你现在可以成功注册了。 ``` root@kerneltalks # /usr/bin/cmk-update-agent register -s monitor.kerneltalks.com -i master -H `hostname` -p http -U omdadmin -S ASFKWEFUNSHEFKG -v @@ -51,11 +55,11 @@ via: https://kerneltalks.com/troubleshooting/check_mk-register-cannot-fetch-depl 作者：[kerneltalks][a] 译者：[lujun9972](https://github.com/lujun9972) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 [a]:https://kerneltalks.com -[1]:https://c4.kerneltalks.com/wp-content/uploads/2017/10/resolve-check_mk-error.png +[1]:https://a4.kerneltalks.com/wp-content/uploads/2017/10/resolve-check_mk-error.png [2]:https://www.nagios.org/ [3]:https://kerneltalks.com/linux/understanding-etc-hosts-file/ diff --git a/published/201801/20171019 3 Simple Excellent Linux Network Monitors.md b/published/201801/20171019 3 Simple Excellent Linux Network Monitors.md new file mode 100644 index 0000000000..e9e35fa708 --- /dev/null +++ b/published/201801/20171019 3 Simple Excellent Linux Network Monitors.md @@ -0,0 +1,181 @@ +三款简单而优秀的 Linux 网络监视工具 +============================================================ + +![network](https://www.linux.com/sites/lcom/files/styles/rendered_file/public/banner_3.png?itok=iuPcSN4k "network") + +> 通过 `iftop`、 `nethogs` 和 `vnstat` 详细了解你的网络连接状态。 + +你可以通过这三个 Linux 命令了解当前网络的大量信息。`iftop` 通过进程号跟踪网络连接，`nethogs` 快速告知你哪些进程在占用你的带宽，而 `vnstat` 以一个良好的轻量级守护进程在后台运行，并实时记录你的网络使用情况。 + +### iftop + +令人称赞的 `iftop` 可以监听您指定的网络接口，并以 top 的样式呈现。 + +这是一个不错的小工具，可以用于找出网络拥塞，测速和维持网络流量总量。看到自己到底在用多少带宽往往是非常惊人的，尤其是对于我们这些仍然记得电话线路、调制解调器，“高速”到令人惊叫的 kb 和实时波特率的老人们。我们在很久之前就不再使用波特率，转而钟情于比特率。波特率用于衡量信号变化，尽管有时候与比特率相同，但大多数情况下并非如此。 + +如果你只有一个网络接口，直接运行 `iftop` 即可。不过 `iftop` 需要 root 权限： + +``` +$ sudo iftop +``` + +如果你有多个网络接口，那就指定你要监控的接口： + +``` +$ sudo iftop -i wlan0 +``` + +就像 `top` 命令一样，你可以在命令运行时更改显示选项： + +* `h` 切换帮助界面。 +* `n` 是否解析域名。 +* `s` 切换源地址的显示，`d` 则切换目的地址的显示。 +* `S` 是否显示端口号。 +* `N` 是否解析端口；若关闭解析则显示端口号。 +* `t` 切换文本显示界面。默认的显示方式需要 ncurses。我个人认为图 1 的显示方式在组织性和可读性都更加良好。 +* `p` 暂停显示更新。 +* `q` 退出程序。 + +![text display](https://www.linux.com/sites/lcom/files/styles/rendered_file/public/fig-1_8.png?itok=luKHS5ve "text display") + +*图 1:组织性和可读性良好的文本显示。* + +当你切换显示设置的时候，`iftop` 并不会中断监测流量。当然你也可以单独监测一台主机。而这需要该主机的 IP 地址和子网掩码。现在，我很好奇 Pandora（LCTT 译注：一家美国的电台公司）能给我贫瘠的带宽带来多大的负载。因此我首先使用 dig 命令找到他们的 IP 地址： + +``` +$ dig A pandora.com +[...] +;; ANSWER SECTION: +pandora.com. 267 IN A 208.85.40.20 +pandora.com. 267 IN A 208.85.40.50 +``` + +那子网掩码呢？[ipcalc][9] 会告诉我们： + +``` +$ ipcalc -b 208.85.40.20 +Address: 208.85.40.20 +Netmask: 255.255.255.0 = 24 +Wildcard: 0.0.0.255 +=> +Network: 208.85.40.0/24 +``` + +现在，将 IP 地址和子网掩码提供给 `iftop`： + +``` +$ sudo iftop -F 208.85.40.20/24 -i wlan0 +``` + +很棒的不是么？而我也很惊奇地发现，Pandora 在我的网络上，每小时大约使用 500kb。并且就像大多数流媒体服务一样，Pandora 的流量在迅速增长，并依靠缓存稳定下来。 + +你可以使用 `-G` 选项对 IPv6 地址执行相同的操作。查阅友好的 man 可以帮助你了解 `iftop` 的其他功能，包括使用个人配置文件自定义你的默认选项，以及使用自定义过滤（请参阅 [PCAP-FILTER][10] 来获取过滤指南）。 + +### nethogs + +当你想要快速了解是谁在吸取你的带宽的时候，`nethogs` 是个快速而简单的方法。你需要以 root 身份运行并指定要监听的接口。它会给你显示大量的应用程序及其进程号，所以如果你想的话，你可以借此杀死任一进程。 + +``` +$ sudo nethogs wlan0 + +nethogs version 0.8.1 + +PID USER PROGRAM DEV SENT RECEIVED +7690 carla /usr/lib/firefox wlan0 12.494 556.580 KB/sec +5648 carla .../chromium-browser wlan0 0.052 0.038 KB/sec +TOTAL 12.546 556.618 KB/sec +``` + +`nethogs` 并没有多少选项：在 kb/s、kb、b、mb之间循环，按接收和发送的数据包排序，调整刷新延迟。具体请看`man nethogs`，或者是运行 `nethogs -h`。 + +### vnstat + +[vnstat][11]是最容易使用的网络数据收集工具。它十分轻量并且不需要 root 权限。它以守护进程在后台运行，因此可以实时地记录你的网络数据。单个 `vnstat` 命令就可以显示所累计的数据。 + +``` +$ vnstat -i wlan0 +Database updated: Tue Oct 17 08:36:38 2017 + + wlan0 since 10/17/2017 + + rx: 45.27 MiB tx: 3.77 MiB total: 49.04 MiB + + monthly + rx | tx | total | avg. rate + ------------------------+-------------+-------------+--------------- + Oct '17 45.27 MiB | 3.77 MiB | 49.04 MiB | 0.28 kbit/s + ------------------------+-------------+-------------+--------------- + estimated 85 MiB | 5 MiB | 90 MiB | + + daily + rx | tx | total | avg. rate + ------------------------+-------------+-------------+--------------- + today 45.27 MiB | 3.77 MiB | 49.04 MiB | 12.96 kbit/s + ------------------------+-------------+-------------+--------------- + estimated 125 MiB | 8 MiB | 133 MiB | +``` + +默认情况下它会显示所有的网络接口。使用 `-i` 选项来选择某个接口。也可以像这样合并多个接口的数据： + +``` +$ vnstat -i wlan0+eth0+eth1 +``` + +你可以通过这几种方式过滤数据显示： + +* `-h` 按小时显示统计信息。 +* `-d` 按天显示统计信息. +* `-w` 和 `-m` 分别按周和月份来显示统计信息。 +* 使用 `-l` 选项查看实时更新。 + +以下这条命令将会删除 wlan1 的数据库并不再监视它： + +``` +$ vnstat -i wlan1 --delete +``` + +而下面这条命令将会为你的一个网络接口创建一个别名。这个例子使用了 Ubuntu 16.04 的一个有线接口名称： + +``` +$ vnstat -u -i enp0s25 --nick eth0 +``` + +默认情况下，vnstat 会监视 eth0。你可以在 `/etc/vnstat.conf` 对它进行修改，或者在你的家目录下创建你自己的个人配置文件。请参阅 `man vnstat` 以获取完整的指南。 + +你也可以安装 `vnstati` 来创建简单的彩图（图 2）: + +``` +$ vnstati -s -i wlx7cdd90a0a1c2 -o vnstat.png +``` + +![vnstati](https://www.linux.com/sites/lcom/files/styles/rendered_file/public/fig-2_5.png?itok=HsWJMcW0 "vnstati") + +*图 2:你可以使用vnstati来创建简单的彩图。* + +请参阅 `man vnstati` 以获取完整的选项。 + + +_欲了解 Linux 的更多信息，可以通过学习 Linux 基金会和 edX 的免费课程，[“Linux 入门”][7]。_ + +-------------------------------------------------------------------------------- + +via: https://www.linux.com/learn/intro-to-linux/2017/10/3-simple-excellent-linux-network-monitors + +作者：[CARLA SCHRODER][a] +译者：[KeyLD](https://github.com/KeyLD) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.linux.com/users/cschroder +[1]:https://www.linux.com/licenses/category/used-permission +[2]:https://www.linux.com/licenses/category/used-permission +[3]:https://www.linux.com/licenses/category/used-permission +[4]:https://www.linux.com/files/images/fig-1png-8 +[5]:https://www.linux.com/files/images/fig-2png-5 +[6]:https://www.linux.com/files/images/bannerpng-3 +[7]:https://training.linuxfoundation.org/linux-courses/system-administration-training/introduction-to-linux +[8]:http://www.ex-parrot.com/pdw/iftop/ +[9]:https://www.linux.com/learn/intro-to-linux/2017/8/how-calculate-network-addresses-ipcalc +[10]:http://www.tcpdump.org/manpages/pcap-filter.7.html +[11]:http://humdi.net/vnstat/ diff --git a/translated/tech/20171024 Run Linux On Android Devices, No Rooting Required.md b/published/201801/20171024 Run Linux On Android Devices, No Rooting Required.md similarity index 68% rename from translated/tech/20171024 Run Linux On Android Devices, No Rooting Required.md rename to published/201801/20171024 Run Linux On Android Devices, No Rooting Required.md index 929c3ecdf8..83350e65a8 100644 --- a/translated/tech/20171024 Run Linux On Android Devices, No Rooting Required.md +++ b/published/201801/20171024 Run Linux On Android Devices, No Rooting Required.md @@ -1,8 +1,9 @@ -无需 Root 实现在 Android 设备上运行 Linux +无需 root 实现在 Android 设备上运行 Linux ====== + ![](https://www.ostechnix.com/wp-content/uploads/2017/10/Termux-720x340.jpg) -曾今，我尝试过搜索一种简单的可以在 Android 上运行 Linux 的方法。我当时唯一的意图只是想使用 Linux 以及一些基本的用用程序，比如 SSH，Git，awk 等。要求的并不多！我不不想 root Android 设备。我有一台平板电脑，主要用于阅读电子书，新闻和少数 Linux 博客。除此之外也不怎么用它了。因此我决定用它来实现一些 Linux 的功能。在 Google Play 商店上浏览了几分钟后，一个应用程序瞬间引起了我的注意，勾起了我实验的欲望。如果你也想在 Android 设备上运行 Linux，这个应用可能会有所帮助。 +曾经，我尝试过搜索一种简单的可以在 Android 上运行 Linux 的方法。我当时唯一的意图只是想使用 Linux 以及一些基本的应用程序，比如 SSH，Git，awk 等。要求的并不多！我不想 root 我的 Android 设备。我有一台平板电脑，主要用于阅读电子书、新闻和少数 Linux 博客。除此之外也不怎么用它了。因此我决定用它来实现一些 Linux 的功能。在 Google Play 商店上浏览了几分钟后，一个应用程序瞬间引起了我的注意，勾起了我实验的欲望。如果你也想在 Android 设备上运行 Linux，这个应用可能会有所帮助。 ### Termux - 在 Android 和 Chrome OS 上运行的 Android 终端模拟器 @@ -12,32 +13,32 @@ Termux 提供了许多重要的功能，比您想象的要多。 - * 它允许你通过 openSSH 登陆远程服务器 + * 它允许你通过 openSSH 登录远程服务器。 * 你还能够从远程系统 SSH 到 Android 设备中。 * 使用 rsync 和 curl 将您的智能手机通讯录同步到远程系统。 - * 支持不同的 shell，比如 BASH，ZSH，以及 FISH 等等。 - * 可以选择不同的文本编辑器来编辑/查看文件，支持 Emacs，Nano 和 Vim。 - * 使用 APT 软件包管理器在 Android 设备上安装你想要的软件包。支持 Git，Perl，Python，Ruby 和 Node.js 的最新版本。 - * 可以将 Android 设备与蓝牙键盘，鼠标和外置显示器连接起来，就像是整合在一起的设备一样。Termux 支持键盘快捷键。 + * 支持不同的 shell，比如 BASH、ZSH，以及 FISH 等等。 + * 可以选择不同的文本编辑器来编辑/查看文件，支持 Emacs、Nano 和 Vim。 + * 使用 APT 软件包管理器在 Android 设备上安装你想要的软件包。支持 Git、Perl、Python、Ruby 和 Node.js 的最新版本。 + * 可以将 Android 设备与蓝牙键盘、鼠标和外置显示器连接起来，就像是整合在一起的设备一样。Termux 支持键盘快捷键。 * Termux 支持几乎所有 GNU/Linux 命令。 此外通过安装插件可以启用其他一些功能。例如，**Termux：API** 插件允许你访问 Android 和 Chrome 的硬件功能。其他有用的插件包括： * Termux：Boot - 设备启动时运行脚本 * Termux：Float - 在浮动窗口中运行 Termux - * Termux：Styling - 提供配色方案和支持 powerline 的字体来定制 Termux 终端的外观。 + * Termux：Styling - 提供配色方案和支持 Powerline 的字体来定制 Termux 终端的外观。 * Termux：Task - 提供一种从任务栏类的应用中调用 Termux 可执行文件的简易方法。 * Termux：Widget - 提供一种从主屏幕启动小脚本的建议方法。 -要了解更多有关 termux 的信息，请长按终端上的任意位置并选择“帮助”菜单选项来打开内置的帮助部分。它唯一的缺点就是**需要 Android 5.0 及更高版本**。如果它支持 Android 4.x 和旧版本的话，将会更有用的多。你可以在** Google Play 商店 **和** F-Droid **中找到并安装 Termux。 +要了解更多有关 termux 的信息，请长按终端上的任意位置并选择“帮助”菜单选项来打开内置的帮助部分。它唯一的缺点就是**需要 Android 5.0 及更高版本**。如果它支持 Android 4.x 和旧版本的话，将会更有用的多。你可以在 **Google Play 商店**和 **F-Droid** 中找到并安装 Termux。 要在 Google Play 商店中安装 Termux，点击下面按钮。 -[![termux][1]][2] +[![][1]][2] 若要在 F-Droid 中安装，则点击下面按钮。 -[![][1]][3] +[![][5]][3] 你现在知道如何使用 Termux 在 Android 设备上使用 Linux 了。你有用过其他更好的应用吗？请在下面留言框中留言。我很乐意也去尝试他们！ @@ -45,7 +46,7 @@ Termux 提供了许多重要的功能，比您想象的要多。 相关资源： -+[Termux 官网 ][4] ++ [Termux 官网 ][4] -------------------------------------------------------------------------------- @@ -54,12 +55,13 @@ via: https://www.ostechnix.com/termux-run-linux-android-devices-no-rooting-requi 作者：[SK][a] 译者：[lujun9972](https://github.com/lujun9972) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 [a]:https://www.ostechnix.com/author/sk/ -[1]:data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7 +[1]:https://www.ostechnix.com/wp-content/uploads/2017/05/google-play-icon.png [2]:https://play.google.com/store/apps/details?id=com.termux [3]:https://f-droid.org/packages/com.termux/ [4]:https://termux.com/ +[5]:https://www.ostechnix.com/wp-content/uploads/2017/10/F-droid-1.png \ No newline at end of file diff --git a/published/20171028 Let Us Play Piano In Terminal Using Our PC Keyboard.md b/published/201801/20171028 Let Us Play Piano In Terminal Using Our PC Keyboard.md similarity index 100% rename from published/20171028 Let Us Play Piano In Terminal Using Our PC Keyboard.md rename to published/201801/20171028 Let Us Play Piano In Terminal Using Our PC Keyboard.md diff --git a/published/201801/20171030 How To Create Custom Ubuntu Live CD Image.md b/published/201801/20171030 How To Create Custom Ubuntu Live CD Image.md new file mode 100644 index 0000000000..97ede8771c --- /dev/null +++ b/published/201801/20171030 How To Create Custom Ubuntu Live CD Image.md @@ -0,0 +1,156 @@ +如何创建定制的 Ubuntu Live CD 镜像 +====== + +![](https://www.ostechnix.com/wp-content/uploads/2017/10/Cubic-720x340.png) + +今天让我们来讨论一下如何创建 Ubuntu Live CD 的定制镜像（ISO）。我们以前可以使用 [Pinguy Builder][1] 完成这项工作。但是，现在它似乎停止维护了。最近 Pinguy Builder 的官方网站似乎没有任何更新。幸运的是，我找到了另一种创建 Ubuntu Live CD 镜像的工具。使用 Cubic 即 **C**ustom **Ub**untu **I**SO **C**reator 的首字母缩写，这是一个用来创建定制的可启动的 Ubuntu Live CD（ISO）镜像的 GUI 应用程序。 + +Cubic 正在积极开发，它提供了许多选项来轻松地创建一个定制的 Ubuntu Live CD ，它有一个集成的 chroot 命令行环境（LCTT 译注：chroot —— Change Root，也就是改变程序执行时所参考的根目录位置），在那里你可以定制各种方面，比如安装新的软件包、内核，添加更多的背景壁纸，添加更多的文件和文件夹。它有一个直观的 GUI 界面，在 live 镜像创建过程中可以轻松的利用导航（可以利用点击鼠标来回切换）。您可以创建一个新的自定义镜像或修改现有的项目。因为它可以用来制作 Ubuntu live 镜像，所以我相信它可以用在制作其他 Ubuntu 的发行版和衍生版镜像中，比如 Linux Mint。 + +### 安装 Cubic + +Cubic 的开发人员已经做出了一个 PPA 来简化安装过程。要在 Ubuntu 系统上安装 Cubic ，在你的终端上运行以下命令: + +``` +sudo apt-add-repository ppa:cubic-wizard/release +sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 6494C6D6997C215E +sudo apt update +sudo apt install cubic +``` + +### 利用 Cubic 创建 Ubuntu Live CD 的定制镜像 + +安装完成后，从应用程序菜单或 dock 启动 Cubic。这是在我在 Ubuntu 16.04 LTS 桌面系统中 Cubic 的样子。 + +为新项目选择一个目录。它是保存镜像文件的目录。 + +![][3] + +请注意，Cubic 不是创建您当前系统的 Live CD 镜像，而是利用 Ubuntu 的安装 CD 来创建一个定制的 Live CD，因此，你应该有一个最新的 ISO 镜像。 + +选择您存储 Ubuntu 安装 ISO 镜像的路径。Cubic 将自动填写您定制操作系统的所有细节。如果你愿意，你可以改变细节。单击 Next 继续。 + +![][4] + +接下来，来自源安装介质中的压缩的 Linux 文件系统将被提取到项目的目录（在我们的例子中目录的位置是 `/home/ostechnix/custom_ubuntu`)。 + +![][5] + +一旦文件系统被提取出来，将自动加载到 chroot 环境。如果你没有看到终端提示符，请按几次回车键。 + +![][6] + +在这里可以安装任何额外的软件包，添加背景图片，添加软件源列表，添加最新的 Linux 内核和所有其他定制到你的 Live CD 。 + +例如，我希望 `vim` 安装在我的 Live CD 中，所以现在就要安装它。 + +![][7] + +我们不需要使用 `sudo`，因为我们已经在具有最高权限（root）的环境中了。 + +类似地，如果需要，可以安装更多的任何版本 Linux 内核。 + +``` +apt install linux-image-extra-4.10.0-24-generic +``` + +此外，您还可以更新软件源列表（添加或删除软件存储库列表）： + +![][8] + +修改源列表后，不要忘记运行 `apt update` 命令来更新源列表： + +``` +apt update +``` + +另外，您还可以向 Live CD 中添加文件或文件夹。复制文件或文件夹（右击它们并选择复制或者利用 `CTRL+C`），在终端右键单击（在 Cubic 窗口内)，选择 “Paste file(s)”，最后点击 Cubic 向导底部的 “Copy”。 + +![][9] + +**Ubuntu 17.10 用户注意事项** + +> 在 Ubuntu 17.10 系统中，DNS 查询可能无法在 chroot 环境中工作。如果您正在制作一个定制的 Ubuntu 17.10 Live 镜像，您需要指向正确的 `resolve.conf` 配置文件： + +>``` +ln -sr /run/systemd/resolve/resolv.conf /run/systemd/resolve/stub-resolv.conf +``` + +> 要验证 DNS 解析工作，运行: + +> ``` +cat /etc/resolv.conf +ping google.com +``` + +如果你想的话，可以添加你自己的壁纸。要做到这一点，请切换到 `/usr/share/backgrounds/` 目录， + +``` +cd /usr/share/backgrounds +``` + +并将图像拖放到 Cubic 窗口中。或复制图像，右键单击 Cubic 终端窗口并选择 “Paste file(s)” 选项。此外，确保你在 `/usr/share/gnome-backproperties` 的XML文件中添加了新的壁纸，这样你可以在桌面上右键单击新添加的图像选择 “Change Desktop Background” 进行交互。完成所有更改后，在 Cubic 向导中单击 “Next”。 + +接下来，选择引导到新的 Live ISO 镜像时使用的 Linux 内核版本。如果已经安装了其他版本内核，它们也将在这部分中被列出。然后选择您想在 Live CD 中使用的内核。 + +![][10] + +在下一节中，选择要从您的 Live 映像中删除的软件包。在使用定制的 Live 映像安装完 Ubuntu 操作系统后，所选的软件包将自动删除。在选择要删除的软件包时，要格外小心，您可能在不知不觉中删除了一个软件包，而此软件包又是另外一个软件包的依赖包。 + +![][11] + +接下来， Live 镜像创建过程将开始。这里所要花费的时间取决于你定制的系统规格。 + +![][12] + +镜像创建完成后后，单击 “Finish”。Cubic 将显示新创建的自定义镜像的细节。 + +如果你想在将来修改刚刚创建的自定义 Live 镜像，不要选择“ Delete all project files, except the generated disk image and the corresponding MD5 checksum file”（除了生成的磁盘映像和相应的 MD5 校验和文件之外，删除所有的项目文件**） ，Cubic 将在项目的工作目录中保留自定义图像，您可以在将来进行任何更改。而不用从头再来一遍。 + +要为不同的 Ubuntu 版本创建新的 Live 镜像，最好使用不同的项目目录。 + +### 利用 Cubic 修改 Ubuntu Live CD 的定制镜像 + +从菜单中启动 Cubic ，并选择一个现有的项目目录。单击 “Next” 按钮，您将看到以下三个选项： + +1. Create a disk image from the existing project. （从现有项目创建一个磁盘映像。） +2. Continue customizing the existing project.（继续定制现有项目。） +3. Delete the existing project.（删除当前项目。） + +![][13] + +第一个选项将允许您从现有项目中使用之前所做的自定义设置创建一个新的 Live ISO 镜像。如果您丢失了 ISO 镜像，您可以使用第一个选项来创建一个新的。 + +第二个选项允许您在现有项目中进行任何其他更改。如果您选择此选项，您将再次进入 chroot 环境。您可以添加新的文件或文件夹，安装任何新的软件，删除任何软件，添加其他的 Linux 内核，添加桌面背景等等。 + +第三个选项将删除现有的项目，所以您可以从头开始。选择此选项将删除所有文件，包括新生成的 ISO 镜像文件。 + +我用 Cubic 做了一个定制的 Ubuntu 16.04 LTS 桌面 Live CD 。就像这篇文章里描述的一样。如果你想创建一个 Ubuntu Live CD， Cubic 可能是一个不错的选择。 + +就这些了，再会！ + + +-------------------------------------------------------------------------------- + +via: https://www.ostechnix.com/create-custom-ubuntu-live-cd-image/ + +作者：[SK][a] +译者：[stevenzdg988](https://github.com/stevenzdg988) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.ostechnix.com/author/sk/ +[1]:https://www.ostechnix.com/pinguy-builder-build-custom-ubuntu-os/ +[2]:data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7 +[3]:http://www.ostechnix.com/wp-content/uploads/2017/10/Cubic-1.png +[4]:http://www.ostechnix.com/wp-content/uploads/2017/10/Cubic-2.png +[5]:http://www.ostechnix.com/wp-content/uploads/2017/10/Cubic-3.png +[6]:http://www.ostechnix.com/wp-content/uploads/2017/10/Cubic-4.png +[7]:http://www.ostechnix.com/wp-content/uploads/2017/10/Cubic-6.png +[8]:http://www.ostechnix.com/wp-content/uploads/2017/10/Cubic-5.png +[9]:http://www.ostechnix.com/wp-content/uploads/2017/10/Cubic-7.png +[10]:http://www.ostechnix.com/wp-content/uploads/2017/10/Cubic-8.png +[11]:http://www.ostechnix.com/wp-content/uploads/2017/10/Cubic-10-1.png +[12]:http://www.ostechnix.com/wp-content/uploads/2017/10/Cubic-12-1.png +[13]:http://www.ostechnix.com/wp-content/uploads/2017/10/Cubic-13.png diff --git a/published/20171031 Migrating to Linux- An Introduction.md b/published/201801/20171031 Migrating to Linux- An Introduction.md similarity index 100% rename from published/20171031 Migrating to Linux- An Introduction.md rename to published/201801/20171031 Migrating to Linux- An Introduction.md diff --git a/published/20171106 Autorandr- automatically adjust screen layout.md b/published/201801/20171106 Autorandr- automatically adjust screen layout.md similarity index 100% rename from published/20171106 Autorandr- automatically adjust screen layout.md rename to published/201801/20171106 Autorandr- automatically adjust screen layout.md diff --git a/translated/tech/20171107 How To Protect Server Against Brute Force Attacks With Fail2ban On Linux.md b/published/201801/20171107 How To Protect Server Against Brute Force Attacks With Fail2ban On Linux.md similarity index 63% rename from translated/tech/20171107 How To Protect Server Against Brute Force Attacks With Fail2ban On Linux.md rename to published/201801/20171107 How To Protect Server Against Brute Force Attacks With Fail2ban On Linux.md index 1d90ea333c..2a4547fef3 100644 --- a/translated/tech/20171107 How To Protect Server Against Brute Force Attacks With Fail2ban On Linux.md +++ b/published/201801/20171107 How To Protect Server Against Brute Force Attacks With Fail2ban On Linux.md @@ -1,64 +1,65 @@ -如何在Linux上用Fail2ban保护服务器免受暴力攻击 +如何在 Linux 上用 Fail2Ban 保护服务器免受暴力攻击 ====== -Linux管理员的一个重要任务是保护服务器免受非法攻击或访问。 默认情况下，Linux系统带有配置良好的防火墙，比如Iptables，Uncomplicated Firewall（UFW），ConfigServer Security Firewall（CSF）等，可以防止多种攻击。 +Linux 管理员的一个重要任务是保护服务器免受非法攻击或访问。 默认情况下，Linux 系统带有配置良好的防火墙，比如iptables、Uncomplicated Firewall（UFW），ConfigServer Security Firewall（CSF）等，可以防止多种攻击。 -任何连接到互联网的机器都是恶意攻击的潜在目标。 有一个名为fail2ban的工具可用来缓解服务器上的非法访问。 +任何连接到互联网的机器都是恶意攻击的潜在目标。 有一个名为 Fail2Ban 的工具可用来缓解服务器上的非法访问。 -### 什么是Fail2ban？ +### 什么是 Fail2Ban？ -[Fail2ban][1]是一款入侵防御软件，可以保护服务器免受暴力攻击。 它是用Python编程语言编写的。 Fail2ban基于auth日志文件工作，默认情况下它会扫描所有auth日志文件，如`/var/log/auth.log`，`/var/log/apache/access.log`等，并禁止带有恶意标志的IP，比如密码失败太多，寻找漏洞等等标志。 +[Fail2Ban][1] 是一款入侵防御软件，可以保护服务器免受暴力攻击。 它是用 Python 编程语言编写的。 Fail2Ban 基于auth 日志文件工作，默认情况下它会扫描所有 auth 日志文件，如 `/var/log/auth.log`、`/var/log/apache/access.log` 等，并禁止带有恶意标志的IP，比如密码失败太多，寻找漏洞等等标志。 -通常，fail2Ban用于更新防火墙规则，用于在指定的时间内拒绝IP地址。 它也会发送邮件通知。 Fail2Ban为各种服务提供了许多过滤器，如ssh，apache，nginx，squid，named，mysql，nagios等。 +通常，Fail2Ban 用于更新防火墙规则，用于在指定的时间内拒绝 IP 地址。 它也会发送邮件通知。 Fail2Ban 为各种服务提供了许多过滤器，如 ssh、apache、nginx、squid、named、mysql、nagios 等。 -Fail2Ban能够降低错误认证尝试的速度，但是它不能消除弱认证带来的风险。 这只是服务器防止暴力攻击的安全手段之一。 +Fail2Ban 能够降低错误认证尝试的速度，但是它不能消除弱认证带来的风险。 这只是服务器防止暴力攻击的安全手段之一。 -### 如何在Linux中安装Fail2ban +### 如何在 Linux 中安装 Fail2Ban -Fail2ban已经与大部分Linux发行版打包在一起了，所以只需使用你的发行包版的包管理器来安装它。 +Fail2Ban 已经与大部分 Linux 发行版打包在一起了，所以只需使用你的发行包版的包管理器来安装它。 -对于**`Debian / Ubuntu`**，使用[APT-GET命令][2]或[APT命令][3]安装。 +对于 Debian / Ubuntu，使用 [APT-GET 命令][2]或 [APT 命令][3]安装。 ``` $ sudo apt install fail2ban ``` -对于**`Fedora`**，使用[DNF命令][4]安装。 +对于 Fedora，使用 [DNF 命令][4]安装。 ``` $ sudo dnf install fail2ban ``` -对于 **`CentOS/RHEL`**，启用[EPEL库][5]或[RPMForge][6]库，使用[YUM命令][7]安装。 +对于 CentOS/RHEL，启用 [EPEL 库][5]或 [RPMForge][6] 库，使用 [YUM 命令][7]安装。 ``` $ sudo yum install fail2ban ``` -对于**`Arch Linux`**，使用[Pacman命令][8]安装。 +对于 Arch Linux，使用 [Pacman 命令][8]安装。 ``` $ sudo pacman -S fail2ban ``` -对于 **`openSUSE`** , 使用[Zypper命令][9]安装. +对于 openSUSE , 使用 [Zypper命令][9]安装。 + ``` $ sudo zypper in fail2ban ``` -### 如何配置Fail2ban +### 如何配置 Fail2Ban -默认情况下，Fail2ban将所有配置文件保存在`/etc/fail2ban/` 目录中。 主配置文件是`jail.conf`，它包含一组预定义的过滤器。 所以，不要编辑文件，这是不可取的，因为只要有新的更新配置就会重置为默认值。 +默认情况下，Fail2Ban 将所有配置文件保存在 `/etc/fail2ban/` 目录中。 主配置文件是 `jail.conf`，它包含一组预定义的过滤器。 所以，不要编辑该文件，这是不可取的，因为只要有新的更新，配置就会重置为默认值。 -只需在同一目录下创建一个名为`jail.local`的新配置文件，并根据您的意愿进行修改。 +只需在同一目录下创建一个名为 `jail.local` 的新配置文件，并根据您的意愿进行修改。 ``` # cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local ``` -默认情况下，大多数选项都已经配置的很完美了，如果要启用对任何特定IP的访问，则可以将IP地址添加到`ignoreip` 区域，对于多个ip的情况，用空格隔开ip地址。 +默认情况下，大多数选项都已经配置的很完美了，如果要启用对任何特定 IP 的访问，则可以将 IP 地址添加到 `ignoreip` 区域，对于多个 IP 的情况，用空格隔开 IP 地址。 -配置文件中的`DEFAULT`部分包含Fail2Ban遵循的基本规则集，您可以根据自己的意愿调整任何参数。 +配置文件中的 `DEFAULT` 部分包含 Fail2Ban 遵循的基本规则集，您可以根据自己的意愿调整任何参数。 ``` # nano /etc/fail2ban/jail.local @@ -71,16 +72,14 @@ maxretry = 3 destemail = 2daygeek@gmail.com ``` - * **ignoreip：**本部分允许我们列出IP地址列表，Fail2ban不会禁止与列表中的地址匹配的主机 -* **bantime：**主机被禁止的秒数 -* **findtime：**如果在上次“findtime”秒期间已经发生了“maxretry”次重试，则主机会被禁止 -* **maxretry：**“maxretry”是主机被禁止之前的失败次数 - - +* `ignoreip`：本部分允许我们列出 IP 地址列表，Fail2Ban 不会禁止与列表中的地址匹配的主机 +* `bantime`：主机被禁止的秒数 +* `findtime`：如果在最近 `findtime` 秒期间已经发生了 `maxretry` 次重试，则主机会被禁止 +* `maxretry`：是主机被禁止之前的失败次数 ### 如何配置服务 -Fail2ban带有一组预定义的过滤器，用于各种服务，如ssh，apache，nginx，squid，named，mysql，nagios等。 我们不希望对配置文件进行任何更改，只需在服务区域中添加`enabled = true`这一行就可以启用任何服务。 禁用服务时将true改为false即可。 +Fail2Ban 带有一组预定义的过滤器，用于各种服务，如 ssh、apache、nginx、squid、named、mysql、nagios 等。 我们不希望对配置文件进行任何更改，只需在服务区域中添加 `enabled = true` 这一行就可以启用任何服务。 禁用服务时将 `true` 改为 `false` 即可。 ``` # SSH servers @@ -91,16 +90,15 @@ logpath = %(sshd_log)s backend = %(sshd_backend)s ``` - * **enabled：** 确定服务是打开还是关闭。 -* **port ：**指的是特定的服务。 如果使用默认端口，则服务名称可以放在这里。 如果使用非传统端口，则应该是端口号。 -* **logpath：**提供服务日志的位置 -* **backend：**“后端”指定用于获取文件修改的后端。 +* `enabled`： 确定服务是打开还是关闭。 +* `port`：指明特定的服务。 如果使用默认端口，则服务名称可以放在这里。 如果使用非传统端口，则应该是端口号。 +* `logpath`：提供服务日志的位置 +* `backend`：指定用于获取文件修改的后端。 +### 重启 Fail2Ban +进行更改后，重新启动 Fail2Ban 才能生效。 -### 重启Fail2Ban - -进行更改后，重新启动Fail2Ban才能生效。 ``` [For SysVinit Systems] # service fail2ban restart @@ -109,9 +107,10 @@ backend = %(sshd_backend)s # systemctl restart fail2ban.service ``` -### 验证Fail2Ban iptables规则 +### 验证 Fail2Ban iptables 规则 + +你可以使用下面的命令来确认是否在防火墙中成功添加了Fail2Ban iptables 规则。 -你可以使用下面的命令来确认是否在防火墙中成功添加了Fail2Ban iptables规则。 ``` # iptables -L Chain INPUT (policy ACCEPT) @@ -135,9 +134,9 @@ target prot opt source destination RETURN all -- anywhere anywhere ``` -### 如何测试Fail2ban +### 如何测试 Fail2Ban -我做了一些失败的尝试来测试这个。 为了证实这一点，我要验证`/var/log/fail2ban.log` 文件。 +我做了一些失败的尝试来测试这个。 为了证实这一点，我要验证 `/var/log/fail2ban.log` 文件。 ``` 2017-11-05 14:43:22,901 fail2ban.server [7141]: INFO Changed logging target to /var/log/fail2ban.log for Fail2ban v0.9.6 @@ -184,6 +183,7 @@ RETURN all -- anywhere anywhere ``` 要查看启用的监狱列表，请运行以下命令。 + ``` # fail2ban-client status Status @@ -191,7 +191,8 @@ Status `- Jail list: apache-auth, sshd ``` -通过运行以下命令来获取禁止的IP地址。 +通过运行以下命令来获取禁止的 IP 地址。 + ``` # fail2ban-client status ssh Status for the jail: ssh @@ -205,18 +206,19 @@ Status for the jail: ssh `- Total banned: 1 ``` -要从Fail2Ban中删除禁止的IP地址，请运行以下命令。 +要从 Fail2Ban 中删除禁止的 IP 地址，请运行以下命令。 + ``` # fail2ban-client set ssh unbanip 192.168.1.115 ``` -------------------------------------------------------------------------------- -via: https://www.2daygeek.com/how-to-install-setup-configure-fail2ban-on-linux/# +via: https://www.2daygeek.com/how-to-install-setup-configure-fail2ban-on-linux/ 作者：[Magesh Maruthamuthu][a] 译者：[Flowsnow](https://github.com/Flowsnow) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 diff --git a/published/201801/20171107 The long goodbye to C.md b/published/201801/20171107 The long goodbye to C.md new file mode 100644 index 0000000000..436c01021f --- /dev/null +++ b/published/201801/20171107 The long goodbye to C.md @@ -0,0 +1,87 @@ +与 C 语言长别离 +========================================== + +这几天来，我在思考那些正在挑战 C 语言的系统编程语言领袖地位的新潮语言，尤其是 Go 和 Rust。思考的过程中，我意识到了一个让我震惊的事实 —— 我有着 35 年的 C 语言经验。每周我都要写很多 C 代码，但是我已经记不清楚上一次我 _创建一个新的 C 语言项目_ 是在什么时候了。 + +如果你完全不认为这种情况令人震惊，那你很可能不是一个系统程序员。我知道有很多程序员使用更高级的语言工作。但是我把大部分时间都花在了深入打磨像 NTPsec、 GPSD 以及 giflib 这些东西上。熟练使用 C 语言在这几十年里一直就是我的专长。但是，现在我不仅是不再使用 C 语言写新的项目，甚至我都记不清我是什么时候开始这样做的了，而且……回头想想，我觉得这都不是本世纪发生的事情。 + +这个对于我来说是件大事，因为如果你问我，我的五个最核心软件开发技能是什么，“C 语言专家” 一定是你最有可能听到的之一。这也激起了我的思考。C 语言的未来会怎样 ？C 语言是否正像当年的 COBOL 语言一样，在辉煌之后，走向落幕？ + +我恰好是在 C 语言迅猛发展，并把汇编语言以及其它许多编译型语言挤出主流存在的前几年开始编程的。那场过渡大约是在 1982 到 1985 年之间。在那之前，有很多编译型语言争相吸引程序员的注意力，那些语言中还没有明确的领导者；但是在那之后，小众的语言就直接毫无声息的退出了舞台。主流的语言（FORTRAN、Pascal、COBOL）则要么只限于老代码，要么就是固守单一领域，再就是在 C 语言的边缘领域顶着愈来愈大的压力苟延残喘。 + +而在那以后，这种情形持续了近 30 年。尽管在应用程序开发上出现了新的动向： Java、 Perl、 Python， 以及许许多多不是很成功的竞争者。起初我很少关注这些语言，这很大一部分是因为在它们的运行时的开销对于当时的实际硬件来说太大。因此，这就使得 C 的成功无可撼动；为了使用和对接大量已有的 C 语言代码，你得使用 C 语言写新代码（一部分脚本语言尝试过打破这种壁垒，但是只有 Python 有可能取得成功）。 + +回想起来，我在 1997 年使用脚本语言写应用时本应该注意到这些语言的更重要的意义的。当时我写的是一个名为 SunSITE 的帮助图书管理员做源码分发的辅助软件，当时使用的是 Perl 语言。 + +这个应用完全是用来处理文本输入的，而且只需要能够应对人类的反应速度即可（大概 0.1 秒），因此使用 C 或者别的没有动态内存分配以及字符串类型的语言来写就会显得很傻。但是在当时，我仅仅是把其视为一个试验，而完全没有想到我几乎再也不会在一个新项目的第一个文件里敲下 `int main(int argc, char **argv)` 这样的 C 语言代码了。 + +我说“几乎”，主要是因为 1999 年的 [SNG][3]。 我想那是我最后一个用 C 从头开始写的项目了。 + +在那之后我写的所有的 C 代码都是在为那些上世纪已经存在的老项目添砖加瓦，或者是在维护诸如 GPSD 以及 NTPsec 一类的项目。 + +当年我本不应该使用 C 语言写 SNG 的。因为在那个年代，摩尔定律的快速迭代使得硬件愈加便宜，使得像 Perl 这样的语言的执行效率也不再是问题。仅仅三年以后，我可能就会毫不犹豫地使用 Python 而不是 C 语言来写 SNG。 + +在 1997 年我学习了 Python， 这对我来说是一道分水岭。这个语言很美妙 —— 就像我早年使用的 Lisp 一样，而且 Python 还有很酷的库！甚至还完全遵循了 POSIX！还有一个蛮好用的对象系统！Python 没有把 C 语言挤出我的工具箱，但是我很快就习惯了在只要能用 Python 时就写 Python ，而只在必须使用 C 语言时写 C。 + +（在此之后，我开始在我的访谈中指出我所谓的 “Perl 的教训” ，也就是任何一个没能实现和 C 语言语义等价的遵循 POSIX 的语言_都注定要失败_。在计算机科学的发展史上，很多学术语言的骨骸俯拾皆是，原因是这些语言的设计者没有意识到这个重要的问题。） + +显然，对我来说，Python 的主要优势之一就是它很简单，当我写 Python 时，我不再需要担心内存管理问题或者会导致核心转储的程序崩溃 —— 对于 C 程序员来说，处理这些问题烦的要命。而不那么明显的优势恰好在我更改语言时显现，我在 90 年代末写应用程序和非核心系统服务的代码时，为了平衡成本与风险都会倾向于选择具有自动内存管理但是开销更大的语言，以抵消之前提到的 C 语言的缺陷。而在仅仅几年之前（甚至是 1990 年），那些语言的开销还是大到无法承受的；那时硬件产业的发展还在早期阶段，没有给摩尔定律足够的时间来发挥威力。 + +尽量地在 C 语言和 Python 之间选择 C —— 只要是能的话我就会从 C 语言转移到 Python 。这是一种降低工程复杂程度的有效策略。我将这种策略应用在了 GPSD 中，而针对 NTPsec , 我对这个策略的采用则更加系统化。这就是我们能把 NTP 的代码库大小削减四分之一的原因。 + +但是今天我不是来讲 Python 的。尽管我觉得它在竞争中脱颖而出，Python 也未必真的是在 2000 年之前彻底结束我在新项目上使用 C 语言的原因，因为在当时任何一个新的学院派的动态语言都可以让我不再选择使用 C 语言。也有可能是在某段时间里在我写了很多 Java 之后，我才慢慢远离了 C 语言。 + +我写这个回忆录是因为我觉得我并非特例，在世纪之交，同样的发展和转变也改变了不少 C 语言老手的编码习惯。像我一样，他们在当时也并没有意识到这种转变正在发生。 + +在 2000 年以后，尽管我还在使用 C/C++ 写之前的项目，比如 GPSD ，游戏韦诺之战以及 NTPsec，但是我的所有新项目都是使用 Python 的。 + +有很多程序是在完全无法在 C 语言下写出来的，尤其是 [reposurgeon][4] 以及 [doclifter][5] 这样的项目。由于 C 语言受限的数据类型本体论以及其脆弱的底层数据管理问题，尝试用 C 写的话可能会很恐怖，并注定失败。 + +甚至是对于更小的项目 —— 那些可以在 C 中实现的东西 —— 我也使用 Python 写，因为我不想花不必要的时间以及精力去处理内核转储问题。这种情况一直持续到去年年底，持续到我创建我的第一个 Rust 项目，以及成功写出第一个[使用 Go 语言的项目][6]。 + +如前文所述，尽管我是在讨论我的个人经历，但是我想我的经历体现了时代的趋势。我期待新潮流的出现，而不是仅仅跟随潮流。在 98 年的时候，我就是 Python 的早期使用者。来自 [TIOBE][7] 的数据则表明，在 Go 语言脱胎于公司的实验项目并刚刚从小众语言中脱颖而出的几个月内，我就开始实现自己的第一个 Go 语言项目了。 + +总而言之：直到现在第一批有可能挑战 C 语言的传统地位的语言才出现。我判断这个的标准很简单 —— 只要这个语言能让我等 C 语言老手接受不再写 C 的事实，这个语言才 “有可能” 挑战到 C 语言的地位 —— 来看啊，这有个新编译器，能把 C 转换到新语言，现在你可以让他完成你的_全部工作_了 —— 这样 C 语言的老手就会开心起来。 + +Python 以及和其类似的语言对此做的并不够好。使用 Python 实现 NTPsec（以此举例）可能是个灾难，最终会由于过高的运行时开销以及由于垃圾回收机制导致的延迟变化而烂尾。如果需求是针对单个用户且只需要以人类能接受的速度运行，使用 Python 当然是很好的，但是对于以 _机器的速度_ 运行的程序来说就不总是如此了 —— 尤其是在很高的多用户负载之下。这不只是我自己的判断 —— 因为拿 Go 语言来说，它的存在主要就是因为当时作为 Python 语言主要支持者的 Google 在使用 Python 实现一些工程的时候也遭遇了同样的效能痛点。 + +Go 语言就是为了解决 Python 搞不定的那些大多由 C 语言来实现的任务而设计的。尽管没有一个全自动语言转换软件让我很是不爽，但是使用 Go 语言来写系统程序对我来说不算麻烦，我发现我写 Go 写的还挺开心的。我的很多 C 编码技能还可以继续使用，我还收获了垃圾回收机制以及并发编程机制，这何乐而不为？ + +（[这里][8]有关于我第一次写 Go 的经验的更多信息） + +本来我想把 Rust 也视为 “C 语言要过时了” 的例证，但是在学习并尝试使用了这门语言编程之后，我觉得[这种语言现在还没有做好准备][9]。也许 5 年以后，它才会成为 C 语言的对手。 + +随着 2017 的尾声来临，我们已经发现了一个相对成熟的语言，其和 C 类似，能够胜任 C 语言的大部分工作场景（我在下面会准确描述），在几年以后，这个语言界的新星可能就会取得成功。 + +这件事意义重大。如果你不长远地回顾历史，你可能看不出来这件事情的伟大性。_三十年了_ —— 这几乎就是我作为一个程序员的全部生涯，我们都没有等到一个 C 语言的继任者，也无法遥望 C 之后的系统编程会是什么样子的。而现在，我们面前突然有了后 C 时代的两种不同的展望和未来…… + +……另一种展望则是下面这个语言留给我们的。我的一个朋友正在开发一个他称之为 “Cx” 的语言，这个语言在 C 语言上做了很少的改动，使得其能够支持类型安全；他的项目的目的就是要创建一个能够在最少人力参与的情况下把古典 C 语言修改为新语言的程序。我不会指出这位朋友的名字，免得给他太多压力，让他做出太多不切实际的保证。但是他的实现方法真的很是有意思，我会尽量给他募集资金。 + +现在，我们看到了可以替代 C 语言实现系统编程的三种不同的可能的道路。而就在两年之前，我们的眼前还是一片漆黑。我重复一遍：这件事情意义重大。 + +我是在说 C 语言将要灭绝吗？不是这样的，在可预见的未来里，C 语言还会是操作系统的内核编程以及设备固件编程的主流语言，在这些场景下，尽力压榨硬件性能的古老规则还在奏效，尽管它可能不是那么安全。 + +现在那些将要被 C 的继任者攻破的领域就是我之前提到的我经常涉及的领域 —— 比如 GPSD 以及 NTPsec、系统服务以及那些因为历史原因而使用 C 语言写的进程。还有就是以 DNS 服务器以及邮件传输代理 —— 那些需要以机器速度而不是人类的速度运行的系统程序。 + +现在我们可以对后 C 时代的未来窥见一斑，即上述这类领域的代码都可以使用那些具有强大内存安全特性的 C 语言的替代者实现。Go 、Rust 或者 Cx ，无论是哪个，都可能使 C 的存在被弱化。比如，如果我现在再来重新实现一遍 NTP ，我可能就会毫不犹豫的使用 Go 语言去完成。 + +-------------------------------------------------------------------------------- + +via: http://esr.ibiblio.org/?p=7711 + +作者：[Eric Raymond][a] +译者：[name1e5s](https://github.com/name1e5s) +校对：[yunfengHe](https://github.com/yunfengHe), [wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://esr.ibiblio.org/?author=2 +[1]:http://esr.ibiblio.org/?author=2 +[2]:http://esr.ibiblio.org/?p=7711 +[3]:http://sng.sourceforge.net/ +[4]:http://www.catb.org/esr/reposurgeon/ +[5]:http://www.catb.org/esr/doclifter/ +[6]:http://www.catb.org/esr/loccount/ +[7]:https://www.tiobe.com/tiobe-index/ +[8]:https://blog.ntpsec.org/2017/02/07/grappling-with-go.html +[9]:http://esr.ibiblio.org/?p=7303 diff --git a/published/20171109 Learn how to use tcpdump command with examples.md b/published/201801/20171109 Learn how to use tcpdump command with examples.md similarity index 100% rename from published/20171109 Learn how to use tcpdump command with examples.md rename to published/201801/20171109 Learn how to use tcpdump command with examples.md diff --git a/translated/tech/20171112 Love Your Bugs.md b/published/201801/20171112 Love Your Bugs.md similarity index 97% rename from translated/tech/20171112 Love Your Bugs.md rename to published/201801/20171112 Love Your Bugs.md index 9203cf52bb..b983fae177 100644 --- a/translated/tech/20171112 Love Your Bugs.md +++ b/published/201801/20171112 Love Your Bugs.md @@ -15,19 +15,17 @@ 好，让我们直接来看第一个 bug。这是我在 Dropbox 工作时遇到的一个 bug。你们或许听说过，Dropbox 是一个将你的文件从一个电脑上同步到云端和其他电脑上的应用。 - - ``` +--------------+ +---------------+ | | | | - | METASERVER | | BLOCKSERVER | + | 元数据服务器 | | 块服务器 | | | | | +-+--+---------+ +---------+-----+ ^ | ^ | | | | | +----------+ | | +---> | | | - | | CLIENT +--------+ + | | 客户端 +--------+ +--------+ | +----------+ ``` @@ -79,7 +77,7 @@ l \x0c < $ ( . - ``` -英文逗号的 ASCII 码是44。`l` 的 ASCII 码是 108。它们的二进制表示如下： +英文逗号的 ASCII 码是 44。`l` 的 ASCII 码是 108。它们的二进制表示如下： ``` bin(ord(',')): 0101100 @@ -101,8 +99,7 @@ $ : 0100100 - : 0101101 ``` - -### 位反转是真的！ +#### 位反转是真的！ 我爱这个 bug 因为它证明了位反转是可能真实发生的事情，而不只是一个理论上的问题。实际上，它在某些情况下会比平时更容易发生。其中一种情况是用户使用的是低配或者老旧的硬件，而运行 Dropbox 的电脑很多都是这样。另外一种会造成很多位反转的地方是外太空——在太空中没有大气层来保护你的内存不受高能粒子和辐射的影响，所以位反转会十分常见。 @@ -110,31 +107,31 @@ $ : 0100100 在刚才那种情况下，Dropbox 并不需要处理位反转。出现内存损坏的是用户的电脑，所以即使我们可以检测到逗号字符的位反转，但如果这发生在其他字符上我们就不一定能检测到了，而且如果从硬盘中读取的文件本身发生了位反转，那我们根本无从得知。我们能改进的地方很少，于是我们决定无视这个异常并继续程序的运行。这种 bug 一般都会在客户端重启之后自动解决。 -### 不常见的 bug 并非不可能发生 +#### 不常见的 bug 并非不可能发生 这是我最喜欢的 bug 之一，有几个原因。第一，它提醒我注意不常见和不可能之间的区别。当规模足够大的时候，不常见的现象会以值得注意的频率发生。 -### 覆盖面广的 bug +#### 覆盖面广的 bug 这个 bug 第二个让我喜欢的地方是它覆盖面非常广。每当桌面客户端和服务器交流的时候，这个 bug 都可能悄然出现，而这可能会发生在系统里很多不同的端点和组件当中。这意味着许多不同的 Dropbox 工程师会看到这个 bug 的各种版本。你第一次看到它的时候，你 _真的_ 会满头雾水，但在那之后诊断这个 bug 就变得很容易了，而调查过程也非常简短：你只需找到中间的字母，看它是不是个 `l`。 -### 文化差异 +#### 文化差异 这个 bug 的一个有趣的副作用是它展示了服务器组和客户端组之间的文化差异。有时候这个 bug 会被服务器组的成员发现并展开调查。如果你的 _服务器_ 上发生了位反转，那应该不是个偶然——这很可能是内存损坏，你需要找到受影响的主机并尽快把它从集群中移除，不然就会有损坏大量用户数据的风险。这是个事故，而你必须迅速做出反应。但如果是用户的电脑在破坏数据，你并没有什么可以做的。 -### 分享你的 bug +#### 分享你的 bug 如果你在调试一个难搞的 bug，特别是在大型系统中，不要忘记跟别人讨论。也许你的同事以前就遇到过类似的 bug。若是如此，你可能会节省很多时间。就算他们没有见过，也不要忘记在你解决了问题之后告诉他们解决方法——写下来或者在组会中分享。这样下次你们组遇到类似的问题时，你们都会早有准备。 ### Bug 如何帮助你进步 -### Recurse Center +#### Recurse Center 在加入 Dropbox 之前，我曾在 Recurse Center 工作。它的理念是建立一个社区让正在自学的程序员们聚到一起来提高能力。这就是 Recurse Center 的全部了：我们没有大纲、作业、截止日期等等。唯一的前提条件是我们都想要成为更好的程序员。参与者中有的人有计算机学位但对自己的实际编程能力不够自信，有的人已经写了十年 Java 但想学 Clojure 或者 Haskell，还有各式各样有着其他的背景的参与者。 我在那里是一位导师，帮助人们更好地利用这个自由的环境，并参考我们从以前的参与者那里学到的东西来提供指导。所以我的同事们和我本人都非常热衷于寻找对成年自学者最有帮助的学习方法。 -### 刻意练习 +#### 刻意练习 在学习方法这个领域有很多不同的研究，其中我觉得最有意思的研究之一是刻意练习的概念。刻意练习理论意在解释专业人士和业余爱好者的表现的差距。它的基本思想是如果你只看内在的特征——不论先天与否——它们都无法非常好地解释这种差距。于是研究者们，包括最初的 Ericsson、Krampe 和 Tesch-Romer，开始寻找能够解释这种差距的理论。他们最终的答案是在刻意练习上所花的时间。 @@ -189,18 +186,15 @@ $ : 0100100 所有这些 bug 都很容易修复。前两个 bug 出在客户端上，所以我们在 alpha 版本修复了它们，但大部分的客户端还没有获得这些改动。我们在服务器代码中修复了第三个 bug 并部署了新版的服务器。 -### 📈 +#### 激增 突然日志服务器集群的流量开始激增。客服团队找到我们并问我们是否知道原因。我花了点时间把所有的部分拼到一起。 在修复之前，这四件事情会发生： 1. 日志文件从最早的开始发送 - 2. 日志文件从最新的开始删除 - 3. 如果服务器无法解码日志文件，它会返回 500 - 4. 如果客户端收到 500，它会停止发送日志 一个存有损坏的日志文件的客户端会试着发送这个文件，服务器会返回 500，客户端会放弃发送日志。在下一次运行时，它会尝试再次发送同样的文件，再次失败，并再次放弃。最终日志目录会被填满，然后客户端会开始删除最新的日志文件，而把损坏的文件继续保留在硬盘上。 @@ -209,27 +203,27 @@ $ : 0100100 问题是，处于这种状态的客户端比我们想象的要多很多。任何有一个损坏文件的客户端都会像被关在堤坝里一样，无法再发送日志。现在这个堤坝被清除了，所有这些客户端都开始发送它们的日志目录的剩余内容。 -###　我们的选择 +#### 我们的选择 -好的，现在文件从世界各地的电脑如洪水般涌来。我们能做什么？（当你在一个有 Dropbox 这种规模，尤其是这种桌面客户端的规模的公司工作时，会遇到这种有趣的事情：你可以非常轻易地对自己造成 DDOS 攻击）。 +好的，现在文件从世界各地的电脑如洪水般涌来。我们能做什么？（当你在一个有 Dropbox 这种规模，尤其是这种桌面客户端的规模的公司工作时，会遇到这种有趣的事情：你可以非常轻易地对自己造成 DDoS 攻击）。 当你部署的新版本发生问题时，第一个选项是回滚。这是非常合理的选择，但对于这个问题，它无法帮助我们。我们改变的不是服务器的状态而是客户端的——我们删除了那些出错文件。将服务器回滚可以防止更多客户端进入这种状态，但它并不能解决根本问题。 -那扩大日志集群的规模呢？我们试过了——然后因为处理能力增加了，我们开始收到更多的请求。我们又扩大了一次，但你不可能一直这么下去。为什么不能？因为这个集群并不是独立的。它会向另一个集群发送请求，在这里是为了处理异常。如果你的一个集群正在被 DDOS，而你持续扩大那个集群，你最终会把它依赖的集群也弄坏，然后你就有两个问题了。 +那扩大日志集群的规模呢？我们试过了——然后因为处理能力增加了，我们开始收到更多的请求。我们又扩大了一次，但你不可能一直这么下去。为什么不能？因为这个集群并不是独立的。它会向另一个集群发送请求，在这里是为了处理异常。如果你的一个集群正在被 DDoS，而你持续扩大那个集群，你最终会把它依赖的集群也弄坏，然后你就有两个问题了。 我们考虑过的另一个选择是减低负载——你不需要每一个日志文件，所以我们可以直接无视一些请求。一个难点是我们并没有一个很好的方法来区分好的请求和坏的请求。我们无法快速地判断哪些日志文件是旧的，哪些是新的。 我们最终使用的是一个 Dropbox 里许多不同场合都用过的一个解决方法：我们有一个自定义的头字段，`chillout`，全世界所有的客户端都遵守它。如果客户端收到一个有这个头字段的响应，它将在字段所标注的时间内不再发送任何请求。很早以前一个英明的程序员把它加到了 Dropbox 客户端里，在之后这些年中它已经不止一次地起了作用。 -### 了解你的系统 +#### 了解你的系统 这个 bug 的第一个教训是要了解你的系统。我对于客户端和服务器之间的交互有不错的理解，但我并没有考虑到当服务器和所有这些客户端同时交互的时候会发生什么。这是一个我没有完全搞懂的层面。 -### 了解你的工具 +#### 了解你的工具 第二个教训是要了解你的工具。如果出了差错，你有哪些选项？你能撤销你做的迁移吗？你如何知道事情出了差错，你又如何发现更多信息？所有这些事情都应该在危机发生之前就了解好——但如果你没有，你会在危机发生时学到它们并不会再忘记。 -### 功能开关 & 服务器端功能控制 +#### 功能开关 & 服务器端功能控制 第三个教训是专门针对移动端和桌面应用开发者的：_你需要服务器端功能控制和功能开关_。当你发现一个问题时如果你没有服务器端的功能控制，你可能需要几天或几星期来推送新版本或者提交新版本到应用商店中，然后问题才能得到解决。这是个很糟糕的处境。Dropbox 桌面客户端不需要经过应用商店的审查过程，但光是把一个版本推送给上千万的用户就已经要花很多时间。相比之下，如果你能在新功能遇到问题的时候在服务器上翻转一个开关：十分钟之后你的问题就已经解决了。 @@ -237,7 +231,7 @@ $ : 0100100 但是它的好处——啊，当你需要它的时候，你真的是很需要它。 -# 如何去爱 bug +### 如何去爱 bug 我讲了几个我爱的 bug，也讲了为什么要爱 bug。现在我想告诉你如何去爱 bug。如果你现在还不爱 bug，我知道唯一一种改变的方法，那就是要有成长型心态。 @@ -261,7 +255,7 @@ Dweck 发现一个人看待智力的方式——固定型还是成长型心态 这些发现表明成长型心态对 debug 至关重要。我们必须从从困惑中重整旗鼓，诚实地面对我们理解上的不足，并时不时地在寻找答案的路上努力奋斗——成长型心态会让这些都变得更简单而且不那么痛苦。 -### 热爱你的 bug +#### 热爱你的 bug 我在 Recurse Center 工作时会直白地欢迎挑战，我就是这样学会热爱我的 bug 的。有时参与者会坐到我身边说“唉，我觉得我遇到了个奇怪的 Python bug”，然后我会说“太棒了，我 _爱_ 奇怪的 Python bug！” 首先，这百分之百是真的，但更重要的是，我这样是在对参与者强调，找到让自己觉得困难的事情是一种成就，而他们做到了这一点，这是件好事。 @@ -274,22 +268,18 @@ Dweck 发现一个人看待智力的方式——固定型还是成长型心态 在此向给我的演讲提出反馈以及给我的演讲提供其他帮助的人士表示感谢： * Sasha Laundy - * Amy Hanlon - * Julia Evans - * Julian Cooper - * Raphael Passini Diniz 以及其他的 Python Brasil 组织团队成员 -------------------------------------------------------------------------------- via: http://akaptur.com/blog/2017/11/12/love-your-bugs/ -作者：[Allison Kaptur ][a] +作者：[Allison Kaptur][a] 译者：[yixunx](https://github.com/yixunx) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 diff --git a/translated/tech/20171114 Restore Corrupted USB Drive To Original State In Linux.md b/published/201801/20171114 Restore Corrupted USB Drive To Original State In Linux.md similarity index 72% rename from translated/tech/20171114 Restore Corrupted USB Drive To Original State In Linux.md rename to published/201801/20171114 Restore Corrupted USB Drive To Original State In Linux.md index 71aa6d05ec..fd6422e2b9 100644 --- a/translated/tech/20171114 Restore Corrupted USB Drive To Original State In Linux.md +++ b/published/201801/20171114 Restore Corrupted USB Drive To Original State In Linux.md @@ -1,8 +1,6 @@ 在 Linux 上恢复一个损坏的 USB 设备至初始状态 ====== - - ![](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/restore-corrupted-usb-drive-to-original-state-in-linux_orig.jpg) 很多时候我们诸如 SD 卡和 U 盘这样的储存器可能会被损坏，并且因此或其他原因不能继续使用。 @@ -13,56 +11,52 @@ [![Linux 系统磁盘管理器](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/published/usb.png?1510665746)][1] -警告：接下来的操作会将你设备上的所有数据格式化 +**警告：接下来的操作会将你设备上的所有数据格式化。** +无论是上面提及的什么原因，最终的结果是我们无法继续使用这个设备。 - -无论什么原因，最终的结果是我们无法继续使用这个设备。 - -所以这里是一个恢复一个 USB 设备或者是 SD 卡到出厂状态的方法。 +所以这里有一个恢复 USB 设备或者是 SD 卡到出厂状态的方法。 大多数时候通过文件浏览器进行一次简单格式化可以解决问题，但是在一些极端情况下，比如文件管理器没有作用，而你又需要你的设备可以继续工作时，你可以使用下面的指导： -我们将会使用一个叫做 mkusb 的小工具来实现目标，这个工具的安装非常简单。 +我们将会使用一个叫做 `mkusb` 的小工具来实现目标，这个工具的安装非常简单。 +添加 mkusb 的仓库： +``` +sudo apt add repository ppa:mkusb/ppa +``` +现在更新你的包列表： +``` +sudo apt-get update +``` -1. 添加 mkusb 的仓库 +安装 `mkusb： -`sudo apt add repository ppa:mkusb/ppa` +``` +sudo apt-get install mkusb +``` -2. 现在更新你的包列表 - -`sudo apt-get update` - -3. 安装 mkusb - -`sudo apt-get install mkusb` - -现在运行 mkusb 你将会看到这个提示，点击 ‘Yes’。 +现在运行 `mkusb` 你将会看到这个提示，点击 ‘Yes’。 [![运行 mkusb dus](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/editor/run-mkusb.png?1510498592)][2] -现在 mkusb 将会最后一次询问你是否希望继续格式化你的数据，‘Stop’是被默认选择的，你现在选择 ‘Go’并点击‘OK’。 +现在 `mkusb` 将会最后一次询问你是否希望继续格式化你的数据，‘Stop’是被默认选择的，你现在选择 ‘Go’ 并点击 ‘OK’。 [![Linux mkusb](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/editor/final-checkpoint_1.png?1510499627)][3] -窗口将会关闭，摒弃人此时你的终端看起来是这样的。 +窗口将会关闭，此时你的终端看起来是这样的。 [![mkusb usb 控制台](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/editor/mkusb.png?1510499982)][4] 在几秒钟之后，整个过程将会完成，并且你将看到一个这样的弹出窗口。 - - [![恢复损坏的 USB 设备](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/editor/usb_1.png?1510500412)][5] 你现在需要把你的设备从系统推出，然后再重新插进去。你的设备将被恢复成为一个普通设备而且还能像原来一样的工作。 - - [![Linux 磁盘管理器](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/editor/usb_2.png?1510500457)][6] 我们现在所做的操作本可以通过终端命令或是 gparted 或者其他的软件来完成，但是那将会需要一些关于分区管理的知识。 @@ -71,23 +65,19 @@ ### 结论 -**mkusb** - -是一个很容易使用的程序，它可以修复你的 USB 储存设备和 SD 卡。mkusb通过 mkusb 的 PPA 来下载。所有在 mkusb 上的操作都需要超级管理员的权限，并且你在这个设备上的所有数据将会被格式化。 +`mkusb` 是一个很容易使用的程序，它可以修复你的 USB 储存设备和 SD 卡。`mkusb` 通过 mkusb 的 PPA 来下载。所有在 `mkusb` 上的操作都需要超级管理员的权限，并且你在这个设备上的所有数据将会被格式化。 一旦操作完成，你将会重置这个设备并让它继续工作。 如果你感到任何疑惑，你可以在下面的评论栏里免费发表。 - - -------------------------------------------------------------------------------- via: http://www.linuxandubuntu.com/home/restore-corrupted-usb-drive-to-original-state-in-linux 作者：[LINUXANDUBUNTU][a] 译者：[Drshu](https://github.com/Drshu) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 diff --git a/published/20171115 How to Fix the ‘No Space Left on Device- Error on Linux.md b/published/201801/20171115 How to Fix the ‘No Space Left on Device- Error on Linux.md similarity index 100% rename from published/20171115 How to Fix the ‘No Space Left on Device- Error on Linux.md rename to published/201801/20171115 How to Fix the ‘No Space Left on Device- Error on Linux.md diff --git a/published/20171115 Security Jobs Are Hot Get Trained and Get Noticed.md b/published/201801/20171115 Security Jobs Are Hot Get Trained and Get Noticed.md similarity index 100% rename from published/20171115 Security Jobs Are Hot Get Trained and Get Noticed.md rename to published/201801/20171115 Security Jobs Are Hot Get Trained and Get Noticed.md diff --git a/published/20171119 10 Best LaTeX Editors For Linux.md b/published/201801/20171119 10 Best LaTeX Editors For Linux.md similarity index 100% rename from published/20171119 10 Best LaTeX Editors For Linux.md rename to published/201801/20171119 10 Best LaTeX Editors For Linux.md diff --git a/published/20171120 Adopting Kubernetes step by step.md b/published/201801/20171120 Adopting Kubernetes step by step.md similarity index 100% rename from published/20171120 Adopting Kubernetes step by step.md rename to published/201801/20171120 Adopting Kubernetes step by step.md diff --git a/published/20171120 Save Files Directly To Google Drive And Download 10 Times Faster.md b/published/201801/20171120 Save Files Directly To Google Drive And Download 10 Times Faster.md similarity index 100% rename from published/20171120 Save Files Directly To Google Drive And Download 10 Times Faster.md rename to published/201801/20171120 Save Files Directly To Google Drive And Download 10 Times Faster.md diff --git a/translated/tech/20171123 Check Cryptocurrency Prices From Commandline.md b/published/201801/20171123 Check Cryptocurrency Prices From Commandline.md similarity index 65% rename from translated/tech/20171123 Check Cryptocurrency Prices From Commandline.md rename to published/201801/20171123 Check Cryptocurrency Prices From Commandline.md index ba4a7bdefe..ec340bb0a5 100644 --- a/translated/tech/20171123 Check Cryptocurrency Prices From Commandline.md +++ b/published/201801/20171123 Check Cryptocurrency Prices From Commandline.md @@ -1,13 +1,16 @@ -从命令行查看加密货币价格 +用命令行查看比特币等加密货币的价格 ====== + ![配图](https://www.ostechnix.com/wp-content/uploads/2017/11/bitcoin-1-720x340.jpg) -前段时间，我们发布了一个关于 **[Cli-Fyi][1] ** 的指南 - 一个潜在有用的命令行查询工具。使用 Cli-Fyi，我们可以很容易地了解加密货币的最新价格和许多其他有用的细节。今天，我们将看到另一个名为 **“Coinmon”** 的加密货币价格查看工具。不像 Cli.Fyi，Coinmon 只能用来查看不同加密货币的价格。没有其他功能！Coinmon 会检查加密货币的价格，并立即直接从你的终端修改价格。它将从 [coinmarketcap.com][2] API 获取所有详细信息。对于那些 **加密货币投资者**和**工程师**来说是非常有用的。 + +前段时间，我们发布了一个关于 [Cli-Fyi][1] 的指南 - 一个可能有用的命令行查询工具。使用 Cli-Fyi，我们可以很容易地了解加密货币的最新价格和许多其他有用的细节。今天，我们将看到另一个名为 “Coinmon” 的加密货币价格查看工具。不像 Cli.Fyi，Coinmon 只能用来查看不同加密货币的价格。没有其他功能！Coinmon 会在终端上检查加密货币的价格。它将从 [coinmarketcap.com][2] API 获取所有详细信息。对于那些 **加密货币投资者**和**工程师**来说是非常有用的。 ### 安装 Coinmon 确保你的系统上安装了 Node.js 和 Npm。如果你的机器上没有安装 Node.js 和/或 npm，请参考以下链接进行安装。 安装完 Node.js 和 Npm 后，从终端运行以下命令安装 Coinmon。 + ``` sudo npm install -g coinmon ``` @@ -15,38 +18,42 @@ sudo npm install -g coinmon ### 从命令行查看加密货币价格 运行以下命令查看市值排名的前 10 位的加密货币： + ``` coinmon ``` 示例输出： -[![][3]][4] +![][4] + +如我所说，如果你不带任何参数运行 Coinmon，它将显示前 10 位加密货币。你还可以使用 `-t` 标志查看最高的 n 位加密货币，例如 20。 -如我所说，如果你不带任何参数运行 coinmon，它将显示前 10 位加密货币。你还可以使用 “-t” 标志查看最高的 n 位加密货币，例如 20。 ``` coinmon -t 20 ``` -所有价格默认以美元显示。你还可以使用 “-c” 标志将价格从美元转换为另一种货币。 +所有价格默认以美元显示。你还可以使用 `-c` 标志将价格从美元转换为另一种货币。 例如，要将价格转换为 INR（印度卢比），运行： + ``` coinmon -c inr ``` -[![][3]][5] +![][5] 目前，Coinmon 支持 AUD、BRL、CAD、CHF、CLP、CNY、CZK、DKK、EUR、GBP、HKD、HUF、IDR、ILS、INR、JPY、KRW、MXN、MYR、NOK、NZD、PHP、PKR、PLN、RUB、SEK、SGD、THB、TRY、TWD、ZAR 这些货币。 也可以使用加密货币的符号来搜索价格。 + ``` coinmon -f btc ``` -这里，**btc** 是比特币的符号。你可以在[**这**][6]查看所有可用的加密货币的符号。 +这里，`btc` 是比特币的符号。你可以在[**这里**][6]查看所有可用的加密货币的符号。 -有关更多详情，请参阅coinmon的帮助部分： +有关更多详情，请参阅 coinmon 的帮助部分： ``` $ coinmon -h @@ -67,15 +74,13 @@ Options: 干杯! - - -------------------------------------------------------------------------------- via: https://www.ostechnix.com/coinmon-check-cryptocurrency-prices-commandline/ 作者：[SK][a] 译者：[geekpi](https://github.com/geekpi) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 @@ -83,6 +88,6 @@ via: https://www.ostechnix.com/coinmon-check-cryptocurrency-prices-commandline/ [1]:https://www.ostechnix.com/cli-fyi-quick-easy-way-fetch-information-ips-emails-domains-lots/ [2]:https://coinmarketcap.com/ [3]:data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7 -[4]:http://www.ostechnix.com/wp-content/uploads/2017/11/coinmon-1.png () -[5]:http://www.ostechnix.com/wp-content/uploads/2017/11/coinmon-2.png () +[4]:http://www.ostechnix.com/wp-content/uploads/2017/11/coinmon-1.png +[5]:http://www.ostechnix.com/wp-content/uploads/2017/11/coinmon-2.png [6]:https://en.wikipedia.org/wiki/List_of_cryptocurrencies diff --git a/published/20171127 Migrating to Linux Disks Files and Filesystems.md b/published/201801/20171127 Migrating to Linux Disks Files and Filesystems.md similarity index 100% rename from published/20171127 Migrating to Linux Disks Files and Filesystems.md rename to published/201801/20171127 Migrating to Linux Disks Files and Filesystems.md diff --git a/published/20171128 A generic introduction to Gitlab CI.md b/published/201801/20171128 A generic introduction to Gitlab CI.md similarity index 100% rename from published/20171128 A generic introduction to Gitlab CI.md rename to published/201801/20171128 A generic introduction to Gitlab CI.md diff --git a/published/201801/20171201 Launching an Open Source Project A Free Guide.md b/published/201801/20171201 Launching an Open Source Project A Free Guide.md new file mode 100644 index 0000000000..69b648b5de --- /dev/null +++ b/published/201801/20171201 Launching an Open Source Project A Free Guide.md @@ -0,0 +1,76 @@ +启动开源项目：免费指导！ +============================================================ + +![](https://www.linuxfoundation.org/wp-content/uploads/2017/11/project-launch-1024x645.jpg) + +启动项目、组建社区支持可能会比较复杂，但是这个全新的指南可以帮助你开启开源项目。 + +各种规模的组织、技术人员和 DecOps 工作者选择使用开源项目，甚至去开发自己的开源项目，开源程序变得越来越普遍。从 Google 到 Netflix 再到 Facebook ，这些公司都在将它们的开源创作发布到开源社区。经常见到起于内部的开源项目，然后受惠于外部开发人员的合作开发。 + +然而，开启一个开源项目、组建社区支持比你想的还要复杂。一些前期准备可以帮助事情开展变得顺利，这就是这个全新的“[启动开源项目指南][1]”所能为你做的。 + +这个免费指南是为了帮助那些深谙开源的组织启动自己的开源项目而诞生。在本文的开始，介绍包括了决定开源什么项目，预计项目费用，考虑开源协议和一些其他方面。开源项目这种方式可能源自国外，但是从 Google 到 Facebook 这样一些主要的开源公司都已经开放提供了开源项目指导资源。事实上，Google 拥有的[丰富的在线资源][2]对开源项目实践和启动开源项目上做出了贡献。 + +Capital One 开源社区经理 Jared Smith 指出，“无论公司在内雇佣了多少聪明人，总还是有聪明人在公司之外”， “我们发现开放我们的源代码给外面世界的专业且愿意分享的人士交流经验是非常值得的，我们能从中获取一些非常好的建议”。 + +在这个新指南中，开源专家 Ibrahim Haddad 提供了五条关于一个组织为什么要开源新项目的原因： + +1. 促成开放式问题解决方案；为标准提供参照实现；分担战略功能的开发开销。 +2. 商品化市场；减少非战略的软件成本费用。 +3. 通过建立产品生态来驱动需求。 +4. 协同合作；吸引客户；深化共同目标间的关系。 +5. 提供用户自我支持的能力：无需等待即可调整代码 + +本文指出：“做出发布或创建一个新的开源项目的决定和自身境况相关。你的公司应该在使用或贡献既有的开源项目上拥有一定程度的熟练度。这是因为消费能够指导你，通过外部工程使开发者对自己的产品构建变得省力（参见我们的指南[使用开源代码][3]和[加入开源社区 ][4] 上）。但是当一旦你顺利的参与过开源，那这将是启动你自己的开源项目的最佳时机。” + +该指南还指出, 规划可以使您和您的组织摆脱法律麻烦。如果您希望您的项目蓬勃发展, 则与许可、分发、支持选项甚至品牌相关的问题都需要提前考虑。 + +“我认为, 对于一家公司来说, 至关重要的是要考虑他们希望通过新的开源项目实现的目标，” Linux 基金会的项目管理主任 John Mertic 说。"他们必须考虑它对社区和开发者的价值，以及他们希望从中得到什么结果。然后, 他们必须了解所有的部分，以正确的方式去完成，包括法律、治理、基础设施和一个启动社区。当你把一个开源项目放在那里时，我总是最强调这些。” + +这个“启动开源项目指南”可以帮助您了解从许可证问题到最佳开发实践的所有内容，并探讨如何无缝地将现有的开放组件编织到您的开源项目中。它是来自 Linux 基金会和 TODO 组的免费指南的新集合之一，对于任何运作开源程序的组织来说都非常有价值。现在可以使用该指南来帮助您运行开源计划办公室，以支持、分享和利用开源。有了这样一个办公室， 组织就可以有效地建立并执行其开放源码战略，并有明确的条款。 + +这些免费的教程是基于开源领导人的经验而来。[在这里可以查看所有指南][7]，然后关注我们的后续文章。 + +也别错过了本系列早些的文章： + +- [ 如何创建开源计划 ][8] +- [ 开源计划管理工具 ][9] +- [ 衡量你的开源项目成功性 ][10] +- [ 吸引开源开发者的高效策略 ][11] +- [ 加入开源社区 ][12] +- [ 使用开源代码 ][13] + +-------------------------------------------------------------------------------- + +via: https://www.linuxfoundation.org/blog/launching-open-source-project-free-guide/ + +作者：[Sam Dean][a] +译者：[CYLeft](https://github.com/CYLeft) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.linuxfoundation.org/author/sdean/ +[1]:https://www.linuxfoundation.org/resources/open-source-guides/starting-open-source-project/ +[2]:https://www.linux.com/blog/learn/chapter/open-source-management/2017/5/googles-new-home-all-things-open-source-runs-deep +[3]:https://www.linuxfoundation.org/using-open-source-code/ +[4]:https://www.linuxfoundation.org/participating-open-source-communities/ +[5]:https://www.linuxfoundation.org/resources/open-source-guides/starting-open-source-project/ +[6]:https://github.com/todogroup/guides +[7]:https://github.com/todogroup/guides +[8]:https://github.com/todogroup/guides/blob/master/creating-an-open-source-program.md +[9]:https://www.linuxfoundation.org/blog/managing-open-source-programs-free-guide/ +[10]:https://www.linuxfoundation.org/measuring-your-open-source-program-success/ +[11]:https://www.linuxfoundation.org/blog/effective-strategies-recruiting-open-source-developers/ +[12]:https://www.linuxfoundation.org/participating-open-source-communities/ +[13]:https://www.linuxfoundation.org/using-open-source-code/ +[14]:https://www.linuxfoundation.org/author/sdean/ +[15]:https://www.linuxfoundation.org/category/audience/attorneys/ +[16]:https://www.linuxfoundation.org/category/blog/ +[17]:https://www.linuxfoundation.org/category/audience/c-level/ +[18]:https://www.linuxfoundation.org/category/audience/developer-influencers/ +[19]:https://www.linuxfoundation.org/category/audience/entrepreneurs/ +[20]:https://www.linuxfoundation.org/category/content-placement/lf-brand/ +[21]:https://www.linuxfoundation.org/category/audience/open-source-developers/ +[22]:https://www.linuxfoundation.org/category/audience/open-source-professionals/ +[23]:https://www.linuxfoundation.org/category/audience/open-source-users/ diff --git a/published/20171205 How to Use the Date Command in Linux.md b/published/201801/20171205 How to Use the Date Command in Linux.md similarity index 100% rename from published/20171205 How to Use the Date Command in Linux.md rename to published/201801/20171205 How to Use the Date Command in Linux.md diff --git a/published/20171205 Using sudo to delegate permissions in Linux.md b/published/201801/20171205 Using sudo to delegate permissions in Linux.md similarity index 100% rename from published/20171205 Using sudo to delegate permissions in Linux.md rename to published/201801/20171205 Using sudo to delegate permissions in Linux.md diff --git a/published/20171207 Cheat – A Collection Of Practical Linux Command Examples.md b/published/201801/20171207 Cheat – A Collection Of Practical Linux Command Examples.md similarity index 100% rename from published/20171207 Cheat – A Collection Of Practical Linux Command Examples.md rename to published/201801/20171207 Cheat – A Collection Of Practical Linux Command Examples.md diff --git a/published/201801/20171207 How To Find Files Based On their Permissions.md b/published/201801/20171207 How To Find Files Based On their Permissions.md new file mode 100644 index 0000000000..4e8021bf35 --- /dev/null +++ b/published/201801/20171207 How To Find Files Based On their Permissions.md @@ -0,0 +1,188 @@ +如何根据文件权限查找文件 +====== + +![](https://www.ostechnix.com/wp-content/uploads/2017/12/Find-Files-Based-On-their-Permissions-720x340.png) + +在 Linux 中查找文件并不是什么大问题。市面上也有很多可靠的自由开源的可视化查找工具。但对我而言，查找文件，用命令行的方式会更快更简单。我们已经知道 [如何根据访问和修改文件的时间寻找或整理文件][1]。今天，在基于 Unix 的操作系统中，我们将见识如何通过权限查找文件。 + +本段教程中，我将创建三个文件名为 `file1`，`file2` 和 `file3` 分别赋予 `777`，`766` 和 `655` 文件权限，并分别置于名为 `ostechnix` 的文件夹中。 + +``` +mkdir ostechnix && cd ostechnix/ +install -b -m 777 /dev/null file1 +install -b -m 766 /dev/null file2 +install -b -m 655 /dev/null file3 +``` + +![][3] + +现在，让我们通过权限来查找一下文件。 + +### 根据权限查找文件 + +根据权限查找文件最具代表性的语法： + +``` +find -perm mode +``` + +mode 可以是代表权限的八进制数字（777、666 …）也可以是权限符号（u=x，a=r+x）。 + +在深入之前，我们就以下三点详细说明 mode 参数。 + +1. 如果我们不指定任何参数前缀，它将会寻找**具体**权限的文件。 +2. 如果我们使用 `-` 参数前缀， 寻找到的文件至少拥有 mode 所述的权限，而不是具体的权限（大于或等于此权限的文件都会被查找出来）。 +3. 如果我们使用 `/` 参数前缀，那么所有者、组或者其他人任意一个应当享有此文件的权限。 + +为了让你更好的理解，让我举些例子。 + +首先，我们将要看到基于数字权限查找文件。 + +### 基于数字（八进制）权限查找文件 + +让我们运行下列命令： + +``` +find -perm 777 +``` + +这条命令将会查找到当前目录权限为**确切为 777** 权限的文件。 + +![1][4] + +如你看见的屏幕输出，file1 是唯一一个拥有**确切为 777 权限**的文件。 + +现在，让我们使用 `-` 参数前缀，看看会发生什么。 + +``` +find -perm -766 +``` + +![][5] + +如你所见，命令行上显示两个文件。我们给 file2 设置了 766 权限，但是命令行显示两个文件，什么鬼？因为，我们设置了 `-` 参数前缀。它意味着这条命令将在所有文件中查找文件所有者的“读/写/执行”权限，文件用户组的“读/写”权限和其他用户的“读/写”权限。本例中，file1 和 file2 都符合要求。换句话说，文件并不一样要求时确切的 766 权限。它将会显示任何属于（高于）此权限的文件 。 + +然后，让我们使用 `/` 参数前置，看看会发生什么。 + +``` +find -perm /222 +``` + +![][6] + +上述命令将会查找某些人（要么是所有者、用户组，要么是其他人）拥有写权限的文件。这里有另外一个例子： + +``` +find -perm /220 +``` + +这条命令会查找所有者或用户组中拥有写权限的文件。这意味着匹配所有者和用户组任一可写的文件，而其他人的权限随意。 + +如果你使用 `-` 前缀运行相同的命令，你只会看到所有者和用户组都拥有写权限的文件。 + +``` +find -perm -220 +``` + +下面的截图会告诉你这两个参数前缀的不同。 + +![][7] + +如我之前说过的一样，我们也可以使用符号表示文件权限。 + +请阅读： + +- [如何在 Linux 中找到最大和最小的目录和文件][10] +- [如何在 Linux 的目录树中找到最老的文件][11] +- [如何在 Linux 中找到超过或小于某个大小的文件][12] + +### 基于符号的文件权限查找文件 + +在下面的例子中，我们使用例如 `u`（所有者）、`g`（用户组） 和 `o`（其他） 的符号表示法。我们也可以使用字母 `a` 代表上述三种类型。我们可以通过特指的 `r` （读）、 `w` （写）、 `x` （执行）分别代表它们的权限。 + +例如，寻找用户组中拥有 `写` 权限的文件，执行： + +``` +find -perm -g=w +``` + +![][8] + +上面的例子中，file1 和 file2 都拥有 `写` 权限。请注意，你可以等效使用 `=` 或 `+` 两种符号标识。例如，下列两行相同效果的代码。 + +``` +find -perm -g=w +find -perm -g+w +``` + +查找文件所有者中拥有写权限的文件，执行： + +``` +find -perm -u=w +``` + +查找所有用户中拥有写权限的文件，执行： + +``` +find -perm -a=w +``` + +查找所有者和用户组中同时拥有写权限的文件，执行： + +``` +find -perm -g+w,u+w +``` + +上述命令等效与 `find -perm -220`。 + +查找所有者或用户组中拥有写权限的文件，执行： + +``` +find -perm /u+w,g+w +``` + +或者, + +``` +find -perm /u=w,g=w +``` + +上述命令等效于 `find -perm /220`。 + +更多详情，参照 man 手册。 + +``` +man find +``` + +了解更多简化案例或其他 Linux 命令，查看[man 手册][9]。 + +然后，这就是所有的内容。希望这个教程有用。更多干货，敬请关注。 + +干杯！ + + +-------------------------------------------------------------------------------- + +via: https://www.ostechnix.com/find-files-based-permissions/ + +作者：[SK][a] +译者：[CYLeft](https://github.com/CYLeft) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.ostechnix.com/author/sk/ +[1]:https://www.ostechnix.com/find-sort-files-based-access-modification-date-time-linux/ +[2]:data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7/ +[3]:https://www.ostechnix.com/wp-content/uploads/2017/12/find-files-1-1.png +[4]:https://www.ostechnix.com/wp-content/uploads/2017/12/find-files-2.png +[5]:https://www.ostechnix.com/wp-content/uploads/2017/12/find-files-3.png + +[6]:https://www.ostechnix.com/wp-content/uploads/2017/12/find-files-6.png +[7]:https://www.ostechnix.com/wp-content/uploads/2017/12/find-files-7.png +[8]:https://www.ostechnix.com/wp-content/uploads/2017/12/find-files-8.png +[9]:https://www.ostechnix.com/3-good-alternatives-man-pages-every-linux-user-know/ +[10]:https://www.ostechnix.com/how-to-find-largest-and-smallest-directories-and-files-in-linux/ +[11]:https://www.ostechnix.com/find-oldest-file-directory-tree-linux/ +[12]:https://www.ostechnix.com/find-files-bigger-smaller-x-size-linux/ \ No newline at end of file diff --git a/published/20171212 How to Search PDF Files from the Terminal with pdfgrep.md b/published/201801/20171212 How to Search PDF Files from the Terminal with pdfgrep.md similarity index 100% rename from published/20171212 How to Search PDF Files from the Terminal with pdfgrep.md rename to published/201801/20171212 How to Search PDF Files from the Terminal with pdfgrep.md diff --git a/published/20171213 How To Allow-Permit User To Access A Specific File or Folder In Linux Using ACL.md b/published/201801/20171213 How To Allow-Permit User To Access A Specific File or Folder In Linux Using ACL.md similarity index 100% rename from published/20171213 How To Allow-Permit User To Access A Specific File or Folder In Linux Using ACL.md rename to published/201801/20171213 How To Allow-Permit User To Access A Specific File or Folder In Linux Using ACL.md diff --git a/published/20171214 A step-by-step guide to building open culture.md b/published/201801/20171214 A step-by-step guide to building open culture.md similarity index 100% rename from published/20171214 A step-by-step guide to building open culture.md rename to published/201801/20171214 A step-by-step guide to building open culture.md diff --git a/published/20171214 How to Install Moodle on Ubuntu 16.04.md b/published/201801/20171214 How to Install Moodle on Ubuntu 16.04.md similarity index 100% rename from published/20171214 How to Install Moodle on Ubuntu 16.04.md rename to published/201801/20171214 How to Install Moodle on Ubuntu 16.04.md diff --git a/published/20171214 How to squeeze the most out of Linux file compression.md b/published/201801/20171214 How to squeeze the most out of Linux file compression.md similarity index 100% rename from published/20171214 How to squeeze the most out of Linux file compression.md rename to published/201801/20171214 How to squeeze the most out of Linux file compression.md diff --git a/published/20171214 The Most Famous Classic Text-based Adventure Game.md b/published/201801/20171214 The Most Famous Classic Text-based Adventure Game.md similarity index 100% rename from published/20171214 The Most Famous Classic Text-based Adventure Game.md rename to published/201801/20171214 The Most Famous Classic Text-based Adventure Game.md diff --git a/published/20171215 5 of the Best Bitcoin Clients for Linux.md b/published/201801/20171215 5 of the Best Bitcoin Clients for Linux.md similarity index 100% rename from published/20171215 5 of the Best Bitcoin Clients for Linux.md rename to published/201801/20171215 5 of the Best Bitcoin Clients for Linux.md diff --git a/published/20171215 How to find and tar files into a tar ball.md b/published/201801/20171215 How to find and tar files into a tar ball.md similarity index 100% rename from published/20171215 How to find and tar files into a tar ball.md rename to published/201801/20171215 How to find and tar files into a tar ball.md diff --git a/published/20171216 Saving window position in Xfce session.md b/published/201801/20171216 Saving window position in Xfce session.md similarity index 100% rename from published/20171216 Saving window position in Xfce session.md rename to published/201801/20171216 Saving window position in Xfce session.md diff --git a/published/20171218 How to Create a .Desktop File For Your Application in Linux.md b/published/201801/20171218 How to Create a .Desktop File For Your Application in Linux.md similarity index 100% rename from published/20171218 How to Create a .Desktop File For Your Application in Linux.md rename to published/201801/20171218 How to Create a .Desktop File For Your Application in Linux.md diff --git a/published/201801/20171219 Migrating to Linux- Graphical Environments.md b/published/201801/20171219 Migrating to Linux- Graphical Environments.md new file mode 100644 index 0000000000..2a8991b8bf --- /dev/null +++ b/published/201801/20171219 Migrating to Linux- Graphical Environments.md @@ -0,0 +1,118 @@ +迁移到 Linux：图形操作环境 +====== + +![](https://www.linux.com/sites/lcom/files/styles/rendered_file/public/linux-migration_0.jpg?itok=0tviWTqd) + +> 这篇文章对 Linux 图形环境做了一番介绍，展示了在不同的 Linux 发行版上的各种选择。 + +这是我们迁移到 Linux 系统系列的第三篇文章。如果你错过了先前的两篇，这里有两文的链接《[入门介绍][1]》 和 《[磁盘、文件、和文件系统][2]》。本文中，我们将讨论图形操作环境。在 Linux 系统中，你可以依照喜好选择并且定制一个图形界面，你有很大的选择余地，这也是 Linux 优越的体验之一。 + +一些主流的 Linux 图形界面包括：Cinnamon、Gnome、KDE Plasma、Xfce 和 MATE，总之这里有很多选择。 + +有一点经常混淆 Linux 新手，虽然某个 Linux 系统分配了一个缺省的图形环境，但是一般你可以随时更换这个图形界面。这和 Windows 或 Mac OS 的惯用者的定势思维不同。安装图形环境是一件独立的事情，很多时候，Linux 和其图形环境的连接并不紧密。此外，你在一个图形环境构建运行的程序同样适用于另一个图形环境。比如说一个为 KDE Plasma 图形环境编写的应用程序完全适用于 Gnome 桌面图形环境。 + +由于人们熟悉 Windows 和 MacOS 系统，部分 Linux 操作系统的图形环境在一定程度上尝试着去模仿它们，但另一些 Linux 图形界面则是独特的。 + +下面，我将就一些不同的 Linux 发行版，展示几个 Linux 图形环境。如果你不确定应该采用那个 Linux 发行版，那我建议你从 [Ubuntu][3] 开始，获取其长期支持（LTS）的版本（Ubuntu 16.04.3 LTS 正在开发）。Ubuntu 稳定且真的非常好用。 + +### 由 Mac 过渡 + +Elementary OS 发行版提供了和 Mac 系统风格很接近的界面。它的默认图形环境被称作 Pantheon，是一款很适合 Mac 用户过渡使用的图形环境。这款图形界面发行版的屏幕底部有一个停靠栏，专为极简者使用而设计。为了保持简约的风格，很多默认的应用程序甚至都不会有自己的菜单。相反，它们的按钮和控件在应用程序的标题栏上（图 1）。 + +![Elementary OS][5] + +*图 1： Elementary OS Pantheon.* + +Ubuntu 发行版提供的一个默认的图形界面，也和 Mac 相似。Ubuntu 17.04 或者更老的版本都使用 Unity 图形环境，Unity 停靠栏的默认位置在屏幕的左边，屏幕顶部有一个全局应用程序共享的菜单栏。 + +### 由 Windows 过渡 + +ChaletOS 亦步亦趋模仿 Windows 界面，可以帮助 Windows 用户轻松的过渡到 Linux。ChaletOS 使用的图形环境是 Xfce（图 2）。在屏幕的左下角有一个开始（主）菜单和搜索栏。屏幕的右下角是一个桌面图标和一些通知信息。这看起来和 Windows 非常像，乍一看，可能都会以为桌面跑的是 Windows。 + +![ChaletOS][6] + +*图 2： ChaletOS with Xfce.* + +Zorin OS 发行版同样尝试模仿 Windows。 Zorin OS 使用的 Gnome 的改进桌面，工作起来和 Windows 的图形界面很相似。左下角的开始按钮、右下角的通知栏和信息通知栏。开始按钮会弹出一个和 Windows 无异的应用程序列表和搜索框。 + +### 独有的图形环境 + +Gnome 桌面（图 3）是最常用的图形环境之一。许多发行版将 Gnome 作为默认的图形环境。Gnome 并不打算模仿 Windows 或是 MacOS，而是以其自身的优雅和易用为目标。 + +![][7] + +*图 3：openSUSE with Gnome.* + +Gnome 桌面环境从版本 2 到 版本 3 发生了巨变，Cinnamon 环境为消除该改变带来的不利影响而创造。尽管 Cinnamon 和前辈 Gnome 2 外观不相似，但是它依旧尝试提供一个简约的界面，而且它的功能和 Windows XP 类似。 + +MATE 图形环境直接模仿于 Gnome 2，在它的屏幕顶部有一个用作设置和存放应用的菜单栏，底部提供了一个应用程序运行选项卡和一些其他组件。 + +KDE plasma 围绕组件界面而构建，组件可以安装在桌面或是面板上（图 4）。 + +![KDE Plasma][8] + +*图 4: 安装了 KDE Plasma 的 Kubuntu 操作系统。* + +没有那个图形环境比另外一个更好。不同的风格适用不同的用户风格。另外，如果选择太多无从下手，那就从 [Ubuntu][3] 开始吧。 + +### 相似与不同 + +不同的操作系统处理方式不同，这会给使用者的过渡带来挑战。比如说，菜单栏可能出现在不同的位置，然后设置有可能用不同的选项入口路径。我列举了一些相似或不同地方来帮助减少 Linux 调整。 + +#### 鼠标 + +Linux 的鼠标通常和 Windows 与 MacOS 的工作方式有所差异。在 Windows 或 Mac，双击标签，你几乎可以打开任何事物，而这在 Linux 图形界面中很多都被设置为单击。 + +此外在 Windows 系统中，你通常通过单击窗口获取焦点。在 Linux，很多窗口的焦点获取方式被设置为鼠标悬停，即便鼠标悬停下的窗口并不在屏幕顶端。这种微妙的差异有时候会让人很吃惊。比如说，在 Windows 中，假如有一个后台应用（不在屏幕顶层），你移动鼠标到它的上面，不点击鼠标仅仅转动鼠标滚轮，顶层窗口会滚动。而在 Linux 中，后台窗口（鼠标悬停的那个窗口）会滚动。 + +#### 菜单 + +应用菜单是电脑程序的一个主要集中位置，最近似乎可以调整移动菜单栏到不碍事的地方，甚至干脆完全删除。大概，当你迁移到 Linux，你可能找不到你期待的菜单。应用程序菜单会像 MacOS一样出现在全局共享菜单栏内。和很多移动应用程序一样，该菜单可能在“更多选项”的图标里。或者，这个菜单干脆被完全移除被一个按钮取代，正如在 Elementary OS Pantheon 环境里的一些程序一样。 + +#### 工作空间 + +很多 Linux 图形环境提供了多个工作空间。一个工作空间包含的正在运行的程序窗口充盈了整个屏幕。切换到不同的工作空间将会改变程序的可见性。这个概念是把当前项目运行使用的全部应用程序分组到一个工作空间，而一些为另一个项目使用的应用程序会被分组到不同的工作空间。 + +不是每一个人都需要甚至是喜欢工作空间，但是我提到它是因为，作为一个新手，你可能无意间通过一个组合键切换了工作空间，然后，“喂！我的应用程序哪去了?” 如果你看到的还是你熟悉的桌面壁纸，那你可能只是切换了工作空间，你所有的应用程序还在一个工作空间运行，只是现在不可见而已。在很多 Linux 环境中，通过敲击 `Alt+Ctrl` 和一个箭头（上、下、左或右）可以切换工作空间。很有可能发现你的应用程序一直都在另一个工作空间里运行。 + +当然，如果你刚好喜欢工作空间（很多人都喜欢），然后你就发现了一个 Linux 很有用的默认功能。 + +#### 设置 + +许多 Linux 图形环境提供一些类型的设置程序或是面板让你在机器上配置设置。值得注意的是类似 Windows 和 MacOS，Linux 可以配置好很多细节，但不是所有的详细设置都可以在设置程序上找到。但是这些设置项已经足够满足大部分典型的桌面系统，比如选择桌面壁纸，改变熄屏时间，连接打印机等其他一些设置。 + +和 Windows 或者 MacOS 相比，Linux 的应用程序设置的分组或是命名都有不同的方式。甚至同是 Linux 系统，不同的图形界面也会出现不同的设置，这可能需要时间适应。当然，在你的图形环境中设置配置的问题可以通过在线查询这样不错的方法解决。 + +#### 应用程序 + +最后，Linux 的应用程序也可能不同。你可能会发现一些熟悉的应用程序，但是对你来说更多的将会是崭新的应用。比如说，你能在 Linux 上找到 Firefox、Chrome 和 Skype。如果不能找到特定的应用程序，通常你能使用一些替代程序。如果还是不能，那你可以使用 WINE 这样的兼容层来运行 Windows 的应用程序。 + +在很多 Linux 图形环境中，你可以通过敲击 Windows 的标志键来打开应用程序菜单栏。在其他一些情况中，你不得不点击开始（主）按钮或应用程序菜单。很多图形环境中，你可以通过分类搜索到应用程序而不一定需要它的特定程序名。比如说，你要使用一个你不清楚名字的编辑器程序，这时候，你可以在应用程序菜单栏键的搜索框中键入“editor”字样，它将为你展示一个甚至更多的被认为是编辑器类的应用程序。 + +为帮你起步，这里列举了一点可能成为 Linux 系统使用的替代程序。 + +![linux][10] + +请注意，Linux 提供了大量的满足你需求的选择，这张表里列举的一点也不完整。 + +-------------------------------------------------------------------------------- + +via: https://www.linux.com/blog/learn/2017/12/migrating-linux-graphical-environments + +作者：[John Bonesio][a] +译者：[CYLeft](https://github.com/CYLeft) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.linux.com/users/johnbonesio +[1]:https://linux.cn/article-9212-1.html +[2]:https://linux.cn/article-9213-1.html +[3]:https://www.evernote.com/OutboundRedirect.action?dest=https%3A%2F%2Fwww.ubuntu.com%2Fdownload%2Fdesktop +[5]:https://www.linux.com/sites/lcom/files/styles/rendered_file/public/elementaryos.png?itok=kJk2-BsL (Elementary OS) +[6]:https://www.linux.com/sites/lcom/files/styles/rendered_file/public/chaletos.png?itok=Zdm2rRgu +[7]:https://www.linux.com/sites/lcom/files/styles/rendered_file/public/opensuse.png?itok=TM0Q9AyH +[8]:https://www.linux.com/sites/lcom/files/styles/rendered_file/public/kubuntu.png?itok=a2E7ttaa (KDE Plasma) +[9]:https://training.linuxfoundation.org/linux-courses/system-administration-training/introduction-to-linux + +[10]: https://www.linux.com/sites/lcom/files/styles/rendered_file/public/linux-options.png?itok=lkqD1UMj diff --git a/published/20171219 Surf anonymously- Learn to install TOR network on Linux.md b/published/201801/20171219 Surf anonymously- Learn to install TOR network on Linux.md similarity index 100% rename from published/20171219 Surf anonymously- Learn to install TOR network on Linux.md rename to published/201801/20171219 Surf anonymously- Learn to install TOR network on Linux.md diff --git a/published/20171219 The Linux commands you should NEVER use.md b/published/201801/20171219 The Linux commands you should NEVER use.md similarity index 100% rename from published/20171219 The Linux commands you should NEVER use.md rename to published/201801/20171219 The Linux commands you should NEVER use.md diff --git a/published/20171220 The current state of kernel page-table isolation.md b/published/201801/20171220 The current state of kernel page-table isolation.md similarity index 100% rename from published/20171220 The current state of kernel page-table isolation.md rename to published/201801/20171220 The current state of kernel page-table isolation.md diff --git a/published/20171226 Check Linux filesystem for errors- FSCK command with examples.md b/published/201801/20171226 Check Linux filesystem for errors- FSCK command with examples.md similarity index 100% rename from published/20171226 Check Linux filesystem for errors- FSCK command with examples.md rename to published/201801/20171226 Check Linux filesystem for errors- FSCK command with examples.md diff --git a/published/20171226 How to Configure Linux for Children.md b/published/201801/20171226 How to Configure Linux for Children.md similarity index 100% rename from published/20171226 How to Configure Linux for Children.md rename to published/201801/20171226 How to Configure Linux for Children.md diff --git a/translated/tech/20171226 How to use-run bash aliases over ssh based session.md b/published/201801/20171226 How to use-run bash aliases over ssh based session.md similarity index 62% rename from translated/tech/20171226 How to use-run bash aliases over ssh based session.md rename to published/201801/20171226 How to use-run bash aliases over ssh based session.md index e93f9be95e..03c60be3d7 100644 --- a/translated/tech/20171226 How to use-run bash aliases over ssh based session.md +++ b/published/201801/20171226 How to use-run bash aliases over ssh based session.md @@ -1,7 +1,8 @@ 通过 ssh 会话执行 bash 别名 ====== -我在远程主机上[上设置过一个叫做 file_repl 的 bash 别名 ][1] . 当我使用 ssh 命令登陆远程主机后，可以很正常的使用这个别名。然而这个 bash 别名却无法通过 ssh 来运行，像这样： +我在远程主机上[上设置过一个叫做 file_repl 的 bash 别名 ][1]。当我使用 ssh 命令登录远程主机后，可以很正常的使用这个别名。然而这个 bash 别名却无法通过 ssh 来运行，像这样： + ``` $ ssh vivek@server1.cyberciti.biz file_repl bash：file_repl：command not found @@ -9,38 +10,48 @@ bash：file_repl：command not found 我要怎样做才能通过 ssh 命令运行 bash 别名呢？ -SSH 客户端 (ssh) 是一个登陆远程服务器并在远程系统上执行 shell 命令的 Linux/Unix 命令。它被设计用来在两个非信任的机器上通过不安全的网络(比如互联网)提供安全的加密通讯。 +SSH 客户端 (ssh) 是一个登录远程服务器并在远程系统上执行 shell 命令的 Linux/Unix 命令。它被设计用来在两个非信任的机器上通过不安全的网络（比如互联网）提供安全的加密通讯。 -## 如何用 ssh 客户端执行命令 +### 如何用 ssh 客户端执行命令 + +通过 ssh 运行 `free` 命令或 [date 命令][2] 可以这样做： + +``` +$ ssh vivek@server1.cyberciti.biz date +``` -通过 ssh 运行 free 命令或 [date 命令 ][2] 可以这样做： -`$ ssh vivek@server1.cyberciti.biz date` 结果为： + ``` Tue Dec 26 09：02：50 UTC 2017 ``` -或者 -`$ ssh vivek@server1.cyberciti.biz free -h` -结果为： +或者： + +``` +$ ssh vivek@server1.cyberciti.biz free -h +``` + +结果为： + ``` -  total used free shared buff/cache available Mem：2.0G 428M 138M 145M 1.4G 1.1G Swap：0B 0B 0B ``` -## 理解 bash shell 以及命令的类型 +### 理解 bash shell 以及命令的类型 [bash shell][4] 共有下面几类命令： - 1。别名，比如 ll - 2。关键字，比如 if - 3。函数(用户自定义函数，比如 genpasswd) - 4。内置命令，比如 pwd - 5。外部文件，比如 /bin/date +1. 别名，比如 `ll` +2. 关键字，比如 `if` +3. 函数 （用户自定义函数，比如 `genpasswd`） +4. 内置命令，比如 `pwd` +5. 外部文件，比如 `/bin/date` + +[type 命令][5] 和 [command 命令][6] 可以用来查看命令类型： -The [type 命令 ][5] 和 [command 命令 ][6] 可以用来查看命令类型： ``` $ type -a date date is /bin/date @@ -51,33 +62,38 @@ pwd is a shell builtin $ type -a file_repl is aliased to `sudo -i /shared/takes/master.replication' ``` -date 和 free 都是外部命令而 file_repl 是 `sudo -i /shared/takes/master.replication` 的别名。你不能直接执行像 file_repl 这样的别名： +`date` 和 `free` 都是外部命令，而 `file_repl` 是 `sudo -i /shared/takes/master.replication` 的别名。你不能直接执行像 `file_repl` 这样的别名： ``` $ ssh user@remote file_repl ``` -## 在 Unix 系统上无法直接通过 ssh 客户端执行 bash 别名 +### 在 Unix 系统上无法直接通过 ssh 客户端执行 bash 别名 要解决这个问题可以用下面方法运行 ssh 命令： + ``` $ ssh -t user@remote /bin/bash -ic 'your-alias-here' $ ssh -t user@remote /bin/bash -ic 'file_repl' ``` -ssh 命令选项： - - 1。**-t**：[强制分配伪终端。可以用来在远程机器上执行任意的 ][7] 基于屏幕的程序，有时这非常有用。当使用 `-t` 时你可能会收到一个类似" bash：cannot set terminal process group (-1)：Inappropriate ioctl for device。bash：no job control in this shell ." 的错误。 +`ssh` 命令选项： +- `-t`：[强制分配伪终端。可以用来在远程机器上执行任意的][7] 基于屏幕的程序，有时这非常有用。当使用 `-t` 时你可能会收到一个类似“bash: cannot set terminal process group (-1): Inappropriate ioctl for device. bash: no job control in this shell .”的错误。 bash shell 的选项： - 1。**-i**：运行交互 shell，这样 shell 才能运行 bash 别名 - 2。**-c**：要执行的命令取之于第一个非选项参数的命令字符串。若在命令字符串后面还有其他参数，这些参会会作为位置参数传递给命令，参数从 $0 开始。 +- `-i`：运行交互 shell，这样 shell 才能运行 bash 别名。 +- `-c`：要执行的命令取之于第一个非选项参数的命令字符串。若在命令字符串后面还有其他参数，这些参数会作为位置参数传递给命令，参数从 `$0` 开始。 总之，要运行一个名叫 `ll` 的 bash 别名，可以运行下面命令： -`$ ssh -t [[email protected]][3] -ic 'll'` + +``` +$ ssh -t vivek@server1.cyberciti.biz -ic 'll' +``` + 结果为： + [![Running bash aliases over ssh based session when using Unix or Linux ssh cli][8]][8] 下面是我的一个 shell 脚本的例子： @@ -100,9 +116,10 @@ ssh ${box} /usr/bin/lxc file push /tmp/https.www.cyberciti.biz.410.url.conf ngin ssh -t ${box} /bin/bash -ic 'push_config_job' ``` -## 相关资料 +### 相关资料 + +更多信息请输入下面命令查看 [OpenSSH 客户端][9] 和 [bash 的 man 帮助 ][10]： -更多信息请输入下面命令查看 [OpenSSH client][9] 和 [bash 的 man 帮助 ][10]： ``` $ man ssh $ man bash @@ -110,14 +127,13 @@ $ help type $ help command ``` - -------------------------------------------------------------------------------- via: https://www.cyberciti.biz/faq/use-bash-aliases-ssh-based-session/ 作者：[Vivek Gite][a] 译者：[lujun9972](https://github.com/lujun9972) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 diff --git a/published/20171227 Best Programming Languages To Learn In 2018.md b/published/201801/20171227 Best Programming Languages To Learn In 2018.md similarity index 100% rename from published/20171227 Best Programming Languages To Learn In 2018.md rename to published/201801/20171227 Best Programming Languages To Learn In 2018.md diff --git a/published/201801/20171228 How to exclude file when using scp command recursively.md b/published/201801/20171228 How to exclude file when using scp command recursively.md new file mode 100644 index 0000000000..20523b6c7c --- /dev/null +++ b/published/201801/20171228 How to exclude file when using scp command recursively.md @@ -0,0 +1,102 @@ +如何在使用 scp 命令时递归地排除文件 +====== + +Q：我需要将所有包含 *.c 文件的文件夹从名为 hostA 的本地笔记本复制到 hostB。我使用的是下面的 `scp` 命令，但不知道如何排除特定的文件（如 *.out）： + +``` +$ scp -r ~/projects/ user@hostB:/home/delta/projects/ +``` + +如何告诉 `scp` 命令在 Linux/Unix 命令行中排除特定的文件或目录？ + +人们可以使用 `scp` 命令在网络主机之间安全地复制文件。它使用 ssh 进行数据传输和身份验证。典型的语法是： + +``` +scp file1 user@host:/path/to/dest/ +scp -r /path/to/source/ user@host:/path/to/dest/ +``` + +### scp 排除文件 + +我不认为你可以在使用 scp 命令时过滤或排除文件。但是，有一个很好的解决方法来排除文件并使用 ssh 安全地复制它。本页面说明如何在使用 `scp` 递归复制目录时过滤或排除文件。 + +### 如何使用 rsync 命令排除文件 + +语法是： + +``` +rsync av -e ssh --exclude='*.out' /path/to/source/ user@hostB:/path/to/dest/ +``` + +这里： + +1. `-a` ：递归到目录，即复制所有文件和子目录。另外，打开归档模式和所有其他选项（相当于 `-rlptgoD`） +2. `-v` ：详细输出 +3. `-e ssh` ：使用 ssh 作为远程 shell，这样所有的东西都被加密 +4. `--exclude='*.out'` ：排除匹配模式的文件，例如 *.out 或 *.c 等。 + + +### rsync 命令的例子 + +在这个例子中，从 `~/virt/` 目录递归地复制所有文件，但排除所有 *.new 文件： + +``` +$ rsync -av -e ssh --exclude='*.new' ~/virt/ root@centos7:/tmp +``` + +示例输出： + +[![Scp exclude files but using rsync exclude command][2]][2] + +如果远程服务器上找不到 `rsync`，那么 `rsync` 命令将失败。在这种情况下，请尝试使用以下 `scp` 命令，该命令在当前目录中使用 [bash shell 模式匹配] [3] （它不能与 `-r` 选项一起使用）： + +``` +$ ls +``` + +示例输出： + +``` +centos71.log centos71.qcow2 centos71.qcow2.new centos71.v2.qcow2.new meta-data user-data +``` + +复制除 .new 之外的当前目录中的所有内容： + +``` +$ shopt -s extglob +$ scp !(*.new) root@centos7:/tmp/ +``` + +示例输出： + +``` +centos71.log 100 % 4262 1.3MB/s 00:00 +centos71.qcow2 100 % 836MB 32.7MB/s 00: 25 +meta-data 100 % 47 18.5KB/s 00:00 +user-data 100 % 1543 569.7KB/s 00:00 +``` + +有关更多信息，请参阅以下手册页： + +``` +$ man rsync +$ man bash +$ man scp +``` + +-------------------------------------------------------------------------------- + +via: https://www.cyberciti.biz/faq/scp-exclude-files-when-using-command-recursively-on-unix-linux/ + +作者：[Vivek Gite][a] +译者：[geekpi](https://github.com/geekpi) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.cyberciti.biz +[1]:https://www.cyberciti.biz/cdn-cgi/l/email-protection +[2]:https://www.cyberciti.biz/media/new/faq/2017/12/scp-exclude-files-on-linux-unix-macos-bash-shell-command-line.jpg +[3]:https://www.gnu.org/software/bash/manual/html_node/Pattern-Matching.html#Pattern-Matching +[4]:https://www.samba.org/ftp/rsync/rsync.html +[5]:https://man.openbsd.org/scp diff --git a/published/20171229 Set Ubuntu Derivatives Back to Default with Resetter.md b/published/201801/20171229 Set Ubuntu Derivatives Back to Default with Resetter.md similarity index 100% rename from published/20171229 Set Ubuntu Derivatives Back to Default with Resetter.md rename to published/201801/20171229 Set Ubuntu Derivatives Back to Default with Resetter.md diff --git a/published/20171230 How To Sync Time Between Linux And Windows Dual Boot.md b/published/201801/20171230 How To Sync Time Between Linux And Windows Dual Boot.md similarity index 100% rename from published/20171230 How To Sync Time Between Linux And Windows Dual Boot.md rename to published/201801/20171230 How To Sync Time Between Linux And Windows Dual Boot.md diff --git a/published/20180101 The mysterious case of the Linux Page Table Isolation patches.md b/published/201801/20180101 The mysterious case of the Linux Page Table Isolation patches.md similarity index 100% rename from published/20180101 The mysterious case of the Linux Page Table Isolation patches.md rename to published/201801/20180101 The mysterious case of the Linux Page Table Isolation patches.md diff --git a/published/201801/20180103 Linux-Unix desktop fun- Simulates the display from -The Matrix.md b/published/201801/20180103 Linux-Unix desktop fun- Simulates the display from -The Matrix.md new file mode 100644 index 0000000000..14efe969de --- /dev/null +++ b/published/201801/20180103 Linux-Unix desktop fun- Simulates the display from -The Matrix.md @@ -0,0 +1,139 @@ +Linux/Unix 桌面盛典：模仿 “黑客帝国” 界面！ +====== + +《黑客帝国》是 1999 年，由 Wachowki 兄弟编导的科幻动作片。这部电影的荧屏里有无尽的绿色字符降落。数字雨模拟着《黑客帝国》中的虚拟现实活动。现在，Linux 和 Unix 终端上，你也可以通过 CMatrix 模仿出矩阵数字雨。 + +### 安装 cmatrix + +根据你的 Linux/Unix 发行版或操作系统安装并且设置 CMatrix。 + +#### 如何在 Debian/Ubuntu Linux 发行版中安装 cmatrix + +在 Debian/Ubuntu/Mint 系统中键入以下命令 [apt-get 命令][1]/[apt 命令][2]： + +``` +$ sudo apt install cmatrix +``` + +示例输出： + +``` +[sudo] password for vivek: +Reading package lists... Done +Building dependency tree +Reading state information... Done +Suggested packages: + cmatrix-xfont +The following NEW packages will be installed: + cmatrix +0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. +Need to get 15.8 kB of archives. +After this operation, 50.2 kB of additional disk space will be used. +Get:1 http://in.archive.ubuntu.com/ubuntu artful/universe amd64 cmatrix amd64 1.2a-5build2 [15.8 kB] +Fetched 15.8 kB in 0s (19.7 kB/s) +Selecting previously unselected package cmatrix. +(Reading database ... 205388 files and directories currently installed.) +Preparing to unpack .../cmatrix_1.2a-5build2_amd64.deb ... +Unpacking cmatrix (1.2a-5build2) ... +Setting up cmatrix (1.2a-5build2) ... +Processing triggers for man-db (2.7.6.1-2) ... +``` + +#### 如何在 Arch Linux 发行版安装 cmatrix + +键入 pacman 命令： + +``` +$ sudo pacman -S cmatrix +``` + +#### 如何在 FreeBCD 系统中安装 cmatrix + +运行如下命令安装 port： + +``` +# cd /usr/ports/misc/cmatrix/ && make install clean +``` + +或者使用 pkg 命令添加二进制包： + +``` +# pkg install cmatrix +``` + +#### 如何在 macOS Unix 发行版中安装 cmatrix + +键入下列命令： + +``` +$ brew install cmatrix +``` + +#### 如何在 OpenBSD 系统中安装 cmatrix + +键入 pkg_add 命令： + +``` +# pkg_add cmatrix +``` + +### 使用 cmatrix + +简单键入命令： + +``` +$ cmatrix +``` + +[![cmtarix 运转中][3]][3] + +#### 使用键盘 + +在执行期间，下列按键有效（`-s` 模式下，按键无效）： + +| 按键 | 说明 | +|---|---| +| `a` | 切换异步滚动 | +| `b` | 随机字符加粗 | +| `B` | 全部字符加粗 | +| `n` | 关闭字符加粗 | +| `0`-`9` | 调整更新时间 | +| `!` `@` `#` `$` `%` `^` `&` `)` | 改变对应的矩阵颜色: `!` – 红、`@` – 绿、`#` – 黄、`$` – 蓝、`%` – 洋红、`^` – 青、 `&` – 白、 `)` – 黑。 | +| `q` | 退出程序 | + +你可以通过以下命令获取 cmatrix 选项： + +``` +$ cmatrix -h +``` + +- `-a`： 异步滚动 +- `-b`： 开启字符加粗 +- `-B`： 所有字符加粗（优先于 -b 选项） +- `-f`： 强制开启 Linux $TERM 模式 +- `-l`： Linux 模式（使用 matrix 控制台字体） +- `-o`： 启用旧式滚动 +- `-h`： 输出使用说明并退出 +- `-n`： 关闭字符加粗 （优先于 -b and -B，默认） +- `-s`： “屏保”模式， 第一次按键时退出 +- `-x`： X 窗口模式，如果你使用的时 mtx.pcf 终端 +- `-V`： 输出版本信息并且退出 +- `-u delay` （0 - 10，默认 4）： 屏幕更新延时 +- `-C [color]`： 调整 matrix 颜色（默认绿色） + +现在，你拥有了一款最炫酷的终端应用！ + +-------------------------------------------------------------------------------- + +via: https://www.cyberciti.biz/open-source/command-line-hacks/matrix-digital-rain-on-linux-macos-unix-terminal/ + +作者：[nixCraft][a] +译者：[CYLeft](https://github.com/CYLeft) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.cyberciti.biz +[1]:https://www.cyberciti.biz/tips/linux-debian-package-management-cheat-sheet.html (See Linux/Unix apt-get command examples for more info) +[2]:https://www.cyberciti.biz/faq/ubuntu-lts-debian-linux-apt-command-examples/ (See Linux/Unix apt command examples for more info) +[3]:https://www.cyberciti.biz/media/new/cms/2018/01/small-cmtarix-file.gif diff --git a/published/201801/20180104 4 artificial intelligence trends to watch.md b/published/201801/20180104 4 artificial intelligence trends to watch.md new file mode 100644 index 0000000000..534575cebe --- /dev/null +++ b/published/201801/20180104 4 artificial intelligence trends to watch.md @@ -0,0 +1,71 @@ +2018 年 4 个需要关注的人工智能趋势 +====== + +> 今年人工智能决策将变得更加透明？ + +![](https://enterprisersproject.com/sites/default/files/styles/620x350/public/images/CIO%20Mentor.png?itok=K-6s_q2C) + + +无论你的 IT 业务现在使用了多少[人工智能][1]，预计你将会在 2018 年使用更多。即便你从来没有涉猎过 AI 项目，这也可能是将谈论转变为行动的一年，[德勤][2]董事总经理 David Schatsky 说。他说：“与 AI 开展合作的公司数量正在上升。” + +看看他对未来一年的AI预测： + +### 1、预期更多的企业 AI 试点项目 + +如今我们经常使用的许多现成的应用程序和平台都将 AI 结合在一起。 Schatsky 说：“除此之外，越来越多的公司正在试验机器学习或自然语言处理来解决特定的问题，或者帮助理解他们的数据，或者使内部流程自动化，或者改进他们自己的产品和服务。 + +“除此之外，公司与人工智能的合作强度将会上升。”他说，“早期采纳它的公司已经有五个或略少的项目正在进行中，但是我们认为这个数字会上升到十个或有更多正在进行的计划。” 他说，这个预测的一个原因是人工智能技术正在变得越来越好，也越来越容易使用。 + +### 2、人工智能将缓解数据科学人才紧缺的现状 + +人才是数据科学中的一个大问题，大多数大公司都在努力聘用他们所需要的数据科学家。 Schatsky 说，AI 可以承担一些负担。他说：“数据科学的实践，逐渐成为由创业公司和大型成熟的技术供应商提供的自动化的工具。”他解释说，大量的数据科学工作是重复的、乏味的，自动化的时机已经成熟。 “数据科学家不会消亡，但他们将会获得更高的生产力，所以一家只能做一些数据科学项目而没有自动化的公司将能够使用自动化来做更多的事情，虽然它不能雇用更多的数据科学家”。 + +### 3、合成数据模型将缓解瓶颈 + +Schatsky 指出，在你训练机器学习模型之前，你必须得到数据来训练它。 这并不容易，他说：“这通常是一个商业瓶颈，而不是生产瓶颈。 在某些情况下，由于有关健康记录和财务信息的规定，你无法获取数据。” + +他说，合成数据模型可以采集一小部分数据，并用它来生成可能需要的较大集合。 “如果你以前需要 10000 个数据点来训练一个模型，但是只能得到 2000 个，那么现在就可以产生缺少的 8000 个数据点，然后继续训练你的模型。” + +### 4、人工智能决策将变得更加透明 + +AI 的业务问题之一就是它经常作为一个黑匣子来操作。也就是说，一旦你训练了一个模型，它就会吐出你不能解释的答案。 Schatsky 说：“机器学习可以自动发现人类无法看到的数据模式，因为数据太多或太复杂。 “发现了这些模式后，它可以预测未见的新数据。” + +问题是，有时你确实需要知道 AI 发现或预测背后的原因。 “以医学图像为例子来说，模型说根据你给我的数据，这个图像中有 90％ 的可能性是肿瘤。 “Schatsky 说，“你说，‘你为什么这么认为？’ 模型说：‘我不知道，这是数据给的建议。’” + +Schatsky 说，如果你遵循这些数据，你将不得不对患者进行探查手术。 当你无法解释为什么时，这是一个艰难的请求。 “但在很多情况下，即使模型产生了非常准确的结果，如果不能解释为什么，也没有人愿意相信它。” + +还有一些情况是由于规定，你确实不能使用你无法解释的数据。 Schatsky 说：“如果一家银行拒绝贷款申请，就需要能够解释为什么。 这是一个法规，至少在美国是这样。传统上来说，人类分销商会打个电话做回访。一个机器学习模式可能会更准确，但如果不能解释它的答案，就不能使用。” + +大多数算法不是为了解释他们的推理而设计的。 他说：“所以研究人员正在找到聪明的方法来让 AI 泄漏秘密，并解释哪些变量使得这个病人更可能患有肿瘤。 一旦他们这样做，人们可以发现答案，看看为什么会有这样的结论。” + +他说，这意味着人工智能的发现和决定可以用在许多今天不可能的领域。 “这将使这些模型更加值得信赖，在商业世界中更具可用性。” + +-------------------------------------------------------------------------------- + +via: https://enterprisersproject.com/article/2018/1/4-ai-trends-watch + +作者：[Minda Zetlin][a] +译者：[Wuod3n](https://github.com/Wuod3n) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://enterprisersproject.com/user/minda-zetlin +[1]:https://enterprisersproject.com/tags/artificial-intelligence +[2]:https://www2.deloitte.com/us/en.html +[3]:https://enterprisersproject.com/article/2017/12/8-emerging-ai-jobs-it-pros?sc_cid=70160000000h0aXAAQ + + + + + + + + + + + + + + + diff --git a/published/20180105 yum find out path where is package installed to on CentOS-RHEL.md b/published/201801/20180105 yum find out path where is package installed to on CentOS-RHEL.md similarity index 100% rename from published/20180105 yum find out path where is package installed to on CentOS-RHEL.md rename to published/201801/20180105 yum find out path where is package installed to on CentOS-RHEL.md diff --git a/published/20180112 City of Barcelona Kicks Out Microsoft in Favor of Linux and Open Source.md b/published/201801/20180112 City of Barcelona Kicks Out Microsoft in Favor of Linux and Open Source.md similarity index 100% rename from published/20180112 City of Barcelona Kicks Out Microsoft in Favor of Linux and Open Source.md rename to published/201801/20180112 City of Barcelona Kicks Out Microsoft in Favor of Linux and Open Source.md diff --git a/published/201801/20180119 No More Ubuntu Debian is the New Choice For Googles In-house Linux Distribution.md b/published/201801/20180119 No More Ubuntu Debian is the New Choice For Googles In-house Linux Distribution.md new file mode 100644 index 0000000000..b5d1b8637f --- /dev/null +++ b/published/201801/20180119 No More Ubuntu Debian is the New Choice For Googles In-house Linux Distribution.md @@ -0,0 +1,98 @@ +Debian 取代 Ubuntu 成为 Google 内部 Linux 发行版的新选择 +============================================================ + +> 摘要：Google 多年来一直使用基于 Ubuntu 的内部操作系统 Goobuntu。如今，Goobuntu 正在被基于 Debian Testing 的 gLinux 所取代。 + +如果你读过那篇《[Ubuntu 十个令人惊奇的事实][18]》，你可能知道 Google 使用了一个名为 [Goobuntu][19] 的 Linux 发行版作为开发平台。这是一个定制化的 Linux 发行版，不难猜到，它是基于 Ubuntu 的。 + +Goobuntu 基本上是一个 [采用轻量级的界面的 Ubuntu][20]，它是基于 Ubuntu LTS 版本的。如果你认为 Google 对 Ubuntu 的测试或开发做出了贡献，那么你就错了。Google 只是 Canonical 公司的 [Ubuntu Advantage Program][21] 计划的付费客户而已。[Canonical][22] 是 Ubuntu 的母公司。 + +### 遇见 gLinux：Google 基于 Debian Buster 的新 Linux 发行版 + +![gLinux from Goobuntu](https://itsfoss.com/wp-content/uploads/2018/01/glinux-announcement-800x450.jpg) + +在使用 Ubuntu 五年多以后，Google 正在用一个基于 Debian Testing 版本的 Linux 发行版 —— gLinux 取代 Goobuntu。 + +正如 [MuyLinux][23] 所报道的，gLinux 是从软件包的源代码中构建出来的，然后 Google 对其进行了修改，这些改动也将为上游做出贡献。 + +这个“新闻”并不是什么新鲜事，它早在去年八月就在 Debconf'17 开发者大会上宣布了。但不知为何，这件事并没有引起应有的关注。 + +请点击 [这里][24] 观看 Debconf 视频中的演示。gLinux 的演示从 12:00 开始。 + +[推荐阅读：微软出局，巴塞罗那青睐 Linux 系统和开源软件][25] + +### 从 Ubuntu 14.04 LTS 转移到 Debian 10 Buster + +Google 曾经看重 Ubuntu LTS 的稳定性，现在为了及时测试软件而转移到 Debian Testing 上。但目前尚不清楚 Google 为什么决定从 Ubuntu 切换到 Debian。 + +Google 计划如何转移到 Debian Testing？目前的 Debian Testing 版本是即将发布的 Debian 10 Buster。Google 开发了一个内部工具，用于将现有系统从 Ubuntu 14.04 LTS 迁移到 Debian 10 Buster。项目负责人 Margarita 在 Debconf 中声称，经过测试，该工具工作正常。 + +Google 还计划将这些改动发到 Debian 的上游项目中，从而为其发展做出贡献。 + +![gLinux testing plan from Google](https://itsfoss.com/wp-content/uploads/2018/01/glinux-testing-plan.jpg) + +*gLinux 的开发计划* + +### Ubuntu 丢失了一个大客户！ + +回溯到 2012 年，Canonical 公司澄清说 Google 不是他们最大的商业桌面客户。但至少可以说，Google 是他们的大客户。当 Google 准备切换到 Debian 时，必然会使 Canonical 蒙受损失。 + +[推荐阅读：Mandrake Linux Creator 推出新的开源移动操作系统][26] + +### 你怎么看？ + +请记住，Google 不会限制其开发者使用任何操作系统，但鼓励使用 Linux。 + +如果你想使用 Goobuntu 或 gLinux，那得成为 Google 公司的雇员才行。因为这是 Google 的内部项目，不对公众开放。 + +总的来说，这对 Debian 来说是一个好消息，尤其是他们成为了上游发行版的话。对 Ubuntu 来说可就不同了。我已经联系了 Canonical 公司征求意见，但至今没有回应。 + +更新：Canonical 公司回应称，他们“不共享与单个客户关系的细节”，因此他们不能提供有关收入和任何其他的细节。 + +你对 Google 抛弃 Ubuntu 而选择 Debian 有什么看法？ + + +![](https://secure.gravatar.com/avatar/20749c268f5d3e4d2c785499eb6a17c0?s=125&d=mm&r=g) + +#### 关于作者 Abhishek Prakash + +我是一名专业的软件开发人员，也是 FOSS 的创始人。我是一个狂热的 Linux 爱好者和开源爱好者。我使用 Ubuntu 并相信知识共享。除了 Linux 之外，我还喜欢经典的侦探推理故事。我是阿加莎·克里斯蒂（Agatha Christie）作品的忠实粉丝。 + + +-------------------------------------------------------------------------------- + +via: https://itsfoss.com/goobuntu-glinux-google/ + +作者：[Abhishek Prakash][a] +译者：[jessie-pang](https://github.com/jessie-pang) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://itsfoss.com/author/abhishek/ +[1]:https://itsfoss.com/author/abhishek/ +[2]:https://itsfoss.com/goobuntu-glinux-google/#comments +[3]:https://www.facebook.com/share.php?u=https%3A%2F%2Fitsfoss.com%2Fgoobuntu-glinux-google%2F%3Futm_source%3Dfacebook%26utm_medium%3Dsocial%26utm_campaign%3DSocialWarfare +[4]:https://twitter.com/share?original_referer=/&text=No+More+Ubuntu%21+Debian+is+the+New+Choice+For+Google%E2%80%99s+In-house+Linux+Distribution&url=https://itsfoss.com/goobuntu-glinux-google/%3Futm_source%3Dtwitter%26utm_medium%3Dsocial%26utm_campaign%3DSocialWarfare&via=abhishek_foss +[5]:https://plus.google.com/share?url=https%3A%2F%2Fitsfoss.com%2Fgoobuntu-glinux-google%2F%3Futm_source%3DgooglePlus%26utm_medium%3Dsocial%26utm_campaign%3DSocialWarfare +[6]:https://www.linkedin.com/cws/share?url=https%3A%2F%2Fitsfoss.com%2Fgoobuntu-glinux-google%2F%3Futm_source%3DlinkedIn%26utm_medium%3Dsocial%26utm_campaign%3DSocialWarfare +[7]:http://www.stumbleupon.com/submit?url=https://itsfoss.com/goobuntu-glinux-google/&title=No+More+Ubuntu%21+Debian+is+the+New+Choice+For+Google%26%238217%3Bs+In-house+Linux+Distribution +[8]:https://www.reddit.com/submit?url=https://itsfoss.com/goobuntu-glinux-google/&title=No+More+Ubuntu%21+Debian+is+the+New+Choice+For+Google%26%238217%3Bs+In-house+Linux+Distribution +[9]:https://www.facebook.com/share.php?u=https%3A%2F%2Fitsfoss.com%2Fgoobuntu-glinux-google%2F%3Futm_source%3Dfacebook%26utm_medium%3Dsocial%26utm_campaign%3DSocialWarfare +[10]:https://twitter.com/share?original_referer=/&text=No+More+Ubuntu%21+Debian+is+the+New+Choice+For+Google%E2%80%99s+In-house+Linux+Distribution&url=https://itsfoss.com/goobuntu-glinux-google/%3Futm_source%3Dtwitter%26utm_medium%3Dsocial%26utm_campaign%3DSocialWarfare&via=abhishek_foss +[11]:https://plus.google.com/share?url=https%3A%2F%2Fitsfoss.com%2Fgoobuntu-glinux-google%2F%3Futm_source%3DgooglePlus%26utm_medium%3Dsocial%26utm_campaign%3DSocialWarfare +[12]:https://www.linkedin.com/cws/share?url=https%3A%2F%2Fitsfoss.com%2Fgoobuntu-glinux-google%2F%3Futm_source%3DlinkedIn%26utm_medium%3Dsocial%26utm_campaign%3DSocialWarfare +[13]:http://www.stumbleupon.com/submit?url=https://itsfoss.com/goobuntu-glinux-google/&title=No+More+Ubuntu%21+Debian+is+the+New+Choice+For+Google%26%238217%3Bs+In-house+Linux+Distribution +[14]:https://www.reddit.com/submit?url=https://itsfoss.com/goobuntu-glinux-google/&title=No+More+Ubuntu%21+Debian+is+the+New+Choice+For+Google%26%238217%3Bs+In-house+Linux+Distribution +[15]:https://itsfoss.com/category/news/ +[16]:https://itsfoss.com/tag/glinux/ +[17]:https://itsfoss.com/tag/goobuntu/ +[18]:https://itsfoss.com/facts-about-ubuntu/ +[19]:https://en.wikipedia.org/wiki/Goobuntu +[20]:http://www.zdnet.com/article/the-truth-about-goobuntu-googles-in-house-desktop-ubuntu-linux/ +[21]:https://www.ubuntu.com/support +[22]:https://www.canonical.com/ +[23]:https://www.muylinux.com/2018/01/15/goobuntu-glinux-google/ +[24]:https://debconf17.debconf.org/talks/44/ +[25]:https://linux.cn/article-9236-1.html +[26]:https://itsfoss.com/eelo-mobile-os/ diff --git a/published/201801/20180123 What Is bashrc and Why Should You Edit It.md b/published/201801/20180123 What Is bashrc and Why Should You Edit It.md new file mode 100644 index 0000000000..0b80202140 --- /dev/null +++ b/published/201801/20180123 What Is bashrc and Why Should You Edit It.md @@ -0,0 +1,113 @@ +什么是 .bashrc，为什么要编辑 .bashrc？ +====== + +![](https://www.maketecheasier.com/assets/uploads/2018/01/what-is-bashrc-hero.png) + +你的 home 目录下藏着很多隐藏文件。如果你在运行 macOS 或者主流的 Linux 发行版的话，你就会在靠近隐藏文件列表的上方看见一个名为 `.bashrc` 的文件。那么什么是 `.bashrc`，编辑 `.bashrc` 又有什么用呢？ + +![finder-find-bashrc][1] + +如果你运行一个基于 Unix 或者类 Unix 的操作系统，bash 很有可能是作为默认终端被安装的。虽然存在很多[不同的 shell][2]，bash 却是最常见或许也是最主流的。如果你不明白那意味着什么，bash 是一个能解释你输入进终端程序的东西，并且基于你的输入来运行命令。它在一定程度上支持使用脚本来定制功能，这时候就要用到 `.bashrc` 了。 + +为了加载你的配置，bash 在每次启动时都会加载 `.bashrc` 文件的内容。每个用户的 home 目录都有这个 shell 脚本。它用来存储并加载你的终端配置和环境变量。 + +终端配置可以包含很多不同的东西。最常见的，`.bashrc` 文件包含用户想要用的别名。别名允许用户通过更短的名字或替代的名字来指向命令，对于经常在终端下工作的人来说这可是一个省时利器。 + +![terminal-edit-bashrc-1][3] + +你可以在任何终端文本编辑器上编辑 `.bashrc`。在接下来的例子中我们将使用 `nano`。 + +要使用 `nano` 来编辑 `.bashrc`，在终端中调用以下命令： + +``` +nano ~/.bashrc +``` + +如果你之前从没有编辑过 `.bashrc` 的话，你也许会发现它是空的。这没关系！如果不是的话，你可以随意在任一行添加你的配置。 + +你对 bashrc 所做的任何修改将在下一次启动终端时生效。如果你想立刻生效的话，运行下面的命令： + +``` +source ~/.bashrc +``` + +你可以添加到任何 `.bashrc` 的位置，随意使用命令（通过 `#`）来组织你的代码。 + +编辑 `.bashrc` 需要遵循 [bash 脚本格式][4]。如果你不知道如何用 bash 编写脚本的话，有很多在线资料可供查阅。这是一本相当全面的[介绍指南][5]，包含一些我们没能在这里提及的 bashrc 的方面。 + +**相关**： [如何在 Linux 启动时以 root 权限运行 bash 脚本][6] + +有一些有用的小技巧能使你的终端体验将更高效，也更用户友好。 + +### 为什么我要编辑 bashrc ？ + +#### Bash 提示符 + +bash 提示符允许你自定义你的终端，并让它在你运行命令时显示提示。自定义的 bash 提示符着实能提高你在终端的工作效率。 + +看看这些即[有用][7]又[有趣][8]的 bash 提示符，你可以把它们添加到你的 `.bashrc` 里。 + +#### 别名 + +![terminal-edit-bashrc-3][9] + +别名允许你使用简写的代码来执行你想要的某种格式的某个命令。让我们用 `ls` 命令来举个例子吧。`ls` 命令默认显示你目录里的内容。这挺有用的，不过显示目录的更多信息，或者显示目录下的隐藏内容，往往更加有用。因此，有个常见的别名就是 `ll`，用来运行 `ls -lha` 或者其他类似的命令。这样就能显示文件的大部分信息，找出隐藏的文件，并能以“能被人类阅读”的单位显示文件大小，而不是用“块”作为单位。 + +你需要按照下面这样的格式书写别名： + +``` +alias ll = "ls -lha" +``` + +左边输入你想设置的别名，右边引号里是要执行的命令。你可以用这种方法来创建命令的短版本，防止出现常见的拼写错误，或者让一个命令总是带上你想要的参数来运行。你也可以用你喜欢的缩写来规避讨厌或容易忘记的语法。这是一些[常见的别名的用法][10]，你可以添加到你的 `.bashrc` 里。 + +#### 函数 + +![terminal-edit-bashrc-2][11] + +除了缩短命令名，你也可以用 bash 函数组合多个命令到一个操作。这些命令可以很复杂，但是它们大多遵循这种语法： + +``` +function_name () { + command_1 + command_2 +} +``` + +下面的命令组合了 `mkdir` 和 `cd` 命令。输入 `md folder_name` 可以在你的工作目录创建一个名为“folder_name”的目录并立刻导航进入。 + +``` +md () { + mkdir -p $1 + cd $1 +} +``` + +如你所见，函数中的 `$1` 代表第一个参数，就是你在函数名后紧跟着输入的文本。 + +### 总结 + +不像某些自定义终端的方法，变动 bashrc 是非常直接且低风险的。即使你一不小心全搞砸了，你也可以随时删掉 bashrc 文件然后重新来一遍。试试看吧，你会惊叹于你提高的生产力的。 + +-------------------------------------------------------------------------------- + +via: https://www.maketecheasier.com/what-is-bashrc/ + +作者：[Alexander Fox][a] +译者：[heart4lor](https://github.com/heart4lor) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.maketecheasier.com/author/alexfox/ +[1]:https://www.maketecheasier.com/assets/uploads/2018/01/finder-find-bashrc.png "finder-find-bashrc" +[2]:https://www.maketecheasier.com/alternative-linux-shells/ +[3]:https://www.maketecheasier.com/assets/uploads/2018/01/terminal-edit-bashrc-1.png "terminal-edit-bashrc-1" +[4]:http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html +[5]:https://www.digitalocean.com/community/tutorials/an-introduction-to-useful-bash-aliases-and-functions +[6]:https://www.maketecheasier.com/run-bash-script-as-root-during-startup-linux/ "How to Run Bash Script as Root During Startup on Linux" +[7]:https://www.maketecheasier.com/8-useful-and-interesting-bash-prompts/ +[8]:https://www.maketecheasier.com/more-useful-and-interesting-bash-prompts/ +[9]:https://www.maketecheasier.com/assets/uploads/2018/01/terminal-edit-bashrc-3.png "terminal-edit-bashrc-3" +[10]:https://www.maketecheasier.com/install-software-in-various-linux-distros/#aliases +[11]:https://www.maketecheasier.com/assets/uploads/2018/01/terminal-edit-bashrc-2.png "terminal-edit-bashrc-2" diff --git a/published/20180103 Creating an Offline YUM repository for LAN.md b/published/20180103 Creating an Offline YUM repository for LAN.md new file mode 100644 index 0000000000..5a0695cb97 --- /dev/null +++ b/published/20180103 Creating an Offline YUM repository for LAN.md @@ -0,0 +1,106 @@ +创建局域网内的离线 Yum 仓库 +====== + +在早先的教程中，我们讨论了[如何使用 ISO 镜像和在线 Yum 仓库的方式来创建自己的 Yum 仓库 ][1]。创建自己的 Yum 仓库是一个不错的想法，但若网络中只有 2-3 台 Linux 机器那就没啥必要了。不过若你的网络中有大量的 Linux 服务器，而且这些服务器还需要定时进行升级，或者你有大量服务器无法直接访问互联网，那么创建自己的 Yum 仓库就很有必要了。 + +当我们有大量的 Linux 服务器，而每个服务器都直接从互联网上升级系统时，数据消耗会很可观。为了节省数据量，我们可以创建个离线 Yum 源并将之分享到本地网络中。网络中的其他 Linux 机器就可以直接从本地 Yum 上获取系统更新，从而节省数据量，而且传输速度也会很好。 + +我们可以使用下面两种方法来分享 Yum 仓库： + +* 使用 Web 服务器（Apache） +* 使用 FTP 服务器（VSFTPD） + +在开始讲解这两个方法之前，我们需要先根据[之前的教程][1]创建一个 Yum 仓库。 + +### 使用 Web 服务器 + +首先在 Yum 服务器上安装 Web 服务器（Apache），我们假设服务器 IP 是 `192.168.1.100`。我们已经在这台系统上配置好了 Yum 仓库，现在我们来使用 `yum` 命令安装 Apache Web 服务器， + +``` +$ yum install httpd +``` + +下一步，拷贝所有的 rpm 包到默认的 Apache 根目录下，即 `/var/www/html`，由于我们已经将包都拷贝到了 `/YUM` 下，我们也可以创建一个软连接来从 `/var/www/html` 指向 `/YUM`。 + +``` +$ ln -s /var/www/html/Centos /YUM +``` + +重启 Web 服务器应用改变： + +``` +$ systemctl restart httpd +``` + +#### 配置客户端机器 + +服务端的配置就完成了，现在需要配置下客户端来从我们创建的离线 Yum 中获取升级包，这里假设客户端 IP 为 `192.168.1.101`。 + +在 `/etc/yum.repos.d` 目录中创建 `offline-yum.repo` 文件，输入如下信息， + +``` +$ vi /etc/yum.repos.d/offline-yum.repo +``` + +``` +name=Local YUM +baseurl=http://192.168.1.100/CentOS/7 +gpgcheck=0 +enabled=1 +``` + +客户端也配置完了。试一下用 `yum` 来安装/升级软件包来确认仓库是正常工作的。 + +### 使用 FTP 服务器 + +在 FTP 上分享 Yum，首先需要安装所需要的软件包，即 vsftpd。 + +``` +$ yum install vsftpd +``` + +vsftp 的默认根目录为 `/var/ftp/pub`，因此你可以拷贝 rpm 包到这个目录，或者为它创建一个软连接： + +``` +$ ln -s /var/ftp/pub /YUM +``` + +重启服务应用改变： + +``` +$ systemctl restart vsftpd +``` + +#### 配置客户端机器 + +像上面一样，在 `/etc/yum.repos.d` 中创建 `offline-yum.repo` 文件，并输入下面信息， + +``` +$ vi /etc/yum.repos.d/offline-yum.repo +``` + +``` +[Offline YUM] +name=Local YUM +baseurl=ftp://192.168.1.100/pub/CentOS/7 +gpgcheck=0 +enabled=1 +``` + +现在客户机可以通过 ftp 接收升级了。要配置 vsftpd 服务器为其他 Linux 系统分享文件，请[阅读这篇指南][2]。 + +这两种方法都很不错，你可以任意选择其中一种方法。有任何疑问或这想说的话，欢迎在下面留言框中留言。 + +-------------------------------------------------------------------------------- + +via: http://linuxtechlab.com/offline-yum-repository-for-lan/ + +作者：[Shusain][a] +译者：[lujun9972](https://github.com/lujun9972) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://linuxtechlab.com/author/shsuain/ +[1]:https://linux.cn/article-9296-1.html +[2]:http://linuxtechlab.com/ftp-secure-installation-configuration/ diff --git a/published/20180124 8 ways to generate random password in Linux.md b/published/20180124 8 ways to generate random password in Linux.md new file mode 100644 index 0000000000..cb91d79d57 --- /dev/null +++ b/published/20180124 8 ways to generate random password in Linux.md @@ -0,0 +1,279 @@ +八种在 Linux 上生成随机密码的方法 +====== + +> 学习使用 8 种 Linux 原生命令或第三方实用程序来生成随机密码。 + +![][1] + +在这篇文章中，我们将引导你通过几种不同的方式在 Linux 终端中生成随机密码。其中几种利用原生 Linux 命令，另外几种则利用极易在 Linux 机器上安装的第三方工具或实用程序实现。在这里我们利用像 `openssl`, [dd][2], `md5sum`, `tr`, `urandom` 这样的原生命令和 mkpasswd，randpw，pwgen，spw，gpg，xkcdpass，diceware，revelation，keepaasx，passwordmaker 这样的第三方工具。 + +其实这些方法就是生成一些能被用作密码的随机字母字符串。随机密码可以用于新用户的密码，不管用户基数有多大，这些密码都是独一无二的。话不多说，让我们来看看 8 种不同的在 Linux 上生成随机密码的方法吧。 + +### 使用 mkpasswd 实用程序生成密码 + +`mkpasswd` 在基于 RHEL 的系统上随 `expect` 软件包一起安装。在基于 Debian 的系统上 `mkpasswd` 则在软件包 `whois` 中。直接安装 `mkpasswd` 软件包将会导致错误： + +- RHEL 系统：软件包 mkpasswd 不可用。 +- Debian 系统：错误：无法定位软件包 mkpasswd。 + +所以按照上面所述安装他们的父软件包，就没问题了。 + +运行 `mkpasswd` 来获得密码 + +```bash +root@kerneltalks# mkpasswd << on RHEL +zt*hGW65c + +root@kerneltalks# mkpasswd teststring << on Ubuntu +XnlrKxYOJ3vik +``` + +这个命令在不同的系统上表现得不一样，所以工作方式各异。你也可以通过参数来控制长度等选项，可以查阅 man 手册来探索。 + +### 使用 openssl 生成密码 + +几乎所有 Linux 发行版都包含 openssl。我们可以利用它的随机功能来生成可以用作密码的随机字母字符串。 + +```bash +root@kerneltalks # openssl rand -base64 10 +nU9LlHO5nsuUvw== +``` + +这里我们使用 `base64` 编码随机函数，最后一个数字参数表示长度。 + +### 使用 urandom 生成密码 + +设备文件 `/dev/urandom` 是另一个获得随机字符串的方法。我们使用 `tr` 功能并裁剪输出来获得随机字符串，并把它作为密码。 + +```bash +root@kerneltalks # strings /dev/urandom |tr -dc A-Za-z0-9 | head -c20; echo +UiXtr0NAOSIkqtjK4c0X +``` + +### 使用 dd 命令生成密码 + +我们甚至可以使用 `/dev/urandom` 设备配合 [dd 命令][2] 来获取随机字符串。 + +```bash +root@kerneltalks# dd if=/dev/urandom bs=1 count=15|base64 -w 0 +15+0 records in +15+0 records out +15 bytes (15 B) copied, 5.5484e-05 s, 270 kB/s +QMsbe2XbrqAc2NmXp8D0 +``` + +我们需要将结果通过 `base64` 编码使它能被人类可读。你可以使用数值来获取想要的长度。想要获得更简洁的输出的话，可以将“标准错误输出”重定向到 `/dev/null`。简洁输出的命令是： + +```bash +root@kerneltalks # dd if=/dev/urandom bs=1 count=15 2>/dev/null|base64 -w 0 +F8c3a4joS+a3BdPN9C++ +``` + +### 使用 md5sum 生成密码 + +另一种获取可用作密码的随机字符串的方法是计算 MD5 校验值！校验值看起来确实像是随机字符串组合在一起，我们可以用作密码。确保你的计算源是个变量，这样的话每次运行命令时生成的校验值都不一样。比如 `date` ！[date 命令][3] 总会生成不同的输出。 + +```bash +root@kerneltalks # date |md5sum +4d8ce5c42073c7e9ca4aeffd3d157102 - +``` + +在这里我们将 `date` 命令的输出通过 `md5sum` 得到了校验和！你也可以用 [cut 命令][4] 裁剪你需要的长度。 + +### 使用 pwgen 生成密码 + +`pwgen` 软件包在类似 [EPEL 软件仓库][5]（LCTT 译注：企业版 Linux 附加软件包）中。`pwgen` 更专注于生成可发音的密码，但它们不在英语词典中，也不是纯英文的。标准发行版仓库中可能并不包含这个工具。安装这个软件包然后运行 `pwgen` 命令行。Boom ! + +```bash +root@kerneltalks # pwgen +thu8Iox7 ahDeeQu8 Eexoh0ai oD8oozie ooPaeD9t meeNeiW2 Eip6ieph Ooh1tiet +cootad7O Gohci0vo wah9Thoh Ohh3Ziur Ao1thoma ojoo6aeW Oochai4v ialaiLo5 +aic2OaDa iexieQu8 Aesoh4Ie Eixou9ph ShiKoh0i uThohth7 taaN3fuu Iege0aeZ +cah3zaiW Eephei0m AhTh8guo xah1Shoo uh8Iengo aifeev4E zoo4ohHa fieDei6c +aorieP7k ahna9AKe uveeX7Hi Ohji5pho AigheV7u Akee9fae aeWeiW4a tiex8Oht +``` + +你的终端会呈现出一个密码列表！你还想要什么呢？好吧。你还想再仔细探索的话， `pwgen` 还有很多自定义选项，这些都可以在 man 手册里查阅到。 + +### 使用 gpg 工具生成密码 + +GPG 是一个遵循 OpenPGP 标准的加密及签名工具。大部分 gpg 工具都预先被安装好了（至少在我的 RHEL7 上是这样）。但如果没有的话你可以寻找 `gpg` 或 `gpg2` 软件包并[安装][6]它。 + +使用下面的命令以从 gpg 工具生成密码。 + +```bash +root@kerneltalks # gpg --gen-random --armor 1 12 +mL8i+PKZ3IuN6a7a +``` + +这里我们传了生成随机字节序列选项（`--gen-random`），质量为 1（第一个参数），次数 12 （第二个参数）。选项 `--armor` 保证以 `base64` 编码输出。 + +### 使用 xkcdpass 生成密码 + +著名的极客幽默网站 [xkcd][7]，发表了一篇非常有趣的文章，是关于好记但又复杂的密码的。你可以在[这里][8]阅读。所以 `xkcdpass` 工具就受这篇文章启发，做了这样的工作！这是一个 Python 软件包，可以在[这里][9]的 Python 的官网上找到它。 + +![](https://imgs.xkcd.com/comics/password_strength.png) + +所有的安装使用说明都在上面那个页面提及了。这里是安装步骤和我的测试 RHEL 服务器的输出，以供参考。 + +```bash +root@kerneltalks # wget https://pypi.python.org/packages/b4/d7/3253bd2964390e034cf0bba227db96d94de361454530dc056d8c1c096abc/xkcdpass-1.14.3.tar.gz#md5=5f15d52f1d36207b07391f7a25c7965f +--2018-01-23 19:09:17-- https://pypi.python.org/packages/b4/d7/3253bd2964390e034cf0bba227db96d94de361454530dc056d8c1c096abc/xkcdpass-1.14.3.tar.gz +Resolving pypi.python.org (pypi.python.org)... 151.101.32.223, 2a04:4e42:8::223 +Connecting to pypi.python.org (pypi.python.org)|151.101.32.223|:443... connected. +HTTP request sent, awaiting response... 200 OK +Length: 871848 (851K) [binary/octet-stream] +Saving to: ‘xkcdpass-1.14.3.tar.gz’ + +100%[==============================================================================================================================>] 871,848 --.-K/s in 0.01s + +2018-01-23 19:09:17 (63.9 MB/s) - ‘xkcdpass-1.14.3.tar.gz’ saved [871848/871848] + + +root@kerneltalks # tar -xvf xkcdpass-1.14.3.tar.gz +xkcdpass-1.14.3/ +xkcdpass-1.14.3/examples/ +xkcdpass-1.14.3/examples/example_import.py +xkcdpass-1.14.3/examples/example_json.py +xkcdpass-1.14.3/examples/example_postprocess.py +xkcdpass-1.14.3/LICENSE.BSD +xkcdpass-1.14.3/MANIFEST.in +xkcdpass-1.14.3/PKG-INFO +xkcdpass-1.14.3/README.rst +xkcdpass-1.14.3/setup.cfg +xkcdpass-1.14.3/setup.py +xkcdpass-1.14.3/tests/ +xkcdpass-1.14.3/tests/test_list.txt +xkcdpass-1.14.3/tests/test_xkcdpass.py +xkcdpass-1.14.3/tests/__init__.py +xkcdpass-1.14.3/xkcdpass/ +xkcdpass-1.14.3/xkcdpass/static/ +xkcdpass-1.14.3/xkcdpass/static/eff-long +xkcdpass-1.14.3/xkcdpass/static/eff-short +xkcdpass-1.14.3/xkcdpass/static/eff-special +xkcdpass-1.14.3/xkcdpass/static/fin-kotus +xkcdpass-1.14.3/xkcdpass/static/ita-wiki +xkcdpass-1.14.3/xkcdpass/static/legacy +xkcdpass-1.14.3/xkcdpass/static/spa-mich +xkcdpass-1.14.3/xkcdpass/xkcd_password.py +xkcdpass-1.14.3/xkcdpass/__init__.py +xkcdpass-1.14.3/xkcdpass.1 +xkcdpass-1.14.3/xkcdpass.egg-info/ +xkcdpass-1.14.3/xkcdpass.egg-info/dependency_links.txt +xkcdpass-1.14.3/xkcdpass.egg-info/entry_points.txt +xkcdpass-1.14.3/xkcdpass.egg-info/not-zip-safe +xkcdpass-1.14.3/xkcdpass.egg-info/PKG-INFO +xkcdpass-1.14.3/xkcdpass.egg-info/SOURCES.txt +xkcdpass-1.14.3/xkcdpass.egg-info/top_level.txt + + +root@kerneltalks # cd xkcdpass-1.14.3 + +root@kerneltalks # python setup.py install +running install +running bdist_egg +running egg_info +writing xkcdpass.egg-info/PKG-INFO +writing top-level names to xkcdpass.egg-info/top_level.txt +writing dependency_links to xkcdpass.egg-info/dependency_links.txt +writing entry points to xkcdpass.egg-info/entry_points.txt +reading manifest file 'xkcdpass.egg-info/SOURCES.txt' +reading manifest template 'MANIFEST.in' +writing manifest file 'xkcdpass.egg-info/SOURCES.txt' +installing library code to build/bdist.linux-x86_64/egg +running install_lib +running build_py +creating build +creating build/lib +creating build/lib/xkcdpass +copying xkcdpass/xkcd_password.py -> build/lib/xkcdpass +copying xkcdpass/__init__.py -> build/lib/xkcdpass +creating build/lib/xkcdpass/static +copying xkcdpass/static/eff-long -> build/lib/xkcdpass/static +copying xkcdpass/static/eff-short -> build/lib/xkcdpass/static +copying xkcdpass/static/eff-special -> build/lib/xkcdpass/static +copying xkcdpass/static/fin-kotus -> build/lib/xkcdpass/static +copying xkcdpass/static/ita-wiki -> build/lib/xkcdpass/static +copying xkcdpass/static/legacy -> build/lib/xkcdpass/static +copying xkcdpass/static/spa-mich -> build/lib/xkcdpass/static +creating build/bdist.linux-x86_64 +creating build/bdist.linux-x86_64/egg +creating build/bdist.linux-x86_64/egg/xkcdpass +copying build/lib/xkcdpass/xkcd_password.py -> build/bdist.linux-x86_64/egg/xkcdpass +copying build/lib/xkcdpass/__init__.py -> build/bdist.linux-x86_64/egg/xkcdpass +creating build/bdist.linux-x86_64/egg/xkcdpass/static +copying build/lib/xkcdpass/static/eff-long -> build/bdist.linux-x86_64/egg/xkcdpass/static +copying build/lib/xkcdpass/static/eff-short -> build/bdist.linux-x86_64/egg/xkcdpass/static +copying build/lib/xkcdpass/static/eff-special -> build/bdist.linux-x86_64/egg/xkcdpass/static +copying build/lib/xkcdpass/static/fin-kotus -> build/bdist.linux-x86_64/egg/xkcdpass/static +copying build/lib/xkcdpass/static/ita-wiki -> build/bdist.linux-x86_64/egg/xkcdpass/static +copying build/lib/xkcdpass/static/legacy -> build/bdist.linux-x86_64/egg/xkcdpass/static +copying build/lib/xkcdpass/static/spa-mich -> build/bdist.linux-x86_64/egg/xkcdpass/static +byte-compiling build/bdist.linux-x86_64/egg/xkcdpass/xkcd_password.py to xkcd_password.pyc +byte-compiling build/bdist.linux-x86_64/egg/xkcdpass/__init__.py to __init__.pyc +creating build/bdist.linux-x86_64/egg/EGG-INFO +copying xkcdpass.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO +copying xkcdpass.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO +copying xkcdpass.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO +copying xkcdpass.egg-info/entry_points.txt -> build/bdist.linux-x86_64/egg/EGG-INFO +copying xkcdpass.egg-info/not-zip-safe -> build/bdist.linux-x86_64/egg/EGG-INFO +copying xkcdpass.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO +creating dist +creating 'dist/xkcdpass-1.14.3-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it +removing 'build/bdist.linux-x86_64/egg' (and everything under it) +Processing xkcdpass-1.14.3-py2.7.egg +creating /usr/lib/python2.7/site-packages/xkcdpass-1.14.3-py2.7.egg +Extracting xkcdpass-1.14.3-py2.7.egg to /usr/lib/python2.7/site-packages +Adding xkcdpass 1.14.3 to easy-install.pth file +Installing xkcdpass script to /usr/bin + +Installed /usr/lib/python2.7/site-packages/xkcdpass-1.14.3-py2.7.egg +Processing dependencies for xkcdpass==1.14.3 +Finished processing dependencies for xkcdpass==1.14.3 +``` + +现在运行 `xkcdpass` 命令，将会随机给出你几个像下面这样的字典单词： + +```bash +root@kerneltalks # xkcdpass +broadside unpadded osmosis statistic cosmetics lugged +``` + +你可以用这些单词作为其他命令，比如 `md5sum` 的输入，来获取随机密码（就像下面这样），甚至你也可以用每个单词的第 N 个字母来生成你的密码！ + +```bash +root@kerneltalks # xkcdpass |md5sum +45f2ec9b3ca980c7afbd100268c74819 - + +root@kerneltalks # xkcdpass |md5sum +ad79546e8350744845c001d8836f2ff2 - +``` + +或者你甚至可以把所有单词串在一起作为一个超长的密码，不仅非常好记，也不容易被电脑程序攻破。 + +Linux 上还有像 [Diceware][10]、 [KeePassX][11]、 [Revelation][12]、 [PasswordMaker][13] 这样的工具，也可以考虑用来生成强随机密码。 + +-------------------------------------------------------------------------------- + +via: https://kerneltalks.com/tips-tricks/8-ways-to-generate-random-password-in-linux/ + +作者：[kerneltalks][a] +译者：[heart4lor](https://github.com/heart4lor) +校对：[Locez](https://github.com/locez) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://kerneltalks.com +[1]:https://a1.kerneltalks.com/wp-content/uploads/2018/01/different-ways-to-generate-password-in-linux.png +[2]:https://kerneltalks.com/commands/learn-dd-command-with-examples/ +[3]:https://kerneltalks.com/commands/date-time-management-using-timedatectl-command/ +[4]:https://kerneltalks.com/linux/cut-command-examples/ +[5]:https://kerneltalks.com/package/how-to-install-epel-repository/ +[6]:https://kerneltalks.com/tools/package-installation-linux-yum-apt/ +[7]:https://xkcd.com/ +[8]:https://xkcd.com/936/ +[9]:https://pypi.python.org/pypi/xkcdpass/ +[10]:http://world.std.com/~reinhold/diceware.html +[11]:https://www.keepassx.org/ +[12]:https://packages.debian.org/sid/gnome/revelation +[13]:https://passwordmaker.org/ diff --git a/published/20180131 Why you should use named pipes on Linux.md b/published/20180131 Why you should use named pipes on Linux.md new file mode 100644 index 0000000000..ac50e96c9a --- /dev/null +++ b/published/20180131 Why you should use named pipes on Linux.md @@ -0,0 +1,103 @@ +为什么应该在 Linux 上使用命名管道 +====== + +> 命名管道并不常用，但是它们为进程间通讯提供了一些有趣的特性。 + +![](https://images.techhive.com/images/article/2017/05/blue-1845806_1280-100722976-large.jpg) + +估计每一位 Linux 使用者都熟悉使用 “|” 符号将数据从一个进程传输到另一个进程的操作。它使用户能简便地从一个命令输出数据到另一个命令，并筛选出想要的数据而无须写脚本进行选择、重新格式化等操作。 + +还有另一种管道， 虽然也叫“管道”这个名字却有着非常不同的性质。即您可能尚未使用甚至尚未知晓的——命名管道。 + +普通管道与命名管道的一个主要区别就是命名管道是以文件形式实实在在地存在于文件系统中的，没错，它们表现出来就是文件。但是与其它文件不同的是，命名管道文件似乎从来没有文件内容。即使用户往命名管道中写入大量数据，该文件看起来还是空的。 + +### 如何在 Linux 上创建命名管道 + +在我们研究这些空空如也的命名管道之前，先追根溯源来看看命名管道是如何被创建的。您应该使用名为 `mkfifo` 的命令来创建它们。为什么提及“FIFO”？是因为命名管道也被认为是一种 FIFO 特殊文件。术语 “FIFO” 指的是它的先进先出first-in, first-out特性。如果你将冰淇淋盛放到碟子中，然后可以品尝它，那么你执行的就是一个LIFO（后进先出last-in, first-out操作。如果你通过吸管喝奶昔，那你就在执行一个 FIFO 操作。好，接下来是一个创建命名管道的例子。 + +``` +$ mkfifo mypipe +$ ls -l mypipe +prw-r-----. 1 shs staff 0 Jan 31 13:59 mypipe +``` + +注意一下特殊的文件类型标记 “p” 以及该文件大小为 0。您可以将重定向数据写入命名管道文件，而文件大小依然为 0。 + +``` +$ echo "Can you read this?" > mypipe +``` + +正如上面所说，敲击回车后似乎什么都没有发生（LCTT 译注：没有返回命令行提示符）。 + +另外再开一个终端，查看该命名管道的大小，依旧是 0： + +``` +$ ls -l mypipe +prw-r-----. 1 shs staff 0 Jan 31 13:59 mypipe +``` + +也许这有违直觉，用户输入的文本已经进入该命名管道，而你仍然卡在输入端。你或者其他人应该等在输出端，并准备读取放入管道的数据。现在让我们读取看看。 + +``` +$ cat mypipe +Can you read this? +``` + +一旦被读取之后，管道中的内容就没有了。 + +另一种研究命名管道如何工作的方式是通过将放入数据的操作置入后台来执行两个操作（将数据放入管道，而在另外一段读取它）。 + +``` +$ echo "Can you read this?" > mypipe & +[1] 79302 +$ cat mypipe +Can you read this? +[1]+ Done echo "Can you read this?" > mypipe +``` + +一旦管道被读取或“耗干”，该管道就清空了，尽管我们还能看见它并再次使用。可为什么要费此周折呢？ + +### 为何要使用命名管道？ + +命名管道很少被使用的理由似乎很充分。毕竟在 Unix 系统上，总有多种不同的方式完成同样的操作。有多种方式写文件、读文件、清空文件，尽管命名管道比它们来得更高效。 + +值得注意的是，命名管道的内容驻留在内存中而不是被写到硬盘上。数据内容只有在输入输出端都打开时才会传送。用户可以在管道的输出端打开之前向管道多次写入。通过使用命名管道，用户可以创建一个进程写入管道并且另外一个进程读取管道的流程，而不用关心协调二者时间上的同步。 + +用户可以创建一个单纯等待数据出现在管道输出端的进程，并在拿到输出数据后对其进行操作。下列命令我们采用 `tail` 来等待数据出现。 + +``` +$ tail -f mypipe +``` + +一旦供给管道数据的进程结束了，我们就可以看到一些输出。 + +``` +$ tail -f mypipe +Uranus replicated to WCDC7 +Saturn replicated to WCDC8 +Pluto replicated to WCDC9 +Server replication operation completed +``` + +如果研究一下向命名管道写入的进程，用户也许会惊讶于它的资源消耗之少。在下面的 `ps` 命令输出中，唯一显著的资源消耗是虚拟内存（VSZ 那一列）。 + +``` +ps u -P 80038 +USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND +shs 80038 0.0 0.0 108488 764 pts/4 S 15:25 0:00 -bash +``` + +命名管道与 Unix/Linux 系统上更常用的管道相比足以不同到拥有另一个名号，但是“管道”确实能反映出它们如何在进程间传送数据的形象，故将称其为“命名管道”还真是恰如其分。也许您在执行操作时就能从这个聪明的 Unix/Linux 特性中获益匪浅呢。 + +-------------------------------------------------------------------------------- + +via: https://www.networkworld.com/article/3251853/linux/why-use-named-pipes-on-linux.html + +作者：[Sandra Henry-Stocker][a] +译者：[YPBlib](https://github.com/YPBlib) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.networkworld.com/author/Sandra-Henry_Stocker/ +[1]:http://www.networkworld.com/article/2926630/linux/11-pointless-but-awesome-linux-terminal-tricks.html#tk.nww-fsb diff --git a/sources/talk/20170523 Best Websites to Download Linux Games.md b/sources/talk/20170523 Best Websites to Download Linux Games.md new file mode 100644 index 0000000000..d3b2870738 --- /dev/null +++ b/sources/talk/20170523 Best Websites to Download Linux Games.md @@ -0,0 +1,141 @@ +申请翻译  WangYueScream +================================ +Best Websites to Download Linux Games +====== +Brief: New to Linux gaming and wondering where to **download Linux games** from? We list the best resources from where you can **download free Linux games** as well as buy premium Linux games. + +Linux and Games? Once upon a time, it was hard to imagine these two going together. Then time passed and a lot of things happened. Fast-forward to the present, there are thousands and thousands of games available for Linux and more are being developed by both big game companies and independent developers. + +[Gaming on Linux][1] is real now and today we are going to see where you can find games for Linux platform and hunt down the games that you like. + +### Where to download Linux games? + +![Websites to download Linux games][2] + +First and foremost, look into your Linux distribution's software center (if it has one). You should find plenty of games there already. + +But that doesn't mean you should restrict yourself to the software center. Let me list you some websites to download Linux games. + +#### 1. Steam + +If you are a seasoned gamer, you have heard about Steam. Yes, if you don't know it already, Steam is available for Linux. Steam recommends Ubuntu but it should run on other major distributions too. And if you are really psyched up about Steam, there is even a dedicated operating system for playing Steam games - [SteamOS][3]. We covered it last year in the [Best Linux Gaming Distribution][4] article. + +![Steam Store][5] + +Steam has the largest games store for Linux. While writing this article, it has exactly 3487 games on Linux platform and that's really huge. You can find games from wide range of genre. As for [Digital Rights Management][6], most of the Steam games have some kind of DRM. + +For using Steam either you will have to install the [Steam client][7] on your Linux distribution or use SteamOS. One of the advantages of Steam is that, after your initial setup, for most of the games you wouldn't need to worry about dependencies and complex installation process. Steam client will do the heavy tasks for you. + +[Steam Store][8] + +#### 2. GOG + +If you are solely interested in DRM-free games, GOG has a pretty large collection of it. At this moment, GOG has 1978 DRM-free games in their library. GOG is kind of famous for its vast collection of DRM-free games. + +![GOG Store][9] + +Officially, GOG games support Ubuntu LTS versions and Linux Mint. So, Ubuntu and its derivatives will have no problem installing them. Installing them on other distributions might need some extra works, such as - installing correct dependencies. + +You will not need any extra clients for downloading games from GOG. All the purchased games will be available in your accounts section. You can download them directly with your favorite download manager. + +[GOG Store][10] + +#### 3. Humble Store + +The Humble Store is another place where you can find various games for Linux. There are both DRM-free and non-DRM-free games available on Humble Store. The non-DRM-free games are generally from the Steam. Currently there are about 1826 games for Linux in the Humble Store. + +![The Humble Store][11] + +Humble Store is famous for another reason though. They have a program called [**Humble Indie Bundle**][12] where they offer a bunch of games together with a compelling discount for a limited time period. Another thing about Humble is that when you make a purchase, 10% of the revenue from your purchase goes to charities. + +Humble doesn't have any extra clients for downloading their games. + +[The Humble Store][13] + +#### 4. itch.io + +itch.io is an open marketplace for independent digital creators with a focus on independent video games. itch.io has some of the most interesting and unique games that you can find. Most games available on itch.io are DRM-free. + +![itch.io Store][14] + +Right now, itch.io has 9514 games available in their store for Linux platform. + +itch.io has their own [client][15] for effortlessly downloading, installing, updating and playing their games. + +[itch.io Store][16] + +#### 5. LGDB + +LGDB is an abbreviation for Linux Game Database. Though technically not a game store, it has a large collection of games for Linux along with various information about them. Every game is documented with links of where you can find them. + +![Linux Game Database][17] + +As of now, there are 2046 games entries in the database. They also have very long lists for [Emulators][18], [Tools][19] and [Game Engines][20]. + +[LGDB][21] + +[Annoying Experiences Every Linux Gamer Never Wanted!][27] + +#### 6. Game Jolt + +Game Jolt has a very impressive collection with about 5000 indie games for Linux under their belt. + +![GameJolt Store][22] + +Game Jolt has an (pre-release) [client][23] for downloading, installing, updating and playing games with ease. + +[Game Jolt Store][24] + +### Others + +There are many other stores that sells Linux Games. Also there are many places you can find free games too. Here are a couple of them: + + * [**Bundle Stars**][25]: Bundle Stars currently has 814 Linux games and 31 games bundles. + * [**GamersGate**][26]: GamersGate has 595 Linux games as for now. There are both DRM-free and non-DRM-free games. + + + +#### App Stores, Software Center & Repositories + +Linux distribution has their own application stores or repositories. Though not many, but there you can find various games too. + +That's all for today. Did you know there are this many games available for Linux? How do you feel about this? Do you use some other websites to download Linux games? Do share your favorites with us. + +-------------------------------------------------------------------------------- + +via: https://itsfoss.com/download-linux-games/ + +作者：[Munif Tanjim][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://itsfoss.com/author/munif/ +[1]:https://itsfoss.com/linux-gaming-guide/ +[2]:https://itsfoss.com/wp-content/uploads/2017/05/download-linux-games-800x450.jpg +[3]:http://store.steampowered.com/steamos/ +[4]:https://itsfoss.com/linux-gaming-distributions/ +[5]:https://itsfoss.com/wp-content/uploads/2017/05/Steam-Store-800x382.jpg +[6]:https://www.wikiwand.com/en/Digital_rights_management +[7]:http://store.steampowered.com/about/ +[8]:http://store.steampowered.com/linux +[9]:https://itsfoss.com/wp-content/uploads/2017/05/GOG-Store-800x366.jpg +[10]:https://www.gog.com/games?system=lin_mint,lin_ubuntu +[11]:https://itsfoss.com/wp-content/uploads/2017/05/The-Humble-Store-800x393.jpg +[12]:https://www.humblebundle.com/?partner=itsfoss +[13]:https://www.humblebundle.com/store?partner=itsfoss +[14]:https://itsfoss.com/wp-content/uploads/2017/05/itch.io-Store-800x485.jpg +[15]:https://itch.io/app +[16]:https://itch.io/games/platform-linux +[17]:https://itsfoss.com/wp-content/uploads/2017/05/LGDB-800x304.jpg +[18]:https://lgdb.org/emulators +[19]:https://lgdb.org/tools +[20]:https://lgdb.org/engines +[21]:https://lgdb.org/games +[22]:https://itsfoss.com/wp-content/uploads/2017/05/GameJolt-Store-800x357.jpg +[23]:http://gamejolt.com/client +[24]:http://gamejolt.com/games/best?os=linux +[25]:https://www.bundlestars.com/en/games?page=1&platforms=Linux +[26]:https://www.gamersgate.com/games?state=available +[27]:https://itsfoss.com/linux-gaming-problems/ diff --git a/sources/talk/20170915 Deep learning wars- Facebook-backed PyTorch vs Google-s TensorFlow.md b/sources/talk/20170915 Deep learning wars- Facebook-backed PyTorch vs Google-s TensorFlow.md deleted file mode 100644 index 5b0014246c..0000000000 --- a/sources/talk/20170915 Deep learning wars- Facebook-backed PyTorch vs Google-s TensorFlow.md +++ /dev/null @@ -1,76 +0,0 @@ -Deep learning wars: Facebook-backed PyTorch vs Google's TensorFlow -====== -The rapid rise of tools and techniques in Artificial Intelligence and Machine learning of late has been astounding. Deep Learning, or "Machine learning on steroids" as some say, is one area where data scientists and machine learning experts are spoilt for choice in terms of the libraries and frameworks available. A lot of these frameworks are Python-based, as Python is a more general-purpose and a relatively easier language to work with. [Theano][1], [Keras][2] [TensorFlow][3] are a few of the popular deep learning libraries built on Python, developed with an aim to make the life of machine learning experts easier. - -Google's TensorFlow is a widely used machine learning and deep learning framework. Open sourced in 2015 and backed by a huge community of machine learning experts, TensorFlow has quickly grown to be THE framework of choice by many organizations for their machine learning and deep learning needs. PyTorch, on the other hand, a recently developed Python package by Facebook for training neural networks is adapted from the Lua-based deep learning library Torch. PyTorch is one of the few available DL frameworks that uses tape-based autograd system to allow building dynamic neural networks in a fast and flexible manner. - -In this article, we pit PyTorch against TensorFlow and compare different aspects where one edges the other out. - -Let's get started! - -### What programming languages support PyTorch and TensorFlow? - -Although primarily written in C++ and CUDA, Tensorflow contains a Python API sitting over the core engine, making it easier for Pythonistas to use. Additional APIs for C++, Haskell, Java, Go, and Rust are also included which means developers can code in their preferred language. - -Although PyTorch is a Python package, there's provision for you to code using the basic C/ C++ languages using the APIs provided. If you are comfortable using Lua programming language, you can code neural network models in PyTorch using the Torch API. - -### How easy are PyTorch and TensorFlow to use? - -TensorFlow can be a bit complex to use if used as a standalone framework, and can pose some difficulty in training Deep Learning models. To reduce this complexity, one can use the Keras wrapper which sits on top of TensorFlow's complex engine and simplifies the development and training of deep learning models. TensorFlow also supports [Distributed training][4], which PyTorch currently doesn't. Due to the inclusion of Python API, TensorFlow is also production-ready i.e., it can be used to train and deploy enterprise-level deep learning models. - -PyTorch was rewritten in Python due to the complexities of Torch. This makes PyTorch more native to developers. It has an easy to use framework that provides maximum flexibility and speed. It also allows quick changes within the code during training without hampering its performance. If you already have some experience with deep learning and have used Torch before, you will like PyTorch even more, because of its speed, efficiency, and ease of use. PyTorch includes custom-made GPU allocator, which makes deep learning models highly memory efficient. Due to this, training large deep learning models becomes easier. Hence, large organizations such as Facebook, Twitter, Salesforce, and many more are embracing Pytorch. - -### Training Deep Learning models with PyTorch and TensorFlow - -Both TensorFlow and PyTorch are used to build and train Neural Network models. - -TensorFlow works on SCG (Static Computational Graph) that includes defining the graph statically before the model starts execution. However, once the execution starts the only way to tweak changes within the model is using [tf.session and tf.placeholder tensors][5]. - -PyTorch is well suited to train RNNs( Recursive Neural Networks) as they run faster in [PyTorch ][6]than in TensorFlow. It works on DCG (Dynamic Computational Graph) and one can define and make changes within the model on the go. In a DCG, each block can be debugged separately, which makes training of neural networks easier. - -TensorFlow has recently come up with TensorFlow Fold, a library designed to create TensorFlow models that works on structured data. Like PyTorch, it implements the DCGs and gives massive computational speeds of up to 10x on CPU and more than 100x on GPU! With the help of [Dynamic Batching][7], you can now implement deep learning models which vary in size as well as structure. - -### Comparing GPU and CPU optimizations - -TensorFlow has faster compile times than PyTorch and provides flexibility for building real-world applications. It can run on literally any kind of processor from a CPU, GPU, TPU, mobile devices, to a Raspberry Pi (IoT Devices). - -PyTorch, on the other hand, includes Tensor computations which can speed up deep neural network models upto [50x or more][8] using GPUs. These tensors can dwell on CPU or GPU. Both CPU and GPU are written as independent libraries; making PyTorch efficient to use, irrespective of the Neural Network size. - -### Community Support - -TensorFlow is one of the most popular Deep Learning frameworks today, and with this comes a huge community support. It has great documentation, and an eloquent set of online tutorials. TensorFlow also includes numerous pre-trained models which are hosted and available on [github][9]. These models aid developers and researchers who are keen to work with TensorFlow with some ready-made material to save their time and efforts. - -PyTorch, on the other hand, has a relatively smaller community since it has been developed fairly recently. As compared to TensorFlow, the documentation isn't that great, and codes are not readily available. However, PyTorch does allow individuals to share their pre-trained models with others. - -### PyTorch and TensorFlow - A David & Goliath story - -As it stands, Tensorflow is clearly favoured and used more than PyTorch for a variety of reasons. - -Tensorflow is vast, experienced, and best suited for practical purposes. It is easily the obvious choice of most of the machine learning and deep learning experts because of the vast array of features it offers, and most importantly, its maturity in the market. It has a better community support along with multiple language APIs available. It has a good documentation and is production-ready due to the availability of ready-to-use code. Hence, it is better suited for someone who wants to get started with Deep Learning, or for organizations wanting to productize their Deep Learning models. - -Although PyTorch is relatively newer and has a smaller community, it is fast and efficient. In short, it gives you all the power of Torch wrapped in the usefulness and ease of Python. Because of its efficiency and speed, it is a good option to have for small, research based projects. As mentioned earlier, companies such as Facebook, Twitter, and many others are using Pytorch to train deep learning models. However, its adoption is yet to go mainstream. The potential is evident, PyTorch is just not ready yet to challenge the beast that is TensorFlow. However considering its growth, the day is not far when PyTorch is further optimized and offers more functionalities - to the point that it becomes the David to TensorFlow's Goliath. - -### Savia Lobo -A Data science fanatic. Loves to be updated with the tech happenings around the globe. Loves singing and composing songs. Believes in putting the art in smart. - - --------------------------------------------------------------------------------- - -via: https://datahub.packtpub.com/deep-learning/dl-wars-pytorch-vs-tensorflow/ - -作者：[Savia Lobo][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://datahub.packtpub.com/author/savial/ -[1]:https://www.packtpub.com/web-development/deep-learning-theano -[2]:https://www.packtpub.com/big-data-and-business-intelligence/deep-learning-keras -[3]:https://www.packtpub.com/big-data-and-business-intelligence/deep-learning-tensorflow -[4]:https://www.tensorflow.org/deploy/distributed -[5]:https://www.tensorflow.org/versions/r0.12/get_started/basic_usage -[6]:https://www.reddit.com/r/MachineLearning/comments/66rriz/d_rnns_are_much_faster_in_pytorch_than_tensorflow/ -[7]:https://arxiv.org/abs/1702.02181 -[8]:https://github.com/jcjohnson/pytorch-examples#pytorch-tensors -[9]:https://github.com/tensorflow/models diff --git a/sources/talk/20171128 Your API is missing Swagger.md b/sources/talk/20171128 Your API is missing Swagger.md new file mode 100644 index 0000000000..af0106a121 --- /dev/null +++ b/sources/talk/20171128 Your API is missing Swagger.md @@ -0,0 +1,56 @@ +Your API is missing Swagger +====== + +![](https://ryanmccue.ca/content/images/2017/11/top-20mobileapps--3-.png) + +We have all struggled through thrown together, convoluted API documentation. It is frustrating, and in the worst case, can lead to bad requests. The process of understanding an API is something most developers go through on a regular basis, so it is any wonder that the majority of APIs have horrific documentation. + +[Swagger][1] is the solution to this problem. Swagger came out in 2011 and is an open source software framework which has many tools that help developers design, build, document, and consume RESTful APIs. Designing an API using Swagger, or documenting it after with Swagger helps everyone consumers of your API seamlessly. One of the amazing features which many people do not know about Swagger is that you can actually **generate** a client from it! That's right, if a service you're consuming has Swagger documentation you can generate a client to consume it! + +All major languages support Swagger and connect it to your API. Depending on the language you're writing your API in you can have the Swagger documentation generated from the actual code. Here are some of the standout Swagger libraries I've seen recently. + +### Golang + +Golang has a couple great tools for integrating Swagger into your API. The first is [go-swagger][2], which is a tool that lets you generate the scaffolding for an API from a Swagger file. This is a fundamentally different way of thinking about APIs. Instead of building the endpoints and thinking about new ones on the fly, go-swagger gets you to think through your API before you write a single line of code. This can help visualize what you want the API to do first. Another tool which Golang has is called [Goa][3]. A quote from their website sums up what Goa is: + +> goa provides a novel approach for developing microservices that saves time when working on independent services and helps with keeping the overall system consistent. goa uses code generation to handle both the boilerplate and ancillary artifacts such as documentation, client modules, and client tools. + +They take designing the API before implementing it to a new level. Goa has a DSL to help you programmatically describe your entire API, from endpoints to payloads, to responses. From this DSL Goa generates a Swagger file for anyone that consumes your API, and it will enforce your endpoints output the correct data, which will keep your API and documentation in sync. This is counter-intuitive when you start, but after actually implementing an API with Goa, you will not know how you ever did it before. + +### Python + +[Flask][4] has a great extension for building an API with Swagger called [Flask-RESTPlus][5]. + +> If you are familiar with Flask, Flask-RESTPlus should be easy to pick up. It provides a coherent collection of decorators and tools to describe your API and expose its documentation properly using Swagger. + +It uses python decorators to generate swagger documentation and can be used to enforce endpoint output similar to Goa. It can be very powerful and makes generating swagger from an API stupid easy. + +### NodeJS + +Finally, NodeJS has a powerful tool for working with Swagger called [swagger-js-codegen][6]. It can generate both servers and clients from a swagger file. + +> This package generates a nodejs, reactjs or angularjs class from a swagger specification file. The code is generated using mustache templates and is quality checked by jshint and beautified by js-beautify. + +It is not quite as easy to use as Goa and Flask-RESTPlus, but if Node is your thing, this will do the job. It shines when it comes to generating frontend code to interface with your API, which is perfect if you're developing a web app to go along with the API. + +### Conclusion + +Swagger is a simple yet powerful representation of your RESTful API. When used properly it can help flush out your API design and make it easier to consume. Harnessing its full power can save you time by forming and visualizing your API before you write a line of code, then generate the boilerplate surrounding the core logic. And with tools like [Goa][3], [Flask-RESTPlus][5], and [swagger-js-codegen][6] which will make the whole experience of architecting and implementing an API painless, there is no excuse not to have Swagger. + +-------------------------------------------------------------------------------- + +via: https://ryanmccue.ca/your-api-is-missing-swagger/ + +作者：[Ryan McCue][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://ryanmccue.ca/author/ryan/ +[1]:http://swagger.io +[2]:https://github.com/go-swagger/go-swagger +[3]:https://goa.design/ +[4]:http://flask.pocoo.org/ +[5]:https://github.com/noirbizarre/flask-restplus +[6]:https://github.com/wcandillon/swagger-js-codegen diff --git a/sources/talk/20171201 5 Podcasts Every Dev Should Listen to.md b/sources/talk/20171201 5 Podcasts Every Dev Should Listen to.md new file mode 100644 index 0000000000..72586c8f35 --- /dev/null +++ b/sources/talk/20171201 5 Podcasts Every Dev Should Listen to.md @@ -0,0 +1,54 @@ +5 Podcasts Every Dev Should Listen to +====== + +![](https://ryanmccue.ca/content/images/2017/11/Electric-Love.png) + +Being a developer is a tough job, the landscape is constantly changing, and new frameworks and best practices come out every month. Having a great go-to list of podcasts keeping you up to date on the industry can make a huge difference. I've done some of the hard work and created a list of the top 5 podcasts I personally listen too. + +### This Developer's Life + +Unlike many developer-focused podcasts, there is no talk of code or explanations of software architecture in [This Developer's Life][1]. There are just relatable stories from other developers. This Developer's Life dives into the issues developers face in their daily lives, from a developers point of view. [Rob Conery][2] and [Scott Hanselman][3] host the show and it focuses on all aspects of a developers life. For example, what it feels like to get fired. To hit a home run. To be competitive. It is a very well made podcast and isn't just for developers, but it can also be enjoyed by those that love and live with them. + +### Developer Tea + +Don’t have a lot of time? [Developer Tea][4] is "A podcast for developers designed to fit inside your tea break." The podcast exists to help driven developers connect with their purpose and excel at their work so that they can make an impact. Hosted by [Jonathan Cutrell][5], the director of technology at Whiteboard, Developer Tea breaks down the news and gives useful insights into all aspects of a developers life in and out of work. Cutrell explains listener questions mixed in with news, interviews, and career advice during his show, which releases multiple episodes every week. + +### Software Engineering Today + +[Software Engineering Daily][6] is a daily podcast which focuses on heavily technical topics like software development and system architecture. It covering a range of topics from load balancing at scale and serverless event-driven architecture to augmented reality. Hosted by [Jeff Meyerson][7], this podcast is great for developers who have a passion for learning about complicated software topics to expand their knowledge base. + +### Talking Code + +The [Talking Code][8] podcast is from 2015, and contains 24 episodes which have "short expert interviews that help you decode what developers are saying." The hosts, [Josh Smith][9] and [Venkat Dinavahi][10], talk about diverse web development topics like how to become an effective junior developer and how to go from junior to senior developer, to topics like building modern web applications and making the most out of your analytics. This podcast is perfect for those getting into web development and those who look to level up their web development skills. + +### The Laracasts Snippet + +[The Laracasts Snippet][11] is a bite-size podcast where each episode offers a single thought on some aspect of web development. The host, [Jeffrey Way][12], is a prominent character in the Laravel community and runs the site [Laracasts][12]. His insights are broad and are useful for developers of all backgrounds. + +### Conclusion + +Podcasts are on the rise and more and more developers are listening to them. With such a rapidly expanding list of new podcasts coming out it can be tough to pick the top 5, but if you listen to these podcasts, you will have a competitive edge as a developer. + +-------------------------------------------------------------------------------- + +via: https://ryanmccue.ca/podcasts-every-developer-should-listen-too/ + +作者：[Ryan McCue][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://ryanmccue.ca/author/ryan/ +[1]:http://thisdeveloperslife.com/ +[2]:https://rob.conery.io/ +[3]:https://www.hanselman.com/ +[4]:https://developertea.com/ +[5]:http://jonathancutrell.com/ +[6]:https://softwareengineeringdaily.com/ +[7]:http://jeffmeyerson.com/ +[8]:http://talkingcode.com/ +[9]:https://twitter.com/joshsmith +[10]:https://twitter.com/venkatdinavahi +[11]:https://laracasts.simplecast.fm/ +[12]:https://laracasts.com diff --git a/sources/talk/20171215 Blueprint for Simple Scalable Microservices.md b/sources/talk/20171215 Blueprint for Simple Scalable Microservices.md new file mode 100644 index 0000000000..8b79458501 --- /dev/null +++ b/sources/talk/20171215 Blueprint for Simple Scalable Microservices.md @@ -0,0 +1,48 @@ +Blueprint for Simple Scalable Microservices +====== + +![](https://ryanmccue.ca/content/images/2017/12/Copy-of-Copy-of-Electric-Love--1-.png) + +When you're building a microservice, what do you value? A fully managed and scalable system? It's hard to know where to start with AWS; there are so many options for hosting code, you can use EC2, ECS, Elastic Beanstalk, Lambda. Everyone has patterns for deploying microservices. Using the pattern below will provide a great structure for a scalable microservice architecture. + +### Elastic Beanstalk + +The first and most important piece is [Elastic Beanstalk][1]. It is a great, simple way to deploy auto-scaling microservices. All you need to do is upload your code to Elastic Beanstalk via their command line tool or management console. Once it's in Elastic Beanstalk the deployment, capacity provisioning, load balancing, auto-scaling is handled by AWS. + +### S3 + +Another important service is [S3][2]; it is an object storage built to store and retrieve data. S3 has lots of uses, from storing images, to backups. Particular use cases are storing sensitive files such as private keys, environment variable files which will be accessed and used by multiple instances or services. Finally, using S3 for less sensitive, publically accessible files like configuration files, Dockerfiles, and images. + +### Kinesis + +[Kinesis][3] is a tool which allows for microservices to communicate with each other and other projects like Lambda, which we will discuss farther down. Kinesis does this by real-time, persistent data streaming, which enables microservices to emit events. Data can be persisted for up to 7 days for persistent and batch processing. + +### RDS + +[Amazon RDS][4] is a great, fully managed relational database hosted by AWS. Using RDS over your own database server is beneficial because AWS manages everything. It makes it easy to set up, operate, and scale a relational databases. + +### Lambda + +Finally, [AWS Lambda][5] lets you run code without provisioning or managing servers. Lambda has many uses; you can even create the whole APIs with it. Some great uses for it in a microservice architecture are cron jobs and image manipulation. Crons can be scheduled with [CloudWatch][6]. + +### Conclusion + +These AWS products you can create fully scalable, stateless microservices that can communicate with each other. Using Elastic Beanstalk to run microservices, S3 to store files, Kinesis to emit events and Lambdas to subscribe to them and run other tasks. Finally, RDS for easily managing and scaling relational databases. + +-------------------------------------------------------------------------------- + +via: https://ryanmccue.ca/blueprint-for-simple-scalable-microservices/ + +作者：[Ryan McCue][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://ryanmccue.ca/author/ryan/ +[1]:https://aws.amazon.com/elasticbeanstalk/?nc2=h_m1 +[2]:https://aws.amazon.com/s3/?nc2=h_m1 +[3]:https://aws.amazon.com/kinesis/?nc2=h_m1 +[4]:https://aws.amazon.com/rds/?nc2=h_m1 +[5]:https://aws.amazon.com/lambda/?nc2=h_m1 +[6]:https://aws.amazon.com/cloudwatch/?nc2=h_m1 diff --git a/sources/talk/20171223 5 Things to Look for When You Contract Out the Backend of Your App.md b/sources/talk/20171223 5 Things to Look for When You Contract Out the Backend of Your App.md new file mode 100644 index 0000000000..e7319850be --- /dev/null +++ b/sources/talk/20171223 5 Things to Look for When You Contract Out the Backend of Your App.md @@ -0,0 +1,60 @@ +5 Things to Look for When You Contract Out the Backend of Your App +====== + +![](https://ryanmccue.ca/content/images/2017/12/Copy-of-Copy-of-Copy-of-Copy-of-Copy-of-Electric-Love.png) + +For many app developers, it can be hard to know what to do when it comes to the backend of your app. There are a few options, Firebase, throw together a quick Node API, contract it out. I am going to make a blog post soon weighing the pros and cons of each of these options, but for now, let's assume you want the API done professionally. + +You are going to want to look for specific things before you give the contract to some freelancer or agency. + +### 1. Documentation + +Documentation is one of the most important pieces here, the API could be amazing, but if it is impossible to understand which endpoints are available, what parameters they provide, and what they respond with you won't have much luck integrating the API into your app. Surprisingly this is one of the pieces with most contractors get wrong. + +So what are you looking for? First, make sure they understand the importance of documentation, this alone makes a huge difference. Second, the should preferably be using an open standard like [Swagger][1] for documentation. If they do both of these things, you should have documentation covered. + +### 2. Communication + +You know the saying "communication is key," well that applies to API development. This is harder to gauge, but sometimes a developer will get the contract, and then disappear. This doesn't mean they aren't working on it, but it means there isn't a good feedback loop to sort out problems before they get too large. + +A good way to get around this is to have a weekly, or however often you want, meeting to go over progress and make sure the API is shaping up the way you want. Even if the meeting is just going over the endpoints and confirming they are returning the data you need. + +### 3. Error Handling + +Error handling is crucial, this basically means if there is an error on the backend, whether it's an invalid request or an unexpected internal server error, it will be handled properly and a useful response is given to the client. It's important that they are handled gracefully. Often this can get overlooked in the API development process. + +This is a tricky thing to look out for, but by letting them know you expect useful error messages and maybe put it into the contract, you should get the error messages you need. This may seem like a small thing but being able to present the user of your app with the actual thing they've done wrong, like "Passwords must be between 6-64 characters" improves the UX immensely. + +### 4. Database + +This section may be a bit controversial, but I think that 90% of apps really just need a SQL database. I know NoSQL is sexy, but you get so many extra benefits from using SQL I feel that's what you should use for the backend of your app. Of course, there are cases where NoSQL is the better option, but broadly speaking you should probably just use a SQL database. + +SQL adds so much added flexibility by being able to add, modify, and remove columns. The option to aggregate data with a simple query is also immensely useful. And finally, the ability to do transactions and be sure all your data is valid will help you sleep better at night. + +The reason I say all the above is because I would recommend looking for someone who is willing to build your API with a SQL database. + +### 5. Infrastructure + +The last major thing to look for when contracting out your backend is infrastructure. This is essential because you want your app to scale. If you get 10,000 users join your app in one day for some reason, you want your backend to handle that. Using services like [AWS Elastic Beanstalk][2] or [Heroku][3] you can create APIs which will scale up automatically with load. That means if your app takes off overnight your API will scale with the load and not buckle under it. + +Making sure your contractor is building it with scalability in mind is key. I wrote a [post on scalable APIs][4] if you're interested in learning more about a good AWS stack. + +### Conclusion + +It is important to get a quality backend when you contract it out. You're paying for a professional to design and build the backend of your app, so if they're lacking in any of the above points it will reduce the chance of success for but the backend, but for your app. If you make a checklist with these points and go over them with contractors, you should be able to weed out the under-qualified applicants and focus your attention on the contractors that know what they're doing. + +-------------------------------------------------------------------------------- + +via: https://ryanmccue.ca/things-to-look-for-when-you-contract-out-the-backend-your-app/ + +作者：[Ryan McCue][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://ryanmccue.ca/author/ryan/ +[1]:https://swagger.io/ +[2]:https://aws.amazon.com/elasticbeanstalk/ +[3]:https://www.heroku.com/ +[4]:https://ryanmccue.ca/blueprint-for-simple-scalable-microservices/ diff --git a/sources/talk/20171225 Where to Get Your App Backend Built.md b/sources/talk/20171225 Where to Get Your App Backend Built.md new file mode 100644 index 0000000000..35d07bac18 --- /dev/null +++ b/sources/talk/20171225 Where to Get Your App Backend Built.md @@ -0,0 +1,88 @@ +Where to Get Your App Backend Built +====== + +![](https://ryanmccue.ca/content/images/2017/12/Copy-of-Copy-of-Copy-of-Copy-of-Copy-of-Copy-of-Electric-Love.png) + +Building a great app takes lots of work. From designing the views to adding the right transitions and images. One thing which is often overlooked is the backend, connecting your app to the outside world. A backend which is not up to the same quality as your app can wreck even the most perfect user interface. That is why choosing the right option for your backend budget and needs is essential. + +There are three main choices you have when you're getting it built. First, you have agencies, they are a company with salespeople, project managers, and developers. Second, you have market rate freelancers, they are developers who charge market rate for their work and are often in North America or western Europe. Finally, there are budget freelancers, they are inexpensive and usually in parts of Asia and South America. + +I am going to break down the pros and cons of each of these options. + +### Agency + +Agencies are often a safe bet if you're looking for a more hands-off approach agencies are often the way to go, they have project managers who will manage your project and communicate your requirements to developers. This takes some of the work off of your plate and can free it up to work on your app. Agencies also often have a team of developers at their disposal, so if the developer working on your project takes a vacation, they can swap another developer in without much hassle. + +With all these upsides there is a downside. Price. Having a sales team, a project management team, and a developer team isn't cheap. Agencies often cost quite a bit of money compared to freelancers. + +So in summary: + +#### Pros + + * Hands Off + * No Single Point of Failure + + + +#### Cons + + * Very expensive + + + +### Market Rate Freelancer + +Another option you have are market rate freelancers, these are highly skilled developers who often have worked in agencies, but decided to go their own way and get clients themselves. They generally produce high-quality work at a lower cost than agencies. + +The downside to freelancers is since they're only one person they might not be available right away to start your work. Especially high demand freelancers you may have to wait a few weeks or months before they start development. They also are hard to replace, if they get sick or go on vacation, it can often be hard to find someone to continue the work, unless you get a good recommendation from the freelancer. + +#### Pros + + * Cost Effective + * Similar quality to agency + * Great for short term + + + +#### Cons + + * May not be available + * Hard to replace + + + +### Budget Freelancer + +The last option I'm going over is budget freelancers who are often found on job boards such as Fiverr and Upwork. They work for very cheap, but that often comes at the cost of quality and communication. Often you will not get what you're looking for, or it will be very brittle code which buckles under strain. + +If you're on a very tight budget, it may be worth rolling the dice on a highly rated budget freelancer, although you must be okay with the risk of potentially throwing the code away. + +#### Pros + + * Very cheap + + + +#### Cons + + * Often low quality + * May not be what you asked for + + + +### Conclusion + +Getting the right backend for your app is important. It is often a good idea to stick with agencies or market rate freelancers due to the predictability and higher quality code, but if you're on a very tight budget rolling the dice with budget freelancers could pay off. At the end of the day, it doesn't matter where the code is from, as long as it works and does what it's supposed to do. + + +-------------------------------------------------------------------------------- + +via: https://ryanmccue.ca/where-to-get-your-app-backend-built/ + +作者：[Ryan McCue][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://ryanmccue.ca/author/ryan/ diff --git a/sources/talk/20180104 4 artificial intelligence trends to watch.md b/sources/talk/20180104 4 artificial intelligence trends to watch.md deleted file mode 100644 index 9c84bba147..0000000000 --- a/sources/talk/20180104 4 artificial intelligence trends to watch.md +++ /dev/null @@ -1,56 +0,0 @@ -4 artificial intelligence trends to watch -====== - -![](https://enterprisersproject.com/sites/default/files/styles/620x350/public/images/CIO%20Mentor.png?itok=K-6s_q2C) - -However much your IT operation is using [artificial intelligence][1] today, expect to be doing more with it in 2018. Even if you have never dabbled in AI projects, this may be the year talk turns into action, says David Schatsky, managing director at [Deloitte][2]. "The number of companies doing something with AI is on track to rise," he says. - -Check out his AI predictions for the coming year: - -### 1. Expect more enterprise AI pilot projects - -Many of today's off-the-shelf applications and platforms that companies already routinely use incorporate AI. "But besides that, a growing number of companies are experimenting with machine learning or natural language processing to solve particular problems or help understand their data, or automate internal processes, or improve their own products and services," Schatsky says. - -**[ What IT jobs will be hot in the AI age? See our related article, [8 emerging AI jobs for IT pros][3]. ]** - -"Beyond that, the intensity with which companies are working with AI will rise," he says. "Companies that are early adopters already mostly have five or fewer projects underway, but we think that number will rise to having 10 or more pilots underway." One reason for this prediction, he says, is that AI technologies are getting better and easier to use. - -### 2. AI will help with data science talent crunch - -Talent is a huge problem in data science, where most large companies are struggling to hire the data scientists they need. AI can take up some of the load, Schatsky says. "The practice of data science is increasingly automatable with tools offered both by startups and large, established technology vendors," he says. A lot of data science work is repetitive and tedious, and ripe for automation, he explains. "Data scientists aren't going away, but they're going to get much more productive. So a company that can only do a few data science projects without automation will be able to do much more with automation, even if it can't hire any more data scientists." - -### 3. Synthetic data models will ease bottlenecks - -Before you can train a machine learning model, you have to get the data to train it on, Schatsky notes. That's not always easy. "That's often a business bottleneck, not a production bottleneck," he says. In some cases you can't get the data because of regulations governing things like health records and financial information. - -Synthetic data models can take a smaller set of data and use it to generate the larger set that may be needed, he says. "If you used to need 10,000 data points to train a model but could only get 2,000, you can now generate the missing 8,000 and go ahead and train your model." - -### 4. AI decision-making will become more transparent - -One of the business problems with AI is that it often operates as a black box. That is, once you train a model, it will spit out answers that you can't necessarily explain. "Machine learning can automatically discover patterns in data that a human can't see because it's too much data or too complex," Schatsky says. "Having discovered these patterns, it can make predictions about new data it hasn't seen." - -The problem is that sometimes you really do need to know the reasons behind an AI finding or prediction. "You feed in a medical image and the model says, based on the data you've given me, there's a 90 percent chance that there's a tumor in this image," Schatsky says. "You say, 'Why do you think so?' and the model says, 'I don't know, that's what the data would suggest.'" - -If you follow that data, you're going to have to do exploratory surgery on a patient, Schatsky says. That's a tough call to make when you can't explain why. "There are a lot of situations where even though the model produces very accurate results, if it can't explain how it got there, nobody wants to trust it." - -There are also situations where because of regulations, you literally can't use data that you can't explain. "If a bank declines a loan application, it needs to be able to explain why," Schatsky says. "That's a regulation, at least in the U.S. Traditionally, a human underwriter makes that call. A machine learning model could be more accurate, but if it can't explain its answer, it can't be used." - -Most algorithms were not designed to explain their reasoning. "So researchers are finding clever ways to get AI to spill its secrets and explain what variables make it more likely that this patient has a tumor," he says. "Once they do that, a human can look at the answers and see why it came to that conclusion." - -That means AI findings and decisions can be used in many areas where they can't be today, he says. "That will make these models more trustworthy and more usable in the business world." - - --------------------------------------------------------------------------------- - -via: https://enterprisersproject.com/article/2018/1/4-ai-trends-watch - -作者：[Minda Zetlin][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://enterprisersproject.com/user/minda-zetlin -[1]:https://enterprisersproject.com/tags/artificial-intelligence -[2]:https://www2.deloitte.com/us/en.html -[3]:https://enterprisersproject.com/article/2017/12/8-emerging-ai-jobs-it-pros?sc_cid=70160000000h0aXAAQ diff --git a/sources/talk/20180107 7 leadership rules for the DevOps age.md b/sources/talk/20180107 7 leadership rules for the DevOps age.md new file mode 100644 index 0000000000..cd202d5591 --- /dev/null +++ b/sources/talk/20180107 7 leadership rules for the DevOps age.md @@ -0,0 +1,124 @@ +7 leadership rules for the DevOps age +====== + +![](https://enterprisersproject.com/sites/default/files/styles/620x350/public/images/CIO_DigitalAcumen_2.png?itok=TGeMQYs4) + +If [DevOps][1] is ultimately more about culture than any particular technology or platform, then remember this: There isn't a finish line. It's about continuous change and improvement - and the C-suite doesn't get a pass. + +Rather, leaders need to [revise some of their traditional approaches][2] if they expect DevOps to help drive the outcomes they seek. Let's consider seven ideas for more effective IT leadership in the DevOps era. + +### 1. Say "yes" to failure + +The word "failure" has long had very specific connotations in IT, and they're almost universally bad: server failure, backup failure, hard drive failure - you get the picture. + +A healthy DevOps culture, however, depends upon redefining failure - IT leaders should rewrite their thesaurus to make the word synonymous with "opportunity." + +"Prior to DevOps, we had a culture of punishing failure," says Robert Reeves, CTO and co-founder of [Datical][3]. "The only learning we had was to avoid mistakes. The number one way to avoid mistakes in IT is to not change anything: Don't accelerate the release schedule, don't move to the cloud, don't do anything differently!" + +That's a playbook for a bygone era and, as Reeves puts plainly, it doesn't work. In fact, that kind of stasis is actual failure. + +"Companies that release slowly and avoid the cloud are paralyzed by fear - and they will fail," Reeves says. "IT leaders must embrace failure as an opportunity. Humans not only learn from their mistakes, they learn from others' mistakes. A culture of openness and ['psychological safety'][4] fosters learning and improvement." + +**[ Related article: [Why agile leaders must move beyond talking about "failure."][5] ]** + +### 2. Live, eat, and breathe DevOps in the C-suite + +While DevOps culture can certainly grow organically in all directions, companies that are shifting from monolithic, siloed IT practices - and likely encountering headwinds en route - need total buy-in from executive leadership. Without it, you're sending mixed messages and likely emboldening those who'd rather push a _but this is the way we 've always done things_ agenda. [Culture change is hard][6]; people need to see leadership fully invested in that change for it to actually happen. + +"Top management must fully support DevOps in order for it to be successful in delivering the benefits," says Derek Choy, CIO at [Rainforest QA][7]. + +Becoming a DevOps shop. Choy notes, touches pretty much everything in the organization, from technical teams to tools to processes to roles and responsibilities. + +"Without unified sponsorship from top management, DevOps implementation will not be successful," Choy says. "Therefore, it is important to have leaders aligned at the top level before transitioning to DevOps." + +### 3. Don 't just declare "DevOps" - define it + +Even in IT organizations that have welcomed DevOps with open arms, it's possible that's not everyone's on the same page. + +**[Read our related article,**[ **3 areas where DevOps and CIOs must get on the same page**][8] **.]** + +One fundamental reason for such disconnects: People might be operating with different definitions for what the term even means. + +"DevOps can mean different things to different people," Choy says. "It is important for C-level [and] VP-level execs to define the goals of DevOps, clearly stating the expected outcome, understand how this outcome can benefit the business and be able to measure and report on success along the way." + +Indeed, beyond the baseline definition and vision, DevOps requires ongoing and frequent communication, not just in the trenches but throughout the organization. IT leaders must make that a priority. + +"Inevitably, there will be hiccups, there will be failures and disruptions to the business," Choy says. "Leaders need to clearly communicate the journey to the rest of the company and what they can expect as part of the process." + +### 4.DevOps is as much about business as technology + +IT leaders running successful DevOps shops have embraced its culture and practices as a business strategy as much as an approach to building and operating software. DevOps culture is a great enabler of IT's shift from support arm to strategic business unit. + +"IT leaders must shift their thinking and approach from being cost/service centers to driving business outcomes, and a DevOps culture helps speed up those outcomes via automation and stronger collaboration," says Mike Kail, CTO and co-founder at [CYBRIC][9]. + +Indeed, this is a strong current that runs through much of these new "rules" for leading in the age of DevOps. + +"Promoting innovation and encouraging team members to take smart risks is a key part of a DevOps culture and IT leaders need to clearly communicate that on a continuous basis," Kail says. + +"An effective IT leader will need to be more engaged with the business than ever before," says Evan Callendar, director, performance services at [West Monroe Partners][10]. "Gone are the days of yearly or quarterly reviews - you need to welcome the [practice of] [bi-weekly backlog grooming][11]. The ability to think strategically at the year level, but interact at the sprint level, will be rewarded when business expectations are met." + +### 5. Change anything that hampers DevOps goals + + +While DevOps veterans generally agree that DevOps is much more a matter of culture than technology, success does depend on enabling that culture with the right processes and tools. Declaring your department a DevOps shop while resisting the necessary changes to processes or technologies is like buying a Ferrari but keeping the engine from your 20-year-old junker that billows smoke each time you turn the key. + +Exhibit A: [Automation][12]. It's critical parallel strategy for DevOps success. + +"IT leadership has to put an emphasis on automation," Callendar says. "This will be an upfront investment, but without it, DevOps simply will engulf itself with inefficiency and lack of delivery." + +Automation is a fundamental, but change doesn't stop there. + +"Leaders need to push for automation, monitoring, and a continuous delivery process. This usually means changes to many existing practices, processes, team structures, [and] roles," Choy says. "Leaders need to be willing to change anything that'll hinder the team's ability to fully automate the process." + +### 6. Rethink team structure and performance metrics + +While we're on the subject of change...if that org chart collecting dust on your desktop is the same one you've been plugging names into for the better part of a decade (or more), it's time for an overhaul. + +"IT executives need to take a completely different approach to organizational structure in this new era of DevOps culture," Kail says. "Remove strict team boundaries, which tend to hamper collaboration, and allow for the teams to be self-organizing and agile." + +Kail says this kind of rethinking can and should extend to other areas in the DevOps age, too, including how you measure individual and team success, and even how you interact with people. + +"Measure initiatives in terms of business outcomes and overall positive impact," Kail advises. "Finally, and something that I believe to be the most important aspect of management: Be empathetic." + +Beware easily collected measurements that are not truly DevOps metrics, writes [Red Hat ][13]technology evangelist Gordon Haff. "DevOps metrics should be tied to business outcomes in some manner," he notes. "You probably don't really care about how many lines of code your developers write, whether a server had a hardware failure overnight, or how comprehensive your test coverage is. In fact, you may not even directly care about the responsiveness of your website or the rapidity of your updates. But you do care to the degree such metrics can be correlated with customers abandoning shopping carts or leaving for a competitor." See his full article, [DevOps metrics: Are you measuring what matters?][14] + +### 7. Chuck conventional wisdom out the window + +If the DevOps age requires new ways of thinking about IT leadership, it follows that some of the old ways need to be retired. But which ones? + +"To be honest, all of them," Kail says. "Get rid of the 'because that's the way we've always done things' mindset. The transition to a culture of DevOps is a complete paradigm shift, not a few subtle changes to the old days of Waterfall and Change Advisory Boards." + +Indeed, IT leaders recognize that real transformation requires more than minor touch-ups to old approaches. Often, it requires a total reboot of a previous process or strategy. + +Callendar of West Monroe Partners shares a parting example of legacy leadership thinking that hampers DevOps: Failing to embrace hybrid IT models and modern infrastructure approaches such as containers and microservices. + +"One of the big rules I see going out the window is architecture consolidation, or the idea that long-term maintenance is cheaper if done within a homogenous environment," Callendar says. + +**Want more wisdom like this, IT leaders? [Sign up for our weekly email newsletter][15].** + +-------------------------------------------------------------------------------- + +via: https://enterprisersproject.com/article/2018/1/7-leadership-rules-devops-age + +作者：[Kevin Casey][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://enterprisersproject.com/user/kevin-casey +[1]:https://enterprisersproject.com/tags/devops +[2]:https://enterprisersproject.com/article/2017/7/devops-requires-dumping-old-it-leadership-ideas +[3]:https://www.datical.com/ +[4]:https://rework.withgoogle.com/guides/understanding-team-effectiveness/steps/foster-psychological-safety/ +[5]:https://enterprisersproject.com/article/2017/10/why-agile-leaders-must-move-beyond-talking-about-failure?sc_cid=70160000000h0aXAAQ +[6]:https://enterprisersproject.com/article/2017/10/how-beat-fear-and-loathing-it-change +[7]:https://www.rainforestqa.com/ +[8]:https://enterprisersproject.com/article/2018/1/3-areas-where-devops-and-cios-must-get-same-page +[9]:https://www.cybric.io/ +[10]:http://www.westmonroepartners.com/ +[11]:https://www.scrumalliance.org/community/articles/2017/february/product-backlog-grooming +[12]:https://www.redhat.com/en/topics/automation?intcmp=701f2000000tjyaAAA +[13]:https://www.redhat.com/en?intcmp=701f2000000tjyaAAA +[14]:https://enterprisersproject.com/article/2017/7/devops-metrics-are-you-measuring-what-matters +[15]:https://enterprisersproject.com/email-newsletter?intcmp=701f2000000tsjPAAQ diff --git a/sources/talk/20180110 8 simple ways to promote team communication.md b/sources/talk/20180110 8 simple ways to promote team communication.md deleted file mode 100644 index dc4ee37f72..0000000000 --- a/sources/talk/20180110 8 simple ways to promote team communication.md +++ /dev/null @@ -1,75 +0,0 @@ -8 simple ways to promote team communication -====== - -![](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/world_remote_teams.png?itok=Wk1yBFv6) - -Image by : opensource.com - -You might be familiar with the expression: So many tools, so little time. In order to try to save you some time, I've outlined some of my favorite tools that help agile teams work better. If you are an agilist, chances are you're aware of similar tools, but I'm specifically narrowing down the list to tools that appeal to open source enthusiasts. - -**Caution!** These tools are a little different than what you may be expecting. There are no project management apps --there is a [great article][1] on that already--so there are no checklists, no integrations with GitHub, just simple ways to organize your thoughts and promote team communication. - -### Building teams - -In an industry where most people are used to giving and receiving negative feedback, it's rare to share positive feedback with coworkers. It's not surprising--while some enjoy giving compliments, many people struggle with telling someone "way to go" or "couldn't have done this without you." But it never hurts to tell someone they're doing a good job, and it often influences people to work better for the team. Here are two tools that help you share kudos with your coworkers. - - * [Management 3.0][2] has a treasure trove of [free resources][3] for building teams. One tool we find compelling is the concept of Feedback Wraps (and not just because it inspires us to think about burritos). [Feedback Wraps][4] is a six-step process to come up with effective feedback for anyone; you might think it is designed for negative feedback, but we find it's perfect for sharing positive comments. - * [Happiness Packets][5] provides a way to share anonymous positive feedback with people in the open source community. It is especially useful for those who aren't comfortable with such a personal interaction or don't know the people they want to reward. Happiness Packets offers a [public archive][6] of comments (from people who agree to share them), so you can look through and get warm fuzzies and ideas on what to say to others if you are struggling to find your own words. As a bonus, its code of conduct process prevents anyone from sending nasty messages. - - - -### Understanding why - -Definitions are hard. In the agile world, keys to success include defining personas, the purpose of a feature, or the product vision, and ensuring the entire agile team understands why they are doing the work they are doing. We are a little disappointed by the limited number of open source tools available that help product managers and owners do their jobs. - -One that we highly respect and use frequently to teach teams at Red Hat is the Product Vision Board. It comes from product management expert Roman Pichler, who offers numerous [tools and templates][7] to help teams develop a better understanding of "the why." (Note that you will need to provide your email address to download these files.) - - * The [Product Vision Board][8] template guides teams by asking simple but effective questions to prompt them to think about what they are doing before they think about how they are going to do it. - * We also like Roman's [Product Management Test][9]. This is a simple and quick web form that guides teams through the traditional role of a product manager and helps uncover where there may be gaps. We recommend that product management teams periodically complete this test to reassess where they fall. - - - -### Visualizing work - -Have you ever been working on a huge assignment, and the steps to complete it are all jumbled up in your head, out of order, and chaotic? Yeah, us, too. Mind mapping is a technique that helps you visually organize all the thoughts in your head. You don't need to start out understanding how everything fits together--you just need your brain, a whiteboard (or a mind-mapping tool), and some time to think. - - * Our favorite open source tool in this space is [Xmind3][10]. It's available for multiple platforms (Linux, MacOS, and Windows), so you can easily share files back and forth with other people. If you need to have the latest & greatest, there is an [updated version][11], which you can download for free if you don't mind sharing your email. - * If you like more variety in your life, Eduard Lucena offers [three additional options][12] in Fedora Magazine. You can find information about these tools' availability in Fedora and other distributions on their project pages. - - * [Labyrinth][13] - * [View Your Mind][14] - * [FreeMind][15] - - - -As we wrote at the start, there are many similar tools out there; if you have a favorite open source tool that helps agile teams work better, please share it in the comments. - -### About the author -Jen Krieger - Jen Krieger is Chief Agile Architect at Red Hat. Most of her 20+ year career has been in software development representing many roles throughout the waterfall and agile lifecycles. At Red Hat, she led a department-wide DevOps movement focusing on CI/CD best practices. Most recently, she worked with with the Project Atomic & OpenShift teams. Now Jen is guiding teams across the company into agility in a way that respects and supports Red Hat's commitment to Open Source. - --------------------------------------------------------------------------------- - -via: https://opensource.com/article/18/1/foss-tools-agile-teams - -作者：[Jen Krieger][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://opensource.com/users/jkrieger -[1]:https://opensource.com/business/16/3/top-project-management-tools-2016 -[2]:https://management30.com/ -[3]:https://management30.com/leadership-resource-hub/ -[4]:https://management30.com/en/practice/feedback-wraps/ -[5]:https://happinesspackets.io/ -[6]:https://www.happinesspackets.io/archive/ -[7]:http://www.romanpichler.com/tools/ -[8]:http://www.romanpichler.com/tools/vision-board/ -[9]:http://www.romanpichler.com/tools/romans-product-management-test/ -[10]:https://sourceforge.net/projects/xmind3/?source=recommended -[11]:http://www.xmind.net/ -[12]:https://fedoramagazine.org/three-mind-mapping-tools-fedora/ -[13]:https://people.gnome.org/~dscorgie/labyrinth.html -[14]:http://www.insilmaril.de/vym/ -[15]:http://freemind.sourceforge.net/wiki/index.php/Main_Page diff --git a/sources/talk/20180110 Why isn-t open source hot among computer science students.md b/sources/talk/20180110 Why isn-t open source hot among computer science students.md deleted file mode 100644 index 282d723949..0000000000 --- a/sources/talk/20180110 Why isn-t open source hot among computer science students.md +++ /dev/null @@ -1,84 +0,0 @@ -Why isn't open source hot among computer science students? -====== - -![](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/EDU_OSDC_OpenClass_520x292_FINAL_JD.png?itok=ly78pMqu) - -Image by : opensource.com - -The technical savvy and inventive energy of young programmers is alive and well. - -This was clear from the diligent work that I witnessed while participating in this year's [PennApps][1], the nation's largest college hackathon. Over the course of 48 hours, my high school- and college-age peers created projects ranging from a [blink-based communication device for shut-in patients][2] to a [burrito maker with IoT connectivity][3]. The spirit of open source was tangible throughout the event, as diverse groups bonded over a mutual desire to build, the free flow of ideas and tech know-how, fearless experimentation and rapid prototyping, and an overwhelming eagerness to participate. - -Why then, I wondered, wasn't open source a hot topic among my tech geek peers? - -To learn more about what college students think when they hear "open source," I surveyed several college students who are members of the same professional computer science organization I belong to. All members of this community must apply during high school or college and are selected based on their computer science-specific achievements and leadership--whether that means leading a school robotics team, founding a nonprofit to bring coding into insufficiently funded classrooms, or some other worthy endeavor. Given these individuals' accomplishments in computer science, I thought that their perspectives would help in understanding what young programmers find appealing (or unappealing) about open source projects. - -The online survey I prepared and disseminated included the following questions: - - * Do you like to code personal projects? Have you ever contributed to an open source project? - * Do you feel like it's more beneficial to you to start your own programming projects, or to contribute to existing open source efforts? - * How would you compare the prestige associated with coding for an organization that produces open source software versus proprietary software? - - - -Though the overwhelming majority said that they at least occasionally enjoyed coding personal projects in their spare time, most had never contributed to an open source project. When I further explored this trend, a few common preconceptions about open source projects and organizations came to light. To persuade my peers that open source projects are worth their time, and to provide educators and open source organizations insight on their students, I'll address the three top preconceptions. - -### Preconception #1: Creating personal projects from scratch is better experience than contributing to an existing open source project. - -Of the college-age programmers I surveyed, 24 out of 26 asserted that starting their own personal projects felt potentially more beneficial than building on open source ones. - -As a bright-eyed freshman in computer science, I believed this too. I had often heard from older peers that personal projects would make me more appealing to intern recruiters. No one ever mentioned the possibility of contributing to open source projects--so in my mind, it wasn't relevant. - -I now realize that open source projects offer powerful preparation for the real world. Contributing to open source projects cultivates [an awareness of how tools and languages piece together][4] in a way that even individual projects cannot. Moreover, open source is an exercise in coordination and collaboration, building students' [professional skills in communication, teamwork, and problem-solving. ][5] - -### Preconception #2: My coding skills just won't cut it. - -A few respondents said they were intimidated by open source projects, unsure of where to contribute, or fearful of stunting project progress. Unfortunately, feelings of inferiority, which too often especially affect female programmers, do not stop at the open source community. In fact, "Imposter Syndrome" may even be magnified, as [open source advocates typically reject bureaucracy][6]--and as difficult as bureaucracy makes internal mobility, it helps newcomers know their place in an organization. - -I remember how intimidated I felt by contribution guidelines while looking through open source projects on GitHub for the first time. However, guidelines are not intended to encourage exclusivity, but to provide a [guiding hand][7]. To that end, I think of guidelines as a way of establishing expectations without relying on a hierarchical structure. - -Several open source projects actively carve a place for new project contributors. [TEAMMATES][8], an educational feedback management tool, is one of the many open source projects that marks issues "up for grabs" for first-timers. In the comments, programmers of all skill levels iron out implementation details, demonstrating that open source is a place for eager new programmers and seasoned software veterans alike. For young programmers who are still hesitant, [a few open source projects][9] have been thoughtful enough to adopt an [Imposter Syndrome disclaimer][10]. - -### Preconception #3: Proprietary software firms do better work than open source software organizations. - -Only five of the 26 respondents I surveyed thought that open and proprietary software organizations were considered equal in prestige. This is likely due to the misperception that "open" means "profitless," and thus low-quality (see [Doesn't 'open source' just mean something is free of charge?][11]). - -However, open source software and profitable software are not mutually exclusive. In fact, small and large businesses alike often pay for free open source software to receive technical support services. As [Red Hat CEO Jim Whitehurst explains][12], "We have engineering teams that track every single change--a bug fix, security enhancement, or whatever--made to Linux, and ensure our customers' mission-critical systems remain up-to-date and stable." - -Moreover, the nature of openness facilitates rather than hinders quality by enabling more people to examine source code. [Igor Faletski, CEO of Mobify][13], writes that Mobify's team of "25 software developers and quality assurance professionals" is "no match for the all the software developers in the world who might make use of [Mobify's open source] platform. Each of them is a potential tester of, or contributor to, the project." - -Another problem may be that young programmers are not aware of the open source software they interact with every day. I used many tools--including MySQL, Eclipse, Atom, Audacity, and WordPress--for months or even years without realizing they were open source. College students, who often rush to download syllabus-specified software to complete class assignments, may be unaware of which software is open source. This makes open source seem more foreign than it is. - -So students, don't knock open source before you try it. Check out this [list of beginner-friendly projects][14] and [these six starting points][15] to begin your open source journey. - -Educators, remind your students of the open source community's history of successful innovation, and lead them toward open source projects outside the classroom. You will help develop sharper, better-prepared, and more confident students. - -### About the author -Susie Choi - Susie is an undergraduate student studying computer science at Duke University. She is interested in the implications of technological innovation and open source principles for issues relating to education and socioeconomic inequality. - --------------------------------------------------------------------------------- - -via: https://opensource.com/article/17/12/students-and-open-source-3-common-preconceptions - -作者：[Susie Choi][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://opensource.com/users/susiechoi -[1]:http://pennapps.com/ -[2]:https://devpost.com/software/blink-9o2iln -[3]:https://devpost.com/software/daburrito -[4]:https://hackernoon.com/benefits-of-contributing-to-open-source-2c97b6f529e9 -[5]:https://opensource.com/education/16/8/5-reasons-student-involvement-open-source -[6]:https://opensource.com/open-organization/17/7/open-thinking-curb-bureaucracy -[7]:https://opensource.com/life/16/3/contributor-guidelines-template-and-tips -[8]:https://github.com/TEAMMATES/teammates/issues?q=is%3Aissue+is%3Aopen+label%3Ad.FirstTimers -[9]:https://github.com/adriennefriend/imposter-syndrome-disclaimer/blob/master/examples.md -[10]:https://github.com/adriennefriend/imposter-syndrome-disclaimer -[11]:https://opensource.com/resources/what-open-source -[12]:https://hbr.org/2013/01/yes-you-can-make-money-with-op -[13]:https://hbr.org/2012/10/open-sourcing-may-be-worth -[14]:https://github.com/MunGell/awesome-for-beginners -[15]:https://opensource.com/life/16/1/6-beginner-open-source diff --git a/sources/talk/20180117 How technology changes the rules for doing agile.md b/sources/talk/20180117 How technology changes the rules for doing agile.md new file mode 100644 index 0000000000..1b67935509 --- /dev/null +++ b/sources/talk/20180117 How technology changes the rules for doing agile.md @@ -0,0 +1,95 @@ +How technology changes the rules for doing agile +====== + +![](https://enterprisersproject.com/sites/default/files/styles/620x350/public/images/CIO%20Containers%20Ecosystem.png?itok=lDTaYXzk) + +More companies are trying agile and [DevOps][1] for a clear reason: Businesses want more speed and more experiments - which lead to innovations and competitive advantage. DevOps helps you gain that speed. But doing DevOps in a small group or startup and doing it at scale are two very different things. Any of us who've worked in a cross-functional group of 10 people, come up with a great solution to a problem, and then tried to apply the same patterns across a team of 100 people know the truth: It often doesn't work. This path has been so hard, in fact, that it has been easy for IT leaders to put off agile methodology for another year. + +But that time is over. If you've tried and stalled, it's time to jump back in. + +Until now, DevOps required customized answers for many organizations - lots of tweaks and elbow grease. But today, [Linux containers ][2]and Kubernetes are fueling standardization of DevOps tools and processes. That standardization will only accelerate. The technology we are using to practice the DevOps way of working has finally caught up with our desire to move faster. + +Linux containers and [Kubernetes][3] are changing the way teams interact. Moreover, on the Kubernetes platform, you can run any application you now run on Linux. What does that mean? You can run a tremendous number of enterprise apps (and handle even previously vexing coordination issues between Windows and Linux.) Finally, containers and Kubernetes will handle almost all of what you'll run tomorrow. They're being future-proofed to handle machine learning, AI, and analytics workloads - the next wave of problem-solving tools. + +**[ See our related article,[4 container adoption patterns: What you need to know. ] ][4]** + +Think about machine learning, for example. Today, people still find the patterns in much of an enterprise's data. When machines find the patterns (think machine learning), your people will be able to act on them faster. With the addition of AI, machines can not only find but also act on patterns. Today, with people doing everything, three weeks is an aggressive software development sprint cycle. With AI, machines can change code multiple times per second. Startups will use that capability - to disrupt you. + +Consider how fast you have to be to compete. If you can't make a leap of faith now to DevOps and a one week cycle, think of what will happen when that startup points its AI-fueled process at you. It's time to move to the DevOps way of working now, or get left behind as your competitors do. + +### How are containers changing how teams work? + +DevOps has frustrated many groups trying to scale this way of working to a bigger group. Many IT (and business) people are suspicious of agile: They've heard it all before - languages, frameworks, and now models (like DevOps), all promising to revolutionize application development and IT process. + +**[ Want DevOps advice from other CIOs? See our comprehensive resource, [DevOps: The IT Leader's Guide][5]. ]** + +It's not easy to "sell" quick development sprints to your stakeholders, either. Imagine if you bought a house this way. You're not going to pay a fixed amount to your builder anymore. Instead, you get something like: "We'll pour the foundation in 4 weeks and it will cost x. Then we'll frame. Then we'll do electrical. But we only know the timing on the foundation right now." People are used to buying homes with a price up front and a schedule. + +The challenge is that building software is not like building a house. The same builder builds thousands of houses that are all the same. Software projects are never the same. This is your first hurdle to get past. + +Dev and operations teams really do work differently: I know because I've worked on both sides. We incent them differently. Developers are rewarded for changing and creating, while operations pros are rewarded for reducing cost and ensuring security. We put them in different groups and generally minimize interaction. And the roles typically attract technical people who think quite differently. This situation sets IT up to fail. You have to be willing to break down these barriers. + +Think of what has traditionally happened. You throw pieces over the wall, then the business throws requirements over the wall because they are operating in "house-buying" mode: "We'll see you in 9 months." Developers build to those requirements and make changes as needed for technical constraints. Then they throw it over the wall to operations to "figure out how to run this." Operations then works diligently to make a slew of changes to align the software with their infrastructure. And what's the end result? + +More often than not, the end result isn't even recognizable to the business when they see it in its final glory. We've watched this pattern play out time and time again in our industry for the better part of two decades. It's time for a change. + +It's Linux containers that truly crack the problem - because containers close the gap between development and operations. They allow both teams to understand and design to all of the critical requirements, but still uniquely fulfill their team's responsibilities. Basically, we take out the telephone game between developers and operations. With containers, we can have smaller operations teams, even teams responsible for millions of applications, but development teams that can change software as quickly as needed. (In larger organizations, the desired pace may be faster than humans can respond on the operations side.) + +With containers, you're separating what is delivered from where it runs. Your operations teams are responsible for the host that will run the containers and the security footprint, and that's all. What does this mean? + +First, it means you can get going on DevOps now, with the team you have. That's right. Keep teams focused on the expertise they already have: With containers, just teach them the bare minimum of the required integration dependencies. + +If you try and retrain everyone, no one will be that good at anything. Containers let teams interact, but alongside a strong boundary, built around each team's strengths. Your devs know what needs to be consumed, but don't need to know how to make it run at scale. Ops teams know the core infrastructure, but don't need to know the minutiae of the app. Also, Ops teams can update apps to address new security implications, before you become the next trending data breach story. + +Teaching a large IT organization of say 30,000 people both ops and devs skills? It would take you a decade. You don't have that kind of time. + +When people talk about "building new, cloud-native apps will get us out of this problem," think critically. You can build cloud-native apps in 10-person teams, but that doesn't scale for a Fortune 1000 company. You can't just build new microservices one by one until you're somehow not reliant on your existing team: You'll end up with a siloed organization. It's an alluring idea, but you can't count on these apps to redefine your business. I haven't met a company that could fund parallel development at this scale and succeed. IT budgets are already constrained; doubling or tripling them for an extended period of time just isn't realistic. + +### When the remarkable happens: Hello, velocity + +Linux containers were made to scale. Once you start to do so, [orchestration tools like Kubernetes come into play][6] - because you'll need to run thousands of containers. Applications won't consist of just a single container, they will depend on many different pieces, all running on containers, all running as a unit. If they don't, your apps won't run well in production. + +Think of how many small gears and levers come together to run your business: The same is true for any application. Developers are responsible for all the pulleys and levers in the application. (You could have an integration nightmare if developers don't own those pieces.) At the same time, your operations team is responsible for all the pulleys and levers that make up your infrastructure, whether on-premises or in the cloud. With Kubernetes as an abstraction, your operations team can give the application the fuel it needs to run - without being experts on all those pieces. + +Developers get to experiment. The operations team keeps infrastructure secure and reliable. This combination opens up the business to take small risks that lead to innovation. Instead of having to make only a couple of bet-the-farm size bets, real experimentation happens inside the company, incrementally and quickly. + +In my experience, this is where the remarkable happens inside organizations: Because people say "How do we change planning to actually take advantage of this ability to experiment?" It forces agile planning. + +For example, KeyBank, which uses a DevOps model, containers, and Kubernetes, now deploys code every day. (Watch this [video][7] in which John Rzeszotarski, director of Continuous Delivery and Feedback at KeyBank, explains the change.) Similarly, Macquarie Bank uses DevOps and containers to put something in production every day. + +Once you push software every day, it changes every aspect of how you plan - and [accelerates the rate of change to the business][8]. "An idea can get to a customer in a day," says Luis Uguina, CDO of Macquarie's banking and financial services group. (See this [case study][9] on Red Hat's work with Macquarie Bank). + +### The right time to build something great + +The Macquarie example demonstrates the power of velocity. How would that change your approach to your business? Remember, Macquarie is not a startup. This is the type of disruptive power that CIOs face, not only from new market entrants but also from established peers. + +The developer freedom also changes the talent equation for CIOs running agile shops. Suddenly, individuals within huge companies (even those not in the hottest industries or geographies) can have great impact. Macquarie uses this dynamic as a recruiting tool, promising developers that all new hires will push something live within the first week. + +At the same time, in this day of cloud-based compute and storage power, we have more infrastructure available than ever. That's fortunate, considering the [leaps that machine learning and AI tools will soon enable][10]. + +This all adds up to this being the right time to build something great. Given the pace of innovation in the market, you need to keep building great things to keep customers loyal. So if you've been waiting to place your bet on DevOps, now is the right time. Containers and Kubernetes have changed the rules - in your favor. + +**Want more wisdom like this, IT leaders? [Sign up for our weekly email newsletter][11].** + +-------------------------------------------------------------------------------- + +via: https://enterprisersproject.com/article/2018/1/how-technology-changes-rules-doing-agile + +作者：[Matt Hicks][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://enterprisersproject.com/user/matt-hicks +[1]:https://enterprisersproject.com/tags/devops +[2]:https://www.redhat.com/en/topics/containers?intcmp=701f2000000tjyaAAA +[3]:https://www.redhat.com/en/topics/containers/what-is-kubernetes?intcmp=701f2000000tjyaAAA +[4]:https://enterprisersproject.com/article/2017/8/4-container-adoption-patterns-what-you-need-know?sc_cid=70160000000h0aXAAQ +[5]:https://enterprisersproject.com/devops?sc_cid=70160000000h0aXAAQ +[6]:https://enterprisersproject.com/article/2017/11/how-enterprise-it-uses-kubernetes-tame-container-complexity +[7]:https://www.redhat.com/en/about/videos/john-rzeszotarski-keybank-red-hat-summit-2017?intcmp=701f2000000tjyaAAA +[8]:https://enterprisersproject.com/article/2017/11/dear-cios-stop-beating-yourselves-being-behind-transformation +[9]:https://www.redhat.com/en/resources/macquarie-bank-case-study?intcmp=701f2000000tjyaAAA +[10]:https://enterprisersproject.com/article/2018/1/4-ai-trends-watch +[11]:https://enterprisersproject.com/email-newsletter?intcmp=701f2000000tsjPAAQ diff --git a/sources/talk/20180117 How to get into DevOps.md b/sources/talk/20180117 How to get into DevOps.md index 09e50ae4f2..b549c3b32e 100644 --- a/sources/talk/20180117 How to get into DevOps.md +++ b/sources/talk/20180117 How to get into DevOps.md @@ -1,3 +1,4 @@ +XLCYun 翻译中 How to get into DevOps ====== ![](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/rh_003784_02_os.comcareers_resume_rh1x.png?itok=S3HGxi6E) diff --git a/sources/talk/20180122 How to price cryptocurrencies.md b/sources/talk/20180122 How to price cryptocurrencies.md new file mode 100644 index 0000000000..19d785164c --- /dev/null +++ b/sources/talk/20180122 How to price cryptocurrencies.md @@ -0,0 +1,74 @@ +translating by wyxplus +How to price cryptocurrencies +====== + +![](https://tctechcrunch2011.files.wordpress.com/2018/01/fabian-blank-78637.jpg?w=1279&h=727&crop=1) + +Predicting cryptocurrency prices is a fool's game, yet this fool is about to try. The drivers of a single cryptocurrency's value are currently too varied and vague to make assessments based on any one point. News is trending up on Bitcoin? Maybe there's a hack or an API failure that is driving it down at the same time. Ethereum looking sluggish? Who knows: Maybe someone will build a new smarter DAO tomorrow that will draw in the big spenders. + +So how do you invest? Or, more correctly, on which currency should you bet? + +The key to understanding what to buy or sell and when to hold is to use the tools associated with assessing the value of open-source projects. This has been said again and again, but to understand the current crypto boom you have to go back to the quiet rise of Linux. + +Linux appeared on most radars during the dot-com bubble. At that time, if you wanted to set up a web server, you had to physically ship a Windows server or Sun Sparc Station to a server farm where it would do the hard work of delivering Pets.com HTML. At the same time, Linux, like a freight train running on a parallel path to Microsoft and Sun, would consistently allow developers to build one-off projects very quickly and easily using an OS and toolset that were improving daily. In comparison, then, the massive hardware and software expenditures associated with the status quo solution providers were deeply inefficient, and very quickly all of the tech giants that made their money on software now made their money on services or, like Sun, folded. + +From the acorn of Linux an open-source forest bloomed. But there was one clear problem: You couldn't make money from open source. You could consult and you could sell products that used open-source components, but early builders built primarily for the betterment of humanity and not the betterment of their bank accounts. + +Cryptocurrencies have followed the Linux model almost exactly, but cryptocurrencies have cash value. Therefore, when you're working on a crypto project you're not doing it for the common good or for the joy of writing free software. You're writing it with the expectation of a big payout. This, therefore, clouds the value judgements of many programmers. The same folks that brought you Python, PHP, Django and Node.js are back… and now they're programming money. + +### Check the codebase + +This year will be the year of great reckoning in the token sale and cryptocurrency space. While many companies have been able to get away with poor or unusable codebases, I doubt developers will let future companies get away with so much smoke and mirrors. It's safe to say we can [expect posts like this one detailing Storj's anemic codebase to become the norm][1] and, more importantly, that these commentaries will sink many so-called ICOs. Though massive, the money trough that is flowing from ICO to ICO is finite and at some point there will be greater scrutiny paid to incomplete work. + +What does this mean? It means to understand cryptocurrency you have to treat it like a startup. Does it have a good team? Does it have a good product? Does the product work? Would someone want to use it? It's far too early to assess the value of cryptocurrency as a whole, but if we assume that tokens or coins will become the way computers pay each other in the future, this lets us hand wave away a lot of doubt. After all, not many people knew in 2000 that Apache was going to beat nearly every other web server in a crowded market or that Ubuntu instances would be so common that you'd spin them up and destroy them in an instant. + +The key to understanding cryptocurrency pricing is to ignore the froth, hype and FUD and instead focus on true utility. Do you think that some day your phone will pay another phone for, say, an in-game perk? Do you expect the credit card system to fold in the face of an Internet of Value? Do you expect that one day you'll move through life splashing out small bits of value in order to make yourself more comfortable? Then by all means, buy and hold or speculate on things that you think will make your life better. If you don't expect the Internet of Value to improve your life the way the TCP/IP internet did (or you do not understand enough to hold an opinion), then you're probably not cut out for this. NASDAQ is always open, at least during banker's hours. + +Still will us? Good, here are my predictions. + +### The rundown + +Here is my assessment of what you should look at when considering an "investment" in cryptocurrencies. There are a number of caveats we must address before we begin: + + * Crypto is not a monetary investment in a real currency, but an investment in a pie-in-the-sky technofuture. That's right: When you buy crypto you're basically assuming that we'll all be on the deck of the Starship Enterprise exchanging them like Galactic Credits one day. This is the only inevitable future for crypto bulls. While you can force crypto into various economic models and hope for the best, the entire platform is techno-utopianist and assumes all sorts of exciting and unlikely things will come to pass in the next few years. If you have spare cash lying around and you like Star Wars, then you're golden. If you bought bitcoin on a credit card because your cousin told you to, then you're probably going to have a bad time. + * Don't trust anyone. There is no guarantee and, in addition to offering the disclaimer that this is not investment advice and that this is in no way an endorsement of any particular cryptocurrency or even the concept in general, we must understand that everything I write here could be wrong. In fact, everything ever written about crypto could be wrong, and anyone who is trying to sell you a token with exciting upside is almost certainly wrong. In short, everyone is wrong and everyone is out to get you, so be very, very careful. + * You might as well hold. If you bought when BTC was $18,000 you'd best just hold on. Right now you're in Pascal's Wager territory. Yes, maybe you're angry at crypto for screwing you, but maybe you were just stupid and you got in too high and now you might as well keep believing because nothing is certain, or you can admit that you were a bit overeager and now you're being punished for it but that there is some sort of bitcoin god out there watching over you. Ultimately you need to take a deep breath, agree that all of this is pretty freaking weird, and hold on. + + + +Now on with the assessments. + +**Bitcoin** - Expect a rise over the next year that will surpass the current low. Also expect [bumps as the SEC and other federal agencies][2] around the world begin regulating the buying and selling of cryptocurrencies in very real ways. Now that banks are in on the joke they're going to want to reduce risk. Therefore, the bitcoin will become digital gold, a staid, boring and volatility proof safe haven for speculators. Although all but unusable as a real currency, it's good enough for what we need it to do and we also can expect quantum computing hardware to change the face of the oldest and most familiar cryptocurrency. + +**Ethereum** - Ethereum could sustain another few thousand dollars on its price as long as Vitalik Buterin, the creator, doesn't throw too much cold water on it. Like a remorseful Victor Frankenstein, Buterin tends to make amazing things and then denigrate them online, a sort of self-flagellation that is actually quite useful in a space full of froth and outright lies. Ethereum is the closest we've come to a useful cryptocurrency, but it is still the Raspberry Pi of distributed computing -- it's a useful and clever hack that makes it easy to experiment but no one has quite replaced the old systems with new distributed data stores or applications. In short, it's a really exciting technology, but nobody knows what to do with it. + +![][3] + +Where will the price go? It will hover around $1,000 and possibly go as high as $1,500 this year, but this is a principled tech project and not a store of value. + +**Altcoins** - One of the signs of a bubble is when average people make statements like "I couldn't afford a Bitcoin so I bought a Litecoin." This is exactly what I've heard multiple times from multiple people and it's akin to saying "I couldn't buy hamburger so I bought a pound of sawdust instead. I think the kids will eat it, right?" Play at your own risk. Altcoins are a very useful low-risk play for many, and if you create an algorithm -- say to sell when the asset hits a certain level -- then you could make a nice profit. Further, most altcoins will not disappear overnight. I would honestly recommend playing with Ethereum instead of altcoins, but if you're dead set on it, then by all means, enjoy. + +**Tokens** - This is where cryptocurrency gets interesting. Tokens require research, education and a deep understanding of technology to truly assess. Many of the tokens I've seen are true crapshoots and are used primarily as pump and dump vehicles. I won't name names, but the rule of thumb is that if you're buying a token on an open market then you've probably already missed out. The value of the token sale as of January 2018 is to allow crypto whales to turn a few cent per token investment into a 100X return. While many founders talk about the magic of their product and the power of their team, token sales are quite simply vehicles to turn 4 cents into 20 cents into a dollar. Multiply that by millions of tokens and you see the draw. + +The answer is simple: find a few projects you like and lurk in their message boards. Assess if the team is competent and figure out how to get in very, very early. Also expect your money to disappear into a rat hole in a few months or years. There are no sure things, and tokens are far too bleeding-edge a technology to assess sanely. + +You are reading this post because you are looking to maintain confirmation bias in a confusing space. That's fine. I've spoken to enough crypto-heads to know that nobody knows anything right now and that collusion and dirty dealings are the rule of the day. Therefore, it's up to folks like us to slowly buy surely begin to understand just what's going on and, perhaps, profit from it. At the very least we'll all get a new Linux of Value when we're all done. + + + +-------------------------------------------------------------------------------- + +via: https://techcrunch.com/2018/01/22/how-to-price-cryptocurrencies/ + +作者：[John Biggs][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://techcrunch.com/author/john-biggs/ +[1]:https://shitcoin.com/storj-not-a-dropbox-killer-1a9f27983d70 +[2]:http://www.businessinsider.com/bitcoin-price-cryptocurrency-warning-from-sec-cftc-2018-1 +[3]:https://tctechcrunch2011.files.wordpress.com/2018/01/vitalik-twitter-1312.png?w=525&h=615 +[4]:https://unsplash.com/photos/pElSkGRA2NU?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText +[5]:https://unsplash.com/search/photos/cash?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText diff --git a/sources/talk/20180122 Raspberry Pi Alternatives.md b/sources/talk/20180122 Raspberry Pi Alternatives.md new file mode 100644 index 0000000000..bf3bca4f61 --- /dev/null +++ b/sources/talk/20180122 Raspberry Pi Alternatives.md @@ -0,0 +1,58 @@ +Raspberry Pi Alternatives +====== +A look at some of the many interesting Raspberry Pi competitors. + +The phenomenon behind the Raspberry Pi computer series has been pretty amazing. It's obvious why it has become so popular for Linux projects—it's a low-cost computer that's actually quite capable for the price, and the GPIO pins allow you to use it in a number of electronics projects such that it starts to cross over into Arduino territory in some cases. Its overall popularity has spawned many different add-ons and accessories, not to mention step-by-step guides on how to use the platform. I've personally written about Raspberry Pis often in this space, and in my own home, I use one to control a beer fermentation fridge, one as my media PC, one to control my 3D printer and one as a handheld gaming device. + +The popularity of the Raspberry Pi also has spawned competition, and there are all kinds of other small, low-cost, Linux-powered Raspberry Pi-like computers for sale—many of which even go so far as to add "Pi" to their names. These computers aren't just clones, however. Although some share a similar form factor to the Raspberry Pi, and many also copy the GPIO pinouts, in many cases, these other computers offer features unavailable in a traditional Raspberry Pi. Some boards offer SATA, Wi-Fi or Gigabit networking; others offer USB3, and still others offer higher-performance CPUs or more RAM. When you are choosing a low-power computer for a project or as a home server, it pays to be aware of these Raspberry Pi alternatives, as in many cases, they will perform much better. So in this article, I discuss some alternatives to Raspberry Pis that I've used personally, their pros and cons, and then provide some examples of where they work best. + +### Banana Pi + +I've mentioned the Banana Pi before in past articles (see "Papa's Got a Brand New NAS" in the September 2016 issue and "Banana Backups" in the September 2017 issue), and it's a great choice when you want a board with a similar form factor, similar CPU and RAM specs, and a similar price (~$30) to a Raspberry Pi but need faster I/O. The Raspberry Pi product line is used for a lot of home server projects, but it limits you to 10/100 networking and a USB2 port for additional storage. Where the Banana Pi product line really shines is in the fact that it includes both a Gigabit network port and SATA port, while still having similar GPIO expansion options and running around the same price as a Raspberry Pi. + +Before I settled on an Odroid XU4 for my home NAS (more on that later), I first experimented with a cluster of Banana Pis. The idea was to attach a SATA disk to each Banana Pi and use software like Ceph or GlusterFS to create a storage cluster shared over the network. Even though any individual Banana Pi wasn't necessarily that fast, considering how cheap they are in aggregate, they should be able to perform reasonably well and allow you to expand your storage by adding another disk and another Banana Pi. In the end, I decided to go a more traditional and simpler route with a single server and software RAID, and now I use one Banana Pi as an image gallery server. I attached a 2.5" laptop SATA drive to the other and use it as a local backup server running BackupPC. It's a nice solution that takes up almost no space and little power to run. + +### Orange Pi Zero + +I was really excited when I first heard about the Raspberry Pi Zero project. I couldn't believe there was such a capable little computer for only $5, and I started imagining all of the cool projects I could use one for around the house. That initial excitement was dampened a bit by the fact that they sold out quickly, and just about every vendor settled into the same pattern: put standalone Raspberry Pi Zeros on backorder but have special $20 starter kits in stock that include various adapter cables, a micro SD card and a plastic case that I didn't need. More than a year after the release, the situation still remains largely the same. Although I did get one Pi Zero and used it for a cool Adafruit "Pi Grrl Zero" gaming project, I had to put the rest of my ideas on hold, because they just never seemed to be in stock when I wanted them. + +The Orange Pi Zero was created by the same company that makes the entire line of Orange Pi computers that compete with the Raspberry Pi. The main thing that makes the Orange Pi Zero shine in my mind is that they have a small, square form factor that is wider than a Raspberry Pi Zero but not as long. It also includes a Wi-Fi card like the more expensive Raspberry Pi Zero W, and it runs between $6 and $9, depending on whether you opt for 256MB of RAM or 512MB of RAM. More important, they are generally in stock, so there's no need to sit on a backorder list when you have a fun project in mind. + +The Orange Pi Zero boards themselves are pretty capable. Out of the box, they include a quad-core ARM CPU, Wi-Fi (as I mentioned before), along with a 10/100 network port and USB2\. They also include Raspberry-Pi-compatible GPIO pins, but even more interesting is that there is a $9 "NAS" expansion board for it that mounts to its 13-pin header and provides extra USB2 ports, a SATA and mSATA port, along with an IR and audio and video ports, which makes it about as capable as a more expensive Banana Pi board. Even without the expansion board, this would make a nice computer you could sit anywhere within range of your Wi-Fi and run any number of services. The main downside is you are limited to composite video, so this isn't the best choice for gaming or video-based projects. + +Although Orange Pi Zeros are capable boards in their own right, what makes them particularly enticing to me is that they are actually available when you want them, unlike some of the other sub-$10 boards out there. There's nothing worse than having a cool idea for a cheap home project and then having to wait for a board to come off backorder. + +![](http://www.linuxjournal.com/files/linuxjournal.com/ufiles/imagecache/large-550px-centered/u1000009/12261f1.jpg) + +Figure 1\. An Orange Pi Zero (right) and an Espressobin (left) + +### Odroid XU4 + +When I was looking to replace my rack-mounted NAS at home, I first looked at all of the Raspberry Pi options, including Banana Pi and other alternatives, but none of them seemed to have quite enough horsepower for my needs. I needed a machine that not only offered Gigabit networking to act as a NAS, but one that had high-speed disk I/O as well. The Odroid XU4 fit the bill with its eight-core ARM CPU, 2GB RAM, Gigabit network and USB3 ports. Although it was around $75 (almost twice the price of a Raspberry Pi), it was a much more capable computer all while being small and low-power. + +The entire Odroid product line is a good one to consider if you want a low-power home server but need more resources than a traditional Raspberry Pi can offer and are willing to spend a little bit extra for the privilege. In addition to a NAS, the Odroid XU4, with its more powerful CPU and extra RAM, is a good all-around server for the home. The USB3 port means you have a lot of storage options should you need them. + +### Espressobin + +Although the Odroid XU4 is a great home server, I still sometimes can see that it gets bogged down in disk and network I/O compared to a traditional higher-powered server. Some of this might be due to the chips that were selected for the board, and perhaps some of it has to do with the fact that I'm using both disk encryption and software RAID over USB3\. In either case, I started looking for another option to help take a bit of the storage burden off this server, and I came across the Espressobin board. + +The Espressobin is a $50 board that launched as a popular Indiegogo campaign and is now a shipping product that you can pick up in a number of places, including Amazon. Although it costs a bit more than a Raspberry Pi 3, it includes a 64-bit dual-core ARM Cortex A53 at 1.2GHz, 1–2Gb of RAM (depending on the configuration), three Gigabit network ports with a built-in switch, a SATA port, a USB3 port, a mini-PCIe port, plus a number of other options, including two sets of GPIO headers and a nice built-in serial console running on the micro-USB port. + +The main benefit to the Espressobin is the fact that it was designed by Marvell with chips that actually can use all of the bandwidth that the board touts. In some other boards, often you'll find a SATA2 port that's hanging off a USB2 interface or other architectural hacks that, although they will let you connect a SATA disk or Gigabit networking port, it doesn't mean you'll get the full bandwidth the spec claims. Although I intend to have my own Espressobin take over home NAS duties, it also would make a great home gateway router, general-purpose server or even a Wi-Fi access point, provided you added the right Wi-Fi card. + +### Conclusion + +A whole world of alternatives to Raspberry Pis exists—this list covers only some of the ones I've used myself. I hope it has encouraged you to think twice before you default to a Raspberry Pi for your next project. Although there's certainly nothing wrong with Raspberry Pis, there are several small computers that run Linux well and, in many cases, offer better hardware or other expansion options beyond the capabilities of a Raspberry Pi for a similar price. + + +-------------------------------------------------------------------------------- + +via: http://www.linuxjournal.com/content/raspberry-pi-alternatives + +作者：[Kyle Rankin][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://www.linuxjournal.com/users/kyle-rankin diff --git a/sources/talk/20180123 Moving to Linux from dated Windows machines.md b/sources/talk/20180123 Moving to Linux from dated Windows machines.md new file mode 100644 index 0000000000..74bf66df68 --- /dev/null +++ b/sources/talk/20180123 Moving to Linux from dated Windows machines.md @@ -0,0 +1,51 @@ +translating by leowang +Moving to Linux from dated Windows machines +====== + +![](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/1980s-computer-yearbook.png?itok=eGOYEKK-) + +Every day, while working in the marketing department at ONLYOFFICE, I see Linux users discussing our office productivity software on the internet. Our products are popular among Linux users, which made me curious about using Linux as an everyday work tool. My old Windows XP-powered computer was an obstacle to performance, so I started reading about Linux systems (particularly Ubuntu) and decided to try it out as an experiment. Two of my colleagues joined me. + +### Why Linux? + +We needed to make a change, first, because our old systems were not enough in terms of performance: we experienced regular crashes, an overload every time more than two apps were active, a 50% chance of freezing when a machine was shut down, and so forth. This was rather distracting to our work, which meant we were considerably less efficient than we could be. + +Upgrading to newer versions of Windows was an option, too, but that is an additional expense, plus our software competes against Microsoft's office suite. So that was an ideological question, too. + +Second, as I mentioned earlier, ONLYOFFICE products are rather popular within the Linux community. By reading about Linux users' experience with our software, we became interested in joining them. + +A week after we asked to change to Linux, we got our shiny new computer cases with [Kubuntu][1] inside. We chose version 16.04, which features KDE Plasma 5.5 and many KDE apps including Dolphin, as well as LibreOffice 5.1 and Firefox 45. + +### What we like about Linux + +Linux's biggest advantage, I believe, is its speed; for instance, it takes just seconds from pushing the machine's On button to starting your work. Everything seemed amazingly rapid from the very beginning: the overall responsiveness, the graphics, and even system updates. + +One other thing that surprised me compared to Windows is that Linux allows you to configure nearly everything, including the entire look of your desktop. In Settings, I found how to change the color and shape of bars, buttons, and fonts; relocate any desktop element; and build a composition of widgets, even including comics and Color Picker. I believe I've barely scratched the surface of the available options and have yet to explore most of the customization opportunities that this system is well known for. + +Linux distributions are generally a very safe environment. People rarely use antivirus apps in Linux, simply because there are so few viruses written for it. You save system speed, time, and, sure enough, money. + +In general, Linux has refreshed our everyday work lives, surprising us with a number of new options and opportunities. Even in the short time we've been using it, we'd characterize it as: + + * Fast and smooth to operate + * Highly customizable + * Relatively newcomer-friendly + * Challenging with basic components, however very rewarding in return + * Safe and secure + * An exciting experience for everyone who seeks to refresh their workplace + + + +Have you switched from Windows or MacOS to Kubuntu or another Linux variant? Or are you considering making the change? Please share your reasons for wanting to adopt Linux, as well as your impressions of going open source, in the comments. + +-------------------------------------------------------------------------------- + +via: https://opensource.com/article/18/1/move-to-linux-old-windows + +作者：[Michael Korotaev][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://opensource.com/users/michaelk +[1]:https://kubuntu.org/ diff --git a/sources/talk/20180124 Containers, the GPL, and copyleft- No reason for concern.md b/sources/talk/20180124 Containers, the GPL, and copyleft- No reason for concern.md new file mode 100644 index 0000000000..814598f7cf --- /dev/null +++ b/sources/talk/20180124 Containers, the GPL, and copyleft- No reason for concern.md @@ -0,0 +1,44 @@ +Containers, the GPL, and copyleft: No reason for concern +====== + +![](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/LAW_patents4abstract_B.png?itok=6RHeRaYh) + +Though open source is thoroughly mainstream, new software technologies and old technologies that get newly popularized sometimes inspire hand-wringing about open source licenses. Most often the concern is about the GNU General Public License (GPL), and specifically the scope of its copyleft requirement, which is often described (somewhat misleadingly) as the GPL's derivative work issue. + +One imperfect way of framing the question is whether GPL-licensed code, when combined in some sense with proprietary code, forms a single modified work such that the proprietary code could be interpreted as being subject to the terms of the GPL. While we haven't yet seen much of that concern directed to Linux containers, we expect more questions to be raised as adoption of containers continues to grow. But it's fairly straightforward to show that containers do _not_ raise new or concerning GPL scope issues. + +Statutes and case law provide little help in interpreting a license like the GPL. On the other hand, many of us give significant weight to the interpretive views of the Free Software Foundation (FSF), the drafter and steward of the GPL, even in the typical case where the FSF is not a copyright holder of the software at issue. In addition to being the author of the license text, the FSF has been engaged for many years in providing commentary and guidance on its licenses to the community. Its views have special credibility and influence based on its public interest mission and leadership in free software policy. + +The FSF's existing guidance on GPL interpretation has relevance for understanding the effects of including GPL and non-GPL code in containers. The FSF has placed emphasis on the process boundary when considering copyleft scope, and on the mechanism and semantics of the communication between multiple software components to determine whether they are closely integrated enough to be considered a single program for GPL purposes. For example, the [GNU Licenses FAQ][1] takes the view that pipes, sockets, and command-line arguments are mechanisms that are normally suggestive of separateness (in the absence of sufficiently "intimate" communications). + +Consider the case of a container in which both GPL code and proprietary code might coexist and execute. A container is, in essence, an isolated userspace stack. In the [OCI container image format][2], code is packaged as a set of filesystem changeset layers, with the base layer normally being a stripped-down conventional Linux distribution without a kernel. As with the userspace of non-containerized Linux distributions, these base layers invariably contain many GPL-licensed packages (both GPLv2 and GPLv3), as well as packages under licenses considered GPL-incompatible, and commonly function as a runtime for proprietary as well as open source applications. The ["mere aggregation" clause][3] in GPLv2 (as well as its counterpart GPLv3 provision on ["aggregates"][4]) shows that this type of combination is generally acceptable, is specifically contemplated under the GPL, and has no effect on the licensing of the two programs, assuming incompatibly licensed components are separate and independent. + +Of course, in a given situation, the relationship between two components may not be "mere aggregation," but the same is true of software running in non-containerized userspace on a Linux system. There is nothing in the technical makeup of containers or container images that suggests a need to apply a special form of copyleft scope analysis. + +It follows that when looking at the relationship between code running in a container and code running outside a container, the "separate and independent" criterion is almost certainly met. The code will run as separate processes, and the whole technical point of using containers is isolation from other software running on the system. + +Now consider the case where two components, one GPL-licensed and one proprietary, are running in separate but potentially interacting containers, perhaps as part of an application designed with a [microservices][5] architecture. In the absence of very unusual facts, we should not expect to see copyleft scope extending across multiple containers. Separate containers involve separate processes. Communication between containers by way of network interfaces is analogous to such mechanisms as pipes and sockets, and a multi-container microservices scenario would seem to preclude what the FSF calls "[intimate][6]" communication by definition. The composition of an application using multiple containers may not be dispositive of the GPL scope issue, but it makes the technical boundaries between the components more apparent and provides a strong basis for arguing separateness. Here, too, there is no technical feature of containers that suggests application of a different and stricter approach to copyleft scope analysis. + +A company that is overly concerned with the potential effects of distributing GPL-licensed code might attempt to prohibit its developers from adding any such code to a container image that it plans to distribute. Insofar as the aim is to avoid distributing code under the GPL, this is a dubious strategy. As noted above, the base layers of conventional container images will contain multiple GPL-licensed components. If the company pushes a container image to a registry, there is normally no way it can guarantee that this will not include the base layer, even if it is widely shared. + +On the other hand, the company might decide to embrace containerization as a means of limiting copyleft scope issues by isolating GPL and proprietary code--though one would hope that technical benefits would drive the decision, rather than legal concerns likely based on unfounded anxiety about the GPL. While in a non-containerized setting the relationship between two interacting software components will often be mere aggregation, the evidence of separateness that containers provide may be comforting to those who worry about GPL scope. + +Open source license compliance obligations may arise when sharing container images. But there's nothing technically different or unique about containers that changes the nature of these obligations or makes them harder to satisfy. With respect to copyleft scope, containerization should, if anything, ease the concerns of the extra-cautious. + +-------------------------------------------------------------------------------- + +via: https://opensource.com/article/18/1/containers-gpl-and-copyleft + +作者：[Richard Fontana][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://opensource.com/users/fontana +[1]:https://www.gnu.org/licenses/gpl-faq.en.html#MereAggregation +[2]:https://github.com/opencontainers/image-spec/blob/master/spec.md +[3]:https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html#section2 +[4]:https://www.gnu.org/licenses/gpl.html#section5 +[5]:https://www.redhat.com/en/topics/microservices +[6]:https://www.gnu.org/licenses/gpl-faq.en.html#GPLPlugins diff --git a/sources/talk/20180124 Security Chaos Engineering- A new paradigm for cybersecurity.md b/sources/talk/20180124 Security Chaos Engineering- A new paradigm for cybersecurity.md new file mode 100644 index 0000000000..35c89150c8 --- /dev/null +++ b/sources/talk/20180124 Security Chaos Engineering- A new paradigm for cybersecurity.md @@ -0,0 +1,87 @@ +Security Chaos Engineering: A new paradigm for cybersecurity +====== +![](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/life_bank_vault_secure_safe.png?itok=YoW93h7C) + +Security is always changing and failure always exists. + +This toxic scenario requires a fresh perspective on how we think about operational security. We must understand that we are often the primary cause of our own security flaws. The industry typically looks at cybersecurity and failure in isolation or as separate matters. We believe that our lack of insight and operational intelligence into our own security control failures is one of the most common causes of security incidents and, subsequently, data breaches. + +> Fall seven times, stand up eight." --Japanese proverb + +The simple fact is that "to err is human," and humans derive their success as a direct result of the failures they encounter. Their rate of failure, how they fail, and their ability to understand that they failed in the first place are important building blocks to success. Our ability to learn through failure is inherent in the systems we build, the way we operate them, and the security we use to protect them. Yet there has been a lack of focus when it comes to how we approach preventative security measures, and the spotlight has trended toward the evolving attack landscape and the need to buy or build new solutions. + +### Security spending is continually rising and so are security incidents + +We spend billions on new information security technologies, however, we rarely take a proactive look at whether those security investments perform as expected. This has resulted in a continual increase in security spending on new solutions to keep up with the evolving attacks. + +Despite spending more on security, data breaches are continuously getting bigger and more frequent across all industries. We have marched so fast down this path of the "get-ahead-of-the-attacker" strategy that we haven't considered that we may be a primary cause of our own demise. How is it that we are building more and more security measures, but the problem seems to be getting worse? Furthermore, many of the notable data breaches over the past year were not the result of an advanced nation-state or spy-vs.-spy malicious advanced persistent threats (APTs); rather the principal causes of those events were incomplete implementation, misconfiguration, design flaws, and lack of oversight. + +The 2017 Ponemon Cost of a Data Breach Study breaks down the [root causes of data breaches][1] into three areas: malicious or criminal attacks, human factors or errors, and system glitches, including both IT and business-process failure. Of the three categories, malicious or criminal attacks comprises the largest distribution (47%), followed by human error (28%), and system glitches (25%). Cybersecurity vendors have historically focused on malicious root causes of data breaches, as it is the largest sole cause, but together human error and system glitches total 53%, a larger share of the overall problem. + +What is not often understood, whether due to lack of insight, reporting, or analysis, is that malicious or criminal attacks are often successful due to human error and system glitches. Both human error and system glitches are, at their root, primary markers of the existence of failure. Whether it's IT system failures, failures in process, or failures resulting from humans, it begs the question: "Should we be focusing on finding a method to identify, understand, and address our failures?" After all, it can be an arduous task to predict the next malicious attack, which often requires investment of time to sift threat intelligence, dig through forensic data, or churn threat feeds full of unknown factors and undetermined motives. Failure instrumentation, identification, and remediation are mostly comprised of things that we know, have the ability to test, and can measure. + +Failures we can analyze consist not only of IT, business, and general human factors but also the way we design, build, implement, configure, operate, observe, and manage security controls. People are the ones designing, building, monitoring, and managing the security controls we put in place to defend against malicious attackers. How often do we proactively instrument what we designed, built, and are operationally managing to determine if the controls are failing? Most organizations do not discover that their security controls were failing until a security incident results from that failure. The worst time to find out your security investment failed is during a security incident at 3 a.m. + +> Security incidents are not detective measures and hope is not a strategy when it comes to operating effective security controls. + +We hypothesize that a large portion of data breaches are caused not by sophisticated nation-state actors or hacktivists, but rather simple things rooted in human error and system glitches. Failure in security controls can arise from poor control placement, technical misconfiguration, gaps in coverage, inadequate testing practices, human error, and numerous other things. + +### The journey into Security Chaos Testing + +Our venture into this new territory of Security Chaos Testing has shifted our thinking about the root cause of many of our notable security incidents and data breaches. + +We were brought together by [Bruce Wong][2], who now works at Stitch Fix with Charles, one of the authors of this article. Prior to Stitch Fix, Bruce was a founder of the Chaos Engineering and System Reliability Engineering (SRE) practices at Netflix, the company commonly credited with establishing the field. Bruce learned about this article's other author, Aaron, through the open source [ChaoSlingr][3] Security Chaos Testing tool project, on which Aaron was a contributor. Aaron was interested in Bruce's perspective on the idea of applying Chaos Engineering to cybersecurity, which led Bruce to connect us to share what we had been working on. As security practitioners, we were both intrigued by the idea of Chaos Engineering and had each begun thinking about how this new method of instrumentation might have a role in cybersecurity. + +Within a short timeframe, we began finishing each other's thoughts around testing and validating security capabilities, which we collectively call "Security Chaos Engineering." We directly challenged many of the concepts we had come to depend on in our careers, such as compensating security controls, defense-in-depth, and how to design preventative security. Quickly we realized that we needed to challenge the status quo "set-it-and-forget-it" model and instead execute on continuous instrumentation and validation of security capabilities. + +Businesses often don't fully understand whether their security capabilities and controls are operating as expected until they are not. We had both struggled throughout our careers to provide measurements on security controls that go beyond simple uptime metrics. Our journey has shown us there is a need for a more pragmatic approach that emphasizes proactive instrumentation and experimentation over blind faith. + +### Defining new terms + +In the security industry, we have a habit of not explaining terms and assuming we are speaking the same language. To correct that, here are a few key terms in this new approach: + + * **(Security) Chaos Experiments** are foundationally rooted in the scientific method, in that they seek not to validate what is already known to be true or already known to be false, rather they are focused on deriving new insights about the current state. + * **Security Chaos Engineering** is the discipline of instrumentation, identification, and remediation of failure within security controls through proactive experimentation to build confidence in the system's ability to defend against malicious conditions in production. + + + +### Security and distributed systems + +Consider the evolving nature of modern application design where systems are becoming more and more distributed, ephemeral, and immutable in how they operate. In this shifting paradigm, it is becoming difficult to comprehend the operational state and health of our systems' security. Moreover, how are we ensuring that it remains effective and vigilant as the surrounding environment is changing its parameters, components, and methodologies? + +What does it mean to be effective in terms of security controls? After all, a single security capability could easily be implemented in a wide variety of diverse scenarios in which failure may arise from many possible sources. For example, a standard firewall technology may be implemented, placed, managed, and configured differently depending on complexities in the business, web, and data logic. + +It is imperative that we not operate our business products and services on the assumption that something works. We must constantly, consistently, and proactively instrument our security controls to ensure they cut the mustard when it matters. This is why Security Chaos Testing is so important. What Security Chaos Engineering does is it provides a methodology for the experimentation of the security of distributed systems in order to build confidence in the ability to withstand malicious conditions. + +In Security Chaos Engineering: + + * Security capabilities must be end-to-end instrumented. + * Security must be continuously instrumented to build confidence in the system's ability to withstand malicious conditions. + * Readiness of a system's security defenses must be proactively assessed to ensure they are battle-ready and operating as intended. + * The security capability toolchain must be instrumented from end to end to drive new insights into not only the effectiveness of the functionality within the toolchain but also to discover where added value and improvement can be injected. + * Practiced instrumentation seeks to identify, detect, and remediate failures in security controls. + * The focus is on vulnerability and failure identification, not failure management. + * The operational effectiveness of incident management is sharpened. + + + +As Henry Ford said, "Failure is only the opportunity to begin again, this time more intelligently." Security Chaos Engineering and Security Chaos Testing give us that opportunity. + +Would you like to learn more? Join the discussion by following [@aaronrinehart][4] and [@charles_nwatu][5] on Twitter. + +-------------------------------------------------------------------------------- + +via: https://opensource.com/article/18/1/new-paradigm-cybersecurity + +作者：[Aaron Rinehart][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://opensource.com/users/aaronrinehart +[1]:https://www.ibm.com/security/data-breach +[2]:https://twitter.com/bruce_m_wong?lang=en +[3]:https://github.com/Optum/ChaoSlingr +[4]:https://twitter.com/aaronrinehart +[5]:https://twitter.com/charles_nwatu diff --git a/sources/talk/20180127 Write Dumb Code.md b/sources/talk/20180127 Write Dumb Code.md new file mode 100644 index 0000000000..acc647b0e5 --- /dev/null +++ b/sources/talk/20180127 Write Dumb Code.md @@ -0,0 +1,54 @@ +Write Dumb Code +====== +The best way you can contribute to an open source project is to remove lines of code from it. We should endeavor to write code that a novice programmer can easily understand without explanation or that a maintainer can understand without significant time investment. + +As students we attempt increasingly challenging problems with increasingly sophisticated technologies. We first learn loops, then functions, then classes, etc.. We are praised as we ascend this hierarchy, writing longer programs with more advanced technology. We learn that experienced programmers use monads while new programmers use for loops. + +Then we graduate and find a job or open source project to work on with others. We search for something that we can add, and implement a solution pridefully, using the all the tricks that we learned in school. + +Ah ha! I can extend this project to do X! And I can use inheritance here! Excellent! + +We implement this feature and feel accomplished, and with good reason. Programming in real systems is no small accomplishment. This was certainly my experience. I was excited to write code and proud that I could show off all of the things that I knew how to do to the world. As evidence of my historical love of programming technology, here is a [linear algebra language][1] built with a another meta-programming language. Notice that no one has touched this code in several years. + +However after maintaining code a bit more I now think somewhat differently. + + 1. We should not seek to build software. Software is the currency that we pay to solve problems, which is our actual goal. We should endeavor to build as little software as possible to solve our problems. + 2. We should use technologies that are as simple as possible, so that as many people as possible can use and extend them without needing to understand our advanced techniques. We should use advanced techniques only when we are not smart enough to figure out how to use more common techniques. + + + +Neither of these points are novel. Most people I meet agree with them to some extent, but somehow we forget them when we go to contribute to a new project. The instinct to contribute by building and to demonstrate sophistication often take over. + +### Software is a cost + +Every line that you write costs people time. It costs you time to write it of course, but you are willing to make this personal sacrifice. However this code also costs the reviewers their time to understand it. It costs future maintainers and developers their time as they fix and modify your code. They could be spending this time outside in the sunshine or with their family. + +So when you add code to a project you should feel meek. It should feel as though you are eating with your family and there isn't enough food on the table. You should take only what you need and no more. The people with you will respect you for your efforts to restrict yourself. Solving problems with less code is a hard, but it is a burden that you take on yourself to lighten the burdens of others. + +### Complex technologies are harder to maintain + +As students, we demonstrate merit by using increasingly advanced technologies. Our measure of worth depends on our ability to use functions, then classes, then higher order functions, then monads, etc. in public projects. We show off our solutions to our peers and feel pride or shame according to our sophistication. + +However when working with a team to solve problems in the world the situation is reversed. Now we strive to solve problems with code that is as simple as possible. When we solve a problem simply we enable junior programmers to extend our solution to solve other problems. Simple code enables others and boosts our impact. We demonstrate our value by solving hard problems with only basic techniques. + +Look! I replaced this recursive function with a for loop and it still does everything that we need it to. I know it's not as clever, but I noticed that the interns were having trouble with it and I thought that this change might help. + +If you are a good programmer then you don't need to demonstrate that you know cool tricks. Instead, you can demonstrate your value by solving a problem in a simple way that enables everyone on your team to contribute in the future. + +### But moderation, of course + +That being said, over-adherence to the "build things with simple tools" dogma can be counter productive. Often a recursive solution can be much simpler than a for-loop solution and often times using a Class or a Monad is the right approach. But we should be mindful when using these technologies that we are building for ourselves our own system; a system with which others have had no experience. + + +-------------------------------------------------------------------------------- + +via: http://matthewrocklin.com/blog/work/2018/01/27/write-dumb-code + +作者：[Matthew Rocklin][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://matthewrocklin.com +[1]:https://github.com/mrocklin/matrix-algebra diff --git a/sources/talk/20180128 Being open about data privacy.md b/sources/talk/20180128 Being open about data privacy.md new file mode 100644 index 0000000000..2deb88922b --- /dev/null +++ b/sources/talk/20180128 Being open about data privacy.md @@ -0,0 +1,107 @@ +Being open about data privacy +====== +![](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/GOV_opendata.png?itok=M8L2HGVx) + + +Image by : opensource.com + +Today is [Data Privacy Day][1], ("Data Protection Day" in Europe), and you might think that those of us in the open source world should think that all data should be free, [as information supposedly wants to be][2], but life's not that simple. That's for two main reasons: + + 1. Most of us (and not just in open source) believe there's at least some data about us that we might not feel happy sharing (I compiled an example list in [a post][3] I published a while ago). + 2. Many of us working in open source actually work for commercial companies or other organisations subject to legal requirements around what they can share. + + + +So actually, data privacy is something that's important for pretty much everybody. + +It turns out that the starting point for what data people and governments believe should be available for organisations to use is somewhat different between the U.S. and Europe, with the former generally providing more latitude for entities--particularly, the more cynical might suggest, large commercial entities--to use data they've collected about us as they will. Europe, on the other hand, has historically taken a more restrictive view, and on the 25th of May, Europe's view arguably will have triumphed. + +### The impact of GDPR + +That's a rather sweeping statement, but the fact remains that this is the date on which a piece of legislation called the General Data Protection Regulation (GDPR), enacted by the European Union in 2016, becomes enforceable. The GDPR basically provides a stringent set of rules about how personal data can be stored, what it can be used for, who can see it, and how long it can be kept. It also describes what personal data is--and it's a pretty broad set of items, from your name and home address to your medical records and on through to your computer's IP address. + +What is important about the GDPR, though, is that it doesn't apply just to European companies, but to any organisation processing data about EU citizens. If you're an Argentinian, Japanese, U.S., or Russian company and you're collecting data about an EU citizen, you're subject to it. + +"Pah!" you may say,1 "I'm not based in the EU: what can they do to me?" The answer is simple: If you want to continue doing any business in the EU, you'd better comply, because if you breach GDPR rules, you could be liable for up to four percent of your global revenues. Yes, that's global revenues: not just revenues in a particular country in Europe or across the EU, not just profits, but global revenues. Those are the sorts of numbers that should lead you to talk to your legal team, who will direct you to your exec team, who will almost immediately direct you to your IT group to make sure you're compliant in pretty short order. + +This may seem like it's not particularly relevant to non-EU citizens, but it is. For most companies, it's going to be simpler and more efficient to implement the same protection measures for data associated with all customers, partners, and employees they deal with, rather than just targeting specific measures at EU citizens. This has got to be a good thing.2 + +However, just because GDPR will soon be applied to organisations across the globe doesn't mean that everything's fine and dandy3: it's not. We give away information about ourselves all the time--and permission for companies to use it. + +There's a telling (though disputed) saying: "If you're not paying, you're the product." What this suggests is that if you're not paying for a service, then somebody else is paying to use your data. Do you pay to use Facebook? Twitter? Gmail? How do you think they make their money? Well, partly through advertising, and some might argue that's a service they provide to you, but actually that's them using your data to get money from the advertisers. You're not really a customer of advertising--it's only once you buy something from the advertiser that you become their customer, but until you do, the relationship is between the the owner of the advertising platform and the advertiser. + +Some of these services allow you to pay to reduce or remove advertising (Spotify is a good example), but on the other hand, advertising may be enabled even for services that you think you do pay for (Amazon is apparently working to allow adverts via Alexa, for instance). Unless we want to start paying to use all of these "free" services, we need to be aware of what we're giving up, and making some choices about what we expose and what we don't. + +### Who's the customer? + +There's another issue around data that should be exercising us, and it's a direct consequence of the amounts of data that are being generated. There are many organisations out there--including "public" ones like universities, hospitals, or government departments4--who generate enormous quantities of data all the time, and who just don't have the capacity to store it. It would be a different matter if this data didn't have long-term value, but it does, as the tools for handling Big Data are developing, and organisations are realising they can be mining this now and in the future. + +The problem they face, though, as the amount of data increases and their capacity to store it fails to keep up, is what to do with it. Luckily--and I use this word with a very heavy dose of irony,5 big corporations are stepping in to help them. "Give us your data," they say, "and we'll host it for free. We'll even let you use the data you collected when you want to!" Sounds like a great deal, yes? A fantastic example of big corporations6 taking a philanthropic stance and helping out public organisations that have collected all of that lovely data about us. + +Sadly, philanthropy isn't the only reason. These hosting deals come with a price: in exchange for agreeing to host the data, these corporations get to sell access to it to third parties. And do you think the public organisations, or those whose data is collected, will get a say in who these third parties are or how they will use it? I'll leave this as an exercise for the reader.7 + +### Open and positive + +It's not all bad news, however. There's a growing "open data" movement among governments to encourage departments to make much of their data available to the public and other bodies for free. In some cases, this is being specifically legislated. Many voluntary organisations--particularly those receiving public funding--are starting to do the same. There are glimmerings of interest even from commercial organisations. What's more, there are techniques becoming available, such as those around differential privacy and multi-party computation, that are beginning to allow us to mine data across data sets without revealing too much about individuals--a computing problem that has historically been much less tractable than you might otherwise expect. + +What does this all mean to us? Well, I've written before on Opensource.com about the [commonwealth of open source][4], and I'm increasingly convinced that we need to look beyond just software to other areas: hardware, organisations, and, relevant to this discussion, data. Let's imagine that you're a company (A) that provides a service to another company, a customer (B).8 There are four different types of data in play: + + 1. Data that's fully open: visible to A, B, and the rest of the world + 2. Data that's known, shared, and confidential: visible to A and B, but nobody else + 3. Data that's company-confidential: visible to A, but not B + 4. Data that's customer-confidential: visible to B, but not A + + + +First of all, maybe we should be a bit more open about data and default to putting it into bucket 1. That data--on self-driving cars, voice recognition, mineral deposits, demographic statistics--could be enormously useful if it were available to everyone.9 Also, wouldn't it be great if we could find ways to make the data in buckets 2, 3, and 4--or at least some of it--available in bucket 1, whilst still keeping the details confidential? That's the hope for some of these new techniques being researched. They're a way off, though, so don't get too excited, and in the meantime, start thinking about making more of your data open by default. + +### Some concrete steps + +So, what can we do around data privacy and being open? Here are a few concrete steps that occurred to me: please use the comments to contribute more. + + * Check to see whether your organisation is taking GDPR seriously. If it isn't, push for it. + * Default to encrypting sensitive data (or hashing where appropriate), and deleting when it's no longer required--there's really no excuse for data to be in the clear to these days except for when it's actually being processed. + * Consider what information you disclose when you sign up to services, particularly social media. + * Discuss this with your non-technical friends. + * Educate your children, your friends' children, and their friends. Better yet, go and talk to their teachers about it and present something in their schools. + * Encourage the organisations you work for, volunteer for, or interact with to make data open by default. Rather than thinking, "why should I make this public?" start with "why shouldn't I make this public?" + * Try accessing some of the open data sources out there. Mine it, create apps that use it, perform statistical analyses, draw pretty graphs,10 make interesting music, but consider doing something with it. Tell the organisations that sourced it, thank them, and encourage them to do more. + + + +1. Though you probably won't, I admit. + +2. Assuming that you believe that your personal data should be protected. + +3. If you're wondering what "dandy" means, you're not alone at this point. + +4. Exactly how public these institutions seem to you will probably depend on where you live: [YMMV][5]. + +5. And given that I'm British, that's a really very, very heavy dose. + +6. And they're likely to be big corporations: nobody else can afford all of that storage and the infrastructure to keep it available. + +7. No. The answer's "no." + +8. Although the example works for people, too. Oh, look: A could be Alice, B could be Bob… + +9. Not that we should be exposing personal data or data that actually needs to be confidential, of course--not that type of data. + +10. A friend of mine decided that it always seemed to rain when she picked her children up from school, so to avoid confirmation bias, she accessed rainfall information across the school year and created graphs that she shared on social media. + +-------------------------------------------------------------------------------- + +via: https://opensource.com/article/18/1/being-open-about-data-privacy + +作者：[Mike Bursell][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://opensource.com/users/mikecamel +[1]:https://en.wikipedia.org/wiki/Data_Privacy_Day +[2]:https://en.wikipedia.org/wiki/Information_wants_to_be_free +[3]:https://aliceevebob.wordpress.com/2017/06/06/helping-our-governments-differently/ +[4]:https://opensource.com/article/17/11/commonwealth-open-source +[5]:http://www.outpost9.com/reference/jargon/jargon_40.html#TAG2036 diff --git a/sources/talk/20180131 An old DOS BBS in a Docker container.md b/sources/talk/20180131 An old DOS BBS in a Docker container.md new file mode 100644 index 0000000000..769b5db1bd --- /dev/null +++ b/sources/talk/20180131 An old DOS BBS in a Docker container.md @@ -0,0 +1,51 @@ +An old DOS BBS in a Docker container +====== +Awhile back, I wrote about [my Debian Docker base images][1]. I decided to extend this concept a bit further: to running DOS applications in Docker. + +But first, a screenshot: + +![][2] + +It turns out this is possible, but difficult. I went through all three major DOS emulators available (dosbox, qemu, and dosemu). I got them all running inside the Docker container, but had a number of, er, fun issues to resolve. + +The general thing one has to do here is present a fake modem to the DOS environment. This needs to be exposed outside the container as a TCP port. That much is possible in various ways -- I wound up using tcpser. dosbox had a TCP modem interface, but it turned out to be too buggy for this purpose. + +The challenge comes in where you want to be able to accept more than one incoming telnet (or TCP) connection at a time. DOS was not a multitasking operating system, so there were any number of hackish solutions back then. One might have had multiple physical computers, one for each incoming phone line. Or they might have run multiple pseudo-DOS instances under a multitasking layer like [DESQview][3], OS/2, or even Windows 3.1. + +(Side note: I just learned of [DESQview/X][4], which integrated DESQview with X11R5 and [replaced the Windows 3 drivers][5] to allow running Windows as an X application). + +For various reasons, I didn't want to try running one of those systems inside Docker. That left me with emulating the original multiple physical node setup. In theory, pretty easy -- spin up a bunch of DOS boxes, each using at most 1MB of emulated RAM, and go to town. But here came the challenge. + +In a multiple-physical-node setup, you need some sort of file sharing, because your nodes have to access the shared message and file store. There were a myriad of clunky ways to do this in the old DOS days - [Netware][6], [LAN manager][7], even some PC NFS clients. I didn't have access to Netware. I tried the Microsoft LM client in DOS, talking to a Samba server running inside the Docker container. This I got working, but the LM client used so much RAM that, even with various high memory tricks, BBS software wasn't going to run. I couldn't just mount an underlying filesystem in multiple dosbox instances either, because dosbox did caching that wasn't going to be compatible. + +This is why I wound up using dosemu. Besides being a more complete emulator than dosbox, it had a way of sharing the host's filesystems that was going to work. + +So, all of this wound up with this: [jgoerzen/docker-bbs-renegade][8]. + +I also prepared building blocks for others that want to do something similar: [docker-dos-bbs][9] and the lower-level [docker-dosemu][10]. + +As a side bonus, I also attempted running this under Joyent's Triton (SmartOS, Solaris-based). I was pleasantly impressed that I got it all almost working there. So yes, a Renegade DOS BBS running under a Linux-based DOS emulator in a container on a Solaris machine. + + + +-------------------------------------------------------------------------------- + +via: http://changelog.complete.org/archives/9836-an-old-dos-bbs-in-a-docker-container + +作者：[John Goerzen][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://changelog.complete.org/archives/author/jgoerzen +[1]:https://changelog.complete.org/archives/9794-fixing-the-problems-with-docker-images +[2]:https://raw.githubusercontent.com/jgoerzen/docker-bbs-renegade/master/renegade-login.png +[3]:https://en.wikipedia.org/wiki/DESQview +[4]:http://toastytech.com/guis/dvx.html +[5]:http://toastytech.com/guis/dvx3.html +[6]:https://en.wikipedia.org/wiki/NetWare +[7]:https://en.wikipedia.org/wiki/LAN_Manager +[8]:https://github.com/jgoerzen/docker-bbs-renegade +[9]:https://github.com/jgoerzen/docker-dos-bbs +[10]:https://github.com/jgoerzen/docker-dosemu diff --git a/sources/talk/20180201 6 pivotal moments in open source history.md b/sources/talk/20180201 6 pivotal moments in open source history.md new file mode 100644 index 0000000000..2d0f80dd1f --- /dev/null +++ b/sources/talk/20180201 6 pivotal moments in open source history.md @@ -0,0 +1,94 @@ +6 pivotal moments in open source history +============================================================ + +### Here's how open source developed from a printer jam solution at MIT to a major development model in the tech industry today. + +![6 pivotal moments in open source history](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/welcome-open-sign-door-osdc-lead.png?itok=i9jCnaiu "6 pivotal moments in open source history") +Image credits : [Alan Levine][4]. [CC0 1.0][5] + +Open source has taken a prominent role in the IT industry today. It is everywhere from the smallest embedded systems to the biggest supercomputer, from the phone in your pocket to the software running the websites and infrastructure of the companies we engage with every day. Let's explore how we got here and discuss key moments from the past 40 years that have paved a path to the current day. + +### 1\. RMS and the printer + +In the late 1970s, [Richard M. Stallman (RMS)][6] was a staff programmer at MIT. His department, like those at many universities at the time, shared a PDP-10 computer and a single printer. One problem they encountered was that paper would regularly jam in the printer, causing a string of print jobs to pile up in a queue until someone fixed the jam. To get around this problem, the MIT staff came up with a nice social hack: They wrote code for the printer driver so that when it jammed, a message would be sent to everyone who was currently waiting for a print job: "The printer is jammed, please fix it." This way, it was never stuck for long. + +In 1980, the lab accepted a donation of a brand-new laser printer. When Stallman asked for the source code for the printer driver, however, so he could reimplement the social hack to have the system notify users on a paper jam, he was told that this was proprietary information. He heard of a researcher in a different university who had the source code for a research project, and when the opportunity arose, he asked this colleague to share it—and was shocked when they refused. They had signed an NDA, which Stallman took as a betrayal of the hacker culture. + +The late '70s and early '80s represented an era where software, which had traditionally been given away with the hardware in source code form, was seen to be valuable. Increasingly, MIT researchers were starting software companies, and selling licenses to the software was key to their business models. NDAs and proprietary software licenses became the norms, and the best programmers were hired from universities like MIT to work on private development projects where they could no longer share or collaborate. + +As a reaction to this, Stallman resolved that he would create a complete operating system that would not deprive users of the freedom to understand how it worked, and would allow them to make changes if they wished. It was the birth of the free software movement. + +### 2\. Creation of GNU and the advent of free software + +By late 1983, Stallman was ready to announce his project and recruit supporters and helpers. In September 1983, [he announced the creation of the GNU project][7] (GNU stands for GNU's Not Unix—a recursive acronym). The goal of the project was to clone the Unix operating system to create a system that would give complete freedom to users. + +In January 1984, he started working full-time on the project, first creating a compiler system (GCC) and various operating system utilities. Early in 1985, he published "[The GNU Manifesto][8]," which was a call to arms for programmers to join the effort, and launched the Free Software Foundation in order to accept donations to support the work. This document is the founding charter of the free software movement. + +### 3\. The writing of the GPL + +Until 1989, software written and released by the [Free Software Foundation][9] and RMS did not have a single license. Emacs was released under the Emacs license, GCC was released under the GCC license, and so on; however, after a company called Unipress forced Stallman to stop distributing copies of an Emacs implementation they had acquired from James Gosling (of Java fame), he felt that a license to secure user freedoms was important. + +The first version of the GNU General Public License was released in 1989, and it encapsulated the values of copyleft (a play on words—what is the opposite of copyright?): You may use, copy, distribute, and modify the software covered by the license, but if you make changes, you must share the modified source code alongside the modified binaries. This simple requirement to share modified software, in combination with the advent of the internet in the 1990s, is what enabled the decentralized, collaborative development model of the free software movement to flourish. + +### 4\. "The Cathedral and the Bazaar" + +By the mid-1990s, Linux was starting to take off, and free software had become more mainstream—or perhaps "less fringe" would be more accurate. The Linux kernel was being developed in a way that was completely different to anything people had been seen before, and was very successful doing it. Out of the chaos of the kernel community came order, and a fast-moving project. + +In 1997, Eric S. Raymond published the seminal essay, "[The Cathedral and the Bazaar][10]," comparing and contrasting the development methodologies and social structure of GCC and the Linux kernel and talking about his own experiences with a "bazaar" development model with the Fetchmail project. Many of the principles that Raymond describes in this essay will later become central to agile development and the DevOps movement—"release early, release often," refactoring of code, and treating users as co-developers are all fundamental to modern software development. + +This essay has been credited with bringing free software to a broader audience, and with convincing executives at software companies at the time that releasing their software under a free software license was the right thing to do. Raymond went on to be instrumental in the coining of the term "open source" and the creation of the Open Source Institute. + +"The Cathedral and the Bazaar" was credited as a key document in the 1998 release of the source code for the Netscape web browser Mozilla. At the time, this was the first major release of an existing, widely used piece of desktop software as free software, which brought it further into the public eye. + +### 5\. Open source + +As far back as 1985, the ambiguous nature of the word "free", used to describe software freedom, was identified as problematic by RMS himself. In the GNU Manifesto, he identified "give away" and "for free" as terms that confused zero price and user freedom. "Free as in freedom," "Speech not beer," and similar mantras were common when free software hit a mainstream audience in the late 1990s, but a number of prominent community figures argued that a term was needed that made the concept more accessible to the general public. + +After Netscape released the source code for Mozilla in 1998 (see #4), a group of people, including Eric Raymond, Bruce Perens, Michael Tiemann, Jon "Maddog" Hall, and many of the leading lights of the free software world, gathered in Palo Alto to discuss an alternative term. The term "open source" was [coined by Christine Peterson][11] to describe free software, and the Open Source Institute was later founded by Bruce Perens and Eric Raymond. The fundamental difference with proprietary software, they argued, was the availability of the source code, and so this was what should be put forward first in the branding. + +Later that year, at a summit organized by Tim O'Reilly, an extended group of some of the most influential people in the free software world at the time gathered to debate various new brands for free software. In the end, "open source" edged out "sourceware," and open source began to be adopted by many projects in the community. + +There was some disagreement, however. Richard Stallman and the Free Software Foundation continued to champion the term "free software," because to them, the fundamental difference with proprietary software was user freedom, and the availability of source code was just a means to that end. Stallman argued that removing the focus on freedom would lead to a future where source code would be available, but the user of the software would not be able to avail of the freedom to modify the software. With the advent of web-deployed software-as-a-service and open source firmware embedded in devices, the battle continues to be waged today. + +### 6\. Corporate investment in open source—VA Linux, Red Hat, IBM + +In the late 1990s, a series of high-profile events led to a huge increase in the professionalization of free and open source software. Among these, the highest-profile events were the IPOs of VA Linux and Red Hat in 1999\. Both companies had massive gains in share price on their opening days as publicly traded companies, proving that open source was now going commercial and mainstream. + +Also in 1999, IBM announced that they were supporting Linux by investing $1 billion in its development, making is less risky to traditional enterprise users. The following year, Sun Microsystems released the source code to its cross-platform office suite, StarOffice, and created the [OpenOffice.org][12] project. + +The combined effect of massive Silicon Valley funding of open source projects, the attention of Wall Street for young companies built around open source software, and the market credibility that tech giants like IBM and Sun Microsystems brought had combined to create the massive adoption of open source, and the embrace of the open development model that helped it thrive have led to the dominance of Linux and open source in the tech industry today. + + _Which pivotal moments would you add to the list? Let us know in the comments._ + +### About the author + + [![](https://opensource.com/sites/default/files/styles/profile_pictures/public/pictures/picture-11423-8ecef7f357341aaa7aee8b43e9b530c9.png?itok=n1snBFq3)][13] Dave Neary - Dave Neary is a member of the Open Source and Standards team at Red Hat, helping make Open Source projects important to Red Hat be successful. Dave has been around the free and open source software world, wearing many different hats, since sending his first patch to the GIMP in 1999.[More about me][2] + +-------------------------------------------------------------------------------- + +via: https://opensource.com/article/18/2/pivotal-moments-history-open-source + +作者：[Dave Neary ][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://opensource.com/users/dneary +[1]:https://opensource.com/article/18/2/pivotal-moments-history-open-source?rate=gsG-JrjfROWACP7i9KUoqmH14JDff8-31C2IlNPPyu8 +[2]:https://opensource.com/users/dneary +[3]:https://opensource.com/user/16681/feed +[4]:https://www.flickr.com/photos/cogdog/6476689463/in/photolist-aSjJ8H-qHAvo4-54QttY-ofm5ZJ-9NnUjX-tFxS7Y-bPPjtH-hPYow-bCndCk-6NpFvF-5yQ1xv-7EWMXZ-48RAjB-5EzYo3-qAFAdk-9gGty4-a2BBgY-bJsTcF-pWXATc-6EBTmq-SkBnSJ-57QJco-ddn815-cqt5qG-ddmYSc-pkYxRz-awf3n2-Rvnoxa-iEMfeG-bVfq5-jXy74D-meCC1v-qx22rx-fMScsJ-ci1435-ie8P5-oUSXhp-xJSm9-bHgApk-mX7ggz-bpsxd7-8ukud7-aEDmBj-qWkytq-ofwhdM-b7zSeD-ddn5G7-ddn5gb-qCxnB2-S74vsk +[5]:https://creativecommons.org/publicdomain/zero/1.0/ +[6]:https://en.wikipedia.org/wiki/Richard_Stallman +[7]:https://groups.google.com/forum/#!original/net.unix-wizards/8twfRPM79u0/1xlglzrWrU0J +[8]:https://www.gnu.org/gnu/manifesto.en.html +[9]:https://www.fsf.org/ +[10]:https://en.wikipedia.org/wiki/The_Cathedral_and_the_Bazaar +[11]:https://opensource.com/article/18/2/coining-term-open-source-software +[12]:http://www.openoffice.org/ +[13]:https://opensource.com/users/dneary +[14]:https://opensource.com/users/dneary +[15]:https://opensource.com/users/dneary +[16]:https://opensource.com/article/18/2/pivotal-moments-history-open-source#comments +[17]:https://opensource.com/tags/licensing \ No newline at end of file diff --git a/sources/talk/20180201 How I coined the term open source.md b/sources/talk/20180201 How I coined the term open source.md new file mode 100644 index 0000000000..5af8d69de7 --- /dev/null +++ b/sources/talk/20180201 How I coined the term open source.md @@ -0,0 +1,103 @@ +How I coined the term 'open source' +============================================================ + +### Christine Peterson finally publishes her account of that fateful day, 20 years ago. + +![How I coined the term 'open source'](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/hello-name-sticker-badge-tag.png?itok=fAgbMgBb "How I coined the term 'open source'") +Image by : opensource.com + +In a few days, on February 3, the 20th anniversary of the introduction of the term "[open source software][6]" is upon us. As open source software grows in popularity and powers some of the most robust and important innovations of our time, we reflect on its rise to prominence. + +I am the originator of the term "open source software" and came up with it while executive director at Foresight Institute. Not a software developer like the rest, I thank Linux programmer Todd Anderson for supporting the term and proposing it to the group. + +This is my account of how I came up with it, how it was proposed, and the subsequent reactions. Of course, there are a number of accounts of the coining of the term, for example by Eric Raymond and Richard Stallman, yet this is mine, written on January 2, 2006. + +It has never been published, until today. + +* * * + +The introduction of the term "open source software" was a deliberate effort to make this field of endeavor more understandable to newcomers and to business, which was viewed as necessary to its spread to a broader community of users. The problem with the main earlier label, "free software," was not its political connotations, but that—to newcomers—its seeming focus on price is distracting. A term was needed that focuses on the key issue of source code and that does not immediately confuse those new to the concept. The first term that came along at the right time and fulfilled these requirements was rapidly adopted: open source. + +This term had long been used in an "intelligence" (i.e., spying) context, but to my knowledge, use of the term with respect to software prior to 1998 has not been confirmed. The account below describes how the term [open source software][7] caught on and became the name of both an industry and a movement. + +### Meetings on computer security + +In late 1997, weekly meetings were being held at Foresight Institute to discuss computer security. Foresight is a nonprofit think tank focused on nanotechnology and artificial intelligence, and software security is regarded as central to the reliability and security of both. We had identified free software as a promising approach to improving software security and reliability and were looking for ways to promote it. Interest in free software was starting to grow outside the programming community, and it was increasingly clear that an opportunity was coming to change the world. However, just how to do this was unclear, and we were groping for strategies. + +At these meetings, we discussed the need for a new term due to the confusion factor. The argument was as follows: those new to the term "free software" assume it is referring to the price. Oldtimers must then launch into an explanation, usually given as follows: "We mean free as in freedom, not free as in beer." At this point, a discussion on software has turned into one about the price of an alcoholic beverage. The problem was not that explaining the meaning is impossible—the problem was that the name for an important idea should not be so confusing to newcomers. A clearer term was needed. No political issues were raised regarding the free software term; the issue was its lack of clarity to those new to the concept. + +### Releasing Netscape + +On February 2, 1998, Eric Raymond arrived on a visit to work with Netscape on the plan to release the browser code under a free-software-style license. We held a meeting that night at Foresight's office in Los Altos to strategize and refine our message. In addition to Eric and me, active participants included Brian Behlendorf, Michael Tiemann, Todd Anderson, Mark S. Miller, and Ka-Ping Yee. But at that meeting, the field was still described as free software or, by Brian, "source code available" software. + +While in town, Eric used Foresight as a base of operations. At one point during his visit, he was called to the phone to talk with a couple of Netscape legal and/or marketing staff. When he was finished, I asked to be put on the phone with them—one man and one woman, perhaps Mitchell Baker—so I could bring up the need for a new term. They agreed in principle immediately, but no specific term was agreed upon. + +Between meetings that week, I was still focused on the need for a better name and came up with the term "open source software." While not ideal, it struck me as good enough. I ran it by at least four others: Eric Drexler, Mark Miller, and Todd Anderson liked it, while a friend in marketing and public relations felt the term "open" had been overused and abused and believed we could do better. He was right in theory; however, I didn't have a better idea, so I thought I would try to go ahead and introduce it. In hindsight, I should have simply proposed it to Eric Raymond, but I didn't know him well at the time, so I took an indirect strategy instead. + +Todd had agreed strongly about the need for a new term and offered to assist in getting the term introduced. This was helpful because, as a non-programmer, my influence within the free software community was weak. My work in nanotechnology education at Foresight was a plus, but not enough for me to be taken very seriously on free software questions. As a Linux programmer, Todd would be listened to more closely. + +### The key meeting + +Later that week, on February 5, 1998, a group was assembled at VA Research to brainstorm on strategy. Attending—in addition to Eric Raymond, Todd, and me—were Larry Augustin, Sam Ockman, and attending by phone, Jon "maddog" Hall. + +The primary topic was promotion strategy, especially which companies to approach. I said little, but was looking for an opportunity to introduce the proposed term. I felt that it wouldn't work for me to just blurt out, "All you technical people should start using my new term." Most of those attending didn't know me, and for all I knew, they might not even agree that a new term was greatly needed, or even somewhat desirable. + +Fortunately, Todd was on the ball. Instead of making an assertion that the community should use this specific new term, he did something less directive—a smart thing to do with this community of strong-willed individuals. He simply used the term in a sentence on another topic—just dropped it into the conversation to see what happened. I went on alert, hoping for a response, but there was none at first. The discussion continued on the original topic. It seemed only he and I had noticed the usage. + +Not so—memetic evolution was in action. A few minutes later, one of the others used the term, evidently without noticing, still discussing a topic other than terminology. Todd and I looked at each other out of the corners of our eyes to check: yes, we had both noticed what happened. I was excited—it might work! But I kept quiet: I still had low status in this group. Probably some were wondering why Eric had invited me at all. + +Toward the end of the meeting, the [question of terminology][8] was brought up explicitly, probably by Todd or Eric. Maddog mentioned "freely distributable" as an earlier term, and "cooperatively developed" as a newer term. Eric listed "free software," "open source," and "sourceware" as the main options. Todd advocated the "open source" model, and Eric endorsed this. I didn't say much, letting Todd and Eric pull the (loose, informal) consensus together around the open source name. It was clear that to most of those at the meeting, the name change was not the most important thing discussed there; a relatively minor issue. Only about 10% of my notes from this meeting are on the terminology question. + +But I was elated. These were some key leaders in the community, and they liked the new name, or at least didn't object. This was a very good sign. There was probably not much more I could do to help; Eric Raymond was far better positioned to spread the new meme, and he did. Bruce Perens signed on to the effort immediately, helping set up [Opensource.org][9] and playing a key role in spreading the new term. + +For the name to succeed, it was necessary, or at least highly desirable, that Tim O'Reilly agree and actively use it in his many projects on behalf of the community. Also helpful would be use of the term in the upcoming official release of the Netscape Navigator code. By late February, both O'Reilly & Associates and Netscape had started to use the term. + +### Getting the name out + +After this, there was a period during which the term was promoted by Eric Raymond to the media, by Tim O'Reilly to business, and by both to the programming community. It seemed to spread very quickly. + +On April 7, 1998, Tim O'Reilly held a meeting of key leaders in the field. Announced in advance as the first "[Freeware Summit][10]," by April 14 it was referred to as the first "[Open Source Summit][11]." + +These months were extremely exciting for open source. Every week, it seemed, a new company announced plans to participate. Reading Slashdot became a necessity, even for those like me who were only peripherally involved. I strongly believe that the new term was helpful in enabling this rapid spread into business, which then enabled wider use by the public. + +A quick Google search indicates that "open source" appears more often than "free software," but there still is substantial use of the free software term, which remains useful and should be included when communicating with audiences who prefer it. + +### A happy twinge + +When an [early account][12] of the terminology change written by Eric Raymond was posted on the Open Source Initiative website, I was listed as being at the VA brainstorming meeting, but not as the originator of the term. This was my own fault; I had neglected to tell Eric the details. My impulse was to let it pass and stay in the background, but Todd felt otherwise. He suggested to me that one day I would be glad to be known as the person who coined the name "open source software." He explained the situation to Eric, who promptly updated his site. + +Coming up with a phrase is a small contribution, but I admit to being grateful to those who remember to credit me with it. Every time I hear it, which is very often now, it gives me a little happy twinge. + +The big credit for persuading the community goes to Eric Raymond and Tim O'Reilly, who made it happen. Thanks to them for crediting me, and to Todd Anderson for his role throughout. The above is not a complete account of open source history; apologies to the many key players whose names do not appear. Those seeking a more complete account should refer to the links in this article and elsewhere on the net. + +### About the author + + [![photo of Christine Peterson](https://opensource.com/sites/default/files/styles/profile_pictures/public/pictures/cp2016_crop2_185.jpg?itok=vUkSjFig)][13] Christine Peterson - Christine Peterson writes, lectures, and briefs the media on coming powerful technologies, especially nanotechnology, artificial intelligence, and longevity. She is Cofounder and Past President of Foresight Institute, the leading nanotech public interest group. Foresight educates the public, technical community, and policymakers on coming powerful technologies and how to guide their long-term impact. She serves on the Advisory Board of the [Machine Intelligence... ][2][more about Christine Peterson][3][More about me][4] + +-------------------------------------------------------------------------------- + +via: https://opensource.com/article/18/2/coining-term-open-source-software + +作者：[ Christine Peterson][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://opensource.com/users/christine-peterson +[1]:https://opensource.com/article/18/2/coining-term-open-source-software?rate=HFz31Mwyy6f09l9uhm5T_OFJEmUuAwpI61FY-fSo3Gc +[2]:http://intelligence.org/ +[3]:https://opensource.com/users/christine-peterson +[4]:https://opensource.com/users/christine-peterson +[5]:https://opensource.com/user/206091/feed +[6]:https://opensource.com/resources/what-open-source +[7]:https://opensource.org/osd +[8]:https://wiki2.org/en/Alternative_terms_for_free_software +[9]:https://opensource.org/ +[10]:http://www.oreilly.com/pub/pr/636 +[11]:http://www.oreilly.com/pub/pr/796 +[12]:https://ipfs.io/ipfs/QmXoypizjW3WknFiJnKLwHCnL72vedxjQkDDP1mXWo6uco/wiki/Alternative_terms_for_free_software.html +[13]:https://opensource.com/users/christine-peterson +[14]:https://opensource.com/users/christine-peterson +[15]:https://opensource.com/users/christine-peterson +[16]:https://opensource.com/article/18/2/coining-term-open-source-software#comments \ No newline at end of file diff --git a/sources/talk/20180201 IT automation- How to make the case.md b/sources/talk/20180201 IT automation- How to make the case.md new file mode 100644 index 0000000000..4b70fbff49 --- /dev/null +++ b/sources/talk/20180201 IT automation- How to make the case.md @@ -0,0 +1,95 @@ +IT automation: How to make the case +====== +At the start of any significant project or change initiative, IT leaders face a proverbial fork in the road. + +Path #1 might seem to offer the shortest route from A to B: Simply force-feed the project to everyone by executive mandate, essentially saying, “You’re going to do this – or else.” + +Path #2 might appear less direct, because on this journey you take the time to explain the strategy and the reasons behind it. In fact, you’re going to be making pit stops along this route, rather than marathoning from start to finish: “Here’s what we’re doing – and why we’re doing it.” + +Guess which path bears better results? + +If you said #2, you’ve traveled both paths before – and experienced the results first-hand. Getting people on board with major changes beforehand is almost always the smarter choice. + +IT leaders know as well as anyone that with significant change often comes [significant fear][1], skepticism, and other challenges. It may be especially true with IT automation. The term alone sounds scary to some people, and it is often tied to misconceptions. Helping people understand the what, why, and how of your company’s automation strategy is a necessary step to achieving your goals associated with that strategy. + +[ **Read our related article,** [**IT automation best practices: 7 keys to long-term success**][2]. ] + +With that in mind, we asked a variety of IT leaders for their advice on making the case for automation in your organization: + +## 1. Show people what’s in it for them + +Let’s face it: Self-interest and self-preservation are natural instincts. Tapping into that human tendency is a good way to get people on board: Show people how your automation strategy will benefit them and their jobs. Will automating a particular process in the software pipeline mean fewer middle-of-the-night calls for team members? Will it enable some people to dump low-skill, manual tasks in favor of more strategic, higher-order work – the sort that helps them take the next step in their career? + +“Convey what’s in it for them, and how it will benefit clients and the whole company,” advises Vipul Nagrath, global CIO at [ADP][3]. “Compare the current state to a brighter future state, where the company enjoys greater stability, agility, efficiency, and security.” + +The same approach holds true when making the case outside of IT; just lighten up on the jargon when explaining the benefits to non-technical stakeholders, Nagrath says. + +Setting up a before-and-after picture is a good storytelling device for helping people see the upside. + +“You want to paint a picture of the current state that people can relate to,” Nagrath says. “Present what’s working, but also highlight what’s causing teams to be less than agile.” Then explain how automating certain processes will improve that current state. + +## 2. Connect automation to specific business goals + +Part of making a strong case entails making sure people understand that you’re not just trend-chasing. If you’re automating simply for the sake of automating, people will sniff that out and become more resistant – perhaps especially within IT. + +“The case for automation needs to be driven by a business demand signal, such as revenue or operating expense,” says David Emerson, VP and deputy CISO at [Cyxtera][4]. “No automation endeavor is self-justifying, and no technical feat, generally, should be a means unto itself, unless it’s a core competency of the company.” + +Like Nagrath, Emerson recommends promoting the incentives associated with achieving the business goals of automation, and working toward these goals (and corresponding incentives) in an iterative, step-by-step fashion. + +## 3. Break the automation plan into manageable pieces + +Even if your automation strategy is literally “automate everything,” that’s a tough sell (and probably unrealistic) for most organizations. You’ll make a stronger case with a plan that approaches automation manageable piece by manageable piece, and that enables greater flexibility to adapt along the way. + +“When making a case for automation, I recommend clearly illustrating the incentive to move to an automated process, and allowing iteration toward that goal to introduce and prove the benefits at lower risk,” Emerson says. + +Sergey Zuev, founder at [GA Connector][5], shares an in-the-trenches account of why automating incrementally is crucial – and how it will help you build a stronger, longer-lasting argument for your strategy. Zuev should know: His company’s tool automates the import of data from CRM applications into Google Analytics. But it was actually the company’s internal experience automating its own customer onboarding process that led to a lightbulb moment. + +“At first, we tried to build the whole onboarding funnel at once, and as a result, the project dragged [on] for months,” Zuev says. “After realizing that it [was] going nowhere, we decided to select small chunks that would have the biggest immediate effect, and start with that. As a result, we managed to implement one of the email sequences in just a week, and are already reaping the benefits of the desecrated manual effort.” + +## 4. Sell the big-picture benefits too + +A step-by-step approach does not preclude painting a bigger picture. Just as it’s a good idea to make the case at the individual or team level, it’s also a good idea for help people understand the company-wide benefits. + +“If we can accelerate the time it takes for the business to get what it needs, it will silence the skeptics.” + +Eric Kaplan, CTO at [AHEAD][6], agrees that using small wins to show automation’s value is a smart strategy for winning people over. But the value those so-called “small” wins reveal can actually help you sharpen the big picture for people. Kaplan points to the value of individual and organizational time as an area everyone can connect with easily. + +“The best place to do this is where you can show savings in terms of time,” Kaplan says. “If we can accelerate the time it takes for the business to get what it needs, it will silence the skeptics.” + +Time and scalability are powerful benefits business and IT colleagues, both charged with growing the business, can grasp. + +“The result of automation is scalability – less effort per person to maintain and grow your IT environment, as [Red Hat][7] VP, Global Services John Allessio recently [noted][8]. “If adding manpower is the only way to grow your business, then scalability is a pipe dream. Automation reduces your manpower requirements and provides the flexibility required for continued IT evolution.” (See his full article, [What DevOps teams really need from a CIO][8].) + +## 5. Promote the heck out of your results + +At the outset of your automation strategy, you’ll likely be making the case based on goals and the anticipated benefits of achieving those goals. But as your automation strategy evolves, there’s no case quite as convincing as one grounded in real-world results. + +“Seeing is believing,” says Nagrath, ADP’s CIO. “Nothing quiets skeptics like a track record of delivery.” + +That means, of course, not only achieving your goals, but also doing so on time – another good reason for the iterative, step-by-step approach. + +While quantitative results such as percentage improvements or cost savings can speak loudly, Nagrath advises his fellow IT leaders not to stop there when telling your automation story. + +“Making a case for automation is also a qualitative discussion, where we can promote the issues prevented, overall business continuity, reductions in failures/errors, and associates taking on [greater] responsibility as they tackle more value-added tasks.” + + +-------------------------------------------------------------------------------- + +via: https://enterprisersproject.com/article/2018/1/how-make-case-it-automation + +作者：[Kevin Casey][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://enterprisersproject.com/user/kevin-casey +[1]:https://enterprisersproject.com/article/2017/10/how-beat-fear-and-loathing-it-change +[2]:https://enterprisersproject.com/article/2018/1/it-automation-best-practices-7-keys-long-term-success?sc_cid=70160000000h0aXAAQ +[3]:https://www.adp.com/ +[4]:https://www.cyxtera.com/ +[5]:http://gaconnector.com/ +[6]:https://www.thinkahead.com/ +[7]:https://www.redhat.com/en?intcmp=701f2000000tjyaAAA +[8]:https://enterprisersproject.com/article/2017/12/what-devops-teams-really-need-cio +[9]:https://enterprisersproject.com/email-newsletter?intcmp=701f2000000tsjPAAQ diff --git a/sources/talk/20180202 -Open source is 20- How it changed programming and business forever.md b/sources/talk/20180202 -Open source is 20- How it changed programming and business forever.md new file mode 100644 index 0000000000..1b7e6b6c37 --- /dev/null +++ b/sources/talk/20180202 -Open source is 20- How it changed programming and business forever.md @@ -0,0 +1,108 @@ +​Open source is 20: How it changed programming and business forever +====== +![][1] + +Every company in the world now uses open-source software. Microsoft, once its greatest enemy, is [now an enthusiastic open supporter][2]. Even [Windows is now built using open-source techniques][3]. And if you ever searched on Google, bought a book from Amazon, watched a movie on Netflix, or looked at your friend's vacation pictures on Facebook, you're an open-source user. Not bad for a technology approach that turns 20 on February 3. + +Now, free software has been around since the first computers, but the philosophy of both free software and open source are both much newer. In the 1970s and 80s, companies rose up which sought to profit by making proprietary software. In the nascent PC world, no one even knew about free software. But, on the Internet, which was dominated by Unix and ITS systems, it was a different story. + +In the late 70s, [Richard M. Stallman][6], also known as RMS, then an MIT programmer, created a free printer utility based on its source code. But then a new laser printer arrived on the campus and he found he could no longer get the source code and so he couldn't recreate the utility. The angry [RMS created the concept of "Free Software."][7] + +RMS's goal was to create a free operating system, [Hurd][8]. To make this happen in September 1983, [he announced the creation of the GNU project][9] (GNU stands for GNU's Not Unix -- a recursive acronym). By January 1984, he was working full-time on the project. To help build it he created the grandfather of all free software/open-source compiler system [GCC][10] and other operating system utilities. Early in 1985, he published "[The GNU Manifesto][11]," which was the founding charter of the free software movement and launched the [Free Software Foundation (FSF)][12]. + +This went well for a few years, but inevitably, [RMS collided with proprietary companies][13]. The company Unipress took the code to a variation of his [EMACS][14] programming editor and turned it into a proprietary program. RMS never wanted that to happen again so he created the [GNU General Public License (GPL)][15] in 1989. This was the first copyleft license. It gave users the right to use, copy, distribute, and modify a program's source code. But if you make source code changes and distribute it to others, you must share the modified code. While there had been earlier free licenses, such as [1980's four-clause BSD license][16], the GPL was the one that sparked the free-software, open-source revolution. + +In 1997, [Eric S. Raymond][17] published his vital essay, "[The Cathedral and the Bazaar][18]." In it, he showed the advantages of the free-software development methodologies using GCC, the Linux kernel, and his experiences with his own [Fetchmail][19] project as examples. This essay did more than show the advantages of free software. The programming principles he described led the way for both [Agile][20] development and [DevOps][21]. Twenty-first century programming owes a large debt to Raymond. + +Like all revolutions, free software quickly divided its supporters. On one side, as John Mark Walker, open-source expert and Strategic Advisor at Glyptodon, recently wrote, "[Free software is a social movement][22], with nary a hint of business interests -- it exists in the realm of religion and philosophy. Free software is a way of life with a strong moral code." + +On the other were numerous people who wanted to bring "free software" to business. They would become the founders of "open source." They argued that such phrases as "Free as in freedom" and "Free speech, not beer," left most people confused about what that really meant for software. + +The [release of the Netscape web browser source code][23] sparked a meeting of free software leaders and experts at [a strategy session held on February 3rd][24], 1998 in Palo Alto, CA. There, Eric S. Raymond, Michael Tiemann, Todd Anderson, Jon "maddog" Hall, Larry Augustin, Sam Ockman, and Christine Peterson hammered out the first steps to open source. + +Peterson created the "open-source term." She remembered: + +> [The introduction of the term "open source software" was a deliberate effort][25] to make this field of endeavor more understandable to newcomers and to business, which was viewed as necessary to its spread to a broader community of users. The problem with the main earlier label, "free software," was not its political connotations, but that -- to newcomers -- its seeming focus on price is distracting. A term was needed that focuses on the key issue of source code and that does not immediately confuse those new to the concept. The first term that came along at the right time and fulfilled these requirements was rapidly adopted: open source. + +To help clarify what open source was, and wasn't, Raymond and Bruce Perens founded the [Open Source Initiative (OSI)][26]. Its purpose was, and still is, to define what are real open-source software licenses and what aren't. + +Stallman was enraged by open source. He wrote: + +> The two terms describe almost the same method/category of software, but they stand for [views based on fundamentally different values][27]. Open source is a development methodology; free software is a social movement. For the free software movement, free software is an ethical imperative, essential respect for the users' freedom. By contrast, the philosophy of open source considers issues in terms of how to make software 'better' -- in a practical sense only. It says that non-free software is an inferior solution to the practical problem at hand. Most discussion of "open source" pays no attention to right and wrong, only to popularity and success. + +He saw open source as kowtowing to business and taking the focus away from the personal freedom of being able to have free access to the code. Twenty years later, he's still angry about it. + +In a recent e-mail to me, Stallman said, it is a "common error is connecting me or my work or free software in general with the term 'Open Source.' That is the slogan adopted in 1998 by people who reject the philosophy of the Free Software Movement." In another message, he continued, "I rejected 'open source' because it was meant to bury the "free software" ideas of freedom. Open source inspired the release ofu seful free programs, but what's missing is the idea that users deserve control of their computing. We libre-software activists say, 'Software you can't change and share is unjust, so let's escape to our free replacement.' Open source says only, 'If you let users change your code, they might fix bugs.' What it does says is not wrong, but weak; it avoids saying the deeper point." + +Philosophical conflicts aside, open source has indeed become the model for practical software development. Larry Augustin, CEO of [SugarCRM][28], the open-source customer relationship management (CRM) Software-as-a-Service (SaaS), was one of the first to practice open-source in a commercial software business. Augustin showed that a successful business could be built on open-source software. + +Other companies quickly embraced this model. Besides Linux companies such as [Canonical][29], [Red Hat][30] and [SUSE][31], technology businesses such as [IBM][32] and [Oracle][33] also adopted it. This, in turn, led to open source's commercial success. More recently companies you would never think of for a moment as open-source businesses like [Wal-Mart][34] and [Verizon][35], now rely on open-source programs and have their own open-source projects. + +As Jim Zemlin, director of [The Linux Foundation][36], observed in 2014: + +> A [new business model][37] has emerged in which companies are joining together across industries to share development resources and build common open-source code bases on which they can differentiate their own products and services. + +Today, Hall looked back and said "I look at 'closed source' as a blip in time." Raymond is unsurprised at open-source's success. In an e-mail interview, Raymond said, "Oh, yeah, it *has* been 20 years -- and that's not a big deal because we won most of the fights we needed to quite a while ago, like in the first decade after 1998." + +"Ever since," he continued, "we've been mainly dealing with the problems of success rather than those of failure. And a whole new class of issues, like IoT devices without upgrade paths -- doesn't help so much for the software to be open if you can't patch it." + +In other words, he concludes, "The reward of victory is often another set of battles." + +These are battles that open source is poised to win. Jim Whitehurst, Red Hat's CEO and president told me: + +> The future of open source is bright. We are on the cusp of a new wave of innovation that will come about because information is being separated from physical objects thanks to the Internet of Things. Over the next decade, we will see entire industries based on open-source concepts, like the sharing of information and joint innovation, become mainstream. We'll see this impact every sector, from non-profits, like healthcare, education and government, to global corporations who realize sharing information leads to better outcomes. Open and participative innovation will become a key part of increasing productivity around the world. + +Others see open source extending beyond software development methods. Nick Hopman, Red Hat's senior director of emerging technology practices, said: + +> Open-source is much more than just a process to develop and expose technology. Open-source is a catalyst to drive change in every facet of society -- government, policy, medical diagnostics, process re-engineering, you name it -- and can leverage open principles that have been perfected through the experiences of open-source software development to create communities that drive change and innovation. Looking forward, open-source will continue to drive technology innovation, but I am even more excited to see how it changes the world in ways we have yet to even consider. + +Indeed. Open source has turned twenty, but its influence, and not just on software and business, will continue on for decades to come. + +-------------------------------------------------------------------------------- + +via: http://www.zdnet.com/article/open-source-turns-20/ + +作者：[Steven J. Vaughan-Nichols][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://www.zdnet.com/meet-the-team/us/steven-j-vaughan-nichols/ +[1]:https://zdnet1.cbsistatic.com/hub/i/r/2018/01/08/d9527281-2972-4cb7-bd87-6464d8ad50ae/thumbnail/570x322/9d4ef9007b3a3ce34de0cc39d2b15b0c/5a4faac660b22f2aba08fc3f-1280x7201jan082018150043poster.jpg +[2]:http://www.zdnet.com/article/microsoft-the-open-source-company/ +[3]:http://www.zdnet.com/article/microsoft-uses-open-source-software-to-create-windows/ +[4]:https://zdnet1.cbsistatic.com/hub/i/r/2016/11/18/a55b3c0c-7a8e-4143-893f-44900cb2767a/resize/220x165/6cd4e37b1904743ff1f579cb10d9e857/linux-open-source-money-penguin.jpg +[5]:http://www.zdnet.com/article/how-do-linux-and-open-source-companies-make-money-from-free-software/ +[6]:https://stallman.org/ +[7]:https://opensource.com/article/18/2/pivotal-moments-history-open-source +[8]:https://www.gnu.org/software/hurd/hurd.html +[9]:https://groups.google.com/forum/#!original/net.unix-wizards/8twfRPM79u0/1xlglzrWrU0J +[10]:https://gcc.gnu.org/ +[11]:https://www.gnu.org/gnu/manifesto.en.html +[12]:https://www.fsf.org/ +[13]:https://www.free-soft.org/gpl_history/ +[14]:https://www.gnu.org/s/emacs/ +[15]:https://www.gnu.org/licenses/gpl-3.0.en.html +[16]:http://www.linfo.org/bsdlicense.html +[17]:http://www.catb.org/esr/ +[18]:http://www.catb.org/esr/writings/cathedral-bazaar/ +[19]:http://www.fetchmail.info/ +[20]:https://www.agilealliance.org/agile101/ +[21]:https://aws.amazon.com/devops/what-is-devops/ +[22]:https://opensource.com/business/16/11/open-source-not-free-software?sc_cid=70160000001273HAAQ +[23]:http://www.zdnet.com/article/the-beginning-of-the-peoples-web-20-years-of-netscape/ +[24]:https://opensource.org/history +[25]:https://opensource.com/article/18/2/coining-term-open-source-software +[26]:https://opensource.org +[27]:https://www.gnu.org/philosophy/open-source-misses-the-point.html +[28]:https://www.sugarcrm.com/ +[29]:https://www.canonical.com/ +[30]:https://www.redhat.com/en +[31]:https://www.suse.com/ +[32]:https://developer.ibm.com/code/open/ +[33]:http://www.oracle.com/us/technologies/open-source/overview/index.html +[34]:http://www.zdnet.com/article/walmart-relies-on-openstack/ +[35]:https://www.networkworld.com/article/3195490/lan-wan/verizon-taps-into-open-source-white-box-fervor-with-new-cpe-offering.html +[36]:http://www.linuxfoundation.org/ +[37]:http://www.zdnet.com/article/it-takes-an-open-source-village-to-make-commercial-software/ diff --git a/sources/talk/20180205 Security Is Not an Absolute - System Overlord.md b/sources/talk/20180205 Security Is Not an Absolute - System Overlord.md new file mode 100644 index 0000000000..d0bd003c8f --- /dev/null +++ b/sources/talk/20180205 Security Is Not an Absolute - System Overlord.md @@ -0,0 +1,62 @@ +Security Is Not an Absolute +====== + +If there’s one thing I wish people from outside the security industry knew when dealing with information security, it’s that **Security is not an absolute**. Most of the time, it’s not even quantifiable. Even in the case of particular threat models, it’s often impossible to make statements about the security of a system with certainty. + +At work, I deal with a lot of very smart people who are not “security people”, but are well-meaning and trying to do the right thing. Online, I sometimes find myself in conversations on [/r/netsec][1], [/r/netsecstudents][2], [/r/asknetsec][3], or [security.stackexchange][4] where someone wants to know something about information security. Either way, it’s quite common that someone asks the fateful question: “Is this secure?”. There are actually only two answers to this question, and neither one is “Yes.” + +The first answer is, fairly obviously, “No.” There are some ideas that are not secure under any reasonable definition of security. Imagine an employer that makes the PIN for your payroll system the day and month on which you started your new job. Clearly, all it takes is someone posting “started my new job today!” to social media, and their PIN has been outed. Consider transporting an encrypted hard drive with the password on a sticky note attached to the outside of the drive. Both of these systems have employed some form of “security control” (even if I use the term loosely), and both are clearly insecure to even the most rudimentary of attacker. Consequently, answering “Is this secure?” with a firm “No” seems appropriate. + +The second answer is more nuanced: “It depends.” What it depends on, and whether those conditions exist in the system in use, are what many security professionals get paid to evaluate. For example, consider the employer in the previous paragraph. Instead of using a fixed scheme for PINs, they now generate a random 4-digit PIN and mail it to each new employee. Is this secure? That all depends on the threat model being applied to the scenario. If we allow an attacker unlimited attempts to log in as that user, then no 4 digit PIN (random or deterministic) is reasonably secure. On average, an attacker will need no more than 5000 requests to find the valid PIN. That can be done by a very basic script in 10s of minutes. If, on the other hand, we lock the account after 10 failed attempts, then we’ve reduced the attacker to a 0.1% chance of success for a given account. Is this secure? For a single account, this is probably reasonably secure (although most users might be uncomfortable at even a 1 in 1000 chance of an attacker succeeding against their personal account) but what if the attacker has a list of 1000 usernames? The attacker now has a **64%** chance of successfully accessing at least 1 account. I think most businesses would find those odds very much against their favor. + +So why can’t we ever come up with an answer of “Yes, this is a secure system”? Well, there’s several factors at play here. The first is that very little in life in general is an absolute: + + * Your doctor cannot tell you with certainty that you will be alive tomorrow. + * A seismologist can’t say that there absolutely won’t be a 9.0 earthquake that levels a big chunk of the West Coast. + * Your car manufacturer cannot guarantee that the 4 wheels on your car do not fall of on your way to work tomorrow. + + + +However, all of these possibilities are very remote events. Most people are comfortable with these probabilities, largely because they do not think much about them, but even if they did, they would believe that it would not happen to them. (And almost always, they would be correct in that assumption.) + +Unfortunately, in information security, we have three things working against us: + + * The risks are much less understood by those seeking to understand them. + * The reality is that there are enough security threats that are **much** more common than the events above. + * The threats against which security must guard are **adaptive**. + + + +Because most people have a hard time reasoning about the likelihood of attacks and threats against them, they seek absolute reassurance. They don’t want to be told “it depends”, they just want to hear “yes, you’re fine.” Many of these individuals are the hypochondriacs of the information security world – they think every possible attack will get them, and they want absolute reassurance they’re safe from those attacks. Alternatively, they don’t understand that there are degrees of security and threat models, and just want to be reassured that they are perfectly secure. Either way, the effect is the same – they don’t understand, but are afraid, and so want the reassurance of complete security. + +We’re in an era where security breaches are unfortunately common, and developers and users alike are hearing about these vulnerabilities and breaches all the time. This causes them to pay far more attention to security then they otherwise would. By itself, this isn’t bad – all of us in the industry have been trying to get everyone’s attention about security issues for decades. Getting it now is better late than never. But because we’re so far behind the curve, the breaches being common, everytone is rushing to find out their risk and get reassurance now. Rather than consider the nuances of the situation, they just want a simple answer to “Am I secure?” + +The last of these issues, however, is also the most unique to information security. For decades, we’ve looked for the formula to make a system perfectly secure. However, each countermeasure or security system is quickly defeated by attackers. We’re in a cat-and-mouse game, rather than an engineering discipline. + +This isn’t to say that security is not an engineering practice – it certainly is in many ways (and my official title claims that I am an engineer), but just that it differs from other engineering areas. The forces faced by a building do not change in face of design changes by the structural engineer. Gravity remains a constant, wind forces are predictible for a given design, the seismic nature of an area is approximately known. Making the building have stronger doors does not suddenly increase the wind forces on the windows. In security, however, when we “strengthen the doors”, the attackers do turn to the “windows” of our system. Our threats are **adaptive** – for each control we implement, they adapt to attempt to circumvent that control. For this reason, a system that was believed secure against the known threats one year is completely broken the next. + +Another form of the security absolutism is those that realize there are degrees of security, but want to take it to an almost ridiculous level of paranoia. Nearly always, these seem to be interested in forms of cryptography – perhaps because cryptography offers numbers that can be tweaked, giving an impression of differing levels of security. + + * Generating RSA encryption keys of over 4k bits in length, even though all cryptographers agree this is pointless. + * Asking why AES-512 doesn’t exist, even though SHA-512 does. (Because the length of a hash and the length of a key do not equal in effective strength against attacks.) + * Setting up bizarre browser settings and then complaining about websites being broken. (Disabling all JavaScript, all cookies, all ciphers that are less than 256 bits and not perfect forward secrecy, etc.) + + + +So the next time you want to know “Is this secure?”, consider the threat model: what are you trying to defend against? Recognize that there are no security absolutes and guarantees, and that good security engineering practice often involves compromise. Sometimes the compromise is one of usability or utility, sometimes the compromise involves working in a less-than-perfect world. + +-------------------------------------------------------------------------------- + +via: https://systemoverlord.com/2018/02/05/security-is-not-an-absolute.html + +作者：[David][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://systemoverlord.com/about +[1]:https://reddit.com/r/netsec +[2]:https://reddit.com/r/netsecstudents +[3]:https://reddit.com/r/asknetsec +[4]:https://security.stackexchange.com diff --git a/sources/tech/20061104 How To Turn On-Off Colors For ls Command In Bash On a Linux-Unix.md b/sources/tech/20061104 How To Turn On-Off Colors For ls Command In Bash On a Linux-Unix.md new file mode 100644 index 0000000000..8923754d76 --- /dev/null +++ b/sources/tech/20061104 How To Turn On-Off Colors For ls Command In Bash On a Linux-Unix.md @@ -0,0 +1,89 @@ +translating---geekpi + +How To Turn On/Off Colors For ls Command In Bash On a Linux/Unix +====== + +How do I turn on or off file name colors (ls command colors) in bash shell on a Linux or Unix like operating systems? + +Most modern Linux distributions and Unix systems comes with alias that defines colors for your file. However, ls command is responsible for displaying color on screen for files, directories and other file system objects. + +By default, color is not used to distinguish types of files. You need to pass --color option to the ls command on Linux. If you are using OS X or BSD based system pass -G option to the ls command. The syntax is as follows to turn on or off colors. + +#### How to turn off colors for ls command + +Type the following command +`$ ls --color=none` +Or just remove alias with the unalias command: +`$ unalias ls` +Please note that the following bash shell aliases are defined to display color with the ls command. Use combination of [alias command][1] and [grep command][2] as follows: +`$ alias | grep ls` +Sample outputs +``` +alias l='ls -CF' +alias la='ls -A' +alias ll='ls -alF' +alias ls='ls --color=auto' +``` + +#### How to turn on colors for ls command + +Use any one of the following command: +``` +$ ls --color=auto +$ ls --color=tty +``` +[Define bash shell aliases ][3]if you want: +`alias ls='ls --color=auto'` +You can add or remove ls command alias to the ~/.bash_profile or [~/.bashrc file][4]. Edit file using a text editor such as vi command: +`$ vi ~/.bashrc` +Append the following code: +``` +# my ls command aliases # +alias ls = 'ls --color=auto' +``` + +[Save and close the file in Vi/Vim text editor][5]. + +#### A note about *BSD/macOS/Apple OS X ls command + +Pass the -G option to ls command to enable colorized output on a {Free,Net,Open}BSD or macOS and Apple OS X Unix family of operating systems: +`$ ls -G` +Sample outputs: +[![How to enable colorized output for the ls command in Mac OS X Terminal][6]][7] +How to enable colorized output for the ls command in Mac OS X Terminal + +#### How do I skip colorful ls command output temporarily? + +You can always [disable bash shell aliases temporarily][8] using any one of the following syntax: +`\ls +/bin/ls +command ls +'ls'` + + +#### About the author + +The author is the creator of nixCraft and a seasoned sysadmin and a trainer for the Linux operating system/Unix shell scripting. He has worked with global clients and in various industries, including IT, education, defense and space research, and the nonprofit sector. Follow him on [Twitter][9], [Facebook][10], [Google+][11]. + +-------------------------------------------------------------------------------- + +via: https://www.cyberciti.biz/faq/how-to-turn-on-or-off-colors-in-bash/ + +作者：[Vivek Gite][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.cyberciti.biz/ +[1]:https://www.cyberciti.biz/tips/bash-aliases-mac-centos-linux-unix.html (See Linux/Unix alias command examples for more info) +[2]:https://www.cyberciti.biz/faq/howto-use-grep-command-in-linux-unix/ (See Linux/Unix grep command examples for more info) +[3]:https://www.cyberciti.biz/tips/bash-aliases-mac-centos-linux-unix.html +[4]:https://bash.cyberciti.biz/guide/~/.bashrc +[5]:https://www.cyberciti.biz/faq/linux-unix-vim-save-and-quit-command/ +[6]:https://www.cyberciti.biz/media/new/faq/2016/01/color-ls-for-Mac-OS-X.jpg +[7]:https://www.cyberciti.biz/faq/apple-mac-osx-terminal-color-ls-output-option/ +[8]:https://www.cyberciti.biz/faq/bash-shell-temporarily-disable-an-alias/ +[9]:https://twitter.com/nixcraft +[10]:https://facebook.com/nixcraft +[11]:https://plus.google.com/+CybercitiBiz diff --git a/sources/tech/20070810 How to use lftp to accelerate ftp-https download speed on Linux-UNIX.md b/sources/tech/20070810 How to use lftp to accelerate ftp-https download speed on Linux-UNIX.md new file mode 100644 index 0000000000..52a9c4c89f --- /dev/null +++ b/sources/tech/20070810 How to use lftp to accelerate ftp-https download speed on Linux-UNIX.md @@ -0,0 +1,84 @@ +translating---geekpi + +How to use lftp to accelerate ftp/https download speed on Linux/UNIX +====== +lftp is a file transfer program. It allows sophisticated FTP, HTTP/HTTPS, and other connections. If the site URL is specified, then lftp will connect to that site otherwise a connection has to be established with the open command. It is an essential tool for all a Linux/Unix command line users. I have already written about [Linux ultra fast command line download accelerator][1] such as Axel and prozilla. lftp is another tool for the same job with more features. lftp can handle seven file access methods: + + 1. ftp + 2. ftps + 3. http + 4. https + 5. hftp + 6. fish + 7. sftp + 8. file + + + +### So what is unique about lftp? + + * Every operation in lftp is reliable, that is any not fatal error is ignored, and the operation is repeated. So if downloading breaks, it will be restarted from the point automatically. Even if FTP server does not support REST command, lftp will try to retrieve the file from the very beginning until the file is transferred completely. + * lftp has shell-like command syntax allowing you to launch several commands in parallel in the background. + * lftp has a builtin mirror which can download or update a whole directory tree. There is also a reverse mirror (mirror -R) which uploads or updates a directory tree on the server. The mirror can also synchronize directories between two remote servers, using FXP if available. + + + +### How to use lftp as download accelerator + +lftp has pget command. It allows you download files in parallel. The syntax is +`lftp -e 'pget -n NUM -c url; exit'` +For example, download file using pget in 5 parts: +``` +$ cd /tmp +$ lftp -e 'pget -n 5 -c http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.22.2.tar.bz2' +``` +Sample outputs: +``` +45108964 bytes transferred in 57 seconds (775.3K/s) +lftp :~>quit + +``` + +Where, + + 1. pget – Download files in parallel + 2. -n 5 – Set maximum number of connections to 5 + 3. -c – Continue broken transfer if lfile.lftp-pget-status exists in the current directory + + + +### How to use lftp to accelerate ftp/https download on Linux/Unix + +Another try with added exit command: +`$ lftp -e 'pget -n 10 -c https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.15.tar.xz; exit'` + +[Linux-lftp-command-demo][https://www.cyberciti.biz/tips/wp-content/uploads/2007/08/Linux-lftp-command-demo.mp4] + +### A note about parallel downloading + +Please note that by using download accelerator you are going to put a load on remote host. Also note that lftp may not work with sites that do not support multi-source downloads or blocks such requests at firewall level. + +NA command offers many other features. Refer to [lftp][2] man page for more information: +`man lftp` + +### about the author + +The author is the creator of nixCraft and a seasoned sysadmin and a trainer for the Linux operating system/Unix shell scripting. He has worked with global clients and in various industries, including IT, education, defense and space research, and the nonprofit sector. Follow him on [Twitter][3], [Facebook][4], [Google+][5]. Get the **latest tutorials on SysAdmin, Linux/Unix and open source topics via[my RSS/XML feed][6]**. + +-------------------------------------------------------------------------------- + +via: https://www.cyberciti.biz/tips/linux-unix-download-accelerator.html + +作者：[Vivek Gite][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.cyberciti.biz +[1]:https://www.cyberciti.biz/tips/download-accelerator-for-linux-command-line-tools.html +[2]:https://lftp.yar.ru/ +[3]:https://twitter.com/nixcraft +[4]:https://facebook.com/nixcraft +[5]:https://plus.google.com/+CybercitiBiz +[6]:https://www.cyberciti.biz/atom/atom.xml diff --git a/sources/tech/20090518 How to use yum-cron to automatically update RHEL-CentOS Linux.md b/sources/tech/20090518 How to use yum-cron to automatically update RHEL-CentOS Linux.md new file mode 100644 index 0000000000..b0ca149c3e --- /dev/null +++ b/sources/tech/20090518 How to use yum-cron to automatically update RHEL-CentOS Linux.md @@ -0,0 +1,141 @@ +How to use yum-cron to automatically update RHEL/CentOS Linux +====== +The yum command line tool is used to install and update software packages under RHEL / CentOS Linux server. I know how to apply updates using [yum update command line][1], but I would like to use cron to update packages where appropriate manually. How do I configure yum to install software patches/updates [automatically with cron][2]? + +You need to install yum-cron package. It provides files needed to run yum updates as a cron job. Install this package if you want auto yum updates nightly via cron. + +### How to install yum cron on a CentOS/RHEL 6.x/7.x + +Type the following [yum command][3] on: +`$ sudo yum install yum-cron` +![](https://www.cyberciti.biz/media/new/faq/2009/05/How-to-install-yum-cron-on-CentOS-RHEL-server.jpg) + +Turn on service using systemctl command on **CentOS/RHEL 7.x** : +``` +$ sudo systemctl enable yum-cron.service +$ sudo systemctl start yum-cron.service +$ sudo systemctl status yum-cron.service +``` +If you are using **CentOS/RHEL 6.x** , run: +``` +$ sudo chkconfig yum-cron on +$ sudo service yum-cron start +``` +![](https://www.cyberciti.biz/media/new/faq/2009/05/How-to-turn-on-yum-cron-service-on-CentOS-or-RHEL-server.jpg) + +yum-cron is an alternate interface to yum. Very convenient way to call yum from cron. It provides methods to keep repository metadata up to date, and to check for, download, and apply updates. Rather than accepting many different command line arguments, the different functions of yum-cron can be accessed through config files. + +### How to configure yum-cron to automatically update RHEL/CentOS Linux + +You need to edit /etc/yum/yum-cron.conf and /etc/yum/yum-cron-hourly.conf files using a text editor such as vi command: +`$ sudo vi /etc/yum/yum-cron.conf` +Make sure updates should be applied when they are available +`apply_updates = yes` +You can set the address to send email messages from. Please note that ‘localhost’ will be replaced with the value of system_name. +`email_from = root@localhost` +List of addresses to send messages to. +`email_to = your-it-support@some-domain-name` +Name of the host to connect to to send email messages. +`email_host = localhost` +If you [do not want to update kernel package add the following on CentOS/RHEL 7.x][4]: +`exclude=kernel*` +For RHEL/CentOS 6.x add [the following to exclude kernel package from updating][5]: +`YUM_PARAMETER=kernel*` +[Save and close the file in vi/vim][6]. You also need to update /etc/yum/yum-cron-hourly.conf file if you want to apply update hourly. Otherwise /etc/yum/yum-cron.conf will run on daily using the following cron job (us [cat command][7]: +`$ cat /etc/cron.daily/0yum-daily.cron` +Sample outputs: +``` +#!/bin/bash +  +# Only run if this flag is set. The flag is created by the yum-cron init +# script when the service is started -- this allows one to use chkconfig and +# the standard "service stop|start" commands to enable or disable yum-cron. +if [[ ! -f /var/lock/subsys/yum-cron ]]; then + exit 0 +fi +  +# Action! +exec /usr/sbin/yum-cron /etc/yum/yum-cron-hourly.conf +[root@centos7-box yum]# cat /etc/cron.daily/0yum-daily.cron +#!/bin/bash +  +# Only run if this flag is set. The flag is created by the yum-cron init +# script when the service is started -- this allows one to use chkconfig and +# the standard "service stop|start" commands to enable or disable yum-cron. +if [[ ! -f /var/lock/subsys/yum-cron ]]; then + exit 0 +fi +  +# Action! +exec /usr/sbin/yum-cron +``` + +That is all. Now your system will update automatically everyday using yum-cron. See man page of yum-cron for more details: +`$ man yum-cron` + +### Method 2 – Use shell scripts + +**Warning** : The following method is outdated. Do not use it on RHEL/CentOS 6.x/7.x. I kept it below for historical reasons only when I used it on CentOS/RHEL version 4.x/5.x. + +Let us see how to configure CentOS/RHEL for yum automatic update retrieval and installation of security packages. You can use yum-updatesd service provided with CentOS / RHEL servers. However, this service provides a few overheads. You can create daily or weekly updates with the following shell script. Create + + * **/etc/cron.daily/yumupdate.sh** to apply updates one a day. + * **/etc/cron.weekly/yumupdate.sh** to apply updates once a week. + + + +#### Sample shell script to update system + +A shell script that instructs yum to update any packages it finds via [cron][8]: +``` +#!/bin/bash +YUM=/usr/bin/yum +$YUM -y -R 120 -d 0 -e 0 update yum +$YUM -y -R 10 -e 0 -d 0 update +``` + +(Code listing -01: /etc/cron.daily/yumupdate.sh) + +Where, + + 1. First command will update yum itself and next will apply system updates. + 2. **-R 120** : Sets the maximum amount of time yum will wait before performing a command + 3. **-e 0** : Sets the error level to 0 (range 0 – 10). 0 means print only critical errors about which you must be told. + 4. -d 0 : Sets the debugging level to 0 – turns up or down the amount of things that are printed. (range: 0 – 10). + 5. **-y** : Assume yes; assume that the answer to any question which would be asked is yes. + + + +Make sure you setup executable permission: +`# chmod +x /etc/cron.daily/yumupdate.sh` + + +### about the author + +Posted by: + +The author is the creator of nixCraft and a seasoned sysadmin and a trainer for the Linux operating system/Unix shell scripting. He has worked with global clients and in various industries, including IT, education, defense and space research, and the nonprofit sector. Follow him on [Twitter][9], [Facebook][10], [Google+][11]. Get the **latest tutorials on SysAdmin, Linux/Unix and open source topics via[my RSS/XML feed][12]**. + +-------------------------------------------------------------------------------- + +via: https://www.cyberciti.biz/faq/fedora-automatic-update-retrieval-installation-with-cron/ + +作者：[Vivek Gite][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.cyberciti.biz/ +[1]:https://www.cyberciti.biz/faq/rhel-centos-fedora-linux-yum-command-howto/ +[2]:https://www.cyberciti.biz/faq/how-do-i-add-jobs-to-cron-under-linux-or-unix-oses +[3]:https://www.cyberciti.biz/faq/rhel-centos-fedora-linux-yum-command-howto/ (See Linux/Unix yum command examples for more info) +[4]:https://www.cyberciti.biz/faq/yum-update-except-kernel-package-command/ +[5]:https://www.cyberciti.biz/faq/redhat-centos-linux-yum-update-exclude-packages/ +[6]:https://www.cyberciti.biz/faq/linux-unix-vim-save-and-quit-command/ +[7]:https://www.cyberciti.biz/faq/linux-unix-appleosx-bsd-cat-command-examples/ (See Linux/Unix cat command examples for more info) +[8]:https://www.cyberciti.biz/faq/how-do-i-add-jobs-to-cron-under-linux-or-unix-oses +[9]:https://twitter.com/nixcraft +[10]:https://facebook.com/nixcraft +[11]:https://plus.google.com/+CybercitiBiz +[12]:https://www.cyberciti.biz/atom/atom.xml diff --git a/sources/tech/20090627 30 Linux System Monitoring Tools Every SysAdmin Should Know.md b/sources/tech/20090627 30 Linux System Monitoring Tools Every SysAdmin Should Know.md index bb527d5519..7e3f5708dd 100644 --- a/sources/tech/20090627 30 Linux System Monitoring Tools Every SysAdmin Should Know.md +++ b/sources/tech/20090627 30 Linux System Monitoring Tools Every SysAdmin Should Know.md @@ -1,3 +1,5 @@ +Translating by jessie-pang + 30 Linux System Monitoring Tools Every SysAdmin Should Know ====== diff --git a/sources/tech/20090724 Top 20 OpenSSH Server Best Security Practices.md b/sources/tech/20090724 Top 20 OpenSSH Server Best Security Practices.md new file mode 100644 index 0000000000..a7ad346af4 --- /dev/null +++ b/sources/tech/20090724 Top 20 OpenSSH Server Best Security Practices.md @@ -0,0 +1,474 @@ +Top 20 OpenSSH Server Best Security Practices +====== +![OpenSSH Security Tips][1] + +OpenSSH is the implementation of the SSH protocol. OpenSSH is recommended for remote login, making backups, remote file transfer via scp or sftp, and much more. SSH is perfect to keep confidentiality and integrity for data exchanged between two networks and systems. However, the main advantage is server authentication, through the use of public key cryptography. From time to time there are [rumors][2] about OpenSSH zero day exploit. This **page shows how to secure your OpenSSH server running on a Linux or Unix-like system to improve sshd security**. + + +#### OpenSSH defaults + + * TCP port - 22 + * OpenSSH server config file - sshd_config (located in /etc/ssh/) + + + +#### 1. Use SSH public key based login + +OpenSSH server supports various authentication. It is recommended that you use public key based authentication. First, create the key pair using following ssh-keygen command on your local desktop/laptop: + +DSA and RSA 1024 bit or lower ssh keys are considered weak. Avoid them. RSA keys are chosen over ECDSA keys when backward compatibility is a concern with ssh clients. All ssh keys are either ED25519 or RSA. Do not use any other type. + +``` +$ ssh-keygen -t key_type -b bits -C "comment" +$ ssh-keygen -t ed25519 -C "Login to production cluster at xyz corp" +$ ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_rsa_aws_$(date +%Y-%m-%d) -C "AWS key for abc corp clients" +``` +Next, install the public key using ssh-copy-id command: +``` +$ ssh-copy-id -i /path/to/public-key-file user@host +$ ssh-copy-id user@remote-server-ip-or-dns-name +$ ssh-copy-id vivek@rhel7-aws-server +``` +When promoted supply user password. Verify that ssh key based login working for you: +`$ ssh vivek@rhel7-aws-server` +[![OpenSSH server security best practices][3]][3] +For more info on ssh public key auth see: + +* [keychain: Set Up Secure Passwordless SSH Access For Backup Scripts][48] + +* [sshpass: Login To SSH Server / Provide SSH Password Using A Shell Script][49] + +* [How To Setup SSH Keys on a Linux / Unix System][50] + +* [How to upload ssh public key to as authorized_key using Ansible DevOPS tool][51] + + +#### 2. Disable root user login + +Before we disable root user login, make sure regular user can log in as root. For example, allow vivek user to login as root using the sudo command. + +##### How to add vivek user to sudo group on a Debian/Ubuntu + +Allow members of group sudo to execute any command. [Add user vivek to sudo group][4]: +`$ sudo adduser vivek sudo` +Verify group membership with [id command][5] +`$ id vivek` + +##### How to add vivek user to sudo group on a CentOS/RHEL server + +Allows people in group wheel to run all commands on a CentOS/RHEL and Fedora Linux server. Use the usermod command to add the user named vivek to the wheel group: +``` +$ sudo usermod -aG wheel vivek +$ id vivek +``` + +##### Test sudo access and disable root login for ssh + +Test it and make sure user vivek can log in as root or run the command as root: +``` +$ sudo -i +$ sudo /etc/init.d/sshd status +$ sudo systemctl status httpd +``` +Once confirmed disable root login by adding the following line to sshd_config: +``` +PermitRootLogin no +ChallengeResponseAuthentication no +PasswordAuthentication no +UsePAM no +``` +See "[How to disable ssh password login on Linux to increase security][6]" for more info. + +#### 3. Disable password based login + +All password-based logins must be disabled. Only public key based logins are allowed. Add the following in your sshd_config file: +``` +AuthenticationMethods publickey +PubkeyAuthentication yes +``` +Older version of SSHD on CentOS 6.x/RHEL 6.x user should use the following setting: +``` +PubkeyAuthentication yes +``` + +#### 4. Limit Users' ssh access + +By default, all systems user can login via SSH using their password or public key. Sometimes you create UNIX / Linux user account for FTP or email purpose. However, those users can log in to the system using ssh. They will have full access to system tools including compilers and scripting languages such as Perl, Python which can open network ports and do many other fancy things. Only allow root, vivek and jerry user to use the system via SSH, add the following to sshd_config: +`AllowUsers vivek jerry` +Alternatively, you can allow all users to login via SSH but deny only a few users, with the following line in sshd_config: +`DenyUsers root saroj anjali foo` +You can also [configure Linux PAM][7] allows or deny login via the sshd server. You can allow [list of group name][8] to access or deny access to the ssh. + +#### 5. Disable Empty Passwords + +You need to explicitly disallow remote login from accounts with empty passwords, update sshd_config with the following line: +`PermitEmptyPasswords no` + +#### 6. Use strong passwords and passphrase for ssh users/keys + +It cannot be stressed enough how important it is to use strong user passwords and passphrase for your keys. Brute force attack works because user goes to dictionary based passwords. You can force users to avoid [passwords against a dictionary][9] attack and use [john the ripper tool][10] to find out existing weak passwords. Here is a sample random password generator (put in your ~/.bashrc): +``` +genpasswd() { + local l=$1 + [ "$l" == "" ] && l=20 + tr -dc A-Za-z0-9_ < /dev/urandom | head -c ${l} | xargs +} +``` + +Run it: +`genpasswd 16` +Output: +``` +uw8CnDVMwC6vOKgW +``` +* [Generating Random Password With mkpasswd / makepasswd / pwgen][52] + +* [Linux / UNIX: Generate Passwords][53] + +* [Linux Random Password Generator Command][54] + +-------------------------------------------------------------------------------- + +#### 7. Firewall SSH TCP port # 22 + +You need to firewall ssh TCP port # 22 by updating iptables/ufw/firewall-cmd or pf firewall configurations. Usually, OpenSSH server must only accept connections from your LAN or other remote WAN sites only. + +##### Netfilter (Iptables) Configuration + +Update [/etc/sysconfig/iptables (Redhat and friends specific file) to accept connection][11] only from 192.168.1.0/24 and 202.54.1.5/29, enter: +``` +-A RH-Firewall-1-INPUT -s 192.168.1.0/24 -m state --state NEW -p tcp --dport 22 -j ACCEPT +-A RH-Firewall-1-INPUT -s 202.54.1.5/29 -m state --state NEW -p tcp --dport 22 -j ACCEPT +``` + +If you've dual stacked sshd with IPv6, edit /etc/sysconfig/ip6tables (Redhat and friends specific file), enter: +``` +-A RH-Firewall-1-INPUT -s ipv6network::/ipv6mask -m tcp -p tcp --dport 22 -j ACCEPT + +``` + +Replace ipv6network::/ipv6mask with actual IPv6 ranges. + +##### UFW for Debian/Ubuntu Linux + +[UFW is an acronym for uncomplicated firewall. It is used for managing a Linux firewall][12] and aims to provide an easy to use interface for the user. Use the [following command to accept port 22 from 202.54.1.5/29][13] only: +`$ sudo ufw allow from 202.54.1.5/29 to any port 22` +Read "[Linux: 25 Iptables Netfilter Firewall Examples For New SysAdmins][14]" for more info. + +##### *BSD PF Firewall Configuration + +If you are using PF firewall update [/etc/pf.conf][15] as follows: +``` +pass in on $ext_if inet proto tcp from {192.168.1.0/24, 202.54.1.5/29} to $ssh_server_ip port ssh flags S/SA synproxy state +``` + +#### 8. Change SSH Port and limit IP binding + +By default, SSH listens to all available interfaces and IP address on the system. Limit ssh port binding and change ssh port (many brutes forcing scripts only try to connect to TCP port # 22). To bind to 192.168.1.5 and 202.54.1.5 IPs and port 300, add or correct the following line in sshd_config: +``` +Port 300 +ListenAddress 192.168.1.5 +ListenAddress 202.54.1.5 +``` + +Port 300 ListenAddress 192.168.1.5 ListenAddress 202.54.1.5 + +A better approach to use proactive approaches scripts such as fail2ban or denyhosts when you want to accept connection from dynamic WAN IP address. + +#### 9. Use TCP wrappers (optional) + +TCP Wrapper is a host-based Networking ACL system, used to filter network access to the Internet. OpenSSH does support TCP wrappers. Just update your /etc/hosts.allow file as follows to allow SSH only from 192.168.1.2 and 172.16.23.12 IP address: +``` +sshd : 192.168.1.2 172.16.23.12 +``` + +See this [FAQ about setting and using TCP wrappers][16] under Linux / Mac OS X and UNIX like operating systems. + +#### 10. Thwart SSH crackers/brute force attacks + +Brute force is a method of defeating a cryptographic scheme by trying a large number of possibilities (combination of users and passwords) using a single or distributed computer network. To prevents brute force attacks against SSH, use the following software: + + * [DenyHosts][17] is a Python based security tool for SSH servers. It is intended to prevent brute force attacks on SSH servers by monitoring invalid login attempts in the authentication log and blocking the originating IP addresses. + * Explains how to setup [DenyHosts][18] under RHEL / Fedora and CentOS Linux. + * [Fail2ban][19] is a similar program that prevents brute force attacks against SSH. + * [sshguard][20] protect hosts from brute force attacks against ssh and other services using pf. + * [security/sshblock][21] block abusive SSH login attempts. + * [ IPQ BDB filter][22] May be considered as a fail2ban lite. + + + +#### 11. Rate-limit incoming traffic at TCP port # 22 (optional) + +Both netfilter and pf provides rate-limit option to perform simple throttling on incoming connections on port # 22. + +##### Iptables Example + +The following example will drop incoming connections which make more than 5 connection attempts upon port 22 within 60 seconds: +``` +#!/bin/bash +inet_if=eth1 +ssh_port=22 +$IPT -I INPUT -p tcp --dport ${ssh_port} -i ${inet_if} -m state --state NEW -m recent --set +$IPT -I INPUT -p tcp --dport ${ssh_port} -i ${inet_if} -m state --state NEW -m recent --update --seconds 60 --hitcount 5 +``` + +Call above script from your iptables scripts. Another config option: +``` +$IPT -A INPUT -i ${inet_if} -p tcp --dport ${ssh_port} -m state --state NEW -m limit --limit 3/min --limit-burst 3 -j ACCEPT +$IPT -A INPUT -i ${inet_if} -p tcp --dport ${ssh_port} -m state --state ESTABLISHED -j ACCEPT +$IPT -A OUTPUT -o ${inet_if} -p tcp --sport ${ssh_port} -m state --state ESTABLISHED -j ACCEPT +# another one line example +# $IPT -A INPUT -i ${inet_if} -m state --state NEW,ESTABLISHED,RELATED -p tcp --dport 22 -m limit --limit 5/minute --limit-burst 5-j ACCEPT +``` + +See iptables man page for more details. + +##### *BSD PF Example + +The following will limits the maximum number of connections per source to 20 and rate limit the number of connections to 15 in a 5 second span. If anyone breaks our rules add them to our abusive_ips table and block them for making any further connections. Finally, flush keyword kills all states created by the matching rule which originate from the host which exceeds these limits. +``` +sshd_server_ip = "202.54.1.5" +table persist +block in quick from +pass in on $ext_if proto tcp to $sshd_server_ip port ssh flags S/SA keep state (max-src-conn 20, max-src-conn-rate 15/5, overload flush) +``` + +#### 12. Use port knocking (optional) + +[Port knocking][23] is a method of externally opening ports on a firewall by generating a connection attempt on a set of prespecified closed ports. Once a correct sequence of connection attempts is received, the firewall rules are dynamically modified to allow the host which sent the connection attempts to connect to the specific port(s). A sample port Knocking example for ssh using iptables: +``` +$IPT -N stage1 +$IPT -A stage1 -m recent --remove --name knock +$IPT -A stage1 -p tcp --dport 3456 -m recent --set --name knock2 + +$IPT -N stage2 +$IPT -A stage2 -m recent --remove --name knock2 +$IPT -A stage2 -p tcp --dport 2345 -m recent --set --name heaven + +$IPT -N door +$IPT -A door -m recent --rcheck --seconds 5 --name knock2 -j stage2 +$IPT -A door -m recent --rcheck --seconds 5 --name knock -j stage1 +$IPT -A door -p tcp --dport 1234 -m recent --set --name knock + +$IPT -A INPUT -m --state ESTABLISHED,RELATED -j ACCEPT +$IPT -A INPUT -p tcp --dport 22 -m recent --rcheck --seconds 5 --name heaven -j ACCEPT +$IPT -A INPUT -p tcp --syn -j door +``` + + +For more info see: +[Debian / Ubuntu: Set Port Knocking With Knockd and Iptables][55] + +#### 13. Configure idle log out timeout interval + +A user can log in to the server via ssh, and you can set an idle timeout interval to avoid unattended ssh session. Open sshd_config and make sure following values are configured: +``` +ClientAliveInterval 300 +ClientAliveCountMax 0 +``` +You are setting an idle timeout interval in seconds (300 secs == 5 minutes). After this interval has passed, the idle user will be automatically kicked out (read as logged out). See [how to automatically log BASH / TCSH / SSH users][24] out after a period of inactivity for more details. + +#### 14. Enable a warning banner for ssh users + +Set a warning banner by updating sshd_config with the following line: +`Banner /etc/issue` +Sample /etc/issue file: +``` +---------------------------------------------------------------------------------------------- +You are accessing a XYZ Government (XYZG) Information System (IS) that is provided for authorized use only. +By using this IS (which includes any device attached to this IS), you consent to the following conditions: + ++ The XYZG routinely intercepts and monitors communications on this IS for purposes including, but not limited to, +penetration testing, COMSEC monitoring, network operations and defense, personnel misconduct (PM), +law enforcement (LE), and counterintelligence (CI) investigations. + ++ At any time, the XYZG may inspect and seize data stored on this IS. + ++ Communications using, or data stored on, this IS are not private, are subject to routine monitoring, +interception, and search, and may be disclosed or used for any XYZG authorized purpose. + ++ This IS includes security measures (e.g., authentication and access controls) to protect XYZG interests--not +for your personal benefit or privacy. + ++ Notwithstanding the above, using this IS does not constitute consent to PM, LE or CI investigative searching +or monitoring of the content of privileged communications, or work product, related to personal representation +or services by attorneys, psychotherapists, or clergy, and their assistants. Such communications and work +product are private and confidential. See User Agreement for details. +---------------------------------------------------------------------------------------------- + +``` + +Above is a standard sample, consult your legal team for specific user agreement and legal notice details. + +#### 15. Disable .rhosts files (verification) + +Don't read the user's ~/.rhosts and ~/.shosts files. Update sshd_config with the following settings: +`IgnoreRhosts yes` +SSH can emulate the behavior of the obsolete rsh command, just disable insecure access via RSH. + +#### 16. Disable host-based authentication (verification) + +To disable host-based authentication, update sshd_config with the following option: +`HostbasedAuthentication no` + +#### 17. Patch OpenSSH and operating systems + +It is recommended that you use tools such as [yum][25], [apt-get][26], [freebsd-update][27] and others to keep systems up to date with the latest security patches: + +#### 18. Chroot OpenSSH (Lock down users to their home directories) + +By default users are allowed to browse the server directories such as /etc/, /bin and so on. You can protect ssh, using os based chroot or use [special tools such as rssh][28]. With the release of OpenSSH 4.8p1 or 4.9p1, you no longer have to rely on third-party hacks such as rssh or complicated chroot(1) setups to lock users to their home directories. See [this blog post][29] about new ChrootDirectory directive to lock down users to their home directories. + +#### 19. Disable OpenSSH server on client computer + +Workstations and laptop can work without OpenSSH server. If you do not provide the remote login and file transfer capabilities of SSH, disable and remove the SSHD server. CentOS / RHEL users can disable and remove openssh-server with the [yum command][30]: +`$ sudo yum erase openssh-server` +Debian / Ubuntu Linux user can disable and remove the same with the [apt command][31]/[apt-get command][32]: +`$ sudo apt-get remove openssh-server` +You may need to update your iptables script to remove ssh exception rule. Under CentOS / RHEL / Fedora edit the files /etc/sysconfig/iptables and /etc/sysconfig/ip6tables. Once done [restart iptables][33] service: +``` +# service iptables restart +# service ip6tables restart +``` + +#### 20. Bonus tips from Mozilla + +If you are using OpenSSH version 6.7+ or newer try [following][34] settings: +``` +#################[ WARNING ]######################## +# Do not use any setting blindly. Read sshd_config # +# man page. You must understand cryptography to # +# tweak following settings. Otherwise use defaults # +#################################################### + +# Supported HostKey algorithms by order of preference. +HostKey /etc/ssh/ssh_host_ed25519_key +HostKey /etc/ssh/ssh_host_rsa_key +HostKey /etc/ssh/ssh_host_ecdsa_key + +# Specifies the available KEX (Key Exchange) algorithms. +KexAlgorithms curve25519-sha256@libssh.org,ecdh-sha2-nistp521,ecdh-sha2-nistp384,ecdh-sha2-nistp256,diffie-hellman-group-exchange-sha256 + +# Specifies the ciphers allowed +Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr + +#Specifies the available MAC (message authentication code) algorithms +MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512,hmac-sha2-256,umac-128@openssh.com + +# LogLevel VERBOSE logs user's key fingerprint on login. Needed to have a clear audit track of which key was using to log in. +LogLevel VERBOSE + +# Log sftp level file access (read/write/etc.) that would not be easily logged otherwise. +Subsystem sftp /usr/lib/ssh/sftp-server -f AUTHPRIV -l INFO +``` + +You can grab list of cipher and alog supported by your OpenSSH server using the following commands: +``` +$ ssh -Q cipher +$ ssh -Q cipher-auth +$ ssh -Q mac +$ ssh -Q kex +$ ssh -Q key +``` +[![OpenSSH Security Tutorial Query Ciphers and algorithms choice][35]][35] + +#### How do I test sshd_config file and restart/reload my SSH server? + +To [check the validity of the configuration file and sanity of the keys][36] for any errors before restarting sshd, run: +`$ sudo sshd -t` +Extended test mode: +`$ sudo sshd -T` +Finally [restart sshd on a Linux or Unix like systems][37] as per your distro version: +``` +$ [sudo systemctl start ssh][38] ## Debian/Ubunt Linux## +$ [sudo systemctl restart sshd.service][39] ## CentOS/RHEL/Fedora Linux## +$ doas /etc/rc.d/sshd restart ## OpenBSD## +$ sudo service sshd restart ## FreeBSD## +``` + +#### Other susggesions + + 1. [Tighter SSH security with 2FA][40] - Multi-Factor authentication can be enabled with [OATH Toolkit][41] or [DuoSecurity][42]. + 2. [Use keychain based authentication][43] - keychain is a special bash script designed to make key-based authentication incredibly convenient and flexible. It offers various security benefits over passphrase-free keys + + + +#### See also: + + * The [official OpenSSH][44] project. + * Man pages: sshd(8),ssh(1),ssh-add(1),ssh-agent(1) + + + +If you have a technique or handy software not mentioned here, please share in the comments below to help your fellow readers keep their OpenSSH based server secure. + +#### About the author + +The author is the creator of nixCraft and a seasoned sysadmin and a trainer for the Linux operating system/Unix shell scripting. He has worked with global clients and in various industries, including IT, education, defense and space research, and the nonprofit sector. Follow him on [Twitter][45], [Facebook][46], [Google+][47]. + +-------------------------------------------------------------------------------- + +via: https://www.cyberciti.biz/tips/linux-unix-bsd-openssh-server-best-practices.html + +作者：[Vivek Gite][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.cyberciti.biz +[1]:https://www.cyberciti.biz/media/new/tips/2009/07/openSSH_logo.png +[2]:https://isc.sans.edu/diary/OpenSSH+Rumors/6742 +[3]:https://www.cyberciti.biz/tips/wp-content/uploads/2009/07/OpenSSH-server-security-best-practices.png +[4]:https://www.cyberciti.biz/faq/how-to-create-a-sudo-user-on-ubuntu-linux-server/ +[5]:https://www.cyberciti.biz/faq/unix-linux-id-command-examples-usage-syntax/ (See Linux/Unix id command examples for more info) +[6]:https://www.cyberciti.biz/faq/how-to-disable-ssh-password-login-on-linux/ +[7]:https://www.cyberciti.biz/tips/linux-pam-configuration-that-allows-or-deny-login-via-the-sshd-server.html +[8]:https://www.cyberciti.biz/tips/openssh-deny-or-restrict-access-to-users-and-groups.html +[9]:https://www.cyberciti.biz/tips/linux-check-passwords-against-a-dictionary-attack.html +[10]:https://www.cyberciti.biz/faq/unix-linux-password-cracking-john-the-ripper/ +[11]:https://www.cyberciti.biz/faq/rhel-fedorta-linux-iptables-firewall-configuration-tutorial/ +[12]:https://www.cyberciti.biz/faq/howto-configure-setup-firewall-with-ufw-on-ubuntu-linux/ +[13]:https://www.cyberciti.biz/faq/ufw-allow-incoming-ssh-connections-from-a-specific-ip-address-subnet-on-ubuntu-debian/ +[14]:https://www.cyberciti.biz/tips/linux-iptables-examples.html +[15]:https://bash.cyberciti.biz/firewall/pf-firewall-script/ +[16]:https://www.cyberciti.biz/faq/tcp-wrappers-hosts-allow-deny-tutorial/ +[17]:https://www.cyberciti.biz/faq/block-ssh-attacks-with-denyhosts/ +[18]:https://www.cyberciti.biz/faq/rhel-linux-block-ssh-dictionary-brute-force-attacks/ +[19]:https://www.fail2ban.org +[20]:https://sshguard.sourceforge.net/ +[21]:http://www.bsdconsulting.no/tools/ +[22]:https://savannah.nongnu.org/projects/ipqbdb/ +[23]:https://en.wikipedia.org/wiki/Port_knocking +[24]:https://www.cyberciti.biz/faq/linux-unix-login-bash-shell-force-time-outs/ +[25]:https://www.cyberciti.biz/faq/rhel-centos-fedora-linux-yum-command-howto/ +[26]:https://www.cyberciti.biz/tips/linux-debian-package-management-cheat-sheet.html +[27]:https://www.cyberciti.biz/tips/howto-keep-freebsd-system-upto-date.html +[28]:https://www.cyberciti.biz/tips/rhel-centos-linux-install-configure-rssh-shell.html +[29]:https://www.debian-administration.org/articles/590 +[30]:https://www.cyberciti.biz/faq/rhel-centos-fedora-linux-yum-command-howto/ (See Linux/Unix yum command examples for more info) +[31]:https://www.cyberciti.biz/faq/ubuntu-lts-debian-linux-apt-command-examples/ (See Linux/Unix apt command examples for more info) +[32]:https://www.cyberciti.biz/tips/linux-debian-package-management-cheat-sheet.html (See Linux/Unix apt-get command examples for more info) +[33]:https://www.cyberciti.biz/faq/howto-rhel-linux-open-port-using-iptables/ +[34]:https://wiki.mozilla.org/Security/Guidelines/OpenSSH +[35]:https://www.cyberciti.biz/tips/wp-content/uploads/2009/07/OpenSSH-Security-Tutorial-Query-Ciphers-and-algorithms-choice.jpg +[36]:https://www.cyberciti.biz/tips/checking-openssh-sshd-configuration-syntax-errors.html +[37]:https://www.cyberciti.biz/faq/howto-restart-ssh/ +[38]:https://www.cyberciti.biz/faq/howto-start-stop-ssh-server/ (Restart sshd on a Debian/Ubuntu Linux) +[39]:https://www.cyberciti.biz/faq/centos-stop-start-restart-sshd-command/ (Restart sshd on a CentOS/RHEL/Fedora Linux) +[40]:https://www.cyberciti.biz/open-source/howto-protect-linux-ssh-login-with-google-authenticator/ +[41]:http://www.nongnu.org/oath-toolkit/ +[42]:https://duo.com +[43]:https://www.cyberciti.biz/faq/ssh-passwordless-login-with-keychain-for-scripts/ +[44]:https://www.openssh.com/ +[45]:https://twitter.com/nixcraft +[46]:https://facebook.com/nixcraft +[47]:https://plus.google.com/+CybercitiBiz +[48]:https://www.cyberciti.biz/faq/ssh-passwordless-login-with-keychain-for-scripts/ +[49]:https://www.cyberciti.biz/faq/noninteractive-shell-script-ssh-password-provider/ +[50]:https://www.cyberciti.biz/faq/how-to-set-up-ssh-keys-on-linux-unix/ +[51]:https://www.cyberciti.biz/faq/how-to-upload-ssh-public-key-to-as-authorized_key-using-ansible/ +[52]:https://www.cyberciti.biz/faq/generating-random-password/ +[53]:https://www.cyberciti.biz/faq/linux-unix-generating-passwords-command/ +[54]:https://www.cyberciti.biz/faq/linux-random-password-generator/ +[55]:https://www.cyberciti.biz/faq/debian-ubuntu-linux-iptables-knockd-port-knocking-tutorial/ diff --git a/sources/tech/20130319 Linux - Unix Bash Shell List All Builtin Commands.md b/sources/tech/20130319 Linux - Unix Bash Shell List All Builtin Commands.md new file mode 100644 index 0000000000..230ca95cba --- /dev/null +++ b/sources/tech/20130319 Linux - Unix Bash Shell List All Builtin Commands.md @@ -0,0 +1,170 @@ +Linux / Unix Bash Shell List All Builtin Commands +====== + +Builtin commands contained within the bash shell itself. How do I list all built-in bash commands on Linux / Apple OS X / *BSD / Unix like operating systems without reading large size bash man page? + +A shell builtin is nothing but command or a function, called from a shell, that is executed directly in the shell itself. The bash shell executes the command directly, without invoking another program. You can view information for Bash built-ins with help command. There are different types of built-in commands. + + +### built-in command types + + 1. Bourne Shell Builtins: Builtin commands inherited from the Bourne Shell. + 2. Bash Builtins: Table of builtins specific to Bash. + 3. Modifying Shell Behavior: Builtins to modify shell attributes and optional behavior. + 4. Special Builtins: Builtin commands classified specially by POSIX. + + + +### How to see all bash builtins + +Type the following command: +``` +$ help +$ help | less +$ help | grep read +``` + +Sample outputs: +``` +GNU bash, version 4.1.5(1)-release (x86_64-pc-linux-gnu) +These shell commands are defined internally. Type `help' to see this list. +Type `help name' to find out more about the function `name'. +Use `info bash' to find out more about the shell in general. +Use `man -k' or `info' to find out more about commands not in this list. + +A star (*) next to a name means that the command is disabled. + + job_spec [&] history [-c] [-d offset] [n] or hist> + (( expression )) if COMMANDS; then COMMANDS; [ elif C> + . filename [arguments] jobs [-lnprs] [jobspec ...] or jobs > + : kill [-s sigspec | -n signum | -sigs> + [ arg... ] let arg [arg ...] + [[ expression ]] local [option] name[=value] ... + alias [-p] [name[=value] ... ] logout [n] + bg [job_spec ...] mapfile [-n count] [-O origin] [-s c> + bind [-lpvsPVS] [-m keymap] [-f filen> popd [-n] [+N | -N] + break [n] printf [-v var] format [arguments] + builtin [shell-builtin [arg ...]] pushd [-n] [+N | -N | dir] + caller [expr] pwd [-LP] + case WORD in [PATTERN [| PATTERN]...)> read [-ers] [-a array] [-d delim] [-> + cd [-L|-P] [dir] readarray [-n count] [-O origin] [-s> + command [-pVv] command [arg ...] readonly [-af] [name[=value] ...] or> + compgen [-abcdefgjksuv] [-o option] > return [n] + complete [-abcdefgjksuv] [-pr] [-DE] > select NAME [in WORDS ... ;] do COMM> + compopt [-o|+o option] [-DE] [name ..> set [--abefhkmnptuvxBCHP] [-o option> + continue [n] shift [n] + coproc [NAME] command [redirections] shopt [-pqsu] [-o] [optname ...] + declare [-aAfFilrtux] [-p] [name[=val> source filename [arguments] + dirs [-clpv] [+N] [-N] suspend [-f] + disown [-h] [-ar] [jobspec ...] test [expr] + echo [-neE] [arg ...] time [-p] pipeline + enable [-a] [-dnps] [-f filename] [na> times + eval [arg ...] trap [-lp] [[arg] signal_spec ...] + exec [-cl] [-a name] [command [argume> true + exit [n] type [-afptP] name [name ...] + export [-fn] [name[=value] ...] or ex> typeset [-aAfFilrtux] [-p] name[=val> + false ulimit [-SHacdefilmnpqrstuvx] [limit> + fc [-e ename] [-lnr] [first] [last] o> umask [-p] [-S] [mode] + fg [job_spec] unalias [-a] name [name ...] + for NAME [in WORDS ... ] ; do COMMAND> unset [-f] [-v] [name ...] + for (( exp1; exp2; exp3 )); do COMMAN> until COMMANDS; do COMMANDS; done + function name { COMMANDS ; } or name > variables - Names and meanings of so> + getopts optstring name [arg] wait [id] + hash [-lr] [-p pathname] [-dt] [name > while COMMANDS; do COMMANDS; done + help [-dms] [pattern ...] { COMMANDS ; } +``` + +### Viewing information for Bash built-ins + +To get detailed info run: +``` +help command +help read +``` +To just get a list of all built-ins with a short description, execute: + +`$ help -d` + +### Find syntax and other options for builtins + +Use the following syntax ' to find out more about the builtins commands: +``` +help name +help cd +help fg +help for +help read +help : +``` + +Sample outputs: +``` +:: : + Null command. +  + No effect; the command does nothing. +  + Exit Status: + Always succeeds +``` + +### Find out if a command is internal (builtin) or external + +Use the type command or command command: +``` +type -a command-name-here +type -a cd +type -a uname +type -a : +type -a ls +``` + + +OR +``` +type -a cd uname : ls uname +``` + +Sample outputs: +``` +cd is a shell builtin +uname is /bin/uname +: is a shell builtin +ls is aliased to `ls --color=auto' +ls is /bin/ls +l is a function +l () +{ + ls --color=auto +} + +``` + +OR +``` +command -V ls +command -V cd +command -V foo +``` + +[![View list bash built-ins command info on Linux or Unix][1]][1] + +### about the author + +The author is the creator of nixCraft and a seasoned sysadmin and a trainer for the Linux operating system/Unix shell scripting. He has worked with global clients and in various industries, including IT, education, defense and space research, and the nonprofit sector. Follow him on [Twitter][2], [Facebook][3], [Google+][4]. + +-------------------------------------------------------------------------------- + +via: https://www.cyberciti.biz/faq/linux-unix-bash-shell-list-all-builtin-commands/ + +作者：[Vivek Gite][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.cyberciti.biz +[1]:https://www.cyberciti.biz/media/new/faq/2013/03/View-list-bash-built-ins-command-info-on-Linux-or-Unix.jpg +[2]:https://twitter.com/nixcraft +[3]:https://facebook.com/nixcraft +[4]:https://plus.google.com/+CybercitiBiz diff --git a/sources/tech/20140225 How To Safely Generate A Random Number - Quarrelsome.md b/sources/tech/20140225 How To Safely Generate A Random Number - Quarrelsome.md new file mode 100644 index 0000000000..39314e7ab4 --- /dev/null +++ b/sources/tech/20140225 How To Safely Generate A Random Number - Quarrelsome.md @@ -0,0 +1,94 @@ +How To Safely Generate A Random Number — Quarrelsome +====== +### Use urandom + +Use [urandom][1]. Use [urandom][2]. Use [urandom][3]. Use [urandom][4]. Use [urandom][5]. Use [urandom][6]. + +### But what about for crypto keys? + +Still [urandom][6]. + +### Why not {SecureRandom, OpenSSL, havaged, &c}? + +These are userspace CSPRNGs. You want to use the kernel’s CSPRNG, because: + + * The kernel has access to raw device entropy. + + * It can promise not to share the same state between applications. + + * A good kernel CSPRNG, like FreeBSD’s, can also promise not to feed you random data before it’s seeded. + + + + +Study the last ten years of randomness failures and you’ll read a litany of userspace randomness failures. [Debian’s OpenSSH debacle][7]? Userspace random. Android Bitcoin wallets [repeating ECDSA k’s][8]? Userspace random. Gambling sites with predictable shuffles? Userspace random. + +Userspace OpenSSL also seeds itself from “from uninitialized memory, magical fairy dust and unicorn horns” generators almost always depend on the kernel’s generator anyways. Even if they don’t, the security of your whole system sure does. **A userspace CSPRNG doesn’t add defense-in-depth; instead, it creates two single points of failure.** + +### Doesn’t the man page say to use /dev/random? + +You But, more on this later. Stay your pitchforks. should ignore the man page. Don’t use /dev/random. The distinction between /dev/random and /dev/urandom is a Unix design wart. The man page doesn’t want to admit that, so it invents a security concern that doesn’t really exist. Consider the cryptographic advice in random(4) an urban legend and get on with your life. + +### But what if I need real random values, not psuedorandom values? + +Both urandom and /dev/random provide the same kind of randomness. Contrary to popular belief, /dev/random doesn’t provide “true random” data. For cryptography, you don’t usually want “true random”. + +Both urandom and /dev/random are based on a simple idea. Their design is closely related to that of a stream cipher: a small secret is stretched into an indefinite stream of unpredictable values. Here the secrets are “entropy”, and the stream is “output”. + +Only on Linux are /dev/random and urandom still meaningfully different. The Linux kernel CSPRNG rekeys itself regularly (by collecting more entropy). But /dev/random also tries to keep track of how much entropy remains in its kernel pool, and will occasionally go on strike if it decides not enough remains. This design is as silly as I’ve made it sound; it’s akin to AES-CTR blocking based on how much “key” is left in the “keystream”. + +If you use /dev/random instead of urandom, your program will unpredictably (or, if you’re an attacker, very predictably) hang when Linux gets confused about how its own RNG works. Using /dev/random will make your programs less stable, but it won’t make them any more cryptographically safe. + +### There’s a catch here, isn’t there? + +No, but there’s a Linux kernel bug you might want to know about, even though it doesn’t change which RNG you should use. + +On Linux, if your software runs immediately at boot, and/or the OS has just been installed, your code might be in a race with the RNG. That’s bad, because if you win the race, there could be a window of time where you get predictable outputs from urandom. This is a bug in Linux, and you need to know about it if you’re building platform-level code for a Linux embedded device. + +This is indeed a problem with urandom (and not /dev/random) on Linux. It’s also a [bug in the Linux kernel][9]. But it’s also easily fixed in userland: at boot, seed urandom explicitly. Most Linux distributions have done this for a long time. But don’t switch to a different CSPRNG. + +### What about on other operating systems? + +FreeBSD and OS X do away with the distinction between urandom and /dev/random; the two devices behave identically. Unfortunately, the man page does a poor job of explaining why this is, and perpetuates the myth that Linux urandom is scary. + +FreeBSD’s kernel crypto RNG doesn’t block regardless of whether you use /dev/random or urandom. Unless it hasn’t been seeded, in which case both block. This behavior, unlike Linux’s, makes sense. Linux should adopt it. But if you’re an app developer, this makes little difference to you: Linux, FreeBSD, iOS, whatever: use urandom. + +### tl;dr + +Use urandom. + +### Epilog + +[ruby-trunk Feature #9569][10] + +> Right now, SecureRandom.random_bytes tries to detect an OpenSSL to use before it tries to detect /dev/urandom. I think it should be the other way around. In both cases, you just need random bytes to unpack, so SecureRandom could skip the middleman (and second point of failure) and just talk to /dev/urandom directly if it’s available. + +Resolution: + +> /dev/urandom is not suitable to be used to generate directly session keys and other application level random data which is generated frequently. +> +> [the] random(4) [man page] on GNU/Linux [says]… + +Thanks to Matthew Green, Nate Lawson, Sean Devlin, Coda Hale, and Alex Balducci for reading drafts of this. Fair warning: Matthew only mostly agrees with me. + +-------------------------------------------------------------------------------- + +via: https://sockpuppet.org/blog/2014/02/25/safely-generate-random-numbers/ + +作者：[Thomas;Erin;Matasano][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://sockpuppet.org/blog +[1]:http://blog.cr.yp.to/20140205-entropy.html +[2]:http://cr.yp.to/talks/2011.09.28/slides.pdf +[3]:http://golang.org/src/pkg/crypto/rand/rand_unix.go +[4]:http://security.stackexchange.com/questions/3936/is-a-rand-from-dev-urandom-secure-for-a-login-key +[5]:http://stackoverflow.com/a/5639631 +[6]:https://twitter.com/bramcohen/status/206146075487240194 +[7]:http://research.swtch.com/openssl +[8]:http://arstechnica.com/security/2013/08/google-confirms-critical-android-crypto-flaw-used-in-5700-bitcoin-heist/ +[9]:https://factorable.net/weakkeys12.extended.pdf +[10]:https://bugs.ruby-lang.org/issues/9569 diff --git a/sources/tech/20150615 Let-s Build A Simple Interpreter. Part 1..md b/sources/tech/20150615 Let-s Build A Simple Interpreter. Part 1..md index 9a815f2852..592e43e067 100644 --- a/sources/tech/20150615 Let-s Build A Simple Interpreter. Part 1..md +++ b/sources/tech/20150615 Let-s Build A Simple Interpreter. Part 1..md @@ -1,4 +1,5 @@ -// Translating by Linchenguang.... +BriFuture is translating this article + Let’s Build A Simple Interpreter. Part 1. ====== diff --git a/sources/tech/20150812 Let-s Build A Simple Interpreter. Part 3..md b/sources/tech/20150812 Let-s Build A Simple Interpreter. Part 3..md deleted file mode 100644 index d9deb9f50e..0000000000 --- a/sources/tech/20150812 Let-s Build A Simple Interpreter. Part 3..md +++ /dev/null @@ -1,342 +0,0 @@ -BriFuture is Translating this article - -Let’s Build A Simple Interpreter. Part 3. -====== - -I woke up this morning and I thought to myself: "Why do we find it so difficult to learn a new skill?" - -I don't think it's just because of the hard work. I think that one of the reasons might be that we spend a lot of time and hard work acquiring knowledge by reading and watching and not enough time translating that knowledge into a skill by practicing it. Take swimming, for example. You can spend a lot of time reading hundreds of books about swimming, talk for hours with experienced swimmers and coaches, watch all the training videos available, and you still will sink like a rock the first time you jump in the pool. - -The bottom line is: it doesn't matter how well you think you know the subject - you have to put that knowledge into practice to turn it into a skill. To help you with the practice part I put exercises into [Part 1][1] and [Part 2][2] of the series. And yes, you will see more exercises in today's article and in future articles, I promise :) - -Okay, let's get started with today's material, shall we? - - -So far, you've learned how to interpret arithmetic expressions that add or subtract two integers like "7 + 3" or "12 - 9". Today I'm going to talk about how to parse (recognize) and interpret arithmetic expressions that have any number of plus or minus operators in it, for example "7 - 3 + 2 - 1". - -Graphically, the arithmetic expressions in this article can be represented with the following syntax diagram: - -![][3] - -What is a syntax diagram? A **syntax diagram** is a graphical representation of a programming language 's syntax rules. Basically, a syntax diagram visually shows you which statements are allowed in your programming language and which are not. - -Syntax diagrams are pretty easy to read: just follow the paths indicated by the arrows. Some paths indicate choices. And some paths indicate loops. - -You can read the above syntax diagram as following: a term optionally followed by a plus or minus sign, followed by another term, which in turn is optionally followed by a plus or minus sign followed by another term and so on. You get the picture, literally. You might wonder what a "term" is. For the purpose of this article a "term" is just an integer. - -Syntax diagrams serve two main purposes: - - * They graphically represent the specification (grammar) of a programming language. - * They can be used to help you write your parser - you can map a diagram to code by following simple rules. - - - -You've learned that the process of recognizing a phrase in the stream of tokens is called **parsing**. And the part of an interpreter or compiler that performs that job is called a **parser**. Parsing is also called **syntax analysis** , and the parser is also aptly called, you guessed it right, a **syntax analyzer**. - -According to the syntax diagram above, all of the following arithmetic expressions are valid: - - * 3 - * 3 + 4 - * 7 - 3 + 2 - 1 - - - -Because syntax rules for arithmetic expressions in different programming languages are very similar we can use a Python shell to "test" our syntax diagram. Launch your Python shell and see for yourself: -``` ->>> 3 -3 ->>> 3 + 4 -7 ->>> 7 - 3 + 2 - 1 -5 -``` - -No surprises here. - -The expression "3 + " is not a valid arithmetic expression though because according to the syntax diagram the plus sign must be followed by a term (integer), otherwise it's a syntax error. Again, try it with a Python shell and see for yourself: -``` ->>> 3 + - File "", line 1 - 3 + - ^ -SyntaxError: invalid syntax -``` - -It's great to be able to use a Python shell to do some testing but let's map the above syntax diagram to code and use our own interpreter for testing, all right? - -You know from the previous articles ([Part 1][1] and [Part 2][2]) that the expr method is where both our parser and interpreter live. Again, the parser just recognizes the structure making sure that it corresponds to some specifications and the interpreter actually evaluates the expression once the parser has successfully recognized (parsed) it. - -The following code snippet shows the parser code corresponding to the diagram. The rectangular box from the syntax diagram (term) becomes a term method that parses an integer and the expr method just follows the syntax diagram flow: -``` -def term(self): - self.eat(INTEGER) - -def expr(self): - # set current token to the first token taken from the input - self.current_token = self.get_next_token() - - self.term() - while self.current_token.type in (PLUS, MINUS): - token = self.current_token - if token.type == PLUS: - self.eat(PLUS) - self.term() - elif token.type == MINUS: - self.eat(MINUS) - self.term() -``` - -You can see that expr first calls the term method. Then the expr method has a while loop which can execute zero or more times. And inside the loop the parser makes a choice based on the token (whether it's a plus or minus sign). Spend some time proving to yourself that the code above does indeed follow the syntax diagram flow for arithmetic expressions. - -The parser itself does not interpret anything though: if it recognizes an expression it's silent and if it doesn't, it throws out a syntax error. Let's modify the expr method and add the interpreter code: -``` -def term(self): - """Return an INTEGER token value""" - token = self.current_token - self.eat(INTEGER) - return token.value - -def expr(self): - """Parser / Interpreter """ - # set current token to the first token taken from the input - self.current_token = self.get_next_token() - - result = self.term() - while self.current_token.type in (PLUS, MINUS): - token = self.current_token - if token.type == PLUS: - self.eat(PLUS) - result = result + self.term() - elif token.type == MINUS: - self.eat(MINUS) - result = result - self.term() - - return result -``` - -Because the interpreter needs to evaluate an expression the term method was modified to return an integer value and the expr method was modified to perform addition and subtraction at the appropriate places and return the result of interpretation. Even though the code is pretty straightforward I recommend spending some time studying it. - -Le's get moving and see the complete code of the interpreter now, okay? - -Here is the source code for your new version of the calculator that can handle valid arithmetic expressions containing integers and any number of addition and subtraction operators: -``` -# Token types -# -# EOF (end-of-file) token is used to indicate that -# there is no more input left for lexical analysis -INTEGER, PLUS, MINUS, EOF = 'INTEGER', 'PLUS', 'MINUS', 'EOF' - - -class Token(object): - def __init__(self, type, value): - # token type: INTEGER, PLUS, MINUS, or EOF - self.type = type - # token value: non-negative integer value, '+', '-', or None - self.value = value - - def __str__(self): - """String representation of the class instance. - - Examples: - Token(INTEGER, 3) - Token(PLUS, '+') - """ - return 'Token({type}, {value})'.format( - type=self.type, - value=repr(self.value) - ) - - def __repr__(self): - return self.__str__() - - -class Interpreter(object): - def __init__(self, text): - # client string input, e.g. "3 + 5", "12 - 5 + 3", etc - self.text = text - # self.pos is an index into self.text - self.pos = 0 - # current token instance - self.current_token = None - self.current_char = self.text[self.pos] - - ########################################################## - # Lexer code # - ########################################################## - def error(self): - raise Exception('Invalid syntax') - - def advance(self): - """Advance the `pos` pointer and set the `current_char` variable.""" - self.pos += 1 - if self.pos > len(self.text) - 1: - self.current_char = None # Indicates end of input - else: - self.current_char = self.text[self.pos] - - def skip_whitespace(self): - while self.current_char is not None and self.current_char.isspace(): - self.advance() - - def integer(self): - """Return a (multidigit) integer consumed from the input.""" - result = '' - while self.current_char is not None and self.current_char.isdigit(): - result += self.current_char - self.advance() - return int(result) - - def get_next_token(self): - """Lexical analyzer (also known as scanner or tokenizer) - - This method is responsible for breaking a sentence - apart into tokens. One token at a time. - """ - while self.current_char is not None: - - if self.current_char.isspace(): - self.skip_whitespace() - continue - - if self.current_char.isdigit(): - return Token(INTEGER, self.integer()) - - if self.current_char == '+': - self.advance() - return Token(PLUS, '+') - - if self.current_char == '-': - self.advance() - return Token(MINUS, '-') - - self.error() - - return Token(EOF, None) - - ########################################################## - # Parser / Interpreter code # - ########################################################## - def eat(self, token_type): - # compare the current token type with the passed token - # type and if they match then "eat" the current token - # and assign the next token to the self.current_token, - # otherwise raise an exception. - if self.current_token.type == token_type: - self.current_token = self.get_next_token() - else: - self.error() - - def term(self): - """Return an INTEGER token value.""" - token = self.current_token - self.eat(INTEGER) - return token.value - - def expr(self): - """Arithmetic expression parser / interpreter.""" - # set current token to the first token taken from the input - self.current_token = self.get_next_token() - - result = self.term() - while self.current_token.type in (PLUS, MINUS): - token = self.current_token - if token.type == PLUS: - self.eat(PLUS) - result = result + self.term() - elif token.type == MINUS: - self.eat(MINUS) - result = result - self.term() - - return result - - -def main(): - while True: - try: - # To run under Python3 replace 'raw_input' call - # with 'input' - text = raw_input('calc> ') - except EOFError: - break - if not text: - continue - interpreter = Interpreter(text) - result = interpreter.expr() - print(result) - - -if __name__ == '__main__': - main() -``` - -Save the above code into the calc3.py file or download it directly from [GitHub][4]. Try it out. See for yourself that it can handle arithmetic expressions that you can derive from the syntax diagram I showed you earlier. - -Here is a sample session that I ran on my laptop: -``` -$ python calc3.py -calc> 3 -3 -calc> 7 - 4 -3 -calc> 10 + 5 -15 -calc> 7 - 3 + 2 - 1 -5 -calc> 10 + 1 + 2 - 3 + 4 + 6 - 15 -5 -calc> 3 + -Traceback (most recent call last): - File "calc3.py", line 147, in - main() - File "calc3.py", line 142, in main - result = interpreter.expr() - File "calc3.py", line 123, in expr - result = result + self.term() - File "calc3.py", line 110, in term - self.eat(INTEGER) - File "calc3.py", line 105, in eat - self.error() - File "calc3.py", line 45, in error - raise Exception('Invalid syntax') -Exception: Invalid syntax -``` - - -Remember those exercises I mentioned at the beginning of the article: here they are, as promised :) - -![][5] - - * Draw a syntax diagram for arithmetic expressions that contain only multiplication and division, for example "7 0_sync_master.sh 1_add_new_article_manual.sh 1_add_new_article_newspaper.sh 2_start_translating.sh 3_continue_the_work.sh 4_finish.sh 5_pause.sh base.sh env format.test lctt.cfg parse_url_by_manual.sh parse_url_by_newspaper.py parse_url_by_newspaper.sh README.org reformat.sh 4 / 2 0_sync_master.sh 1_add_new_article_manual.sh 1_add_new_article_newspaper.sh 2_start_translating.sh 3_continue_the_work.sh 4_finish.sh 5_pause.sh base.sh env format.test lctt.cfg parse_url_by_manual.sh parse_url_by_newspaper.py parse_url_by_newspaper.sh README.org reformat.sh 3". Seriously, just grab a pen or a pencil and try to draw one. - * Modify the source code of the calculator to interpret arithmetic expressions that contain only multiplication and division, for example "7 0_sync_master.sh 1_add_new_article_manual.sh 1_add_new_article_newspaper.sh 2_start_translating.sh 3_continue_the_work.sh 4_finish.sh 5_pause.sh base.sh env format.test lctt.cfg parse_url_by_manual.sh parse_url_by_newspaper.py parse_url_by_newspaper.sh README.org reformat.sh 4 / 2 * 3". - * Write an interpreter that handles arithmetic expressions like "7 - 3 + 2 - 1" from scratch. Use any programming language you're comfortable with and write it off the top of your head without looking at the examples. When you do that, think about components involved: a lexer that takes an input and converts it into a stream of tokens, a parser that feeds off the stream of the tokens provided by the lexer and tries to recognize a structure in that stream, and an interpreter that generates results after the parser has successfully parsed (recognized) a valid arithmetic expression. String those pieces together. Spend some time translating the knowledge you've acquired into a working interpreter for arithmetic expressions. - - - -**Check your understanding.** - - 1. What is a syntax diagram? - 2. What is syntax analysis? - 3. What is a syntax analyzer? - - - - -Hey, look! You read all the way to the end. Thanks for hanging out here today and don't forget to do the exercises. :) I'll be back next time with a new article - stay tuned. - - --------------------------------------------------------------------------------- - -via: https://ruslanspivak.com/lsbasi-part3/ - -作者：[Ruslan Spivak][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://ruslanspivak.com -[1]:http://ruslanspivak.com/lsbasi-part1/ (Part 1) -[2]:http://ruslanspivak.com/lsbasi-part2/ (Part 2) -[3]:https://ruslanspivak.com/lsbasi-part3/lsbasi_part3_syntax_diagram.png -[4]:https://github.com/rspivak/lsbasi/blob/master/part3/calc3.py -[5]:https://ruslanspivak.com/lsbasi-part3/lsbasi_part3_exercises.png diff --git a/sources/tech/20160606 Learn your tools Navigating your Git History.md b/sources/tech/20160606 Learn your tools Navigating your Git History.md new file mode 100644 index 0000000000..2d17cac30b --- /dev/null +++ b/sources/tech/20160606 Learn your tools Navigating your Git History.md @@ -0,0 +1,161 @@ +Learn your tools: Navigating your Git History +============================================================ + +Starting a greenfield application everyday is nearly impossible, especially in your daily job. In fact, most of us are facing (somewhat) legacy codebases on a daily basis, and regaining the context of why some feature, or line of code exists in the codebase is very important. This is where `git`, the distributed version control system, is invaluable. Let’s dive in and see how we can use our `git` history and easily navigate through it. + +### Git history + +First and foremost, what is `git` history? As the name says, it is the commit history of a `git` repo. It contains a bunch of commit messages, with their authors’ name, the commit hash and the date of the commit. The easiest way to see the history of a `git`repo, is the `git log` command. + +Sidenote: For the purpose of this post, we will use Ruby on Rails’ repo, the `master`branch. The reason behind this is because Rails has a very good `git` history, with nice commit messages, references and explanations behind every change. Given the size of the codebase, the age and the number of maintainers, it’s certainly one of the best repositories that I have seen. Of course, I am not saying there are no other repositories built with good `git` practices, but this is one that has caught my eye. + +So back to Rails’ repo. If you run `git log` in the Rails’ repo, you will see something like this: + +``` +commit 66ebbc4952f6cfb37d719f63036441ef98149418Author: Arthur Neves Date: Fri Jun 3 17:17:38 2016 -0400 Dont re-define class SQLite3Adapter on test We were declaring in a few tests, which depending of the order load will cause an error, as the super class could change. see https://github.com/rails/rails/commit/ac1c4e141b20c1067af2c2703db6e1b463b985da#commitcomment-17731383commit 755f6bf3d3d568bc0af2c636be2f6df16c651eb1Merge: 4e85538 f7b850eAuthor: Eileen M. Uchitelle Date: Fri Jun 3 10:21:49 2016 -0400 Merge pull request #25263 from abhishekjain16/doc_accessor_thread [skip ci] Fix grammarcommit f7b850ec9f6036802339e965c8ce74494f731b4aAuthor: Abhishek Jain Date: Fri Jun 3 16:49:21 2016 +0530 [skip ci] Fix grammarcommit 4e85538dddf47877cacc65cea6c050e349af0405Merge: 082a515 cf2158cAuthor: Vijay Dev Date: Fri Jun 3 14:00:47 2016 +0000 Merge branch 'master' of github.com:rails/docrails Conflicts: guides/source/action_cable_overview.mdcommit 082a5158251c6578714132e5c4f71bd39f462d71Merge: 4bd11d4 3bd30d9Author: Yves Senn Date: Fri Jun 3 11:30:19 2016 +0200 Merge pull request #25243 from sukesan1984/add_i18n_validation_test Add i18n_validation_testcommit 4bd11d46de892676830bca51d3040f29200abbfaMerge: 99d8d45 e98caf8Author: Arthur Nogueira Neves Date: Thu Jun 2 22:55:52 2016 -0400 Merge pull request #25258 from alexcameron89/master [skip ci] Make header bullets consistent in engines.mdcommit e98caf81fef54746126d31076c6d346c48ae8e1bAuthor: Alex Kitchens Date: Thu Jun 2 21:26:53 2016 -0500 [skip ci] Make header bullets consistent in engines.md +``` + +As you can see, the `git log` shows the commit hash, the author and his email and the date of when the commit was created. Of course, `git` being super customisable, it allows you to customise the output format of the `git log` command. Let’s say, we want to just see the first line of the commit message, we could run `git log --oneline`, which will produce a more compact log: + +``` +66ebbc4 Dont re-define class SQLite3Adapter on test755f6bf Merge pull request #25263 from abhishekjain16/doc_accessor_threadf7b850e [skip ci] Fix grammar4e85538 Merge branch 'master' of github.com:rails/docrails082a515 Merge pull request #25243 from sukesan1984/add_i18n_validation_test4bd11d4 Merge pull request #25258 from alexcameron89/mastere98caf8 [skip ci] Make header bullets consistent in engines.md99d8d45 Merge pull request #25254 from kamipo/fix_debug_helper_test818397c Merge pull request #25240 from matthewd/reloadable-channels2c5a8ba Don't blank pad day of the month when formatting dates14ff8e7 Fix debug helper test +``` + +To see all of the `git log` options, I recommend checking out manpage of `git log`, available in your terminal via `man git-log` or `git help log`. A tip: if `git log` is a bit scarse or complicated to use, or maybe you are just bored, I recommend checking out various `git` GUIs and command line tools. In the past I’ve used [GitX][1] which was very good, but since the command line feels like home to me, after trying [tig][2] I’ve never looked back. + +### Finding Nemo + +So now, since we know the bare minimum of the `git log` command, let’s see how we can explore the history more effectively in our everyday work. + +Let’s say, hypothetically, we are suspecting an unexpected behaviour in the`String#classify` method and we want to find how and where it has been implemented. + +One of the first commands that you can use, to see where the method is defined, is `git grep`. Simply said, this command prints out lines that match a certain pattern. Now, to find the definition of the method, it’s pretty simple - we can grep for `def classify` and see what we get: + +``` +➜ git grep 'def classify'activesupport/lib/active_support/core_ext/string/inflections.rb: def classifyactivesupport/lib/active_support/inflector/methods.rb: def classify(table_name)tools/profile: def classify +``` + +Now, although we can already see where our method is created, we are not sure on which line it is. If we add the `-n` flag to our `git grep` command, `git` will provide the line numbers of the match: + +``` +➜ git grep -n 'def classify'activesupport/lib/active_support/core_ext/string/inflections.rb:205: def classifyactivesupport/lib/active_support/inflector/methods.rb:186: def classify(table_name)tools/profile:112: def classify +``` + +Much better, right? Having the context in mind, we can easily figure out that the method that we are looking for lives in `activesupport/lib/active_support/core_ext/string/inflections.rb`, on line 205\. The `classify` method, in all of it’s glory looks like this: + +``` +# Creates a class name from a plural table name like Rails does for table names to models.# Note that this returns a string and not a class. (To convert to an actual class# follow +classify+ with +constantize+.)## 'ham_and_eggs'.classify # => "HamAndEgg"# 'posts'.classify # => "Post"def classify ActiveSupport::Inflector.classify(self)end +``` + +Although the method we found is the one we usually call on `String`s, it invokes another method on the `ActiveSupport::Inflector`, with the same name. Having our `git grep` result available, we can easily navigate there, since we can see the second line of the result being`activesupport/lib/active_support/inflector/methods.rb` on line 186\. The method that we are are looking for is: + +``` +# Creates a class name from a plural table name like Rails does for table# names to models. Note that this returns a string and not a Class (To# convert to an actual class follow +classify+ with #constantize).## classify('ham_and_eggs') # => "HamAndEgg"# classify('posts') # => "Post"## Singular names are not handled correctly:## classify('calculus') # => "Calculus"def classify(table_name) # strip out any leading schema name camelize(singularize(table_name.to_s.sub(/.*\./, ''.freeze)))end +``` + +Boom! Given the size of Rails, finding this should not take us more than 30 seconds with the help of `git grep`. + +### So, what changed last? + +Now, since we have the method available, we need to figure out what were the changes that this file has gone through. The since we know the correct file name and line number, we can use `git blame`. This command shows what revision and author last modified each line of a file. Let’s see what were the latest changes made to this file: + +``` +git blame activesupport/lib/active_support/inflector/methods.rb +``` + +Whoa! Although we get the last change of every line in the file, we are more interested in the specific method (lines 176 to 189). Let’s add a flag to the `git blame` command, that will show the blame of just those lines. Also, we will add the `-s` (suppress) option to the command, to skip the author names and the timestamp of the revision (commit) that changed the line: + +``` +git blame -L 176,189 -s activesupport/lib/active_support/inflector/methods.rb9fe8e19a 176) # Creates a class name from a plural table name like Rails does for table5ea3f284 177) # names to models. Note that this returns a string and not a Class (To9fe8e19a 178) # convert to an actual class follow +classify+ with #constantize).51cd6bb8 179) #6d077205 180) # classify('ham_and_eggs') # => "HamAndEgg"9fe8e19a 181) # classify('posts') # => "Post"51cd6bb8 182) #51cd6bb8 183) # Singular names are not handled correctly:5ea3f284 184) #66d6e7be 185) # classify('calculus') # => "Calculus"51cd6bb8 186) def classify(table_name)51cd6bb8 187) # strip out any leading schema name5bb1d4d2 188) camelize(singularize(table_name.to_s.sub(/.*\./, ''.freeze)))51cd6bb8 189) end +``` + +The output of the `git blame` command now shows all of the file lines and their respective revisions. Now, to see a specific revision, or in other words, what each of those revisions changed, we can use the `git show` command. When supplied a revision hash (like `66d6e7be`) as an argument, it will show you the full revision, with the author name, timestamp and the whole revision in it’s glory. Let’s see what actually changed at the latest revision that changed line 188: + +``` +git show 5bb1d4d2 +``` + +Whoa! Did you test that? If you didn’t, it’s an awesome [commit][3] by [Schneems][4] that made a very interesting performance optimization by using frozen strings, which makes sense in our current context. But, since we are on this hypothetical debugging session, this doesn’t tell much about our current problem. So, how can we see what changes has our method under investigation gone through? + +### Searching the logs + +Now, we are back to the `git` log. The question is, how can we see all the revisions that the `classify` method went under? + +The `git log` command is quite powerful, because it has a rich list of options to apply to it. We can try to see what the `git` log has stored for this file, using the `-p`options, which means show me the patch for this entry in the `git` log: + +``` +git log -p activesupport/lib/active_support/inflector/methods.rb +``` + +This will show us a big list of revisions, for every revision of this file. But, just like before, we are interested in the specific file lines. Let’s modify the command a bit, to show us what we need: + +``` +git log -L 176,189:activesupport/lib/active_support/inflector/methods.rb +``` + +The `git log` command accepts the `-L` option, which takes the lines range and the filename as arguments. The format might be a bit weird for you, but it translates to: + +``` +git log -L ,: +``` + +When we run this command, we can see the list of revisions for these lines, which will lead us to the first revision that created the method: + +``` +commit 51xd6bb829c418c5fbf75de1dfbb177233b1b154Author: Foo Bar Date: Tue Jun 7 19:05:09 2011 -0700 Refactordiff --git a/activesupport/lib/active_support/inflector/methods.rb b/activesupport/lib/active_support/inflector/methods.rb--- a/activesupport/lib/active_support/inflector/methods.rb+++ b/activesupport/lib/active_support/inflector/methods.rb@@ -58,0 +135,14 @@+ # Create a class name from a plural table name like Rails does for table names to models.+ # Note that this returns a string and not a Class. (To convert to an actual class+ # follow +classify+ with +constantize+.)+ #+ # Examples:+ # "egg_and_hams".classify # => "EggAndHam"+ # "posts".classify # => "Post"+ #+ # Singular names are not handled correctly:+ # "business".classify # => "Busines"+ def classify(table_name)+ # strip out any leading schema name+ camelize(singularize(table_name.to_s.sub(/.*\./, '')))+ end +``` + +Now, look at that - it’s a commit from 2011\. Practically, `git` allows us to travel back in time. This is a very good example of why a proper commit message is paramount to regain context, because from the commit message we cannot really regain context of how this method came to be. But, on the flip side, you should **never ever** get frustrated about it, because you are looking at someone that basically gives away his time and energy for free, doing open source work. + +Coming back from that tangent, we are not sure how the initial implementation of the `classify` method came to be, given that the first commit is just a refactor. Now, if you are thinking something within the lines of “but maybe, just maybe, the method was not on the line range 176 to 189, and we should look more broadly in the file”, you are very correct. The revision that we saw said “Refactor” in it’s commit message, which means that the method was actually there, but after that refactor it started to exist on that line range. + +So, how can we confirm this? Well, believe it or not, `git` comes to the rescue again. The `git log` command accepts the `-S` option, which looks for the code change (additions or deletions) for the specified string as an argument to the command. This means that, if we call `git log -S classify`, we can see all of the commits that changed a line that contains that string. + +If you call this command in the Rails repo, you will first see `git` slowing down a bit. But, when you realise that `git` actually parses all of the revisions in the repo to match the string, it’s actually super fast. Again, the power of `git` at your fingertips. So, to find the first version of the `classify` method, we can run: + +``` +git log -S 'def classify' +``` + +This will return all of the revisions where this method has been introduced or changed. If you were following along, the last commit in the log that you will see is: + +``` +commit db045dbbf60b53dbe013ef25554fd013baf88134Author: David Heinemeier Hansson Date: Wed Nov 24 01:04:44 2004 +0000 Initial git-svn-id: http://svn-commit.rubyonrails.org/rails/trunk@4 5ecf4fe2-1ee6-0310-87b1-e25e094e27de +``` + +How cool is that? It’s the initial commit to Rails, made on a `svn` repo by DHH! This means that `classify` has been around since the beginning of (Rails) time. Now, to see the commit with all of it’s changes, we can run: + +``` +git show db045dbbf60b53dbe013ef25554fd013baf88134 +``` + +Great, we got to the bottom of it. Now, by using the output from `git log -S 'def classify'` you can track the changes that have happened to this method, combined with the power of the `git log -L` command. + +### Until next time + +Sure, we didn’t really fix any bugs, because we were trying some `git` commands and following along the evolution of the `classify` method. But, nevertheless, `git` is a very powerful tool that we all must learn to use and to embrace. I hope this article gave you a little bit more knowledge of how useful `git` is. + +What are your favourite (or, most effective) ways of navigating through the `git`history? + +-------------------------------------------------------------------------------- + +作者简介： + +Backend engineer, interested in Ruby, Go, microservices, building resilient architectures and solving challenges at scale. I coach at Rails Girls in Amsterdam, maintain a list of small gems and often contribute to Open Source. +This is where I write about software development, programming languages and everything else that interests me. + +------ + +via: https://ieftimov.com/learn-your-tools-navigating-git-history + +作者：[Ilija Eftimov ][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://ieftimov.com/ +[1]:http://gitx.frim.nl/ +[2]:https://github.com/jonas/tig +[3]:https://github.com/rails/rails/commit/5bb1d4d288d019e276335465d0389fd2f5246bfd +[4]:https://twitter.com/schneems \ No newline at end of file diff --git a/sources/tech/20160810 How does gdb work.md b/sources/tech/20160810 How does gdb work.md index 4159f1c65a..56b0cfe7bf 100644 --- a/sources/tech/20160810 How does gdb work.md +++ b/sources/tech/20160810 How does gdb work.md @@ -1,3 +1,5 @@ +translating by ucasFL + How does gdb work? ============================================================ diff --git a/sources/tech/20161106 Myths about -dev-urandom.md b/sources/tech/20161106 Myths about -dev-urandom.md new file mode 100644 index 0000000000..b00b5863d8 --- /dev/null +++ b/sources/tech/20161106 Myths about -dev-urandom.md @@ -0,0 +1,289 @@ +Myths about /dev/urandom +====== + +There are a few things about /dev/urandom and /dev/random that are repeated again and again. Still they are false. + +I'm mostly talking about reasonably recent Linux systems, not other UNIX-like systems. + +### /dev/urandom is insecure. Always use /dev/random for cryptographic purposes. + +Fact: /dev/urandom is the preferred source of cryptographic randomness on UNIX-like systems. + +### /dev/urandom is a pseudo random number generator, a PRNG, while /dev/random is a “true” random number generator. + +Fact: Both /dev/urandom and /dev/random are using the exact same CSPRNG (a cryptographically secure pseudorandom number generator). They only differ in very few ways that have nothing to do with “true” randomness. + +### /dev/random is unambiguously the better choice for cryptography. Even if /dev/urandom were comparably secure, there's no reason to choose the latter. + +Fact: /dev/random has a very nasty problem: it blocks. + +### But that's good! /dev/random gives out exactly as much randomness as it has entropy in its pool. /dev/urandom will give you insecure random numbers, even though it has long run out of entropy. + +Fact: No. Even disregarding issues like availability and subsequent manipulation by users, the issue of entropy “running low” is a straw man. About 256 bits of entropy are enough to get computationally secure numbers for a long, long time. + +And the fun only starts here: how does /dev/random know how much entropy there is available to give out? Stay tuned! + +### But cryptographers always talk about constant re-seeding. Doesn't that contradict your last point? + +Fact: You got me! Kind of. It is true, the random number generator is constantly re-seeded using whatever entropy the system can lay its hands on. But that has (partly) other reasons. + +Look, I don't claim that injecting entropy is bad. It's good. I just claim that it's bad to block when the entropy estimate is low. + +### That's all good and nice, but even the man page for /dev/(u)random contradicts you! Does anyone who knows about this stuff actually agree with you? + +Fact: No, it really doesn't. It seems to imply that /dev/urandom is insecure for cryptographic use, unless you really understand all that cryptographic jargon. + +The man page does recommend the use of /dev/random in some cases (it doesn't hurt, in my opinion, but is not strictly necessary), but it also recommends /dev/urandom as the device to use for “normal” cryptographic use. + +And while appeal to authority is usually nothing to be proud of, in cryptographic issues you're generally right to be careful and try to get the opinion of a domain expert. + +And yes, quite a few experts share my view that /dev/urandom is the go-to solution for your random number needs in a cryptography context on UNIX-like systems. Obviously, their opinions influenced mine, not the other way around. + +Hard to believe, right? I must certainly be wrong! Well, read on and let me try to convince you. + +I tried to keep it out, but I fear there are two preliminaries to be taken care of, before we can really tackle all those points. + +Namely, what is randomness, or better: what kind of randomness am I talking about here? + +And, even more important, I'm really not being condescending. I have written this document to have a thing to point to, when this discussion comes up again. More than 140 characters. Without repeating myself again and again. Being able to hone the writing and the arguments itself, benefitting many discussions in many venues. + +And I'm certainly willing to hear differing opinions. I'm just saying that it won't be enough to state that /dev/urandom is bad. You need to identify the points you're disagreeing with and engage them. + +### You're saying I'm stupid! + +Emphatically no! + +Actually, I used to believe that /dev/urandom was insecure myself, a few years ago. And it's something you and me almost had to believe, because all those highly respected people on Usenet, in web forums and today on Twitter told us. Even the man page seems to say so. Who were we to dismiss their convincing argument about “entropy running low”? + +This misconception isn't so rampant because people are stupid, it is because with a little knowledge about cryptography (namely some vague idea what entropy is) it's very easy to be convinced of it. Intuition almost forces us there. Unfortunately intuition is often wrong in cryptography. So it is here. + +### True randomness + +What does it mean for random numbers to be “truly random”? + +I don't want to dive into that issue too deep, because it quickly gets philosophical. Discussions have been known to unravel fast, because everyone can wax about their favorite model of randomness, without paying attention to anyone else. Or even making himself understood. + +I believe that the “gold standard” for “true randomness” are quantum effects. Observe a photon pass through a semi-transparent mirror. Or not. Observe some radioactive material emit alpha particles. It's the best idea we have when it comes to randomness in the world. Other people might reasonably believe that those effects aren't truly random. Or even that there is no randomness in the world at all. Let a million flowers bloom. + +Cryptographers often circumvent this philosophical debate by disregarding what it means for randomness to be “true”. They care about unpredictability. As long as nobody can get any information about the next random number, we're fine. And when you're talking about random numbers as a prerequisite in using cryptography, that's what you should aim for, in my opinion. + +Anyway, I don't care much about those “philosophically secure” random numbers, as I like to think of your “true” random numbers. + +### Two kinds of security, one that matters + +But let's assume you've obtained those “true” random numbers. What are you going to do with them? + +You print them out, frame them and hang them on your living-room wall, to revel in the beauty of a quantum universe? That's great, and I certainly understand. + +Wait, what? You're using them? For cryptographic purposes? Well, that spoils everything, because now things get a bit ugly. + +You see, your truly-random, quantum effect blessed random numbers are put into some less respectable, real-world tarnished algorithms. + +Because almost all of the cryptographic algorithms we use do not hold up to ### information-theoretic security**. They can “only” offer **computational security. The two exceptions that come to my mind are Shamir's Secret Sharing and the One-time pad. And while the first one may be a valid counterpoint (if you actually intend to use it), the latter is utterly impractical. + +But all those algorithms you know about, AES, RSA, Diffie-Hellman, Elliptic curves, and all those crypto packages you're using, OpenSSL, GnuTLS, Keyczar, your operating system's crypto API, these are only computationally secure. + +What's the difference? While information-theoretically secure algorithms are secure, period, those other algorithms cannot guarantee security against an adversary with unlimited computational power who's trying all possibilities for keys. We still use them because it would take all the computers in the world taken together longer than the universe has existed, so far. That's the level of “insecurity” we're talking about here. + +Unless some clever guy breaks the algorithm itself, using much less computational power. Even computational power achievable today. That's the big prize every cryptanalyst dreams about: breaking AES itself, breaking RSA itself and so on. + +So now we're at the point where you don't trust the inner building blocks of the random number generator, insisting on “true randomness” instead of “pseudo randomness”. But then you're using those “true” random numbers in algorithms that you so despise that you didn't want them near your random number generator in the first place! + +Truth is, when state-of-the-art hash algorithms are broken, or when state-of-the-art block ciphers are broken, it doesn't matter that you get “philosophically insecure” random numbers because of them. You've got nothing left to securely use them for anyway. + +So just use those computationally-secure random numbers for your computationally-secure algorithms. In other words: use /dev/urandom. + +### Structure of Linux's random number generator + +#### An incorrect view + +Chances are, your idea of the kernel's random number generator is something similar to this: + +![image: mythical structure of the kernel's random number generator][1] + +“True randomness”, albeit possibly skewed and biased, enters the system and its entropy is precisely counted and immediately added to an internal entropy counter. After de-biasing and whitening it's entering the kernel's entropy pool, where both /dev/random and /dev/urandom get their random numbers from. + +The “true” random number generator, /dev/random, takes those random numbers straight out of the pool, if the entropy count is sufficient for the number of requested numbers, decreasing the entropy counter, of course. If not, it blocks until new entropy has entered the system. + +The important thing in this narrative is that /dev/random basically yields the numbers that have been input by those randomness sources outside, after only the necessary whitening. Nothing more, just pure randomness. + +/dev/urandom, so the story goes, is doing the same thing. Except when there isn't sufficient entropy in the system. In contrast to /dev/random, it does not block, but gets “low quality random” numbers from a pseudorandom number generator (conceded, a cryptographically secure one) that is running alongside the rest of the random number machinery. This CSPRNG is just seeded once (or maybe every now and then, it doesn't matter) with “true randomness” from the randomness pool, but you can't really trust it. + +In this view, that seems to be in a lot of people's minds when they're talking about random numbers on Linux, avoiding /dev/urandom is plausible. + +Because either there is enough entropy left, then you get the same you'd have gotten from /dev/random. Or there isn't, then you get those low-quality random numbers from a CSPRNG that almost never saw high-entropy input. + +Devilish, right? Unfortunately, also utterly wrong. In reality, the internal structure of the random number generator looks like this. + +#### A better simplification + +##### Before Linux 4.8 + +![image: actual structure of the kernel's random number generator before Linux 4.8][2] This is a pretty rough simplification. In fact, there isn't just one, but three pools filled with entropy. One primary pool, and one for /dev/random and /dev/urandom each, feeding off the primary pool. Those three pools all have their own entropy counts, but the counts of the secondary pools (for /dev/random and /dev/urandom) are mostly close to zero, and “fresh” entropy flows from the primary pool when needed, decreasing its entropy count. Also there is a lot of mixing and re-injecting outputs back into the system going on. All of this is far more detail than is necessary for this document. + +See the big difference? The CSPRNG is not running alongside the random number generator, filling in for those times when /dev/urandom wants to output something, but has nothing good to output. The CSPRNG is an integral part of the random number generation process. There is no /dev/random handing out “good and pure” random numbers straight from the whitener. Every randomness source's input is thoroughly mixed and hashed inside the CSPRNG, before it emerges as random numbers, either via /dev/urandom or /dev/random. + +Another important difference is that there is no entropy counting going on here, but estimation. The amount of entropy some source is giving you isn't something obvious that you just get, along with the data. It has to be estimated. Please note that when your estimate is too optimistic, the dearly held property of /dev/random, that it's only giving out as many random numbers as available entropy allows, is gone. Unfortunately, it's hard to estimate the amount of entropy. + +The Linux kernel uses only the arrival times of events to estimate their entropy. It does that by interpolating polynomials of those arrival times, to calculate “how surprising” the actual arrival time was, according to the model. Whether this polynomial interpolation model is the best way to estimate entropy is an interesting question. There is also the problem that internal hardware restrictions might influence those arrival times. The sampling rates of all kinds of hardware components may also play a role, because it directly influences the values and the granularity of those event arrival times. + +In the end, to the best of our knowledge, the kernel's entropy estimate is pretty good. Which means it's conservative. People argue about how good it really is, but that issue is far above my head. Still, if you insist on never handing out random numbers that are not “backed” by sufficient entropy, you might be nervous here. I'm sleeping sound because I don't care about the entropy estimate. + +So to make one thing crystal clear: both /dev/random and /dev/urandom are fed by the same CSPRNG. Only the behavior when their respective pool runs out of entropy, according to some estimate, differs: /dev/random blocks, while /dev/urandom does not. + +##### From Linux 4.8 onward + +In Linux 4.8 the equivalency between /dev/urandom and /dev/random was given up. Now /dev/urandom output does not come from an entropy pool, but directly from a CSPRNG. + +![image: actual structure of the kernel's random number generator from Linux 4.8 onward][3] + +We will see shortly why that is not a security problem. + +### What's wrong with blocking? + +Have you ever waited for /dev/random to give you more random numbers? Generating a PGP key inside a virtual machine maybe? Connecting to a web server that's waiting for more random numbers to create an ephemeral session key? + +That's the problem. It inherently runs counter to availability. So your system is not working. It's not doing what you built it to do. Obviously, that's bad. You wouldn't have built it if you didn't need it. + +I'm working on safety-related systems in factory automation. Can you guess what the main reason for failures of safety systems is? Manipulation. Simple as that. Something about the safety measure bugged the worker. It took too much time, was too inconvenient, whatever. People are very resourceful when it comes to finding “inofficial solutions”. + +But the problem runs even deeper: people don't like to be stopped in their ways. They will devise workarounds, concoct bizarre machinations to just get it running. People who don't know anything about cryptography. Normal people. + +Why not patching out the call to `random()`? Why not having some guy in a web forum tell you how to use some strange ioctl to increase the entropy counter? Why not switch off SSL altogether? + +In the end you just educate your users to do foolish things that compromise your system's security without you ever knowing about it. + +It's easy to disregard availability, usability or other nice properties. Security trumps everything, right? So better be inconvenient, unavailable or unusable than feign security. + +But that's a false dichotomy. Blocking is not necessary for security. As we saw, /dev/urandom gives you the same kind of random numbers as /dev/random, straight out of a CSPRNG. Use it! + +### The CSPRNGs are alright + +But now everything sounds really bleak. If even the high-quality random numbers from /dev/random are coming out of a CSPRNG, how can we use them for high-security purposes? + +It turns out, that “looking random” is the basic requirement for a lot of our cryptographic building blocks. If you take the output of a cryptographic hash, it has to be indistinguishable from a random string so that cryptographers will accept it. If you take a block cipher, its output (without knowing the key) must also be indistinguishable from random data. + +If anyone could gain an advantage over brute force breaking of cryptographic building blocks, using some perceived weakness of those CSPRNGs over “true” randomness, then it's the same old story: you don't have anything left. Block ciphers, hashes, everything is based on the same mathematical fundament as CSPRNGs. So don't be afraid. + +### What about entropy running low? + +It doesn't matter. + +The underlying cryptographic building blocks are designed such that an attacker cannot predict the outcome, as long as there was enough randomness (a.k.a. entropy) in the beginning. A usual lower limit for “enough” may be 256 bits. No more. + +Considering that we were pretty hand-wavey about the term “entropy” in the first place, it feels right. As we saw, the kernel's random number generator cannot even precisely know the amount of entropy entering the system. Only an estimate. And whether the model that's the basis for the estimate is good enough is pretty unclear, too. + +### Re-seeding + +But if entropy is so unimportant, why is fresh entropy constantly being injected into the random number generator? + +djb [remarked][4] that more entropy actually can hurt. + +First, it cannot hurt. If you've got more randomness just lying around, by all means use it! + +There is another reason why re-seeding the random number generator every now and then is important: + +Imagine an attacker knows everything about your random number generator's internal state. That's the most severe security compromise you can imagine, the attacker has full access to the system. + +You've totally lost now, because the attacker can compute all future outputs from this point on. + +But over time, with more and more fresh entropy being mixed into it, the internal state gets more and more random again. So that such a random number generator's design is kind of self-healing. + +But this is injecting entropy into the generator's internal state, it has nothing to do with blocking its output. + +### The random and urandom man page + +The man page for /dev/random and /dev/urandom is pretty effective when it comes to instilling fear into the gullible programmer's mind: + +> A read from the /dev/urandom device will not block waiting for more entropy. As a result, if there is not sufficient entropy in the entropy pool, the returned values are theoretically vulnerable to a cryptographic attack on the algorithms used by the driver. Knowledge of how to do this is not available in the current unclassified literature, but it is theoretically possible that such an attack may exist. If this is a concern in your application, use /dev/random instead. + +Such an attack is not known in “unclassified literature”, but the NSA certainly has one in store, right? And if you're really concerned about this (you should!), please use /dev/random, and all your problems are solved. + +The truth is, while there may be such an attack available to secret services, evil hackers or the Bogeyman, it's just not rational to just take it as a given. + +And even if you need that peace of mind, let me tell you a secret: no practical attacks on AES, SHA-3 or other solid ciphers and hashes are known in the “unclassified” literature, either. Are you going to stop using those, as well? Of course not! + +Now the fun part: “use /dev/random instead”. While /dev/urandom does not block, its random number output comes from the very same CSPRNG as /dev/random's. + +If you really need information-theoretically secure random numbers (you don't!), and that's about the only reason why the entropy of the CSPRNGs input matters, you can't use /dev/random, either! + +The man page is silly, that's all. At least it tries to redeem itself with this: + +> If you are unsure about whether you should use /dev/random or /dev/urandom, then probably you want to use the latter. As a general rule, /dev/urandom should be used for everything except long-lived GPG/SSL/SSH keys. + +Fine. I think it's unnecessary, but if you want to use /dev/random for your “long-lived keys”, by all means, do so! You'll be waiting a few seconds typing stuff on your keyboard, that's no problem. + +But please don't make connections to a mail server hang forever, just because you “wanted to be safe”. + +### Orthodoxy + +The view espoused here is certainly a tiny minority's opinions on the Internet. But ask a real cryptographer, you'll be hard pressed to find someone who sympathizes much with that blocking /dev/random. + +Let's take [Daniel Bernstein][5], better known as djb: + +> Cryptographers are certainly not responsible for this superstitious nonsense. Think about this for a moment: whoever wrote the /dev/random manual page seems to simultaneously believe that +> +> * (1) we can't figure out how to deterministically expand one 256-bit /dev/random output into an endless stream of unpredictable keys (this is what we need from urandom), but +> +> * (2) we _can_ figure out how to use a single key to safely encrypt many messages (this is what we need from SSL, PGP, etc.). +> +> + +> +> For a cryptographer this doesn't even pass the laugh test. + +Or [Thomas Pornin][6], who is probably one of the most helpful persons I've ever encountered on the Stackexchange sites: + +> The short answer is yes. The long answer is also yes. /dev/urandom yields data which is indistinguishable from true randomness, given existing technology. Getting "better" randomness than what /dev/urandom provides is meaningless, unless you are using one of the few "information theoretic" cryptographic algorithm, which is not your case (you would know it). +> +> The man page for urandom is somewhat misleading, arguably downright wrong, when it suggests that /dev/urandom may "run out of entropy" and /dev/random should be preferred; + +Or maybe [Thomas Ptacek][7], who is not a real cryptographer in the sense of designing cryptographic algorithms or building cryptographic systems, but still the founder of a well-reputed security consultancy that's doing a lot of penetration testing and breaking bad cryptography: + +> Use urandom. Use urandom. Use urandom. Use urandom. Use urandom. Use urandom. + +### Not everything is perfect + +/dev/urandom isn't perfect. The problems are twofold: + +On Linux, unlike FreeBSD, /dev/urandom never blocks. Remember that the whole security rested on some starting randomness, a seed? + +Linux's /dev/urandom happily gives you not-so-random numbers before the kernel even had the chance to gather entropy. When is that? At system start, booting the computer. + +FreeBSD does the right thing: they don't have the distinction between /dev/random and /dev/urandom, both are the same device. At startup /dev/random blocks once until enough starting entropy has been gathered. Then it won't block ever again. + +In the meantime, Linux has implemented a new syscall, originally introduced by OpenBSD as getentropy(2): getrandom(2). This syscall does the right thing: blocking until it has gathered enough initial entropy, and never blocking after that point. Of course, it is a syscall, not a character device, so it isn't as easily accessible from shell or script languages. It is available from Linux 3.17 onward. + +On Linux it isn't too bad, because Linux distributions save some random numbers when booting up the system (but after they have gathered some entropy, since the startup script doesn't run immediately after switching on the machine) into a seed file that is read next time the machine is booting. So you carry over the randomness from the last running of the machine. + +Obviously that isn't as good as if you let the shutdown scripts write out the seed, because in that case there would have been much more time to gather entropy. The advantage is obviously that this does not depend on a proper shutdown with execution of the shutdown scripts (in case the computer crashes, for example). + +And it doesn't help you the very first time a machine is running, but the Linux distributions usually do the same saving into a seed file when running the installer. So that's mostly okay. + +Virtual machines are the other problem. Because people like to clone them, or rewind them to a previously saved check point, this seed file doesn't help you. + +But the solution still isn't using /dev/random everywhere, but properly seeding each and every virtual machine after cloning, restoring a checkpoint, whatever. + +### tldr; + + Just use /dev/urandom! + + +-------------------------------------------------------------------------------- + +via: https://www.2uo.de/myths-about-urandom/ + +作者：[Thomas Hühn][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.2uo.de/ +[1]:https://www.2uo.de/myths-about-urandom/structure-no.png +[2]:https://www.2uo.de/myths-about-urandom/structure-yes.png +[3]:https://www.2uo.de/myths-about-urandom/structure-new.png +[4]:http://blog.cr.yp.to/20140205-entropy.html +[5]:http://www.mail-archive.com/cryptography@randombit.net/msg04763.html +[6]:http://security.stackexchange.com/questions/3936/is-a-rand-from-dev-urandom-secure-for-a-login-key/3939#3939 +[7]:http://sockpuppet.org/blog/2014/02/25/safely-generate-random-numbers/ diff --git a/sources/tech/20170216 25 Free Books To Learn Linux For Free.md b/sources/tech/20170216 25 Free Books To Learn Linux For Free.md deleted file mode 100644 index e549f50ea3..0000000000 --- a/sources/tech/20170216 25 Free Books To Learn Linux For Free.md +++ /dev/null @@ -1,295 +0,0 @@ -25 Free Books To Learn Linux For Free -====== -Brief: In this article, I'll share with you the best resource to **learn Linux for free**. This is a collection of websites, online video courses and free eBooks. - -**How to learn Linux?** - -This is perhaps the most commonly asked question in our Facebook group for Linux users. - -The answer to this simple looking question 'how to learn Linux' is not at all simple. - -Problem is that different people have different meanings of learning Linux. - - * If someone has never used Linux, be it command line or desktop version, that person might be just wondering to know more about it. - * If someone uses Windows as the desktop but have to use Linux command line at work, that person might be interested in learning Linux commands. - * If someone has been using Linux for sometimes and is aware of the basics but he/she might want to go to the next level. - * If someone is just interested in getting your way around a specific Linux distribution. - * If someone is trying to improve or learn Bash scripting which is almost synonymous with Linux command line. - * If someone is willing to make a career as a Linux SysAdmin or trying to improve his/her sysadmin skills. - - - -You see, the answer to "how do I learn Linux" depends on what kind of Linux knowledge you are seeking. And for this purpose, I have collected a bunch of resources that you could use for learning Linux. - -These free resources include eBooks, video courses, websites etc. And these are divided into sub-categories so that you can easily find what you are looking for when you seek to learn Linux. - -Again, there is no **best way to learn Linux**. It totally up to you how you go about learning Linux, by online web portals, downloaded eBooks, video courses or something else. - -Let's see how you can learn Linux. - -**Disclaimer** : All the books listed here are legal to download. The sources mentioned here are the official sources, as per my knowledge. However, if you find it otherwise, please let me know so that I can take appropriate action. - -![Best Free eBooks to learn Linux for Free][1] - -## 1. Free materials to learn Linux for absolute beginners - -So perhaps you have just heard of Linux from your friends or from a discussion online. You are intrigued about the hype around Linux and you are overwhelmed by the vast information available on the internet but just cannot figure out exactly where to look for to know more about Linux. - -Worry not. Most of us, if not all, have been to your stage. - -### Introduction to Linux by Linux Foundation [Video Course] - -If you have no idea about what is Linux and you want to get started with it, I suggest you to go ahead with the free video course provided by the [Linux Foundation][2] on [edX][3]. Consider it an official course by the organization that 'maintains' Linux. And yes, it is endorsed by [Linus Torvalds][4], the father of Linux himself. - -[Introduction To Linux][5] - -### Linux Journey [Online Portal] - -Not official and perhaps not very popular. But this little website is the perfect place for a no non-sense Linux learning for beginners. - -The website is designed beautifully and is well organized based on the topics. It also has interactive quizzes that you can take after reading a section or chapter. My advice, bookmark this website: - -[Linux Journey][6] - -### Learn Linux in 5 Days [eBook] - -This brilliant eBook is available for free exclusively to It's FOSS readers all thanks to [Linux Training Academy][7]. - -Written for absolute beginners in mind, this free Linux eBook gives you a quick overview of Linux, common Linux commands and other things that you need to learn to get started with Linux. - -You can download the book from the page below: - -[Learn Linux In 5 Days][8] - -### The Ultimate Linux Newbie Guide [eBook] - -This is a free to download eBook for Linux beginners. The eBook starts with explaining what is Linux and then go on to provide more practical usage of Linux as a desktop. - -You can download the latest version of this eBook from the link below: - -[The Ultimate Linux Newbie Guide][9] - -## 2. Free Linux eBooks for Beginners to Advanced - -This section lists out those Linux eBooks that are 'complete' in nature. - -What I mean is that these are like academic textbooks that focus on each and every aspects of Linux, well most of it. You can read those as an absolute beginner or you can read those for deeper understanding as an intermediate Linux user. You can also use them for reference even if you are at expert level. - -### Introduction to Linux [eBook] - -Introduction to Linux is a free eBook from [The Linux Documentation Project][10] and it is one of the most popular free Linux books out there. Though I think some parts of this book needs to be updated, it is still a very good book to teach you about Linux, its file system, command line, networking and other related stuff. - -[Introduction To Linux][11] - -### Linux Fundamentals [eBook] - -This free eBook by Paul Cobbaut teaches you about Linux history, installation and focuses on the basic Linux commands you should know. You can get the book from the link below: - -[Linux Fundamentals][12] - -### Advanced Linux Programming [eBook] - -As the name suggests, this is for advanced users who are or want to develop software for Linux. It deals with sophisticated features such as multiprocessing, multi-threading, interprocess communication, and interaction with hardware devices. - -Following the book will help you develop a faster, reliable and secure program that uses the full capability of a GNU/Linux system. - -[Advanced Linux Programming][13] - -### Linux From Scratch [eBook] - -If you think you know enough about Linux and you are a pro, then why not create your own Linux distribution? Linux From Scratch (LFS) is a project that provides you with step-by-step instructions for building your own custom Linux system, entirely from source code. - -Call it DIY Linux but this is a great way to put your Linux expertise to the next level. - -There are various sub-parts of this project, you can check it out on its website and download the books from there. - -[Linux From Scratch][14] - -## 3. Free eBooks to learn Linux command line and Shell scripting - -The real power of Linux lies in the command line and if you want to conquer Linux, you must learn Linux command line and Shell scripting. - -In fact, if you have to work on Linux terminal on your job, having a good knowledge of Linux command line will actually help you in your tasks and perhaps help you in advancing your career as well (as you'll be more efficient). - -In this section, we'll see various Linux commands free eBooks. - -### GNU/Linux Command−Line Tools Summary [eBook] - -This eBook from The Linux Documentation Project is a good place to begin with Linux command line and get acquainted with Shell scripting. - -[GNU/Linux Command−Line Tools Summary][15] - -### Bash Reference Manual from GNU [eBook] - -This is a free eBook to download from [GNU][16]. As the name suggests, it deals with Bash Shell (if I can call that). This book has over 175 pages and it covers a number of topics around Linux command line in Bash. - -You can get it from the link below: - -[Bash Reference Manual][17] - -### The Linux Command Line [eBook] - -This 500+ pages of free eBook by William Shotts is the MUST HAVE for anyone who is serious about learning Linux command line. - -Even if you think you know things about Linux, you'll be amazed at how much this book still teaches you. - -It covers things from beginners to advanced level. I bet that you'll be a hell lot of better Linux user after reading this book. Download it and keep it with you always. - -[The Linux Command Line][18] - -### Bash Guide for Beginners [eBook] - -If you just want to get started with Bash scripting, this could be a good companion for you. The Linux Documentation Project is behind this eBook again and it's the same author who wrote Introduction to Linux eBook (discussed earlier in this article). - -[Bash Guide for Beginners][19] - -### Advanced Bash-Scripting Guide [eBook] - -If you think you already know basics of Bash scripting and you want to take your skills to the next level, this is what you need. This book has over 900+ pages of various advanced commands and their examples. - -[Advanced Bash-Scripting Guide][20] - -### The AWK Programming Language [eBook] - -Not the prettiest book here but if you really need to go deeper with your scripts, this old-yet-gold book could be helpful. - -[The AWK Programming Language][21] - -### Linux 101 Hacks [eBook] - -This 270 pages eBook from The Geek Stuff teaches you the essentials of Linux command lines with easy to follow practical examples. You can get the book from the link below: - -[Linux 101 Hacks][22] - -## 4. Distribution specific free learning material - -This section deals with material that are dedicated to a certain Linux distribution. What we saw so far was the Linux in general, more focused on file systems, commands and other core stuff. - -These books, on the other hand, can be termed as manual or getting started guide for various Linux distributions. So if you are using a certain Linux distribution or planning to use it, you can refer to these resources. And yes, these books are more desktop Linux focused. - -I would also like to add that most Linux distributions have their own wiki or documentation section which are often pretty vast. You can always refer to them when you are online. - -### Ubuntu Manual - -Needless to say that this eBook is for Ubuntu users. It's an independent project that provides Ubuntu manual in the form of free eBook. It is updated for each version of Ubuntu. - -The book is rightly called manual because it is basically a composition of step by step instruction and aimed at absolute beginners to Ubuntu. So, you get to know Unity desktop, how to go around it and find applications etc. - -It's a must have if you never used Ubuntu Unity because it helps you to figure out how to use Ubuntu for your daily usage. - -[Ubuntu Manual][23] - -### For Linux Mint: Just Tell Me Damnit! [eBook] - -A very basic eBook that focuses on Linux Mint. It shows you how to install Linux Mint in a virtual machine, how to find software, install updates and customize the Linux Mint desktop. - -You can download the eBook from the link below: - -[Just Tell Me Damnit!][24] - -### Solus Linux Manual [eBook] - -Caution! This used to be the official manual from Solus Linux but I cannot find its mentioned on Solus Project's website anymore. I don't know if it's outdated or not. But in any case, a little something about Solu Linux won't really hurt, will it? - -[Solus Linux User Guide][25] - -## 5. Free eBooks for SysAdmin - -This section is dedicated to the SysAdmins, the superheroes for developers. I have listed a few free eBooks here for SysAdmin which will surely help anyone who is already a SysAdmin or aspirs to be one. I must add that you should also focus on essential Linux command lines as it will make your job easier. - -### The Debian Administration's Handbook [eBook] - -If you use Debian Linux for your servers, this is your bible. Book starts with Debian history, installation, package management etc and then moves on to cover topics like [LAMP][26], virtual machines, storage management and other core sysadmin stuff. - -[The Debian Administration's Handbook][27] - -### Advanced Linux System Administration [eBook] - -This is an ideal book if you are preparing for [LPI certification][28]. The book deals straightway to the topics essential for sysadmins. So knowledge of Linux command line is a prerequisite in this case. - -[Advanced Linux System Administration][29] - -### Linux System Administration [eBook] - -Another free eBook by Paul Cobbaut. The 370 pages long eBook covers networking, disk management, user management, kernel management, library management etc. - -[Linux System Administration][30] - -### Linux Servers [eBook] - -One more eBook from Paul Cobbaut of [linux-training.be][31]. This book covers web servers, mysql, DHCP, DNS, Samba and other file servers. - -[Linux Servers][32] - -### Linux Networking [eBook] - -Networking is the bread and butter of a SysAdmin, and this book by Paul Cobbaut (again) is a good reference material. - -[Linux Networking][33] - -### Linux Storage [eBook] - -This book by Paul Cobbaut (yes, him again) explains disk management on Linux in detail and introduces a lot of other storage-related technologies. - -[Linux Storage][34] - -### Linux Security [eBook] - -This is the last eBook by Paul Cobbaut in our list here. Security is one of the most important part of a sysadmin's job. This book focuses on file permissions, acls, SELinux, users and passwords etc. - -[Linux Security][35] - -## Your favorite Linux learning material? - -I know that this is a good collection of free Linux eBooks. But this could always be made better. - -If you have some other resources that could be helpful in learning Linux, do share with us. Please note to share only the legal downloads so that I can update this article with your suggestion(s) without any problem. - -I hope you find this article helpful in learning Linux. Your feedback is welcome :) - --------------------------------------------------------------------------------- - -via: https://itsfoss.com/learn-linux-for-free/ - -作者：[Abhishek Prakash][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://itsfoss.com/author/abhishek/ -[1]:https://4bds6hergc-flywheel.netdna-ssl.com/wp-content/uploads/2017/02/free-ebooks-linux-800x450.png -[2]:https://www.linuxfoundation.org/ -[3]:https://www.edx.org -[4]:https://www.youtube.com/watch?v=eE-ovSOQK0Y -[5]:https://www.edx.org/course/introduction-linux-linuxfoundationx-lfs101x-0 -[6]:https://linuxjourney.com/ -[7]:https://www.linuxtrainingacademy.com/ -[8]:https://courses.linuxtrainingacademy.com/itsfoss-ll5d/ -[9]:https://linuxnewbieguide.org/ulngebook/ -[10]:http://www.tldp.org/index.html -[11]:http://tldp.org/LDP/intro-linux/intro-linux.pdf -[12]:http://linux-training.be/linuxfun.pdf -[13]:http://advancedlinuxprogramming.com/alp-folder/advanced-linux-programming.pdf -[14]:http://www.linuxfromscratch.org/ -[15]:http://tldp.org/LDP/GNU-Linux-Tools-Summary/GNU-Linux-Tools-Summary.pdf -[16]:https://www.gnu.org/home.en.html -[17]:https://www.gnu.org/software/bash/manual/bash.pdf -[18]:http://linuxcommand.org/tlcl.php -[19]:http://www.tldp.org/LDP/Bash-Beginners-Guide/Bash-Beginners-Guide.pdf -[20]:http://www.tldp.org/LDP/abs/abs-guide.pdf -[21]:https://ia802309.us.archive.org/25/items/pdfy-MgN0H1joIoDVoIC7/The_AWK_Programming_Language.pdf -[22]:http://www.thegeekstuff.com/linux-101-hacks-ebook/ -[23]:https://ubuntu-manual.org/ -[24]:http://downtoearthlinux.com/resources/just-tell-me-damnit/ -[25]:https://drive.google.com/file/d/0B5Ymf8oYXx-PWTVJR0pmM3daZUE/view -[26]:https://en.wikipedia.org/wiki/LAMP_(software_bundle) -[27]:https://debian-handbook.info/about-the-book/ -[28]:https://www.lpi.org/our-certifications/getting-started -[29]:http://www.nongnu.org/lpi-manuals/manual/pdf/GNU-FDL-OO-LPI-201-0.1.pdf -[30]:http://linux-training.be/linuxsys.pdf -[31]:http://linux-training.be/ -[32]:http://linux-training.be/linuxsrv.pdf -[33]:http://linux-training.be/linuxnet.pdf -[34]:http://linux-training.be/linuxsto.pdf -[35]:http://linux-training.be/linuxsec.pdf diff --git a/sources/tech/20170428 Ultimate guide to securing SSH sessions.md b/sources/tech/20170428 Ultimate guide to securing SSH sessions.md new file mode 100644 index 0000000000..a96c4da6e7 --- /dev/null +++ b/sources/tech/20170428 Ultimate guide to securing SSH sessions.md @@ -0,0 +1,139 @@ +Ultimate guide to securing SSH sessions +====== +Hi Linux-fanatics, in this tutorial we will be discussing some ways with which we make our ssh server more secure. OpenSSH is currently used by default to work on servers as physical access to servers is very limited. We use ssh to copy/backup files/folders, to remotely execute commands etc. But these ssh connections might not be as secure as we believee & we must make some changes to our default settings to make them more secure. + +Here are steps needed to secure our ssh sessions, + +### Use complex username & password + +This is first of the problem that needs to be addressed, I have known users who have '12345' as their password. It seems they are inviting hackers to get themselves hacked. You should always have a complex password. + +It should have at-least 8 characters with numbers & alphabets, lower case & upper case letter, and also special characters. A good example would be " ** ** _vXdrf23#$wd_**** " , it is not a word so dictionary attack will be useless & has uppercase, lowercase characters, numbers & special characters. + +### Limit user logins + +Not all the users are required to have access to ssh in an organization, so we should make changes to our configuration file to limit user logins. Let's say only Bob & Susan are authorized have access to ssh, so open your configuration file + +``` + $ vi /etc/ssh/sshd_config +``` + +& add the allowed users to the bottom of the file + +``` + AllowUsers bob susan +``` + +Save the file & restart the service. Now only Bob & Susan will have access to ssh , others won't be able to access ssh. + +### Configure Idle logout time + + +Once logged into ssh sessions, there is default time before sessions logs out on it own. By default idle logout time is 60 minutes, which according to me is way to much. Consider this, you logged into a session , executed some commands & then went out to get a cup of coffee but you forgot to log-out of the ssh. Just think what could be done in the 60 seconds, let alone in 60 minutes. + +So, its wise to reduce idle log-out time to something around 5 minutes & it can be done in config file only. Open '/etc/ssh/sshd_config' & change the values + +``` +ClientAliveInterval 300 +ClientAliveCountMax 0 +``` + +Its in seconds, so configure them accordingly. + +### Disable root logins + +As we know root have access to anything & everything on the server, so we must disable root access through ssh session. Even if it is needed to complete a task that only root can do, we can escalate the privileges of a normal user. + +To disable root access, open your configuration file & change the following parameter + +``` +PermitRootLogin no +ClientAliveCountMax 0 +``` + +This will disable root access to ssh sessions. + +### Enable Protocol 2 + +SSH protocol 1 had man in the middle attack issues & other security issues as well, all these issues were addressed in Protocol 2. So protocol 1 must not be used at any cost. To change the protocol , open your sshd_config file & change the following parameter + +``` + Protocol 2 +``` + +### Enable a warning screen + +It would be a good idea to enable a warning screen stating a warning about misuse of ssh, just before a user logs into the session. To create a warning screen, create a file named **" warning"** in **/etc/** folder (or any other folder) & write something like "We monitor all our sessions on continuously. Don't misuse your access or else you will be prosecuted" or whatever you wish to warn. You can also consult legal team about this warning to make it more official. + +After this file is create, open sshd_config file & enter the following parameter into the file + +``` + Banner /etc/issue +``` + +now you warning message will be displayed each time someone tries to access the session. + +### Use non-standard ssh port + +By default, ssh uses port 22 & all the brute force scripts are written for port 22 only. So to make your sessions even more secure, use a non-standard port like 15000. But make sure before selecting a port that its not being used by some other service. + +To change port, open sshd_config & change the following parameter + +``` + Port 15000 +``` + +Save & restart the service and you can access the ssh only with this new port. To start a session with custom port use the following command + +``` + $ ssh -p 15000 {server IP} +``` + +** Note:-** If using firewall, open the port on your firewall & we must also change the SELinux settings if using a custom port for ssh. Run the following command to update the SELinux label + +``` +$ semanage port -a -t ssh_port_t -p tcp 15000 +``` + +### Limit IP access + +If you have an environment where your server is accessed by only limited number of IP addresses, you can also allow access to those IP addresses only. Open sshd_config file & enter the following with your custom port + +``` +Port 15000 +ListenAddress 192.168.1.100 +ListenAddress 192.168.1.115 +``` + +Now ssh session will only be available to these mentioned IPs with the custom port 15000. + +### Disable empty passwords + +As mentioned already that you should only use complex username & passwords, so using an empty password for remote login is a complete no-no. To disable empty passwords, open sshd_config file & edit the following parameter + +``` +PermitEmptyPasswords no +``` + +### Use public/private key based authentication + +Using Public/Private key based authentication has its advantages i.e. you no longer need to enter the password when entering into a session (unless you are using a passphrase to decrypt the key) & no one can have access to your server until & unless they have the right authentication key. Process to setup public/private key based authentication is discussed in [**this tutorial here**][1]. + +So, this completes our tutorial on securing your ssh server. If having any doubts or issues, please leave a message in the comment box below. + +-------------------------------------------------------------------------------- + +via: http://linuxtechlab.com/ultimate-guide-to-securing-ssh-sessions/ + +作者：[SHUSAIN][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://linuxtechlab.com/author/shsuain/ +[1]:http://linuxtechlab.com/configure-ssh-server-publicprivate-key/ +[2]:https://www.facebook.com/techlablinux/ +[3]:https://twitter.com/LinuxTechLab +[4]:https://plus.google.com/+linuxtechlab +[5]:http://linuxtechlab.com/contact-us-2/ diff --git a/sources/tech/20170429 Monitoring network bandwidth with iftop command.md b/sources/tech/20170429 Monitoring network bandwidth with iftop command.md new file mode 100644 index 0000000000..1eb7ed24a8 --- /dev/null +++ b/sources/tech/20170429 Monitoring network bandwidth with iftop command.md @@ -0,0 +1,97 @@ +translating---geekpi + +Monitoring network bandwidth with iftop command +====== +System Admins are required to monitor IT infrastructure to make sure that everything is up & running. We have to monitor performance of hardware i.e memory, hdds & CPUs etc & so does we have to monitor our network. We need to make sure that our network is not being over utilised or our applications, websites might not work. In this tutorial, we are going to learn to use IFTOP utility. + +( **Recommended read** :[ **Resource monitoring using Nagios**][1], [**Tools for checking system info**,][2] [**Important logs to monitor**][3]) + +Iftop is network monitoring utility that provides real time real time bandwidth monitoring. Iftop measures total data moving in & out of the individual socket connections i.e. it captures packets moving in and out via network adapter & than sums those up to find the bandwidth being utilized. + +## Installation on Debian/Ubuntu + +Iftop is available with default repositories of Debian/Ubuntu & can be simply installed using the command below, + +``` +$ sudo apt-get install iftop +``` + +## Installation on RHEL/Centos using yum + +For installing iftop on CentOS or RHEL, we need to enable EPEL repository. To enable repository, run the following on your terminal, + +### RHEL/CentOS 7 + +``` +$ rpm -Uvh https://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-10.noarch.rpm +``` + +### RHEL/CentOS 6 (64 Bit) + +``` +$ rpm -Uvh http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm +``` + +### RHEL/CentOS 6 (32 Bit) + +``` +$ rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm +``` + +After epel repository has been installed, we can now install iftop by running, + +``` +$ yum install iftop +``` + +This will install iftop utility on your system. We will now use it to monitor our network, + +## Using IFTOP + +You can start using iftop by opening your terminal windown & type, + +``` +$ iftop +``` + +![network monitoring][5] + +You will now be presented with network activity happening on your machine. You can also use + +``` +$ iftop -n +``` + +Which will present the network information on your screen but with '-n' , you will not be presented with the names related to IP addresses but only ip addresses. This option allows for some bandwidth to be saved, which goes into resolving IP addresses to names. + +Now we can also see all the commands that can be used with iftop. Once you have ran iftop, press 'h' button on the keyboard to see all the commands that can be used with iftop. + +![network monitoring][7] + +To monitor a particular network interface, we can mention interface with iftop, + +``` +$ iftop -I enp0s3 +``` + +You can check further options that are used with iftop using help, as mentioned above. But these mentioned examples are only what you might to monitor network. + + +-------------------------------------------------------------------------------- + +via: http://linuxtechlab.com/monitoring-network-bandwidth-iftop-command/ + +作者：[SHUSAIN][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://linuxtechlab.com/author/shsuain/ +[1]:http://linuxtechlab.com/installing-configuring-nagios-server/ +[2]:http://linuxtechlab.com/commands-system-hardware-info/ +[3]:http://linuxtechlab.com/important-logs-monitor-identify-issues/ +[4]:https://i1.wp.com/linuxtechlab.com/wp-content/plugins/a3-lazy-load/assets/images/lazy_placeholder.gif?resize=661%2C424 +[5]:https://i0.wp.com/linuxtechlab.com/wp-content/uploads/2017/04/iftop-1.jpg?resize=661%2C424 +[6]:https://i1.wp.com/linuxtechlab.com/wp-content/plugins/a3-lazy-load/assets/images/lazy_placeholder.gif?resize=663%2C416 +[7]:https://i0.wp.com/linuxtechlab.com/wp-content/uploads/2017/04/iftop-help.jpg?resize=663%2C416 diff --git a/sources/tech/20170505 Create your first Ansible server (automation) setup.md b/sources/tech/20170505 Create your first Ansible server (automation) setup.md new file mode 100644 index 0000000000..7bcd2997c8 --- /dev/null +++ b/sources/tech/20170505 Create your first Ansible server (automation) setup.md @@ -0,0 +1,164 @@ +Create your first Ansible server (automation) setup +====== +Automation/configuration management tools are the new craze in the IT world, organizations are moving towards adopting them. There are many tools that are available in market like Puppet, Chef, Ansible etc & in this tutorial, we are going to learn about Ansible. + +Ansible is an open source configuration tool; that is used to deploy, configure & manage servers. Ansible is one of the easiest automation tool to learn and master. It does not require you to learn complicated programming language like ruby (used in puppet & chef) & uses YAML, which is a very simple language. Also it does not require any special agent to be installed on client machines & only requires client machines to have python and ssh installed, both of these are usually available on systems. + +## Pre-requisites + +Before we move onto installation part, let's discuss the pre-requisites for Ansible + + 1. For server, we will need a machine with either CentOS or RHEL 7 installed & EPEL repository enabled + +To enable epel repository, use the commands below, + + **RHEL/CentOS 7** + +``` + $ rpm -Uvh https://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-10.noarch.rpm +``` + + **RHEL/CentOS 6 (64 Bit)** + +``` + $ rpm -Uvh http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm +``` + + **RHEL/CentOS 6 (32 Bit)** + +``` + $ rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm +``` + + 2. For client machines, Open SSH & python should be installed. Also we need to configure password less login for ssh session (create public-private keys). To create public-private keys & configure password less login for ssh session, refer to our article " + +[Setting up SSH Server for Public/Private keys based Authentication (Password-less login)][1]" + + + +## Installation + +Once we have epel repository enabled, we can now install anisble using yum, + +``` + $ yum install ansible +``` + +## Configuring Ansible hosts + +We will now configure hosts that we want Ansible to manage. To do that we need to edit the file **/etc/ansible/host** s & add the clients in following syntax, + +``` +[group-name] +alias ansible_ssh_host=host_IP_address +``` + +where, alias is the alias name given to hosts we adding & it can be anything, + +host_IP_address is where we enter the IP address for the hosts. + +For this tutorial, we are going to add 2 clients/hosts for ansible to manage, so let's create an entry for these two hosts in the configuration file, + +``` + $ vi /etc/ansible/hosts + [test_clients] + client1 ansible_ssh_host=192.168.1.101 + client2 ansible_ssh_host=192.168.1.10 +``` + +Save file & exit it. Now as mentioned in pre-requisites, we should have a password less login to these clients from the ansible server. To check if that's the case, ssh into the clients and we should be able to login without password, + +``` + $ ssh root@192.168.1.101 +``` + +If that's working, then we can move further otherwise we need to create Public/Private keys for ssh session (Refer to article mentioned above in pre-requisites). + +We are using root to login to other servers but we can use other local users as well & we need to define it for Ansible whatever user we will be using. To do so, we will first create a folder named 'group_vars' in '/etc/ansible' + +``` + $ cd /etc/ansible + $ mkdir group_vars +``` + +Next, we will create a file named after the group we have created in 'etc/ansible/hosts' i.e. test_clients + +``` + $ vi test_clients +``` + +& add the ifollowing information about the user, + +``` + -- + ansible_ssh_user:root +``` + + **Note :-** File will start with '--' (minus symbol), so keep not of that. + +If we want to use same user for all the groups created, then we can create only a single file named 'all' to mention the user details for ssh login, instead of creating a file for every group. + +``` + $ vi /etc/ansible/group_vars/all + -- + ansible_ssh_user: root +``` + +Similarly, we can setup files for individual hosts as well. + +Now, the setup for the clients has been done. We will now push some simple commands to all the clients being managed by Ansible. + +## Testing hosts + +To check the connectivity of all the hosts, we will issue a command, + +``` + $ ansible -m ping all +``` + +If all the hosts are properly connected, it should return the following output, + +``` + client1 | SUCCESS = > { + " changed": false, + " ping": "pong" + } + client2 | SUCCESS = > { + " changed": false, + " ping": "pong" + } +``` + +We can also issue command to an individual host, + +``` + $ ansible -m ping client1 +``` + +or to the multiple hosts, + +``` + $ ansible -m ping client1:client2 +``` + +or even to a single group, + +``` + $ ansible -m ping test_client +``` + +This complete our tutorial on setting up an Ansible server, in our future posts we will further explore funtinalities offered by Ansible. If any having doubts or queries regarding this post, use the comment box below. + + +-------------------------------------------------------------------------------- + +via: http://linuxtechlab.com/create-first-ansible-server-automation-setup/ + +作者：[SHUSAIN][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://linuxtechlab.com/author/shsuain/ +[1]:http://linuxtechlab.com/configure-ssh-server-publicprivate-key/ diff --git a/sources/tech/20170508 Ansible Tutorial- Intorduction to simple Ansible commands.md b/sources/tech/20170508 Ansible Tutorial- Intorduction to simple Ansible commands.md deleted file mode 100644 index e72d90301c..0000000000 --- a/sources/tech/20170508 Ansible Tutorial- Intorduction to simple Ansible commands.md +++ /dev/null @@ -1,156 +0,0 @@ -Ansible Tutorial: Intorduction to simple Ansible commands -====== -In our earlier Ansible tutorial, we discussed [**the installation & configuration of Ansible**][1]. Now in this ansible tutorial, we will learn some basic examples of ansible commands that we will use to manage our infrastructure. So let us start by looking at the syntax of a complete ansible command, - -``` -$ ansible -m -a -``` - -Here, we can also use a single host or all in place of & are optional to provide. Now let's look at some basic commands to use with ansible, - -### Check connectivity of hosts - -We have used this command in our previous tutorial also. The command to check connectivity of hosts is - -``` -$ ansible -m ping -``` - -### Rebooting hosts - -``` -$ ansible -a "/sbin/reboot" -``` - -### Checking host 's system information - -Ansible collects the system's information for all the hosts connected to it. To display the information of hosts, run - -``` -$ ansible -m setup | less -``` - -Secondly, to check a particular info from the collected information by passing an argument, - -``` -$ ansible -m setup -a "filter=ansible_distribution" -``` - -### Transfering files - -For transferring files we use a module 'copy' & complete command that is used is - -``` -$ ansible -m copy -a "src=/home/dan dest=/tmp/home" -``` - -### Manging users - -So to manage the users on the connected hosts, we use a module named 'user' & comamnds to use it are as follows, - -#### Creating a new user - -``` - $ ansible -m user -a "name=testuser password=" -``` - -#### Deleting a user - -``` -$ ansible -m user -a "name=testuser state=absent" -``` - - **Note:-** To create an encrypted password, use the 'mkpasswd -method=sha-512' command. - -### Changing permissions & ownership - -So for changing ownership of files of connected hosts, we use module named 'file' & commands used are - -### Changing permission of a file - -``` -$ ansible -m file -a "dest=/home/dan/file1.txt mode=777" -``` - -### Changing ownership of a file - -``` - $ ansible -m file -a "dest=/home/dan/file1.txt mode=777 owner=dan group=dan" -``` - -### Managing Packages - -So, we can manage the packages installed on all the hosts connected to ansible by using 'yum' & 'apt' modules & the complete commands used are - -#### Check if package is installed & update it - -``` -$ ansible -m yum -a "name=ntp state=latest" -``` - -#### Check if package is installed & don't update it - -``` -$ ansible -m yum -a "name=ntp state=present" -``` - -#### Check if package is at a specific version - -``` -$ ansible -m yum -a "name= ntp-1.8 state=present" -``` - -#### Check if package is not installed - -``` -$ ansible -m yum -a "name=ntp state=absent" -``` - -### Managing services - -So to manage services with ansible, we use a modules 'service' & complete commands that are used are, - -#### Starting a service - -``` -$ansible -m service -a "name=httpd state=started" -``` - -#### Stopping a service - -``` -$ ansible -m service -a "name=httpd state=stopped" -``` - -#### Restarting a service - -``` -$ ansible -m service -a "name=httpd state=restarted" -``` - -So this completes our tutorial of some simple, one line commands that can be used with ansible. Also, for our future tutorials, we will learn to create plays & playbooks that help us manage our hosts more easliy & efficiently. - -If you think we have helped you or just want to support us, please consider these :- - -Connect to us: [Facebook][2] | [Twitter][3] | [Google Plus][4] - -Become a Supporter - [Make a contribution via PayPal][5] - -Linux TechLab is thankful for your continued support. - --------------------------------------------------------------------------------- - -via: http://linuxtechlab.com/ansible-tutorial-simple-commands/ - -作者：[SHUSAIN][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:http://linuxtechlab.com/author/shsuain/ -[1]:http://linuxtechlab.com/create-first-ansible-server-automation-setup/ -[2]:https://www.facebook.com/linuxtechlab/ -[3]:https://twitter.com/LinuxTechLab -[4]:https://plus.google.com/+linuxtechlab -[5]:http://linuxtechlab.com/contact-us-2/ diff --git a/sources/tech/20170511 Working with VI editor - The Basics.md b/sources/tech/20170511 Working with VI editor - The Basics.md deleted file mode 100644 index 4056c3c9ec..0000000000 --- a/sources/tech/20170511 Working with VI editor - The Basics.md +++ /dev/null @@ -1,138 +0,0 @@ -Working with VI editor : The Basics -====== -VI editor is a powerful command line based text editor that was originally created for Unix but has since been ported to various Unix & Linux distributions. In Linux there exists another, advanced version of VI editor called VIM (also known as VI IMproved ). VIM only adds funtionalities to already powefrul VI editor, some of the added functionalities a - - * Support for many more Linux distributions, - - * Support for various coding languages like python, c++, perl etc with features like code folding , code highlighting etc - - * Can be used to edit files over network protocols like ssh and http, - - * Support to edit files inside a compressed archive, - - * Allows screen to split for editing multiple files. - - - - -Now let's discuss the commands/options that we can use with VI/VIM. For the purposes of this tutorial, we are going to use VI as an example but all the commands with VI can be used with VIM as well. But firstly we will start out with the two modes of VI text editor, - -### Command Mode - -This mode allows to handle tasks like saving files, executing a command within vi, copy/cut/paste operations, & tasks like finding/replacing. When in Insert mode, we can press escape to exit into command mode. - -### Insert Mode - -It's where we insert text into the file. To get into insert mode, we will press 'i' in command line mode. - -### Creating a file - -In order to create a file, use - -``` - $ vi filename -``` - -Once the file is created & opened, we will enter into what's called a command mode & to enter text into the file, we need to use insert mode. Let's learn in brief about these two modes, - -### Exit out of Vi - -To exit out of Vi from insert mode, we will first press 'Esc' key to exit into command mode & here we can perform following tasks to exit out of vi, - - 1. Exit without saving file- to exit out of vi command mode without saving of file, type : `:q!` - - 2. Save file & exit - To save a file & exit, type: `:wq` - -Now let's discuss the commands/options that can be used in command mode. - -### Cursor movement - -Use the keys mentioned below to manipulate the cursor position - - 1. **k** moves cursor one line up - - 2. **j ** moves cursor one line down - - 3. **h ** moves cursor to left one character postion - - 4. **i** moves cursor to right one character position - - - - - **Note :** If want to move multiple line up or down or left or right, we can use 4k or 5j, which will move cursor 4 lines up or 5 characters right respectively. - - 5. **0** cursor will be at begining of the line - - 6. **$** cursor will be at the end of a line - - 7. ** nG** moves to nth line of the file - - 8. **G** moves to last line of the file - - 9. **{ ** moves a paragraph back - - 10. **}** moves a paragraph forward - - - - -There are several other options that can be used to manage the cursor movement but these should get the work done for you. - -### Editing files - -Now we will learn the options that can be used in command mode to change our mode to Insert mode for editing the files - - 1. **i** Inserts text before the current cursor location - - 2. **I** Inserts text at the beginning of the current line - - 3. ** a ** Inserts text after the current cursor location - - 4. **A ** Inserts text at the end of the current line - - 5. **o** Creates a new line for text entry below the cursor location - - 6. ** O** Creates a new line for text entry above the cursor location - - - - -### Deleting file text - -All of these commands will be excuted from command mode, so if you are in Insert mode exit out to command mode using the 'Esc' key - - 1. **dd** will delete the complete line of the cursor, can use a number like 2dd to delete next 2 lines after the cursor - - 2. **d$** deletes from cursor position till end of the line - - 3. **d^** deletes from cursor position till beginning of line - - 4. **dw** deletes from cursor to next word - - -### Copy & paste commands - - 1. **yy** to yank or copy the current line. Can be used with a number to copy a number of lines - - 2. **p** paste the copied lines after cursor position - - 3. **P** paste the copied lines before the cursor postion - - - - -These were some the basic operations that we can use with VI or VIM editor. In our future tutorials, we leanrn to perform some advanced operations with VI/VIM editors. If having any queries or comments, please leave them in the comment box below. - - --------------------------------------------------------------------------------- - -via: http://linuxtechlab.com/working-vi-editor-basics/ - -作者：[Shusain][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:http://linuxtechlab.com/author/shsuain/ diff --git a/sources/tech/20170526 Creating a YUM repository from ISO - Online repo.md b/sources/tech/20170526 Creating a YUM repository from ISO - Online repo.md deleted file mode 100644 index ac11acc6e5..0000000000 --- a/sources/tech/20170526 Creating a YUM repository from ISO - Online repo.md +++ /dev/null @@ -1,118 +0,0 @@ -translating---geekpi - -Creating a YUM repository from ISO & Online repo -====== - -YUM tool is one of the most important tool for Centos/RHEL/Fedora. Though in latest builds of fedora, it has been replaced with DNF but that not at all means that it has ran its course. It is still used widely for installing rpm packages, we have already discussed YUM with examples in our earlier tutorial ([ **READ HERE**][1]). - -In this tutorial, we are going to learn to create a Local YUM repository, first by using ISO image of OS & then by creating a mirror image of an online yum repository. - -### Creating YUM with DVD ISO - -We are using a Centos 7 dvd for this tutorial & same process should work on RHEL 7 as well. - -Firstly create a directory named YUM in root folder - -``` -$ mkdir /YUM- -``` - -then mount Centos 7 ISO , - -``` -$ mount -t iso9660 -o loop /home/dan/Centos-7-x86_x64-DVD.iso /mnt/iso/ -``` - -Next, copy the packages from mounted ISO to /YUM folder. Once all the packages have been copied to the system, we will install the required packages for creating YUM. Open /YUM & install the following RPM packages, - -``` -$ rpm -ivh deltarpm -$ rpm -ivh python-deltarpm -$ rpm -ivh createrepo -``` - -Once these packages have been installed, we will create a file named " **local.repo "** in **/etc/yum.repos.d** folder with all the yum information - -``` -$ vi /etc/yum.repos.d/local.repo -``` - -``` -LOCAL REPO] -Name=Local YUM -baseurl=file:///YUM -gpgcheck=0 -enabled=1 -``` - -Save & exit the file. Next we will create repo-data by running the following command - -``` -$ createrepo -v /YUM -``` - -It will take some time to create the repo data. Once the process finishes, run - -``` -$ yum clean all -``` - -to clean cache & then run - -``` -$ yum repolist -``` - -to check the list of all repositories. You should see repo "local.repo" in the list. - - -### Creating mirror YUM repository with online repository - -Process involved in creating a yum is similar to creating a yum with an ISO image with one exception that we will fetch our rpm packages from an online repository instead of an ISO. - -Firstly, we need to find an online repository to get the latest packages . It is advised to find an online yum that is closest to your location , in order to optimize the download speeds. We will be using below mentioned , you can select one nearest to yours location from [CENTOS MIRROR LIST][2] - -After selecting a mirror, we will sync that mirror with our system using rsync but before you do that, make sure that you plenty of space on your server - -``` -$ rsync -avz rsync://mirror.fibergrid.in/centos/7.2/os/x86_64/Packages/s/ /YUM -``` - -Sync will take quite a while (maybe an hour) depending on your internet speed. After the syncing is completed, we will update our repo-data - -``` -$ createrepo - v /YUM -``` - -Our Yum is now ready to used . We can create a cron job for our repo to be updated automatically at a determined time daily or weekly as per you needs. - -To create a cron job for syncing the repository, run - -``` -$ crontab -e -``` - -& add the following line - -``` -30 12 * * * rsync -avz http://mirror.centos.org/centos/7/os/x86_64/Packages/ /YUM -``` - -This will enable the syncing of yum every night at 12:30 AM. Also remember to create repository configuration file in /etc/yum.repos.d , as we did above. - -That's it guys, you now have your own yum repository to use. Please share this article if you like it & leave your comments/queries in the comment box down below. - - --------------------------------------------------------------------------------- - -via: http://linuxtechlab.com/creating-yum-repository-iso-online-repo/ - -作者：[Shusain][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:http://linuxtechlab.com/author/shsuain/ -[1]:http://linuxtechlab.com/using-yum-command-examples/ -[2]:http://mirror.centos.org/centos/ diff --git a/sources/tech/20170915 How To Install And Setup Vagrant.md b/sources/tech/20170915 How To Install And Setup Vagrant.md deleted file mode 100644 index 1e76600595..0000000000 --- a/sources/tech/20170915 How To Install And Setup Vagrant.md +++ /dev/null @@ -1,265 +0,0 @@ -How To Install And Setup Vagrant -====== -Vagrant is a powerful tool when it comes to virtual machines, here we will look at how to setup and use Vagrant with Virtualbox on Ubuntu to provision reproducible virtual machines. - -## Virtual Machines, not all that complex - -For years, developers have been using virtual machines as part of their workflow, allowing them to swap and change environments that the software is running in, this is generally to prevent conflicts between projects such as project A needing php 5.3 and project b needing php 5.4. - -Also, using Virtual Machines means you only ever need the computer you're working on, you don't need dedicated hardware to mirror the production environment. - -It also comes in handy when multiple developers are working on one project, they can all run an environment which contains all of its requirements, but it can be hard maintaining multiple machines and ensuring all have the same versions of all the requirements, this is where Vagrant comes in. - -### The benefits of using Virtual Machines - - * Your vm is separate from your host environment - * You can have a vm tailor for the requirements of your code - * Anything done in one vm does not effect another VM - * You can run programs in a vm which your host may not be able to run, such as running some windows only software in a - windows vm on top of ubuntu - - - -## What is Vagrant - -In short, it's a tool that works with virtual box to allow you to automate the creation and removal of a virtual machines. - -It revolves around a Config File Called the VagrantFile, which tells vagrant what version of what os you want to install, and some other options such as the IP and Directory Syncing. You can also add a provisioning script of commands to run on the virtual machine. - -By Sharing this VagrantFile around, all developers on a project. You will all be using the exact same virtual machine. - -## Installing the Requirements - -### Install VirtualBox - -VirtualBox is the program which will run the Virtual Machine and is available in the Ubuntu Repos -``` -sudo apt-get install virtualbox -``` - -### Install Vagrant - -For vagrant itself, you need to head to and install the package for your OS. - -### Install Guest Additions - -If you intend to sharing any folders with virtual machine, you need to install the following plugin. -``` -vagrant plugin install vagrant-vbguest -``` - -## Setting Up Vagrant - -### First we need to create an area for vagrant setups. -``` -mkdir ~/Vagrant/test-vm -cd ~/Vagrant/test-vm -``` - -### Create the VagrantFile -``` -vagrant init -``` - -### Start the Virtual Machine -``` -vagrant up -``` - -### Login to the Machine -``` -vagrant-ssh -``` - -By this point you will have the basic vagrant box, and a file called VagrantFile. - -## Customising - -The VagrantFile created in the steps above will look similar to the following - -**VagrantFile** - -``` -# -*- mode: ruby -*- -# vi: set ft=ruby : -# All Vagrant configuration is done below. The "2" in Vagrant.configure -# configures the configuration version (we support older styles for -# backwards compatibility). Please don't change it unless you know what -# you're doing. -Vagrant.configure("2") do |config| - # The most common configuration options are documented and commented below. - # For a complete reference, please see the online documentation at - # https://docs.vagrantup.com. - - # Every Vagrant development environment requires a box. You can search for - # boxes at https://vagrantcloud.com/search. - config.vm.box = "base" - - # Disable automatic box update checking. If you disable this, then - # boxes will only be checked for updates when the user runs - # `vagrant box outdated`. This is not recommended. - # config.vm.box_check_update = false - - # Create a forwarded port mapping which allows access to a specific port - # within the machine from a port on the host machine. In the example below, - # accessing "localhost:8080" will access port 80 on the guest machine. - # NOTE: This will enable public access to the opened port - # config.vm.network "forwarded_port", guest: 80, host: 8080 - - # Create a forwarded port mapping which allows access to a specific port - # within the machine from a port on the host machine and only allow access - # via 127.0.0.1 to disable public access - # config.vm.network "forwarded_port", guest: 80, host: 8080, host_ip: "127.0.0.1" - - # Create a private network, which allows host-only access to the machine - # using a specific IP. - # config.vm.network "private_network", ip: "192.168.33.10" - - # Create a public network, which generally matched to bridged network. - # Bridged networks make the machine appear as another physical device on - # your network. - # config.vm.network "public_network" - - # Share an additional folder to the guest VM. The first argument is - # the path on the host to the actual folder. The second argument is - # the path on the guest to mount the folder. And the optional third - # argument is a set of non-required options. - # config.vm.synced_folder "../data", "/vagrant_data" - - # Provider-specific configuration so you can fine-tune various - # backing providers for Vagrant. These expose provider-specific options. - # Example for VirtualBox: - # - # config.vm.provider "virtualbox" do |vb| - # # Display the VirtualBox GUI when booting the machine - # vb.gui = true - # - # # Customize the amount of memory on the VM: - # vb.memory = "1024" - # end - # - # View the documentation for the provider you are using for more - # information on available options. - - # Enable provisioning with a shell script. Additional provisioners such as - # Puppet, Chef, Ansible, Salt, and Docker are also available. Please see the - # documentation for more information about their specific syntax and use. - # config.vm.provision "shell", inline: <<-SHELL - # apt-get update - # apt-get install -y apache2 - # SHELL -end -``` - -Now this VagrantFile wll create the basic virtual machine. But the concept behind vagrant is to have the virtual machines set up for our specific tasks. So lets remove the comments and tweak the config. - -**VagrantFile** -``` -# -*- mode: ruby -*- -# vi: set ft=ruby : - -Vagrant.configure("2") do |config| - # Set the Linux Version to Debian Jessie - config.vm.box = "debian/jessie64" - # Set the IP of the Box - config.vm.network "private_network", ip: "192.168.33.10" - # Sync Our Projects Directory with the WWW directory - config.vm.synced_folder "~/Projects", "/var/www/" - # Run the following to Provision - config.vm.provision "shell", path: "install.sh" -end -``` - -Now we have a simple VagrantFile, Which sets the box to debian jessie, sets an IP for us to use, syncs the folders we are interested in, and finally runs an install.sh, which is where our shell commands can go. - -**install.sh** -``` -#! /usr/bin/env bash -# Variables -DBHOST=localhost -DBNAME=dbname -DBUSER=dbuser -DBPASSWD=test123 - -echo "[ Provisioning machine ]" -echo "1) Update APT..." -apt-get -qq update - -echo "1) Install Utilities..." -apt-get install -y tidy pdftk curl xpdf imagemagick openssl vim git - -echo "2) Installing Apache..." -apt-get install -y apache2 - -echo "3) Installing PHP and packages..." -apt-get install -y php5 libapache2-mod-php5 libssh2-php php-pear php5-cli php5-common php5-curl php5-dev php5-gd php5-imagick php5-imap php5-intl php5-mcrypt php5-memcached php5-mysql php5-pspell php5-xdebug php5-xmlrpc -#php5-suhosin-extension, php5-mysqlnd - -echo "4) Installing MySQL..." -debconf-set-selections <<< "mysql-server mysql-server/root_password password secret" -debconf-set-selections <<< "mysql-server mysql-server/root_password_again password secret" -apt-get install -y mysql-server -mysql -uroot -p$DBPASSWD -e "CREATE DATABASE $DBNAME" -mysql -uroot -p$DBPASSWD -e "grant all privileges on $DBNAME.* to '$DBUSER'@'localhost' identified by '$DBPASSWD'" - -echo "5) Generating self signed certificate..." -mkdir -p /etc/ssl/localcerts -openssl req -new -x509 -days 365 -nodes -subj "/C=US/ST=Denial/L=Springfield/O=Dis/CN=www.example.com" -out /etc/ssl/localcerts/apache.pem -keyout /etc/ssl/localcerts/apache.key -chmod 600 /etc/ssl/localcerts/apache* - -echo "6) Setup Apache..." -a2enmod rewrite -> /etc/apache2/sites-enabled/000-default.conf -echo " - - ServerAdmin [[email protected]][1] - DocumentRoot /var/www/ - ErrorLog ${APACHE_LOG_DIR}/error.log - CustomLog ${APACHE_LOG_DIR}/access.log combined - - -" >> /etc/apache2/sites-enabled/000-default.conf -service apache2 restart - -echo "7) Composer Install..." -curl --silent https://getcomposer.org/installer | php -mv composer.phar /usr/local/bin/composer - -echo "8) Install NodeJS..." -curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash - -apt-get -qq update -apt-get -y install nodejs - -echo "9) Install NPM Packages..." -npm install -g gulp gulp-cli - -echo "Provisioning Completed" -``` - -By having the above VagrantFile and Install.sh in your directory, running vagrant up will do the following - - * Create a Virtual Machine Using Debian Jessie - * Set the Machines IP to 192.168.33.10 - * Sync ~/Projects with /var/www/ - * Install and Setup Apache, Mysql, PHP, Git, Vim - * Install and Run Composer - * Install Nodejs and gulp - * Create A MySQL Database - * Create Self Sign Certificates - - - -By sharing the VagrantFile and install.sh with others, you can work on the exact same environment, on two different machines. - - --------------------------------------------------------------------------------- - -via: https://www.chris-shaw.com/blog/how-to-install-and-setup-vagrant - -作者：[Christopher Shaw][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://www.chris-shaw.com -[1]:/cdn-cgi/l/email-protection diff --git a/sources/tech/20170920 Easy APT Repository - Iain R. Learmonth.md b/sources/tech/20170920 Easy APT Repository - Iain R. Learmonth.md deleted file mode 100644 index 144cff3d7b..0000000000 --- a/sources/tech/20170920 Easy APT Repository - Iain R. Learmonth.md +++ /dev/null @@ -1,85 +0,0 @@ -translating---geekpi - -Easy APT Repository · Iain R. Learmonth -====== - -The [PATHspider][5] software I maintain as part of my work depends on some features in [cURL][6] and in [PycURL][7] that have [only][8] [just][9] been mereged or are still [awaiting][10] merge. I need to build a docker container that includes these as Debian packages, so I need to quickly build an APT repository. - -A Debian repository can essentially be seen as a static website and the contents are GPG signed so it doesn't necessarily need to be hosted somewhere trusted (unless availability is critical for your application). I host my blog with [Netlify][11], a static website host, and I figured they would be perfect for this use case. They also [support open source projects][12]. - -There is a CLI tool for netlify which you can install with: -``` -sudo apt install npm -sudo npm install -g netlify-cli - -``` - -The basic steps for setting up a repository are: -``` -mkdir repository -cp /path/to/*.deb repository/ - - -cd - - repository -apt-ftparchive packages . > Packages -apt-ftparchive release . > Release -gpg --clearsign -o InRelease Release -netlify deploy - -``` - -Once you've followed these steps, and created a new site on Netlify, you'll be able to manage this site also through the web interface. A few things you might want to do are set up a custom domain name for your repository, or enable HTTPS with Let's Encrypt. (Make sure you have `apt-transport-https` if you're going to enable HTTPS though.) - -To add this repository to your apt sources: -``` -gpg --export -a YOURKEYID | sudo apt-key add - - - -echo - - - -"deb https://SUBDOMAIN.netlify.com/ /" - - | sudo tee -a /etc/apt/sources.list -sudo apt update - -``` - -You'll now find that those packages are installable. Beware of [APT pinning][13] as you may find that the newer versions on your repository are not actually the preferred versions according to your policy. - -**Update** : If you're wanting a solution that would be more suitable for regular use, take a look at [repropro][14]. If you're wanting to have end-users add your apt repository as a third-party repository to their system, please take a look at [this page on the Debian wiki][15] which contains advice on how to instruct users to use your repository. - -**Update 2** : Another commenter has pointed out [aptly][16], which offers a greater feature set and removes some of the restrictions imposed by repropro. I've never use aptly myself so can't comment on specifics, but from the website it looks like it might be a nicely polished tool. - - - --------------------------------------------------------------------------------- - -via: https://iain.learmonth.me/blog/2017/2017w383/ - -作者：[Iain R. Learmonth][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://iain.learmonth.me -[1]:https://iain.learmonth.me/tags/netlify/ -[2]:https://iain.learmonth.me/tags/debian/ -[3]:https://iain.learmonth.me/tags/apt/ -[4]:https://iain.learmonth.me/tags/foss/ -[5]:https://pathspider.net -[6]:http://curl.haxx.se/ -[7]:http://pycurl.io/ -[8]:https://github.com/pycurl/pycurl/pull/456 -[9]:https://github.com/pycurl/pycurl/pull/458 -[10]:https://github.com/curl/curl/pull/1847 -[11]:http://netlify.com/ -[12]:https://www.netlify.com/open-source/ -[13]:https://wiki.debian.org/AptPreferences -[14]:https://mirrorer.alioth.debian.org/ -[15]:https://wiki.debian.org/DebianRepository/UseThirdParty -[16]:https://www.aptly.info/ diff --git a/sources/tech/20170927 Linux directory structure- -lib explained.md b/sources/tech/20170927 Linux directory structure- -lib explained.md deleted file mode 100644 index ff9ec9b72f..0000000000 --- a/sources/tech/20170927 Linux directory structure- -lib explained.md +++ /dev/null @@ -1,79 +0,0 @@ -translate by cy - -Linux directory structure: /lib explained -====== -[![lib folder linux][1]][1] - -We already explained other important system folders like /bin, /boot, /dev, /etc etc folders in our previous posts. Please check below links for more information about other stuff which you are interested. In this post, we will see what is /lib folder all about. - -[**Linux Directory Structure explained: /bin folder**][2] - -[**Linux Directory Structure explained: /boot folder**][3] - -[**Linux Directory Structure explained: /dev folder**][4] - -[**Linux Directory Structure explained: /etc folder**][5] - -[**Linux Directory Structure explained: /lost+found folder**][6] - -[**Linux Directory Structure explained: /home folder**][7] - -### What is /lib folder in Linux? - -The lib folder is a **library files directory** which contains all helpful library files used by the system. In simple terms, these are helpful files which are used by an application or a command or a process for their proper execution. The commands in /bin or /sbin dynamic library files are located just in this directory. The kernel modules are also located here. - -Taken an example of executing pwd command. It requires some library files to execute properly. Let us prove what is happening with pwd command when executing. We will use [the strace command][8] to figure out which library files are used. - -Example: - -If you observe, We just used open kernel call for pwd command. The pwd command to execute properly it will require two lib files. - -Contents of /lib folder in Linux - -As said earlier this folder contains object files and libraries, it's good to know some important subfolders with this directory. And below content are for my system and you may see some variants in your system. - -**/lib/firmware** - This is a folder which contains hardware firmware code. - -### What is the difference between firmware and drivers? - -Many devices software consists of two software piece to make that hardware properly. The piece of code that is loaded into actual hardware is firmware and the software which communicate between this firmware and kernel is called drivers. This way the kernel directly communicate with hardware and make sure hardware is doing the work assigned to it. - -**/lib/modprobe.d** - Configuration directory for modprobe command - -**/lib/modules** - All loadable kernel modules are stored in this directory. If you have more kernels you will see folders within this directory each represents a kernel. - -**/lib/hdparm** - Contains SATA/IDE parameters for disks to run properly. - -**/lib/udev** - Userspace /dev is a device manager for Linux Kernel. This folder contains all udev related files/folders like rules.d folder which contain udev specific rules. - -### The /lib folder sister folders: /lib32 and /lib64 - -These folders contain their specific architecture library files. These folders are almost identical to /lib folder expects architecture level differences. - -### Other library folders in Linux - -**/usr/lib** - All software libraries are installed here. This does not contain system default or kernel libraries. - -**/usr/local/lib** - To place extra system library files here. These library files can be used by different applications. - -**/var/lib** - Holds dynamic data libraries/files like the rpm/dpkg database and game scores. - --------------------------------------------------------------------------------- - -via: https://www.linuxnix.com/linux-directory-structure-lib-explained/ - -作者：[Surendra Anne][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://www.linuxnix.com/author/surendra/ -[1]:https://www.linuxnix.com/wp-content/uploads/2017/09/The-lib-folder-explained.png -[2]:https://www.linuxnix.com/linux-directory-structure-explained-bin-folder/ -[3]:https://www.linuxnix.com/linux-directory-structure-explained-boot-folder/ -[4]:https://www.linuxnix.com/linux-directory-structure-explained-dev-folder/ -[5]:https://www.linuxnix.com/linux-directory-structure-explainedetc-folder/ -[6]:https://www.linuxnix.com/lostfound-directory-linuxunix/ -[7]:https://www.linuxnix.com/linux-directory-structure-home-root-folders/ -[8]:https://www.linuxnix.com/10-strace-command-examples-linuxunix/ diff --git a/sources/tech/20170928 How to Use the ZFS Filesystem on Ubuntu Linux.md b/sources/tech/20170928 How to Use the ZFS Filesystem on Ubuntu Linux.md index 8e0e9df7de..7c909706fd 100644 --- a/sources/tech/20170928 How to Use the ZFS Filesystem on Ubuntu Linux.md +++ b/sources/tech/20170928 How to Use the ZFS Filesystem on Ubuntu Linux.md @@ -1,4 +1,4 @@ -How to Use the ZFS Filesystem on Ubuntu Linux +How to Use the ZFS Filesystem on Ubuntu Linux ====== There are a myriad of [filesystems available for Linux][1]. So why try a new one? They all work, right? They're not all the same, and some have some very distinct advantages, like ZFS. diff --git a/sources/tech/20171004 How To Create A Video From PDF Files In Linux.md b/sources/tech/20171004 How To Create A Video From PDF Files In Linux.md deleted file mode 100644 index 5ecf5da24e..0000000000 --- a/sources/tech/20171004 How To Create A Video From PDF Files In Linux.md +++ /dev/null @@ -1,91 +0,0 @@ -translating---geekpi - -How To Create A Video From PDF Files In Linux -====== -![](https://www.ostechnix.com/wp-content/uploads/2017/10/Video-1-720x340.jpg) - -I have a huge collection of PDF files, mostly Linux tutorials, in my tablet PC. Sometimes I feel too lazy to read them from the tablet. I thought It would be better If I can be able to create a video from PDF files and watch it in a big screen devices like a TV or a Computer. Though I have a little working experience with [**FFMpeg**][1], I am not aware of how to create a movie file using it. After a bit of Google searches, I came up with a good solution. For those who wanted to make a movie file from a set of PDF files, read on. It is not that difficult. - -### Create A Video From PDF Files In Linux - -For this purpose, you need to install **" FFMpeg"** and **" ImageMagick"** software in your system. - -To install FFMpeg, refer the following link. - -Imagemagick is available in the official repositories of most Linux distributions. - -On **Arch Linux** and derivatives such as **Antergos** , **Manjaro Linux** , run the following command to install it. -``` -sudo pacman -S imagemagick -``` - -**Debian, Ubuntu, Linux Mint:** -``` -sudo apt-get install imagemagick -``` - -**Fedora:** -``` -sudo dnf install imagemagick -``` - -**RHEL, CentOS, Scientific Linux:** -``` -sudo yum install imagemagick -``` - -**SUSE, openSUSE:** -``` -sudo zypper install imagemagick -``` - -After installing ffmpeg and imagemagick, convert your PDF file image format such as PNG or JPG like below. -``` -convert -density 400 input.pdf picture.png -``` - -Here, **-density 400** specifies the horizontal resolution of the output image file(s). - -The above command will convert all pages in the given PDF file to PNG format. Each page in the PDF file will be converted into a PNG file and saved in the current directory with file name **picture-1.png** , **picture-2.png** … and so on. It will take a while depending on the number of pages in the input PDF file. - -Once all pages in the PDF converted into PNG format, run the following command to create a video file from the PNG files. -``` -ffmpeg -r 1/10 -i picture-%01d.png -c:v libx264 -r 30 -pix_fmt yuv420p video.mp4 -``` - -Here, - - * **-r 1/10** : Display each image for 10 seconds. - * **-i picture-%01d.png** : Reads all pictures that starts with name **" picture-"**, following with 1 digit (%01d) and ending with **.png**. If the images name comes with 2 digits (I.e picture-10.png, picture11.png etc), use (%02d) in the above command. - * **-c:v libx264** : Output video codec (i.e h264). - * **-r 30** : framerate of output video - * **-pix_fmt yuv420p** : Output video resolution - * **video.mp4** : Output video file with .mp4 format. - - - -Hurrah! The movie file is ready!! You can play it on any devices that supports .mp4 format. Next, I need to find a way to insert a cool music to my video. I hope it won't be difficult either. - -If you wanted it in higher pixel resolution, you don't have to start all over again. Just convert the output video file to any other higher/lower resolution of your choice, say 720p, as shown below. -``` -ffmpeg -i video.mp4 -vf scale=-1:720 video_720p.mp4 -``` - -Please note that creating a video using ffmpeg requires a good configuration PC. While converting videos, ffmpeg will consume most of your system resources. I recommend to do this in high-end system. - -And, that's all for now folks. Hope you find this useful. More good stuffs to come. Stay tuned! - - - --------------------------------------------------------------------------------- - -via: https://www.ostechnix.com/create-video-pdf-files-linux/ - -作者：[SK][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://www.ostechnix.com/author/sk/ -[1]:https://www.ostechnix.com/20-ffmpeg-commands-beginners/ diff --git a/sources/tech/20171016 5 SSH alias examples in Linux.md b/sources/tech/20171016 5 SSH alias examples in Linux.md index ddf92f1cc0..cd7e79db0b 100644 --- a/sources/tech/20171016 5 SSH alias examples in Linux.md +++ b/sources/tech/20171016 5 SSH alias examples in Linux.md @@ -1,3 +1,4 @@ +ch-cn translating 5 SSH alias examples in Linux ====== [![][1]][1] diff --git a/sources/tech/20171027 Easy guide to secure VNC server with TLS encryption.md b/sources/tech/20171027 Easy guide to secure VNC server with TLS encryption.md deleted file mode 100644 index 7548991798..0000000000 --- a/sources/tech/20171027 Easy guide to secure VNC server with TLS encryption.md +++ /dev/null @@ -1,137 +0,0 @@ -translating---geekpi - -Easy guide to secure VNC server with TLS encryption -====== -In this tutorial, we will learn to install VNC server & secure VNC server sessions with TLS encryption. -This method has been tested on CentOS 6 & 7 but should work on other versions/OS as well (RHEL, Scientific Linux etc). - -**(Recommended Read:[Ultimate guide for Securing SSH sessions][1] )** - -### Installing VNC server - -Before we install VNC server on our machines, make sure we are have a working GUI. If GUI is not installed on our machine, we can install it by executing the following command, - -``` -yum groupinstall "GNOME Desktop" -``` - -Now we will tigervnc as our VNC server, to install it run, - -``` -# yum install tigervnc-server -``` - -Once VNC server has been installed, we will create a new user to access the server, - -``` -# useradd vncuser -``` - -& assign it a password for accessing VNC by using following command, - -``` -# vncpasswd vncuser -``` - -Now we have a little change in configuration on CentOS 6 & 7, we will first address the CentOS 6 configuration, - -#### CentOS 6 - -Now we need to edit VNC configuration file, - -``` - **# vim /etc/sysconfig/vncservers** -``` - -& add the following lines, - -``` -[ …] -VNCSERVERS= "1:vncuser" -VNCSERVERARGS[1]= "-geometry 1024×768″ -``` - -Save the file & exit. Next restart the vnc service to implement the changes, - -``` -# service vncserver restart -``` - -& enable it at boot, - -``` -# chkconfig vncserver on -``` - -#### CentOS 7 - -On CentOS 7, /etc/sysconfig/vncservers file has been changed to /lib/systemd/system/vncserver@.service. We will use this configuration file as reference, so create a copy of the file, - -``` -# cp /lib/systemd/system/vncserver@.service /etc/systemd/system/vncserver@:1.service -``` - -Next we will edit the file to include our created user, - -``` -# vim /etc/systemd/system/vncserver@:1.service -``` - -& edit the user on the following 2 lines, - -``` -ExecStart=/sbin/runuser -l vncuser -c "/usr/bin/vncserver %i" -PIDFile=/home/vncuser/.vnc/%H%i.pid -``` - -Save file & exit. Next restart the service & enable it at boot, - -``` -systemctl restart[[email protected]][2]:1.service -systemctl enable[[email protected]][2]:1.service -``` - -We now have our VNC server ready & can connect to it from a client machine using the IP address of VNC server. But we before we do that, we will secure our connections with TLS encryption. - -### Securing the VNC session - -To secure VNC server session, we will first configure the encryption method to secure VNC server sessions. We will be using TLS encryption but can also use SSL encryption. Execute the following command to start using TLS encrytption on VNC server, - -``` -# vncserver -SecurityTypes=VeNCrypt,TLSVnc -``` - -You will asked to enter a password to access VNC (if using any other user, than the above mentioned user) - -![secure vnc server][4] - -We can now access the server using the VNC viewer from the client machine, use the following command to start vnc viewer with secure connection, - - **# vncviewer -SecurityTypes=VeNCrypt,TLSVnc 192.168.1.45:1** - -here, 192.168.1.45 is the IP address of the VNC server. - -![secure vnc server][6] - -Enter the password & we can than access the server remotely & that too with TLS encryption. - -This completes our tutorial, feel free to send your suggestions or queries using the comment box below. - - --------------------------------------------------------------------------------- - -via: http://linuxtechlab.com/secure-vnc-server-tls-encryption/ - -作者：[Shusain][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:http://linuxtechlab.com/author/shsuain/ -[1]:http://linuxtechlab.com/ultimate-guide-to-securing-ssh-sessions/ -[2]:/cdn-cgi/l/email-protection -[3]:https://i1.wp.com/linuxtechlab.com/wp-content/plugins/a3-lazy-load/assets/images/lazy_placeholder.gif?resize=642%2C241 -[4]:https://i1.wp.com/linuxtechlab.com/wp-content/uploads/2017/10/secure_vnc-1.png?resize=642%2C241 -[5]:https://i1.wp.com/linuxtechlab.com/wp-content/plugins/a3-lazy-load/assets/images/lazy_placeholder.gif?resize=665%2C419 -[6]:https://i2.wp.com/linuxtechlab.com/wp-content/uploads/2017/10/secure_vnc-2.png?resize=665%2C419 diff --git a/sources/tech/20171101 -dev-[u]random- entropy explained.md b/sources/tech/20171101 -dev-[u]random- entropy explained.md new file mode 100644 index 0000000000..c89ba22465 --- /dev/null +++ b/sources/tech/20171101 -dev-[u]random- entropy explained.md @@ -0,0 +1,108 @@ +/dev/[u]random: entropy explained +====== +### Entropy + +When the topic of /dev/random and /dev/urandom come up, you always hear this word: “Entropy”. Everyone seems to have their own analogy for it. So why not me? I like to think of Entropy as “Random juice”. It is juice, required for random to be more random. + +If you have ever generated an SSL certificate, or a GPG key, you may have seen something like: +``` +We need to generate a lot of random bytes. It is a good idea to perform +some other action (type on the keyboard, move the mouse, utilize the +disks) during the prime generation; this gives the random number +generator a better chance to gain enough entropy. +++++++++++..+++++.+++++++++++++++.++++++++++...+++++++++++++++...++++++ ++++++++++++++++++++++++++++++.+++++..+++++.+++++.+++++++++++++++++++++++++>. +++++++++++>+++++...........................................................+++++ +Not enough random bytes available. Please do some other work to give +the OS a chance to collect more entropy! (Need 290 more bytes) + +``` + + +By typing on the keyboard, and moving the mouse, you help generate Entropy, or Random Juice. + +You might be asking yourself… Why do I need Entropy? and why it is so important for random to be actually random? Well, lets say our Entropy was limited to keyboard, mouse, and disk IO. But our system is a server, so I know there is no mouse and keyboard input. This means the only factor is your IO. If it is a single disk, that was barely used, you will have low Entropy. This means your systems ability to be random is weak. In other words, I could play the probability game, and significantly decrease the amount of time it would take to crack things like your ssh keys, or decrypt what you thought was an encrypted session. + +Okay, but that is pretty unrealistic right? No, actually it isn’t. Take a look at this [Debian OpenSSH Vulnerability][1]. This particular issue was caused by someone removing some of the code responsible for adding Entropy. Rumor has it they removed it because it was causing valgrind to throw warnings. However, in doing that, random is now MUCH less random. In fact, so much less that Brute forcing the private ssh keys generated is now a fesible attack vector. + +Hopefully by now we understand how important Entropy is to security. Whether you realize you are using it or not. + +### /dev/random & /dev/urandom + + +/dev/urandom is a Psuedo Random Number Generator, and it **does not** block if you run out of Entropy. +/dev/random is a True Random Number Generator, and it **does** block if you run out of Entropy. + +Most often, if we are dealing with something pragmatic, and it doesn’t contain the keys to your nukes, /dev/urandom is the right choice. Otherwise if you go with /dev/random, then when the system runs out of Entropy your application is just going to behave funny. Whether it outright fails, or just hangs until it has enough depends on how you wrote your application. + +### Checking the Entropy + +So, how much Entropy do you have? +``` +[root@testbox test]# cat /proc/sys/kernel/random/poolsize +4096 +[root@testbox test]# cat /proc/sys/kernel/random/entropy_avail +2975 +[root@testbox test]# + +``` + +/proc/sys/kernel/random/poolsize, to state the obvious is the size(in bits) of the Entropy Pool. eg: How much random-juice we should save before we stop pumping more. /proc/sys/kernel/random/entropy_avail, is the amount(in bits) of random-juice in the pool currently. + +### How can we influence this number? + +The number is drained as we use it. The most crude example I can come up with is catting /dev/random into /dev/null: +``` +[root@testbox test]# cat /dev/random > /dev/null & +[1] 19058 +[root@testbox test]# cat /proc/sys/kernel/random/entropy_avail +0 +[root@testbox test]# cat /proc/sys/kernel/random/entropy_avail +1 +[root@testbox test]# + +``` + +The easiest way to influence this is to run [Haveged][2]. Haveged is a daemon that uses the processor “flutter” to add Entropy to the systems Entropy Pool. Installation and basic setup is pretty straight forward +``` +[root@b08s02ur ~]# systemctl enable haveged +Created symlink from /etc/systemd/system/multi-user.target.wants/haveged.service to /usr/lib/systemd/system/haveged.service. +[root@b08s02ur ~]# systemctl start haveged +[root@b08s02ur ~]# + +``` + +On a machine with relatively moderate traffic: +``` +[root@testbox ~]# pv /dev/random > /dev/null + 40 B 0:00:15 [ 0 B/s] [ <=> ] + 52 B 0:00:23 [ 0 B/s] [ <=> ] + 58 B 0:00:25 [5.92 B/s] [ <=> ] + 64 B 0:00:30 [6.03 B/s] [ <=> ] +^C +[root@testbox ~]# systemctl start haveged +[root@testbox ~]# pv /dev/random > /dev/null +7.12MiB 0:00:05 [1.43MiB/s] [ <=> ] +15.7MiB 0:00:11 [1.44MiB/s] [ <=> ] +27.2MiB 0:00:19 [1.46MiB/s] [ <=> ] + 43MiB 0:00:30 [1.47MiB/s] [ <=> ] +^C +[root@testbox ~]# + +``` + +Using pv we are able to see how much data we are passing via pipe. As you can see, before haveged, we were getting 2.1 bits per second(B/s). Whereas after starting haveged, and adding processor flutter to our Entropy pool we get ~1.5MiB/sec. + +-------------------------------------------------------------------------------- + +via: http://jhurani.com/linux/2017/11/01/entropy-explained.html + +作者：[James J][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://jblevins.org/log/ssh-vulnkey +[1]:http://jhurani.com/linux/2017/11/01/%22https://jblevins.org/log/ssh-vulnkey%22 +[2]:http://www.issihosts.com/haveged/ diff --git a/sources/tech/20171103 How To Fully Update And Upgrade Offline Debian-based Systems.md b/sources/tech/20171103 How To Fully Update And Upgrade Offline Debian-based Systems.md deleted file mode 100644 index a0fb133043..0000000000 --- a/sources/tech/20171103 How To Fully Update And Upgrade Offline Debian-based Systems.md +++ /dev/null @@ -1,125 +0,0 @@ -How To Fully Update And Upgrade Offline Debian-based Systems -====== -![](https://www.ostechnix.com/wp-content/uploads/2017/11/Upgrade-Offline-Debian-based-Systems-2-720x340.png) - -A while ago we have shown you how to install softwares in any[ **offline Ubuntu**][1] system and any [**offline Arch Linux**][2] system. Today, we will see how to fully update and upgrade offline Debian-based systems. Unlike the previous methods, we do not update/upgrade a single package, but the whole system. This method can be helpful where you don't have an active Internet connection or slow Internet speed. - -### Fully Update And Upgrade Offline Debian-based Systems - -Let us say, you have a system (Windows or Linux) with high-speed Internet connection at work and a Debian or any Debian derived systems with no internet connection or very slow Internet connection(like dial-up) at home. You want to upgrade your offline home system. What would you do? Buy a high speed Internet connection? Not necessary! You still can update or upgrade your offline system with Internet. This is where **Apt-Offline** comes in help. - -As the name says, apt-offline is an Offline APT Package Manager for APT based systems like Debian and Debian derived distributions such as Ubuntu, Linux Mint. Using apt-offline, we can fully update/upgrade our Debian box without the need of connecting it to the Internet. It is cross-platform tool written in the Python Programming Language and has both CLI and graphical interfaces. - -#### Requirements - - * An Internet connected system (Windows or Linux). We call it online system for the sake of easy understanding throughout this guide. - * An Offline system (Debian and Debian derived system). We call it offline system. - * USB drive or External Hard drive with sufficient space to carry all updated packages. - - - -#### Installation - -Apt-Offline is available in the default repositories of Debian and derivatives. If your Online system is running with Debian, Ubuntu, Linux Mint, and other DEB based systems, you can install Apt-Offline using command: -``` -sudo apt-get install apt-offline -``` - -If your Online runs with any other distro than Debian, git clone Apt-Offline repository: -``` -git clone https://github.com/rickysarraf/apt-offline.git -``` - -Go the directory and run it from there. -``` -cd apt-offline/ -``` -``` -sudo ./apt-offline -``` - -#### Steps to do in Offline system (Non-Internet connected system) - -Go to your offline system and create a directory where you want to store the signature file: -``` -mkdir ~/tmp -``` -``` -cd ~/tmp/ -``` - -You can use any directory of your choice. Then, run the following command to generate the signature file: -``` -sudo apt-offline set apt-offline.sig -``` - -Sample output would be: -``` -Generating database of files that are needed for an update. - -Generating database of file that are needed for operation upgrade -``` - -By default, apt-offline will generate database of files that are needed to be update and upgrade. You can use **--` update`** or `**--upgrade** options to create database for either one of these.` - -Copy the entire **tmp** folder in an USB drive or external drive and go to your online system (Internet-enabled system). - -#### Steps to do in Online system - -Plug in your USB drive and go to the temp directory: -``` -cd tmp/ -``` - -Then, run the following command: -``` -sudo apt-offline get apt-offline.sig --threads 5 --bundle apt-offline-bundle.zip -``` - -Here, "-threads 5" represents the number of APT repositories. You can increase the number if you want to download packages from more repositories. And, "-bundle apt-offline-bundle.zip" option represents all packages will be bundled in a single archive file called **apt-offline-bundle.zip**. This archive file will be saved in your current working directory. - -The above command will download data based on the signature file generated earlier in the offline system. - -[![][3]][4] - -This will take several minutes depending upon the Internet connection speed. Please note that apt-offline is cross platform, so you can use it to download packages on any OS. - -Once completed, copy the **tmp** folder to USB or External drive and return back to the offline system. Make sure your USB device has enough free space to keep all downloaded files, because all packages are available in the tmp folder now. - -#### Steps to do in offline system - -Plug in the device in your offline system and go to the **tmp** directory where you have downloaded all packages earlier. -``` -cd tmp -``` - -Then, run the following command to install all download packages. -``` -sudo apt-offline install apt-offline-bundle.zip -``` - -This will update the APT database, so APT will find all required packages in the APT cache. - -**Note:** If both online and offline systems are in the same local network, you can transfer the **tmp** folder to the offline system using "scp" or any other file transfer applications. If both systems are in different places, copy the folder using USB devices. - -And, that's all for now folks. I hope this guide will useful for you. More good stuffs to come. Stay tuned! - -Cheers! - - - --------------------------------------------------------------------------------- - -via: https://www.ostechnix.com/fully-update-upgrade-offline-debian-based-systems/ - -作者：[SK][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://www.ostechnix.com/author/sk/ -[1]:https://www.ostechnix.com/install-softwares-offline-ubuntu-16-04/ -[2]:https://www.ostechnix.com/install-packages-offline-arch-linux/ -[3]:data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7 -[4]:http://www.ostechnix.com/wp-content/uploads/2017/11/apt-offline.png () diff --git a/sources/tech/20171113 The big break in computer languages.md b/sources/tech/20171113 The big break in computer languages.md deleted file mode 100644 index c6eb8c3bfe..0000000000 --- a/sources/tech/20171113 The big break in computer languages.md +++ /dev/null @@ -1,93 +0,0 @@ -Translated by name1e5s - -The big break in computer languages -============================================================ - - -My last post ([The long goodbye to C][3]) elicited a comment from a C++ expert I was friends with long ago, recommending C++ as the language to replace C. Which ain’t gonna happen; if that were a viable future, Go and Rust would never have been conceived. - -But my readers deserve more than a bald assertion. So here, for the record, is the story of why I don’t touch C++ any more. This is a launch point for a disquisition on the economics of computer-language design, why some truly unfortunate choices got made and baked into our infrastructure, and how we’re probably going to fix them. - -Along the way I will draw aside the veil from a rather basic mistake that people trying to see into the future of programming languages (including me) have been making since the 1980s. Only very recently do we have the field evidence to notice where we went wrong. - -I think I first picked up C++ because I needed GNU eqn to be able to output MathXML, and eqn was written in C++. That project succeeded. Then I was a senior dev on Battle For Wesnoth for a number of years in the 2000s and got comfortable with the language. - -Then came the day we discovered that a person we incautiously gave commit privileges to had fucked up the games’s AI core. It became apparent that I was the only dev on the team not too frightened of that code to go in. And I fixed it all right – took me two weeks of struggle. After which I swore a mighty oath never to go near C++ again. - -My problem with the language, starkly revealed by that adventure, is that it piles complexity on complexity upon chrome upon gingerbread in an attempt to address problems that cannot actually be solved because the foundational abstractions are leaky. It’s all very well to say “well, don’t do that” about things like bare pointers, and for small-scale single-developer projects (like my eqn upgrade) it is realistic to expect the discipline can be enforced. - -Not so on projects with larger scale or multiple devs at varying skill levels (the case I normally deal with). With probability asymptotically approaching one over time and increasing LOC, someone is inadvertently going to poke through one of the leaks. At which point you have a bug which, because of over-layers of gnarly complexity such as STL, is much more difficult to characterize and fix than the equivalent defect in C. My Battle For Wesnoth experience rubbed my nose in this problem pretty hard. - -What works for a Steve Heller (my old friend and C++ advocate) doesn’t scale up when I’m dealing with multiple non-Steve-Hellers and might end up having to clean up their mess. So I just don’t go there any more. Not worth the aggravation. C is flawed, but it does have one immensely valuable property that C++ didn’t keep – if you can mentally model the hardware it’s running on, you can easily see all the way down. If C++ had actually eliminated C’s flaws (that it, been type-safe and memory-safe) giving away that transparency might be a trade worth making. As it is, nope. - -One way we can tell that C++ is not sufficient is to imagine an alternate world in which it is. In that world, older C projects would routinely up-migrate to C++. Major OS kernels would be written in C++, and existing kernel implementations like Linux would be upgrading to it. In the real world, this ain’t happening. Not only has C++ failed to present enough of a value proposition to keep language designers uninterested in imagining languages like D, Go, and Rust, it has failed to displace its own ancestor. There’s no path forward from C++ without breaching its core assumptions; thus, the abstraction leaks won’t go away. - -Since I’ve mentioned D, I suppose this is also the point at which I should explain why I don’t see it as a serious contender to replace C. Yes, it was spun up eight years before Rust and nine years before Go – props to Walter Bright for having the vision. But in 2001 the example of Perl and Python had already been set – the window when a proprietary language could compete seriously with open source was already closing. The wrestling match between the official D library/runtime and Tango hurt it, too. It has never recovered from those mistakes. - -So now there’s Go (I’d say “…and Rust”, but for reasons I’ve discussed before I think it will be years before Rust is fully competitive). It  _is_  type-safe and memory-safe (well, almost; you can partway escape using interfaces, but it’s not normal to have to go to the unsafe places). One of my regulars, Mark Atwood, has correctly pointed out that Go is a language made of grumpy-old-man rage, specifically rage by  _one of the designers of C_  (Ken Thompson) at the bloated mess that C++ became. - -I can relate to Ken’s grumpiness; I’ve been muttering for decades that C++ attacked the wrong problem. There were two directions a successor language to C might have gone. One was to do what C++ did – accept C’s leaky abstractions, bare pointers and all, for backward compatibility, than try to build a state-of-the-art language on top of them. The other would have been to attack C’s problems at their root –  _fix_ the leaky abstractions. That would break backward compatibility, but it would foreclose the class of problems that dominate C/C++ defects. - -The first serious attempt at the second path was Java in 1995\. It wasn’t a bad try, but the choice to build it over a j-code interpreter mode it unsuitable for systems programming. That left a huge hole in the options for systems programming that wouldn’t be properly addressed for another 15 years, until Rust and Go. In particular, it’s why software like my GPSD and NTPsec projects is still predominantly written in C in 2017 despite C’s manifest problems. - -This is in many ways a bad situation. It was hard to really see this because of the lack of viable alternatives, but C/C++ has not scaled well. Most of us take for granted the escalating rate of defects and security compromises in infrastructure software without really thinking about how much of that is due to really fundamental language problems like buffer-overrun vulnerabilities. - -So, why did it take so long to address that? It was 37 years from C (1972) to Go (2009); Rust only launched a year sooner. I think the underlying reasons are economic. - -Ever since the very earliest computer languages it’s been understood that every language design embodies an assertion about the relative value of programmer time vs. machine resources. At one end of that spectrum you have languages like assembler and (later) C that are designed to extract maximum performance at the cost of also pessimizing developer time and costs; at the other, languages like Lisp and (later) Python that try to automate away as much housekeeping detail as possible, at the cost of pessimizing machine performance. - -In broadest terms, the most important discriminator between the ends of this spectrum is the presence or absence of automatic memory management. This corresponds exactly to the empirical observation that memory-management bugs are by far the most common class of defects in machine-centric languages that require programmers to manage that resource by hand. - -A language becomes economically viable where and when its relative-value assertion matches the actual cost drivers of some particular area of software development. Language designers respond to the conditions around them by inventing languages that are a better fit for present or near-future conditions than the languages they have available to use. - -Over time, there’s been a gradual shift from languages that require manual memory management to languages with automatic memory management and garbage collection (GC). This shift corresponds to the Moore’s Law effect of decreasing hardware costs making programmer time relatively more expensive. But there are at least two other relevant dimensions. - -One is distance from the bare metal. Inefficiency low in the software stack (kernels and service code) ripples multiplicatively up the stack. This, we see machine-centric languages down low and programmer-centric languages higher up, most often in user-facing software that only has to respond at human speed (time scale 0.1 sec). - -Another is project scale. Every language also has an expected rate of induced defects per thousand lines of code due to programmers tripping over leaks and flaws in its abstractions. This rate runs higher in machine-centric languages, much lower in programmer-centric ones with GC. As project scale goes up, therefore, languages with GC become more and more important as a strategy against unacceptable defect rates. - -When we view language deployments along these three dimensions, the observed pattern today – C down below, an increasing gallimaufry of languages with GC above – almost makes sense. Almost. But there is something else going on. C is stickier than it ought to be, and used way further up the stack than actually makes sense. - -Why do I say this? Consider the classic Unix command-line utilities. These are generally pretty small programs that would run acceptably fast implemented in a scripting language with a full POSIX binding. Re-coded that way they would be vastly easier to debug, maintain and extend. - -Why are these still in C (or, in unusual exceptions like eqn, in C++)? Transition costs. It’s difficult to translate even small, simple programs between languages and verify that you have faithfully preserved all non-error behaviors. More generally, any area of applications or systems programming can stay stuck to a language well after the tradeoff that language embodies is actually obsolete. - -Here’s where I get to the big mistake I and other prognosticators made. We thought falling machine-resource costs – increasing the relative cost of programmer-hours – would be enough by themselves to displace C (and non-GC languages generally). In this we were not entirely or even mostly wrong – the rise of scripting languages, Java, and things like Node.js since the early 1990s was pretty obviously driven that way. - -Not so the new wave of contending systems-programming languages, though. Rust and Go are both explicitly responses to  _increasing project scale_ . Where scripting languages got started as an effective way to write small programs and gradually scaled up, Rust and Go were positioned from the start as ways to reduce defect rates in  _really large_  projects. Like, Google’s search service and Facebook’s real-time-chat multiplexer. - -I think this is the answer to the “why not sooner” question. Rust and Go aren’t actually late at all, they’re relatively prompt responses to a cost driver that was underweighted until recently. - -OK, so much for theory. What predictions does this one generate? What does it tell us about what comes after C? - -Here’s the big one. The largest trend driving development towards GC languages haven’t reversed, and there’s no reason to expect it will. Therefore: eventually we  _will_  have GC techniques with low enough latency overhead to be usable in kernels and low-level firmware, and those will ship in language implementations. Those are the languages that will truly end C’s long reign. - -There are broad hints in the working papers from the Go development group that they’re headed in this direction – references to academic work on concurrent garbage collectors that never have stop-the-world pauses. If Go itself doesn’t pick up this option, other language designers will. But I think they will – the business case for Google to push them there is obvious (can you say “Android development”?). - -Well before we get to GC that good, I’m putting my bet on Go to replace C anywhere that the GC it has now is affordable – which means not just applications but most systems work outside of kernels and embedded. The reason is simple: there is no path out of C’s defect rates with lower transition costs. - -I’ve been experimenting with moving C code to Go over the last week, and I’m noticing two things. One is that it’s easy to do – C’s idioms map over pretty well. The other is that the resulting code is much simpler. One would expect that, with GC in the language and maps as a first-class data type, but I’m seeing larger reductions in code volume than initially expected – about 2:1, similar to what I see when moving C code to Python. - -Sorry, Rustaceans – you’ve got a plausible future in kernels and deep firmware, but too many strikes against you to beat Go over most of C’s range. No GC, plus Rust is a harder transition from C because of the borrow checker, plus the standardized part of the API is still seriously incomplete (where’s my select(2), again?). - -The only consolation you get, if it is one, is that the C++ fans are screwed worse than you are. At least Rust has a real prospect of dramatically lowering downstream defect rates relative to C anywhere it’s not crowded out by Go; C++ doesn’t have that. - -This entry was posted in [Software][4] by [Eric Raymond][5]. Bookmark the [permalink][6]. - --------------------------------------------------------------------------------- - -via: http://esr.ibiblio.org/?p=7724 - -作者：[Eric Raymond][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:http://esr.ibiblio.org/?author=2 -[1]:http://esr.ibiblio.org/?author=2 -[2]:http://esr.ibiblio.org/?p=7724 -[3]:http://esr.ibiblio.org/?p=7711 -[4]:http://esr.ibiblio.org/?cat=13 -[5]:http://esr.ibiblio.org/?author=2 -[6]:http://esr.ibiblio.org/?p=7724 diff --git a/sources/tech/20171116 How to use a here documents to write data to a file in bash script.md b/sources/tech/20171116 How to use a here documents to write data to a file in bash script.md index 12d15af78f..9c2a636b09 100644 --- a/sources/tech/20171116 How to use a here documents to write data to a file in bash script.md +++ b/sources/tech/20171116 How to use a here documents to write data to a file in bash script.md @@ -1,3 +1,4 @@ +translating by ljgibbslf How to use a here documents to write data to a file in bash script ====== diff --git a/sources/tech/20171127 Protecting Your Website From Application Layer DOS Attacks With mod.md b/sources/tech/20171127 Protecting Your Website From Application Layer DOS Attacks With mod.md deleted file mode 100644 index c640d776c1..0000000000 --- a/sources/tech/20171127 Protecting Your Website From Application Layer DOS Attacks With mod.md +++ /dev/null @@ -1,223 +0,0 @@ -Translating by jessie-pang - -Protecting Your Website From Application Layer DOS Attacks With mod -====== -There exist many ways of maliciously taking a website offline. The more complicated methods involve technical knowledge of databases and programming. A far simpler method is known as a "Denial Of Service", or "DOS" attack. This attack derives its name from its goal which is to deny your regular clients or site visitors normal website service. - -There are, generally speaking, two forms of DOS attack; - - 1. Layer 3,4 or Network-Layer attacks. - 2. Layer 7 or Application-Layer attacks. - - - -The first type of DOS attack, network-layer, is when a huge quantity of junk traffic is directed at the web server. When the quantity of junk traffic exceeds the capacity of the network infrastructure the website is taken offline. - -The second type of DOS attack, application-layer, is where instead of junk traffic legitimate looking page requests are made. When the number of page requests exceeds the capacity of the web server to serve pages legitimate visitors will not be able to use the site. - -This guide will look at mitigating application-layer attacks. This is because mitigating networking-layer attacks requires huge quantities of available bandwidth and the co-operation of upstream providers. This is usually not something that can be protected against through configuration of the web server. - -An application-layer attack, at least a modest one, can be protected against through the configuration of a normal web server. Protecting against this form of attack is important because [Cloudflare][1] have [recently reported][2] that the number of network-layer attacks is diminishing while the number of application-layer attacks is increasing. - -This guide will explain using the Apache2 module [mod_evasive][3] by [zdziarski][4]. - -In addition, mod_evasive will stop an attacker trying to guess a username/password combination by attempting hundreds of combinations i.e. a brute force attack. - -Mod_evasive works by keeping a record of the number of requests arriving from each IP address. When this number exceeds one of the several thresholds that IP is served an error page. Error pages require far fewer resources than a site page keeping the site online for legitimate visitors. - -### Installing mod_evasive on Ubuntu 16.04 - -Mod_evasive is contained in the default Ubuntu 16.04 repositories with the package name "libapache2-mod-evasive". A simple `apt-get` will get it installed: -``` -apt-get update -apt-get upgrade -apt-get install libapache2-mod-evasive - -``` - -We now need to configure mod_evasive. - -It's configuration file is located at `/etc/apache2/mods-available/evasive.conf`. By default, all the modules settings are commented after installation. Therefore, the module won't interfere with site traffic until the configuration file has been edited. -``` - - #DOSHashTableSize 3097 - #DOSPageCount 2 - #DOSSiteCount 50 - #DOSPageInterval 1 - #DOSSiteInterval 1 - #DOSBlockingPeriod 10 - - #DOSEmailNotify you@yourdomain.com - #DOSSystemCommand "su - someuser -c '/sbin/... %s ...'" - #DOSLogDir "/var/log/mod_evasive" - - -``` - -The first block of directives mean as follows: - - * **DOSHashTableSize** - The current list of accessing IP's and their request count. - * **DOSPageCount** - The threshold number of page requests per DOSPageInterval. - * **DOSPageInterval** - The amount of time in which mod_evasive counts up the page requests. - * **DOSSiteCount** - The same as the DOSPageCount but counts requests from the same IP for any page on the site. - * **DOSSiteInterval** - The amount of time that mod_evasive counts up the site requests. - * **DOSBlockingPeriod** - The amount of time in seconds that an IP is blocked for. - - - -If the default configuration shown above is used then an IP will be blocked if it: - - * Requests a single page more than twice a second. - * Requests more than 50 pages different pages per second. - - - -If an IP exceeds these thresholds it is blocked for 10 seconds. - -This may not seem like a lot, however, mod_evasive will continue monitoring the page requests even for blocked IP's and reset their block period. As long as an IP is attempting to DOS the site it will remain blocked. - -The remaining directives are: - - * **DOSEmailNotify** - An email address to receive notification of DOS attacks and IP's being blocked. - * **DOSSystemCommand** - A command to run in the event of a DOS. - * **DOSLogDir** - The directory where mod_evasive keeps some temporary files. - - - -### Configuring mod_evasive - -The default configuration is a good place to start as it should not block any legitimate users. The configuration file with all directives (apart from DOSSystemCommand) uncommented looks like the following: -``` - - DOSHashTableSize 3097 - DOSPageCount 2 - DOSSiteCount 50 - DOSPageInterval 1 - DOSSiteInterval 1 - DOSBlockingPeriod 10 - - DOSEmailNotify JohnW@example.com - #DOSSystemCommand "su - someuser -c '/sbin/... %s ...'" - DOSLogDir "/var/log/mod_evasive" - - -``` - -The log directory must be created and given the same owner as the apache process. Here it is created at `/var/log/mod_evasive` and given the owner and group of the Apache web server on Ubuntu `www-data`: -``` -mkdir /var/log/mod_evasive -chown www-data:www-data /var/log/mod_evasive - -``` - -After editing Apache's configuration, especially on a live website, it is always a good idea to check the syntax of the edits before restarting or reloading. This is because a syntax error will stop Apache from re-starting and taking your site offline. - -Apache comes packaged with a helper command that has a configuration syntax checker. Simply run the following command to check your edits: -``` -apachectl configtest - -``` - -If your configuration is correct you will get the response: -``` -Syntax OK - -``` - -However, if there is a problem you will be told where it occurred and what it was, e.g.: -``` -AH00526: Syntax error on line 6 of /etc/apache2/mods-enabled/evasive.conf: -DOSSiteInterval takes one argument, Set site interval -Action 'configtest' failed. -The Apache error log may have more information. - -``` - -If your configuration passes the configtest then the module can be safely enabled and Apache reloaded: -``` -a2enmod evasive -systemctl reload apache2.service - -``` - -Mod_evasive is now configured and running. - -### Testing - -In order to test mod_evasive, we simply need to make enough web requests to the server that we exceed the threshold and record the response codes from Apache. - -A normal, successful page request will receive the response: -``` -HTTP/1.1 200 OK - -``` - -However, one that has been denied by mod_evasive will return the following: -``` -HTTP/1.1 403 Forbidden - -``` - -The following script will make HTTP requests to `127.0.0.1:80`, that is localhost on port 80, as rapidly as possible and print out the response code of every request. - -All you need to do is to copy the following bash script into a file e.g. `mod_evasive_test.sh`: -``` -#!/bin/bash -set -e - -for i in {1..50}; do - curl -s -I 127.0.0.1 | head -n 1 -done - -``` - -The parts of this script mean as follows: - - * curl - This is a command to make web requests. - * -s - Hide the progress meter. - * -I - Only display the response header information. - * head - Print the first part of a file. - * -n 1 - Only display the first line. - - - -Then make it executable: -``` -chmod 755 mod_evasive_test.sh - -``` - -When the script is run **before** mod_evasive is enabled you will see 50 lines of `HTTP/1.1 200 OK` returned. - -However, after mod_evasive is enabled you will see the following: -``` -HTTP/1.1 200 OK -HTTP/1.1 200 OK -HTTP/1.1 403 Forbidden -HTTP/1.1 403 Forbidden -HTTP/1.1 403 Forbidden -HTTP/1.1 403 Forbidden -HTTP/1.1 403 Forbidden -... - -``` - -The first two requests were allowed, but then once a third in the same second was made mod_evasive denied any further requests. You will also receive an email letting you know that a DOS attempt was detected to the address you set with the `DOSEmailNotify` option. - -Mod_evasive is now protecting your site! - --------------------------------------------------------------------------------- - -via: https://bash-prompt.net/guides/mod_proxy/ - -作者：[Elliot Cooper][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://bash-prompt.net/about/ -[1]:https://www.cloudflare.com -[2]:https://blog.cloudflare.com/the-new-ddos-landscape/ -[3]:https://github.com/jzdziarski/mod_evasive -[4]:https://www.zdziarski.com/blog/ diff --git a/sources/tech/20171203 3 Essential Questions to Ask at Your Next Tech Interview.md b/sources/tech/20171203 3 Essential Questions to Ask at Your Next Tech Interview.md deleted file mode 100644 index 891ef48948..0000000000 --- a/sources/tech/20171203 3 Essential Questions to Ask at Your Next Tech Interview.md +++ /dev/null @@ -1,47 +0,0 @@ -3 Essential Questions to Ask at Your Next Tech Interview -====== -![](https://www.linux.com/sites/lcom/files/styles/rendered_file/public/os-jobs_0.jpg?itok=nDf5j7xC) - -Interviewing can be stressful, but 58 percent of companies tell Dice and the Linux Foundation that they need to hire open source talent in the months ahead. Learn how to ask the right questions. - -The Linux Foundation - -The annual [Open Source Jobs Report][1] from Dice and The Linux Foundation reveals a lot about prospects for open source professionals and hiring activity in the year ahead. In this year's report, 86 percent of tech professionals said that knowing open source has advanced their careers. Yet what happens with all that experience when it comes time for advancing within their own organization or applying for a new roles elsewhere? - -Interviewing for a new job is never easy. Aside from the complexities of juggling your current work while preparing for a new role, there's the added pressure of coming up with the necessary response when the interviewer asks "Do you have any questions for me?" - -At Dice, we're in the business of careers, advice, and connecting tech professionals with employers. But we also hire tech talent at our organization to work on open source projects. In fact, the Dice platform is based on a number of Linux distributions and we leverage open source databases as the basis for our search functionality. In short, we couldn't run Dice without open source software, therefore it's vital that we hire professionals who understand, and love, open source. - -Over the years, I've learned the importance of asking good questions during an interview. It's an opportunity to learn about your potential new employer, as well as better understand if they are a good match for your skills. - -Here are three essential questions to ask and the reason they're important: - -**1\. What is the company 's position on employees contributing to open source projects or writing code in their spare time?** - -The answer to this question will tell you a lot about the company you're interviewing with. In general, companies will want tech pros who contribute to websites or projects as long as they don't conflict with the work you're doing at that firm. Allowing this outside the company also fosters an entrepreneurial spirt among the tech organization, and teaches tech skills that you may not otherwise get in the normal course of your day. - -**2\. How are projects prioritized here?** - -As all companies have become tech companies, there is often a division between innovative customer facing tech projects versus those that improve the platform itself. Will you be working on keeping the existing platform up to date? Or working on new products for the public? Depending on where your interests lie, the answer could determine if the company is a right fit for you. - -**3\. Who primarily makes decisions on new products and how much input do developers have in the decision-making process?** - -This question is one part understanding who is responsible for innovation at the company (and how close you'll be working with him/her) and one part discovering your career path at the firm. A good company will talk to its developers and open source talent ahead of developing new products. It seems like a no brainer, but it's a step that's sometimes missed and will mean the difference between a collaborative environment or chaotic process ahead of new product releases. - -Interviewing can be stressful, however as 58 percent of companies tell Dice and The Linux Foundation that they need to hire open source talent in the months ahead, it's important to remember the heightened demand puts professionals like you in the driver's seat. Steer your career in the direction you desire. - -[Download ][2] the full 2017 Open Source Jobs Report now. - --------------------------------------------------------------------------------- - -via: https://www.linux.com/blog/os-jobs/2017/12/3-essential-questions-ask-your-next-tech-interview - -作者：[Brian Hostetter][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://www.linux.com/users/brianhostetter -[1]:https://www.linuxfoundation.org/blog/2017-jobs-report-highlights-demand-open-source-skills/ -[2]:http://bit.ly/2017OSSjobsreport diff --git a/sources/tech/20171204 Tutorial on how to write basic udev rules in Linux.md b/sources/tech/20171204 Tutorial on how to write basic udev rules in Linux.md deleted file mode 100644 index e4f3d6f537..0000000000 --- a/sources/tech/20171204 Tutorial on how to write basic udev rules in Linux.md +++ /dev/null @@ -1,210 +0,0 @@ -# Tutorial on how to write basic udev rules in Linux - -Contents - -* * [1. Objective][4] - - * [2. Requirements][5] - - * [3. Difficulty][6] - - * [4. Conventions][7] - - * [5. Introduction][8] - - * [6. How rules are organized][9] - - * [7. The rules syntax][10] - - * [8. A test case][11] - - * [9. Operators][12] - * * [9.1.1. == and != operators][1] - - * [9.1.2. The assignment operators: = and :=][2] - - * [9.1.3. The += and -= operators][3] - - * [10. The keys we used][13] - -### Objective - -Understanding the base concepts behind udev, and learn how to write simple rules - -### Requirements - -* Root permissions - -### Difficulty - -MEDIUM - -### Conventions - -* **#** - requires given command to be executed with root privileges either directly as a root user or by use of `sudo` command - -* **$** - given command to be executed as a regular non-privileged user - -### Introduction - -In a GNU/Linux system, while devices low level support is handled at the kernel level, the management of events related to them is managed in userspace by `udev`, and more precisely by the `udevd` daemon. Learning how to write rules to be applied on the occurring of those events can be really useful to modify the behavior of the system and adapt it to our needs. - -### How rules are organized - -Udev rules are defined into files with the `.rules` extension. There are two main locations in which those files can be placed: `/usr/lib/udev/rules.d` it's the directory used for system-installed rules, `/etc/udev/rules.d/`is reserved for custom made rules.  - -The files in which the rules are defined are conventionally named with a number as prefix (e.g `50-udev-default.rules`) and are processed in lexical order independently of the directory they are in. Files installed in `/etc/udev/rules.d`, however, override those with the same name installed in the system default path. - -### The rules syntax - -The syntax of udev rules is not very complicated once you understand the logic behind it. A rule is composed by two main sections: the "match" part, in which we define the conditions for the rule to be applied, using a series of keys separated by a comma, and the "action" part, in which we perform some kind of action, when the conditions are met.  - -### A test case - -What a better way to explain possible options than to configure an actual rule? As an example, we are going to define a rule to disable the touchpad when a mouse is connected. Obviously the attributes provided in the rule definition, will reflect my hardware.  - -We will write our rule in the `/etc/udev/rules.d/99-togglemouse.rules` file with the help of our favorite text editor. A rule definition can span over multiple lines, but if that's the case, a backslash must be used before the newline character, as a line continuation, just as in shell scripts. Here is our rule: -``` -ACTION=="add" \ -, ATTRS{idProduct}=="c52f" \ -, ATTRS{idVendor}=="046d" \ -, ENV{DISPLAY}=":0" \ -, ENV{XAUTHORITY}="/run/user/1000/gdm/Xauthority" \ -, RUN+="/usr/bin/xinput --disable 16" -``` -Let's analyze it. - -### Operators - -First of all, an explanation of the used and possible operators: - -#### == and != operators - -The `==` is the equality operator and the `!=` is the inequality operator. By using them we establish that for the rule to be applied the defined keys must match, or not match the defined value respectively. - -#### The assignment operators: = and := - -The `=` assignment operator, is used to assign a value to the keys that accepts one. We use the `:=` operator, instead, when we want to assign a value and we want to make sure that it is not overridden by other rules: the values assigned with this operator, in facts, cannot be altered. - -#### The += and -= operators - -The `+=` and `-=` operators are used respectively to add or to remove a value from the list of values defined for a specific key. - -### The keys we used - -Let's now analyze the keys we used in the rule. First of all we have the `ACTION` key: by using it, we specified that our rule is to be applied when a specific event happens for the device. Valid values are `add`, `remove` and `change`  - -We then used the `ATTRS` keyword to specify an attribute to be matched. We can list a device attributes by using the `udevadm info` command, providing its name or `sysfs` path: -``` -udevadm info -ap /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.1/0003:046D:C52F.0010/input/input39 - -Udevadm info starts with the device specified by the devpath and then -walks up the chain of parent devices. It prints for every device -found, all possible attributes in the udev rules key format. -A rule to match, can be composed by the attributes of the device -and the attributes from one single parent device. - - looking at device '/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.1/0003:046D:C52F.0010/input/input39': - KERNEL=="input39" - SUBSYSTEM=="input" - DRIVER=="" - ATTR{name}=="Logitech USB Receiver" - ATTR{phys}=="usb-0000:00:1d.0-1.2/input1" - ATTR{properties}=="0" - ATTR{uniq}=="" - - looking at parent device '/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.1/0003:046D:C52F.0010': - KERNELS=="0003:046D:C52F.0010" - SUBSYSTEMS=="hid" - DRIVERS=="hid-generic" - ATTRS{country}=="00" - - looking at parent device '/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.1': - KERNELS=="2-1.2:1.1" - SUBSYSTEMS=="usb" - DRIVERS=="usbhid" - ATTRS{authorized}=="1" - ATTRS{bAlternateSetting}==" 0" - ATTRS{bInterfaceClass}=="03" - ATTRS{bInterfaceNumber}=="01" - ATTRS{bInterfaceProtocol}=="00" - ATTRS{bInterfaceSubClass}=="00" - ATTRS{bNumEndpoints}=="01" - ATTRS{supports_autosuspend}=="1" - - looking at parent device '/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2': - KERNELS=="2-1.2" - SUBSYSTEMS=="usb" - DRIVERS=="usb" - ATTRS{authorized}=="1" - ATTRS{avoid_reset_quirk}=="0" - ATTRS{bConfigurationValue}=="1" - ATTRS{bDeviceClass}=="00" - ATTRS{bDeviceProtocol}=="00" - ATTRS{bDeviceSubClass}=="00" - ATTRS{bMaxPacketSize0}=="8" - ATTRS{bMaxPower}=="98mA" - ATTRS{bNumConfigurations}=="1" - ATTRS{bNumInterfaces}==" 2" - ATTRS{bcdDevice}=="3000" - ATTRS{bmAttributes}=="a0" - ATTRS{busnum}=="2" - ATTRS{configuration}=="RQR30.00_B0009" - ATTRS{devnum}=="12" - ATTRS{devpath}=="1.2" - ATTRS{idProduct}=="c52f" - ATTRS{idVendor}=="046d" - ATTRS{ltm_capable}=="no" - ATTRS{manufacturer}=="Logitech" - ATTRS{maxchild}=="0" - ATTRS{product}=="USB Receiver" - ATTRS{quirks}=="0x0" - ATTRS{removable}=="removable" - ATTRS{speed}=="12" - ATTRS{urbnum}=="1401" - ATTRS{version}==" 2.00" - - [...] -``` -Above is the truncated output received after running the command. As you can read it from the output itself, `udevadm` starts with the specified path that we provided, and gives us information about all the parent devices. Notice that attributes of the device are reported in singular form (e.g `KERNEL`), while the parent ones in plural form (e.g `KERNELS`). The parent information can be part of a rule but only one of the parents can be referenced at a time: mixing attributes of different parent devices will not work. In the rule we defined above, we used the attributes of one parent device: `idProduct` and `idVendor`.  - -The next thing we have done in our rule, is to use the `ENV` keyword: it can be used to both set or try to match environment variables. We assigned a value to the `DISPLAY` and `XAUTHORITY` ones. Those variables are essential when interacting with the X server programmatically, to setup some needed information: with the `DISPLAY` variable, we specify on what machine the server is running, what display and what screen we are referencing, and with `XAUTHORITY` we provide the path to the file which contains Xorg authentication and authorization information. This file is usually located in the users "home" directory.  - -Finally we used the `RUN` keyword: this is used to run external programs. Very important: this is not executed immediately, but the various actions are executed once all the rules have been parsed. In this case we used the `xinput` utility to change the status of the touchpad. I will not explain the syntax of xinput here, it would be out of context, just notice that `16` is the id of the touchpad.  - -Once our rule is set, we can debug it by using the `udevadm test` command. This is useful for debugging but it doesn't really run commands specified using the `RUN` key: -``` -$ udevadm test --action="add" /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.1/0003:046D:C52F.0010/input/input39 -``` -What we provided to the command is the action to simulate, using the `--action` option, and the sysfs path of the device. If no errors are reported, our rule should be good to go. To run it in the real world, we must reload the rules: -``` -# udevadm control --reload -``` -This command will reload the rules files, however, will have effect only on new generated events.  - -We have seen the basic concepts and logic used to create an udev rule, however we only scratched the surface of the many options and possible settings. The udev manpage provides an exhaustive list: please refer to it for a more in-depth knowledge. - --------------------------------------------------------------------------------- - -via: https://linuxconfig.org/tutorial-on-how-to-write-basic-udev-rules-in-linux - -作者：[Egidio Docile ][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://disqus.com/by/egidiodocile/ -[1]:https://linuxconfig.org/tutorial-on-how-to-write-basic-udev-rules-in-linux#h9-1-1-and-operators -[2]:https://linuxconfig.org/tutorial-on-how-to-write-basic-udev-rules-in-linux#h9-1-2-the-assignment-operators-and -[3]:https://linuxconfig.org/tutorial-on-how-to-write-basic-udev-rules-in-linux#h9-1-3-the-and-operators -[4]:https://linuxconfig.org/tutorial-on-how-to-write-basic-udev-rules-in-linux#h1-objective -[5]:https://linuxconfig.org/tutorial-on-how-to-write-basic-udev-rules-in-linux#h2-requirements -[6]:https://linuxconfig.org/tutorial-on-how-to-write-basic-udev-rules-in-linux#h3-difficulty -[7]:https://linuxconfig.org/tutorial-on-how-to-write-basic-udev-rules-in-linux#h4-conventions -[8]:https://linuxconfig.org/tutorial-on-how-to-write-basic-udev-rules-in-linux#h5-introduction -[9]:https://linuxconfig.org/tutorial-on-how-to-write-basic-udev-rules-in-linux#h6-how-rules-are-organized -[10]:https://linuxconfig.org/tutorial-on-how-to-write-basic-udev-rules-in-linux#h7-the-rules-syntax -[11]:https://linuxconfig.org/tutorial-on-how-to-write-basic-udev-rules-in-linux#h8-a-test-case -[12]:https://linuxconfig.org/tutorial-on-how-to-write-basic-udev-rules-in-linux#h9-operators -[13]:https://linuxconfig.org/tutorial-on-how-to-write-basic-udev-rules-in-linux#h10-the-keys-we-used diff --git a/sources/tech/20171211 A tour of containerd 1.0.md b/sources/tech/20171211 A tour of containerd 1.0.md deleted file mode 100644 index 4cf3e2b587..0000000000 --- a/sources/tech/20171211 A tour of containerd 1.0.md +++ /dev/null @@ -1,48 +0,0 @@ -A tour of containerd 1.0 -====== -![containerd][1] - -We have done a few talks in the past on different features of containerd, how it was designed, and some of the problems that we have fixed along the way. Containerd is used by Docker, Kubernetes CRI, and a few other projects but this is a post for people who may not know what containerd actually does within these platforms. I would like to do more posts on the feature set and design of containerd in the future but for now, we will start with the basics. - -I think the container ecosystem can be confusing at times. Especially with the terminology that we use. Whats this? A runtime. And this? A runtime… containerd (pronounced " _container-dee "_) as the name implies, not contain nerd as some would like to troll me with, is a container daemon. It was originally built as an integration point for OCI runtimes like runc but over the past six months it has added a lot of functionality to bring it up to par with the needs of modern container platforms like Docker and orchestration systems like Kubernetes. - -So what do you actually get using containerd? You get push and pull functionality as well as image management. You get container lifecycle APIs to create, execute, and manage containers and their tasks. An entire API dedicated to snapshot management and an openly governed project to depend on. Basically everything that you need to build a container platform without having to deal with the underlying OS details. I think the most important part of containerd is having a versioned and stable API that will have bug fixes and security patches backported. - -![containerd][2] - -Since there is no such thing as Linux containers in the kernel, containers are various kernel features tied together, when you are building a large platform or distributed system you want an abstraction layer between your management code and the syscalls and duct tape of features to run a container. That is where containerd lives. It provides a client a layer of stable types that platforms can build on top of without ever having to drop down to the kernel level. It's so much nicer to work with Container, Task, and Snapshot types than it is to manage calls to clone() or mount(). Balanced with the flexibility to directly interact with the runtime or host-machine, these objects avoid the sacrifice of capabilities that typically come with higher-level abstractions. The result is that easy tasks are simple to complete and hard tasks are possible. - -![containerd][3]Containerd was designed to be used by Docker and Kubernetes as well as any other container system that wants to abstract away syscalls or OS specific functionality to run containers on Linux, Windows, Solaris, or other Operating Systems. With these users in mind, we wanted to make sure that containerd has only what they need and nothing that they don't. Realistically this is impossible but at least that is what we try for. While networking is out of scope for containerd, what it doesn't do lets higher level systems have full control. The reason for this is, when you are building a distributed system, networking is a very central aspect. With SDN and service discovery today, networking is way more platform specific than abstracting away netlink calls on linux. Most of the new overlay networks are route based and require routing tables to be updated each time a new container is created or deleted. Service discovery, DNS, etc all have to be notified of these changes as well. It would be a large chunk of code to be able to support all the different network interfaces, hooks, and integration points to support this if we added networking to containerd. What we did instead is opted for a robust events system inside containerd so that multiple consumers can subscribe to the events that they care about. We also expose a [Task API ][4]that lets users create a running task, have the ability to add interfaces to the network namespace of the container, and then start the container's process without the need for complex hooks in various points of a container's lifecycle. - -Another area that has been added to containerd over the past few months is a complete storage and distribution system that supports both OCI and Docker image formats. You have a complete content addressed storage system across the containerd API that works not only for images but also metadata, checkpoints, and arbitrary data attached to containers. - -We also took the time to [rethink how "graphdrivers" work][5]. These are the overlay or block level filesystems that allow images to have layers and you to perform efficient builds. Graphdrivers were initially written by Solomon and I when we added support for devicemapper. Docker only supported AUFS at the time so we modeled the graphdrivers after the overlay filesystem. However, making a block level filesystem such as devicemapper/lvm act like an overlay filesystem proved to be much harder to do in the long run. The interfaces had to expand over time to support different features than what we originally thought would be needed. With containerd, we took a different approach, make overlay filesystems act like a snapshotter instead of vice versa. This was much easier to do as overlay filesystems provide much more flexibility than snapshotting filesystems like BTRFS, ZFS, and devicemapper as they don't have a strict parent/child relationship. This helped us build out [a smaller interface for the snapshotters][6] while still fulfilling the requirements needed from things [like a builder][7] as well as reduce the amount of code needed, making it much easier to maintain in the long run. - -![][8] - -You can find more details about the architecture of containerd in [Stephen Day's Dec 7th 2017 KubeCon SIG Node presentation][9]. - -In addition to the technical and design changes in the 1.0 codebase, we also switched the containerd [governance model from the long standing BDFL to a Technical Steering Committee][10] giving the community an independent third party resource to rely on. - - --------------------------------------------------------------------------------- - -via: https://blog.docker.com/2017/12/containerd-ga-features-2/ - -作者：[Michael Crosby][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://blog.docker.com/author/michael/ -[1]:https://i0.wp.com/blog.docker.com/wp-content/uploads/950cf948-7c08-4df6-afd9-cc9bc417cabe-6.jpg?resize=400%2C120&ssl=1 -[2]:https://i1.wp.com/blog.docker.com/wp-content/uploads/4a7666e4-ebdb-4a40-b61a-26ac7c3f663e-4.jpg?resize=906%2C470&ssl=1 (containerd) -[3]:https://i1.wp.com/blog.docker.com/wp-content/uploads/2a73a4d8-cd40-4187-851f-6104ae3c12ba-1.jpg?resize=1140%2C680&ssl=1 -[4]:https://github.com/containerd/containerd/blob/master/api/services/tasks/v1/tasks.proto -[5]:https://blog.mobyproject.org/where-are-containerds-graph-drivers-145fc9b7255 -[6]:https://github.com/containerd/containerd/blob/master/api/services/snapshots/v1/snapshots.proto -[7]:https://blog.mobyproject.org/introducing-buildkit-17e056cc5317 -[8]:https://i1.wp.com/blog.docker.com/wp-content/uploads/d0fb5eb9-c561-415d-8d57-e74442a879a2-1.jpg?resize=1140%2C556&ssl=1 -[9]:https://speakerdeck.com/stevvooe/whats-happening-with-containerd-and-the-cri -[10]:https://github.com/containerd/containerd/pull/1748 diff --git a/sources/tech/20171215 Linux Vs Unix.md b/sources/tech/20171215 Linux Vs Unix.md deleted file mode 100644 index 40c411051b..0000000000 --- a/sources/tech/20171215 Linux Vs Unix.md +++ /dev/null @@ -1,143 +0,0 @@ - - translating by HardworkFish - - [![Linux vs. Unix](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/unix-vs-linux_orig.jpg)][1] - -​In computer time, a substantial part of the population has a misconception that the **Unix** and **Linux** operating systems are one and the same. However, the opposite is true. Let's look at it from a closer look. - -### What is Unix? - - [![what is unix](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/unix_orig.png)][2] - -In IT, we come across - -[Unix][3] - -as an operating system (under the trademark), which was created by AT & T in 1969 in New Jersey, USA. Most operating systems are inspired by Unix, but Unix has also been inspired by the Multics system, which has not been completed. Another version of Unix was Plan 9 from Bell Labs. - -### Where is Unix used? - -As an operating system, Unix is used in particular for servers, workstations, and nowadays also for personal computers. It played a very important role in the creation of the Internet, the creation of computer networks or also the client-server model. - -#### Characteristics of the Unix system: - -* supports multitasking (multitasking) - -* Simplicity of control compared to Multics - -* all data is stored as plain text - -* tree saving of a single-root file - -* access to multiple user accounts​ - -#### Unix Operating System Composition: - -​ - -**a)** - -a monolithic operating system kernel that takes care of low-level and user-initiated operations, the total communication takes place via a system call. - -**b)** - -system utilities (or so-called utilities) - -**c)** - -many other applications - -### What is Linux? - - [![what is linux](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/linux_orig.png)][4] - -This is an open source operating system built on the principle of a Unix system. As the name of the open-source description suggests, it is a freely-downloadable system that can be downloaded externally, but it is also possible to interfere with the system's editing, adding, and then extending the source code. It's one of the biggest benefits, unlike today's operating systems that are paid (Windows, Mac OS X, ...). Not only was Unix a model for creating a new operating system, another important factor was the MINIX system. Unlike - -**Linus** - -, this version was used by its creator ( - -**Andrew Tanenbaum** - -) as a commercial system. - -​ - -[Linux][5] - -began to be developed by - -**Linus Torvalds** - -in 1991, which was a system that dealt with as a hobby. One of the main reasons why Linux started to deal with Unix was the simplicity of the system. The first official release of the provisory version of Linux (0.01) occurred on September 17, 1991\. Even though the system was completely imperfect and complete, it was of great interest to him, and within a few days, Linus started to write emails with other ideas about expansion or source codes. - -### Characteristics of Linux - -The cornerstone of Linux is the Unix kernel, which is based on the basic characteristics of Unix and the standards that are - -**POSIX** - - and Single - -**UNIX Specification** - -. As it may seem, the official name of the operating system is taken from the creator of - -**Linus** - -, where the end of the operating system name "x" is just a link to the - -**Unix system** - -. - -#### Main features: - -* run multiple tasks at once (multitasking) - -* programs may consist of one or more processes (multipurpose system), and each process may have one or more threads - -* multiuser, so it can run multiple user programs - -* individual accounts are protected by appropriate authorization - -* so the accounts have precisely defined system control rights - -The author of - -**Tuxe Penguin's** - -logo is Larry Ewing of 1996, who accepted him as a mascot for his open-source - -**Linux operating system** - -. - -**Linux Torvalds** - -proposed the initial name of the new operating system as "Freax" as free + freak + x ( - -**Unix system** - -), but it did not like the - -**FTP server** - -where the provisory version of Linux was running. - --------------------------------------------------------------------------------- - -via: http://www.linuxandubuntu.com/home/linux-vs-unix - -作者：[linuxandubuntu][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:http://www.linuxandubuntu.com -[1]:http://www.linuxandubuntu.com/home/linux-vs-unix -[2]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/unix_orig.png -[3]:http://www.unix.org/what_is_unix.html -[4]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/linux_orig.png -[5]:https://www.linux.com diff --git a/sources/tech/20171218 Internet Chemotherapy.md b/sources/tech/20171218 Internet Chemotherapy.md deleted file mode 100644 index ffe15fb5c1..0000000000 --- a/sources/tech/20171218 Internet Chemotherapy.md +++ /dev/null @@ -1,335 +0,0 @@ -Internet Chemotherapy -====== - -12/10 2017 - -### 1. Internet Chemotherapy - -Internet Chemotherapy was a 13 month project between Nov 2016 - Dec 2017. -It has been known under names such as 'BrickerBot', 'bad firmware -upgrade', 'ransomware', 'large-scale network failure' and even -'unprecedented terrorist actions.' That last one was a little harsh, -Fernandez, but I guess I can't please everybody. - -You can download the module which executes the http and telnet-based -payloads from this router at http://91.215.104.140/mod_plaintext.py. Due to -platform limitations the module is obfuscated single threaded python, but -the payloads are in plain view and should be easy to figure out for any -programmer worth his/her/hir salt. Take a look at the number of payloads, -0-days and techniques and let the reality sink in for a moment. Then -imagine what would've happened to the Internet in 2017 if I had been a -blackhat dedicated to building a massive DDoS cannon for blackmailing the -biggest providers and companies. I could've disrupted them all and caused -extraordinary damage to the Internet in the process. - -My ssh crawler is too dangerous to publish. It contains various levels of -automation for the purpose of moving laterally through poorly designed -ISP networks and taking them over through only a single breached router. -My ability to commandeer and secure hundreds of thousands of ISP routers -was the foundation of my anti-IoT botnet project as it gave me great -visibility of what was happening on the Internet and it gave me an -endless supply of nodes for hacking back. I began my non-destructive ISP -network cleanup project in 2015 and by the time Mirai came around I was -in a good position to react. The decision to willfully sabotage other -people's equipment was nonetheless a difficult one to make, but the -colossally dangerous CVE-2016-10372 situation ultimately left me with no -other choice. From that moment on I was all-in. - -I am now here to warn you that what I've done was only a temporary band- -aid and it's not going to be enough to save the Internet in the future. -The bad guys are getting more sophisticated, the number of potentially -vulnerable devices keep increasing, and it's only a matter of time before -a large scale Internet-disrupting event will occur. If you are willing to -believe that I've disabled over 10 million vulnerable devices over the 13- -month span of the project then it's not far-fetched to say that such a -destructive event could've already happened in 2017. - -YOU SHOULD WAKE UP TO THE FACT THAT THE INTERNET IS ONLY ONE OR TWO -SERIOUS IOT EXPLOITS AWAY FROM BEING SEVERELY DISRUPTED. The damage of -such an event is immeasurable given how digitally connected our societies -have become, yet CERTs, ISPs and governments are not taking the gravity -of the situation seriously enough. ISPs keep deploying devices with -exposed control ports and although these are trivially found using -services like Shodan the national CERTs don't seem to care. A lot of -countries don't even have CERTs. Many of the world's biggest ISPs do not -have any actual security know-how in-house, and are instead relying on -foreign vendors for help in case anything goes wrong. I've watched large -ISPs withering for months under conditioning from my botnet without them -being able to fully mitigate the vulnerabilities (good examples are BSNL, -Telkom ZA, PLDT, from time to time PT Telkom, and pretty much most large -ISPs south of the border). Just look at how slow and ineffective Telkom -ZA was in dealing with its Aztech modem problem and you will begin to -understand the hopelessness of the current situation. In 99% of the -problem cases the solution would have simply been for the ISPs to deploy -sane ACLs and CPE segmentation, yet months later their technical staff -still hasn't figured this out. If ISPs are unable to mitigate weeks and -months of continuous deliberate sabotage of their equipment then what -hope is there that they would notice and fix a Mirai problem on their -networks? Many of the world's biggest ISPs are catastrophically negligent -and this is the biggest danger by a landslide, yet paradoxically it -should also be the easiest problem to fix. - -I've done my part to try to buy the Internet some time, but I've gone as -far as I can. Now it's up to you. Even small actions are important. Among -the things you can do are: - -* Review your own ISP's security through services such as Shodan and take - them to task over exposed telnet, http, httpd, ssh, tr069 etc. ports on - their networks. Refer them to this document if you have to. There's no - good reason why any of these control ports should ever be accessible - from the outside world. Exposing control ports is an amateur mistake. - If enough customers complain they might actually do something about it! - -* Vote with your wallet! Refuse to buy or use 'intelligent' products - unless the manufacturer can prove that the product can and will receive - timely security updates. Find out about the vendor's security track - record before giving them your hard-earned money. Be willing to pay a - little bit more for credible security. - -* Lobby your local politicians and government officials for improved - security legislation for IoT (Internet of Things) devices such as - routers, IP cameras and 'intelligent' devices. Private or public - companies currently lack the incentives for solving this problem in the - immediate term. This matter is as important as minimum safety - requirements for cars and general electrical appliances. - -* Consider volunteering your time or other resources to underappreciated - whitehat organizations such as GDI Foundation or Shadowserver - Foundation. These organizations and people make a big difference and - they can significantly amplify the impact of your skillset in helping - the Internet. - -* Last but not least, consider the long-shot potential of getting IoT - devices designated as an 'attractive nuisance' through precedent- - setting legal action. If a home owner can be held liable for a - burglar/trespasser getting injured then I don't see why a device owner - (or ISP or manufacturer) shouldn't be held liable for the damage that - was caused by their dangerous devices being exploitable through the - Internet. Attribution won't be a problem for Layer 7 attacks. If any - large ISPs with deep pockets aren't willing to fund such precedent - cases (and they might not since they fear that such precedents could - come back to haunt them) we could even crowdfund such initiatives over - here and in the EU. ISPs: consider your volumetric DDoS bandwidth cost - savings in 2017 as my indirect funding of this cause and as evidence - for its potential upside. - -### 2. Timeline - -Here are some of the more memorable events of the project: - -* Deutsche Telekom Mirai disruption in late November 2016. My hastily - assembled initial TR069/64 payload only performed a 'route del default' - but this was enough to get the ISP's attention to the problem and the - resulting headlines alerted other ISPs around the world to the - unfolding disaster. - -* Around January 11-12 some Mirai-infected DVRs with exposed control port - 6789 ended up getting bricked in Washington DC, and this made numerous - headlines. Gold star to Vemulapalli for determining that Mirai combined - with /dev/urandom had to be 'highly sophisticated ransomware'. Whatever - happened to those 2 unlucky souls in Europe? - -* In late January 2017 the first genuine large-scale ISP takedown occured - when Rogers Canada's supplier Hitron carelessly pushed out new firmware - with an unauthenticated root shell listening on port 2323 (presumably - this was a debugging interface that they forgot to disable). This epic - blunder was quickly discovered by Mirai botnets, and the end-result was - a large number of bricked units. - -* In February 2017 I noticed the first Mirai evolution of the year, with - both Netcore/Netis and Broadcom CLI-based modems being attacked. The - BCM CLI would turn out to become one of the main Mirai battlegrounds of - 2017, with both the blackhats and me chasing the massive long tail of - ISP and model-specific default credentials for the rest of the year. - The 'broadcom' payloads in the above source may look strange but - they're statistically the most likely sequences to disable any of the - endless number of buggy BCM CLI firmwares out there. - -* In March 2017 I significantly increased my botnet's node count and - started to add more web payloads in response to the threats from IoT - botnets such as Imeij, Amnesia and Persirai. The large-scale takedown - of these hacked devices created a new set of concerns. For example, - among the leaked credentials of the Avtech and Wificam devices there - were logins which strongly implied airports and other important - facilities, and around April 1 2017 the UK government officials - warned of a 'credible cyber threat' to airports and nuclear - facilities from 'hacktivists.' Oops. - -* The more aggressive scanning also didn't escape the attention of - civilian security researchers, and in April 6 2017 security company - Radware published an article about my project. The company trademarked - it under the name 'BrickerBot.' It became clear that if I were to - continue increasing the scale of my IoT counteroffensive I had to come - up with better network mapping/detection methods for honeypots and - other risky targets. - -* Around April 11th 2017 something very unusual happened. At first it - started like so many other ISP takedowns, with a semi-local ISP called - Sierra Tel running exposed Zyxel devices with the default telnet login - of supervisor/zyad1234. A Mirai runner discovered the exposed devices - and my botnet followed soon after, and yet another clash in the epic - BCM CLI war of 2017 took place. This battle didn't last long. It - would've been just like any of the hundreds of other ISP takedowns in - 2017 were it not for something very unusual occuring right after the - smoke settled. Amazingly, the ISP didn't try to cover up the outage as - some kind of network issue, power spike or a bad firmware upgrade. They - didn't lie to their customers at all. Instead, they promptly published - a press release about their modems having been vulnerable which allowed - their customers to assess their potential risk exposure. What did the - most honest ISP in the world get for its laudable transparency? Sadly - it got little more than criticism and bad press. It's still the most - depressing case of 'why we can't have nice things' to me, and probably - the main reason for why 99% of security mistakes get covered up and the - actual victims get left in the dark. Too often 'responsible disclosure' - simply becomes a euphemism for 'coverup.' - -* On April 14 2017 DHS warned of 'BrickerBot Threat to Internet of - Things' and the thought of my own government labeling me as a cyber - threat felt unfair and myopic. Surely the ISPs that run dangerously - insecure network deployments and the IoT manufacturers that peddle - amateurish security implementations should have been fingered as the - actual threat to Americans rather than me? If it hadn't been for me - millions of us would still be doing their banking and other sensitive - transactions over hacked equipment and networks. If anybody from DHS - ever reads this I urge you to reconsider what protecting the homeland - and its citizens actually means. - -* In late April 2017 I spent some time on improving my TR069/64 attack - methods, and in early May 2017 a company called Wordfence (now Defiant) - reported a significant decline in a TR069-exploiting botnet that had - previously posed a threat to Wordpress installations. It's noteworthy - that the same botnet temporarily returned a few weeks later using a - different exploit (but this was also eventually mitigated). - -* In May 2017 hosting company Akamai reported in its Q1 2017 State of the - Internet report an 89% decrease in large (over 100 Gbps) DDoS attacks - compared with Q1 2016, and a 30% decrease in total DDoS attacks. The - largest attack of Q1 2017 was 120 Gbps vs 517 Gbps in Q4 2016. As large - volumetric DDoS was one of the primary signatures of Mirai this felt - like concrete justification for all the months of hard work in the IoT - trenches. - -* During the summer I kept improving my exploit arsenal, and in late July - I performed some test runs against APNIC ISPs. The results were quite - surprising. Among other outcomes a few hundred thousand BSNL and MTNL - modems were disabled and this outage become headline news in India. - Given the elevated geopolitical tensions between India and China at the - time I felt that there was a credible risk of the large takedown being - blamed on China so I made the rare decision to publically take credit - for it. Catalin, I'm very sorry for the abrupt '2 day vacation' that - you had to take after reporting the news. - -* Previously having worked on APNIC and AfriNIC, on August 9th 2017 I - also launched a large scale cleanup of LACNIC space which caused - problems for various providers across the subcontinent. The attack made - headlines in Venezuela after a few million cell phone users of Movilnet - lost service. Although I'm personally against government surveillance - of the Internet the case of Venezuela is noteworthy. Many of the - LACNIC ISPs and networks have been languishing for months under - persistent conditioning from my botnet, but Venezuelan providers have - been quick to fortify their networks and secure their infrastructure. - I believe this is due to Venezuela engaging in far more invasive deep - packet inspection than the other LACNIC countries. Food for thought. - -* In August 2017 F5 Labs released a report called "The Hunt for IoT: The - Rise of Thingbots" in which the researchers were perplexed over the - recent lull in telnet activity. The researchers speculated that the - lack of activity may be evidence that one or more very large cyber - weapons are being built (which I guess was in fact true). This piece - is to my knowledge the most accurate assessment of the scope of my - project but fascinatingly the researchers were unable to put two and - two together in spite of gathering all the relevant clues on a single - page. - -* In August 2017 Akamai's Q2 2017 State of the Internet report announces - the first quarter in 3 years without the provider observing a single - large (over 100 Gbps) attack, and a 28% decrease in total DDoS attacks - vs Q1 2017. This seems like further validation of the cleanup effort. - This phenomenally good news is completely ignored by the mainstream - media which operates under an 'if it bleeds it leads' mentality even - when it comes to information security. This is yet another reason why - we can't have nice things. - -* After the publication of CVE-2017-7921 and 7923 in September 2017 I - decided to take a closer look at Hikvision devices, and to my horror - I realized that there's a technique for botting most of the vulnerable - firmwares that the blackhats hadn't discovered yet. As a result I - launched a global cleanup initiative around mid-September. Over a - million DVRs and cameras (mainly Hikvision and Dahua) were disabled - over a span of 3 weeks and publications such as IPVM.com wrote several - articles about the attacks. Dahua and Hikvision wrote press releases - mentioning or alluding to the attacks. A huge number of devices finally - got their firmwares upgraded. Seeing the confusion that the cleanup - effort caused I decided to write a quick summary for the CCTV people at - http://depastedihrn3jtw.onion.link/show.php?md5=62d1d87f67a8bf485d43a05ec32b1e6f - (sorry for the NSFW language of the pastebin service). The staggering - number of vulnerable units that were online months after critical - security patches were available should be the ultimate wakeup call to - everyone about the utter dysfunctionality of the current IoT patching - process. - -* Around September 28 2017 Verisign releases a report saying that DDoS - attacks declined 55% in Q2 2017 vs Q1, with a massive 81% attack peak - decline. - -* On November 23rd 2017 the CDN provider Cloudflare reports that 'in - recent months, Cloudflare has seen a dramatic reduction in simple - attempts to flood our network with junk traffic.' Cloudflare speculates - it could've partly been due to their change in policies, but the - reductions also line up well with the IoT cleanup activities. - -* At the end of November 2017 Akamai's Q3 2017 State of the Internet - report sees a small 8% increase in total DDoS attacks for the quarter. - Although this was a significant reduction compared to Q3 2016 the - slight uptick serves as a reminder of the continued risks and dangers. - -* As a further reminder of the dangers a new Mirai strain dubbed 'Satori' - reared its head in November-December of 2017. It's particularly - noteworthy how quickly the botnet managed to grow based on a single - 0-day exploit. This event underlines the current perilous operating - state of the Internet, and why we're only one or two severe IoT - exploits away from widespread disruption. What will happen when nobody - is around to disable the next threat? Sinkholing and other whitehat/ - 'legal' mitigations won't be enough in 2018 just like they weren't - enough in 2016. Perhaps in the future governments will be able to - collaborate on a counterhacking task force with a global mandate for - disabling particularly severe existential threats to the Internet, but - I'm not holding my breath. - -* Late in the year there were also some hysterical headlines regarding a - new botnet that was dubbed 'Reaper' and 'IoTroop'. I know some of you - will eventually ridicule those who estimated its size at 1-2 million - but you should understand that security researchers have very limited - knowledge of what's happening on networks and hardware that they don't - control. In practice the researchers could not possibly have known or - even assumed that most of the vulnerable device pool had already been - disabled by the time the botnet emerged. Give the 'Reaper' one or two - new unmitigated 0-days and it'll become as terrifying as our worst - fears. - -### 3. Parting Thoughts - -I'm sorry to leave you in these circumstances, but the threat to my own -safety is becoming too great to continue. I have made many enemies. If -you want to help look at the list of action items further up. Good luck. - -There will also be those who will criticize me and say that I've acted -irresponsibly, but that's completely missing the point. The real point -is that if somebody like me with no previous hacking background was able -to do what I did, then somebody better than me could've done far worse -things to the Internet in 2017. I'm not the problem and I'm not here to -play by anyone's contrived rules. I'm only the messenger. The sooner you -realize this the better. - --Dr Cyborkian a.k.a. janit0r, conditioner of 'terminally ill' devices. - --------------------------------------------------------------------------------- - -via：https://ghostbin.com/paste/q2vq2 - -作者：janit0r -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译， -[Linux中国](https://linux.cn/) 荣誉推出 diff --git a/sources/tech/20171219 How to generate webpages using CGI scripts.md b/sources/tech/20171219 How to generate webpages using CGI scripts.md deleted file mode 100644 index b452cdee84..0000000000 --- a/sources/tech/20171219 How to generate webpages using CGI scripts.md +++ /dev/null @@ -1,129 +0,0 @@ -How to generate webpages using CGI scripts -====== -Back in the stone age of the Internet when I first created my first business website, life was good. - -I installed Apache and created a few simple HTML pages that stated a few important things about my business and gave important information like an overview of my product and how to contact me. It was a static website because the content seldom changed. Maintenance was simple because of the unchanging nature of my site. - -## Static content - -Static content is easy and still common. Let's take a quick look at a couple sample static web pages. You don't need a working website to perform these little experiments. Just place the files in your home directory and open them with your browser. You will see exactly what you would if the file were served to your browser via a web server. - -The first thing you need on a static website is the index.html file which is usually located in the /var/www/html directory. This file can be as simple as a text phrase such as "Hello world" without any HTML markup at all. This would simply display the text string. Create index.html in your home directory and add "Hello world" (without the quotes) as it's only content. Open the index.html in your browser with the following URL. -``` -file:///home//index.html -``` - -So HTML is not required, but if you had a large amount of text that needed formatting, the results of a web page with no HTML coding would be incomprehensible with everything running together. - -So the next step is to make the content more readable by using a bit of HTML coding to provide some formatting. The following command creates a page with the absolute minimum markup required for a static web page with HTML. You could also use your favorite editor to create the content. -``` -echo "

Hello World

" > test1.html -``` - -Now view index.html and see the difference. - -Of course you can put a lot of additional HTML around the actual content line to make a more complete and standard web page. That more complete version as shown below will still display the same results in the browser, but it also forms the basis for more standardized web site. Go ahead and use this content for your index.html file and display it in your browser. -``` - - - -My Web Page - - -

Hello World

- - -``` - -I built a couple static websites using these techniques, but my life was about to change. - -## Dynamic web pages for a new job - -I took a new job in which my primary task was to create and maintain the CGI ([Common Gateway Interface][6]) code for a very dynamic website. In this context, dynamic means that the HTML needed to produce the web page on a browser was generated from data that could be different every time the page was accessed. This includes input from the user on a web form that is used to look up data in a database. The resulting data is surrounded by appropriate HTML and displayed on the requesting browser. But it does not need to be that complex. - -Using CGI scripts for a website allows you to create simple or complex interactive programs that can be run to provide a dynamic web page that can change based on input, calculations, current conditions in the server, and so on. There are many languages that can be used for CGI scripts. We will look at two of them, Perl and Bash. Other popular CGI languages include PHP and Python. - -This article does not cover installation and setup of Apache or any other web server. If you have access to a web server that you can experiment with, you can directly view the results as they would appear in a browser. Otherwise, you can still run the programs from the command line and view the HTML that would be created. You can also redirect that HTML output to a file and then display the resulting file in your browser. - -### Using Perl - -Perl is a very popular language for CGI scripts. Its strength is that it is a very powerful language for the manipulation of text. - -To get CGI scripts to execute, you need the following line in the in httpd.conf for the website you are using. This tells the web server where your executable CGI files are located. For this experiment, let's not worry about that. -``` -ScriptAlias /cgi-bin/ "/var/www/cgi-bin/" -``` - -Add the following Perl code to the file index.cgi, which should be located in your home directory for your experimentation. Set the ownership of the file to apache.apache when you use a web server, and set the permissions to 755 because it must be executable no matter where it is located. - -``` -#!/usr/bin/perl -print "Content-type: text/html\n\n"; -print "\n"; -print "

Hello World

\n"; -print "Using Perl

\n"; -print "\n"; -``` - -Run this program from the command line and view the results. It should display the HTML code it will generate. - -Now view the index.cgi in your browser. Well, all you get is the contents of the file. Browsers really need to have this delivered as CGI content. Apache does not really know that it needs to run the file as a CGI program unless the Apache configuration for the web site includes the "ScriptAlias" definition as shown above. Without that bit of configuration Apache simply send the data in the file to the browser. If you have access to a web server, you could try this out with your executable index files in the /var/www/cgi-bin directory. - -To see what this would look like in your browser, run the program again and redirect the output to a new file. Name it whatever you want. Then use your browser to view the file that contains the generated content. - -The above CGI program is still generating static content because it always displays the same output. Add the following line to your CGI program immediately after the "Hello World" line. The Perl "system" command executes the commands following it in a system shell, and returns the result to the program. In this case, we simply grep the current RAM usage out of the results from the free command. - -``` -system "free | grep Mem\n"; -``` - -Now run the program again and redirect the output to the results file. Reload the file in the browser. You should see an additional line so that displays the system memory statistics. Run the program and refresh the browser a couple more times and notice that the memory usage should change occasionally. - -### Using Bash - -Bash is probably the simplest language of all for use in CGI scripts. Its primary strength for CGI programming is that it has direct access to all of the standard GNU utilities and system programs. - -Rename the existing index.cgi to Perl.index.cgi and create a new index.cgi with the following content. Remember to set the permissions correctly to executable. - -``` -#!/bin/bash -echo "Content-type: text/html" -echo "" -echo '' -echo '' -echo '' -echo 'Hello World' -echo '' -echo '' -echo '

Hello World

' -echo 'Using Bash

' -free | grep Mem -echo '' -echo '' -exit 0 -``` - -Execute this program from the command line and view the output, then run it and redirect the output to the temporary results file you created before. Then refresh the browser to view what it looks like displayed as a web page. - -## Conclusion - -It is actually very simple to create CGI programs that can be used to generate a wide range of dynamic web pages. This is a trivial example but you should now see some of the possibilities. - --------------------------------------------------------------------------------- - -via: https://opensource.com/article/17/12/cgi-scripts - -作者：[David Both][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://opensource.com/users/dboth -[1]:http://december.com/html/4/element/html.html -[2]:http://december.com/html/4/element/head.html -[3]:http://december.com/html/4/element/title.html -[4]:http://december.com/html/4/element/body.html -[5]:http://december.com/html/4/element/h1.html -[6]:https://en.wikipedia.org/wiki/Common_Gateway_Interface -[7]:http://perldoc.perl.org/functions/system.html diff --git a/sources/tech/20171231 Why You Should Still Love Telnet.md b/sources/tech/20171231 Why You Should Still Love Telnet.md deleted file mode 100644 index 201ee91bd4..0000000000 --- a/sources/tech/20171231 Why You Should Still Love Telnet.md +++ /dev/null @@ -1,161 +0,0 @@ -XYenChi is translating -Why You Should Still Love Telnet -====== -Telnet, the protocol and the command line tool, were how system administrators used to log into remote servers. However, due to the fact that there is no encryption all communication, including passwords, are sent in plaintext meant that Telnet was abandoned in favour of SSH almost as soon as SSH was created. - -For the purposes of logging into a remote server, you should never, and probably have never considered it. This does not mean that the `telnet` command is not a very useful tool when used for debugging remote connection problems. - -In this guide, we will explore using `telnet` to answer the all too common question, "Why can't I ###### connect‽". - -This frustrated question is usually encountered after installing a application server like a web server, an email server, an ssh server, a Samba server etc, and for some reason, the client won't connect to the server. - -`telnet` isn't going to solve your problem but it will, very quickly, narrow down where you need to start looking to fix your problem. - -`telnet` is a very simple command to use for debugging network related issues and has the syntax: -``` -telnet - -``` - -Because `telnet` will initially simply establish a connection to the port without sending any data it can be used with almost any protocol including encrypted protocols. - -There are four main errors that you will encounter when trying to connect to a problem server. We will look at all four, explore what they mean and look at how you should fix them. - -For this guide we will assume that we have just installed a [Samba][1] server at `samba.example.com` and we can't get a local client to connect to the server. - -### Error 1 - The connection that hangs forever - -First, we need to attempt to connect to the Samba server with `telnet`. This is done with the following command (Samba listens on port 445): -``` -telnet samba.example.com 445 - -``` - -Sometimes, the connection will get to this point stop indefinitely: -``` -telnet samba.example.com 445 -Trying 172.31.25.31... - -``` - -This means that `telnet` has not received any response to its request to establish a connection. This can happen for two reasons: - - 1. There is a router down between you and the server. - 2. There is a firewall dropping your request. - - - -In order to rule out **1.** run a quick [`mtr samba.example.com`][2] to the server. If the server is accessible then it's a firewall (note: it's almost always a firewall). - -Firstly, check if there are any firewall rules on the server itself with the following command `iptables -L -v -n`, if there are none then you will get the following output: -``` -iptables -L -v -n -Chain INPUT (policy ACCEPT 0 packets, 0 bytes) - pkts bytes target prot opt in out source destination - -Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) - pkts bytes target prot opt in out source destination - -Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) - pkts bytes target prot opt in out source destination - -``` - -If you see anything else then this is likely the problem. In order to check, stop `iptables` for a moment and run `telnet samba.example.com 445` again and see if you can connect. If you still can't connect see if your provider and/or office has a firewall in place that is blocking you. - -### Error 2 - DNS problems - -A DNS issue will occur if the hostname you are using does not resolve to an IP address. The error that you will see is as follows: -``` -telnet samba.example.com 445 -Server lookup failure: samba.example.com:445, Name or service not known - -``` - -The first step here is to substitute the IP address of the server for the hostname. If you can connect to the IP but not the hostname then the problem is the hostname. - -This can happen for many reasons (I have seen all of the following): - - 1. Is the domain registered? Use `whois` to find out if it is. - 2. Is the domain expired? Use `whois` to find out if it is. - 3. Are you using the correct hostname? Use `dig` or `host` to ensure that the hostname you are using resolves to the correct IP. - 4. Is your **A** record correct? Check that you didn 't accidentally create an **A** record for something like `smaba.example.com`. - - - -Always double check the spelling and the correct hostname (is it `samba.example.com` or `samba1.example.com`) as this will often trip you up especially with long, complicated or foreign hostnames. - -### Error 3 - The server isn't listening on that port - -This error occurs when `telnet` is able to reach to the server but there is nothing listening on the port you specified. The error looks like this: -``` -telnet samba.example.com 445 -Trying 172.31.25.31... -telnet: Unable to connect to remote host: Connection refused - -``` - -This can happen for a couple of reasons: - - 1. Are you **sure** you 're connecting to the right server? - 2. Your application server is not listening on the port you think it is. Check exactly what it's doing by running `netstat -plunt` on the server and see what port it is, in fact, listening on. - 3. The application server isn't running. This can happen when the application server exits immediately and silently after you start it. Start the server and run `ps auxf` or `systemctl status application.service` to check it's running. - - - -### Error 4 - The connection was closed by the server - -This error happens when the connection was successful but the application server has a build in security measure that killed the connection as soon as it was made. This error looks like: -``` -telnet samba.example.com 445 -Trying 172.31.25.31... -Connected to samba.example.com. -Escape character is '^]'. -��Connection closed by foreign host. - -``` - -The last line `Connection closed by foreign host.` indicates that the connection was actively terminated by the server. In order to fix this, you need to look at the security configuration of the application server to ensure your IP or user is allowed to connect to it. - -### A successful connection - -This is what a successful `telnet` connection attempt looks like: -``` -telnet samba.example.com 445 -Trying 172.31.25.31... -Connected to samba.example.com. -Escape character is '^]'. - -``` - -The connection will stay open for a while depending on the timeout of the application server you are connected to. - -A telnet connection is closed by typing `CTRL+]` and then when you see the `telnet>` prompt, type "quit" and hit ENTER i.e.: -``` -telnet samba.example.com 445 -Trying 172.31.25.31... -Connected to samba.example.com. -Escape character is '^]'. -^] -telnet> quit -Connection closed. - -``` - -### Conclusion - -There are a lot of reasons that a client application can't connect to a server. The exact reason can be difficult to establish especially when the client is a GUI that offers little or no error information. Using `telnet` and observing the output will allow you to very rapidly narrow down where the problem lies and save you a whole lot of time. - --------------------------------------------------------------------------------- - -via: https://bash-prompt.net/guides/telnet/ - -作者：[Elliot Cooper][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://bash-prompt.net -[1]:https://www.samba.org/ -[2]:https://www.systutorials.com/docs/linux/man/8-mtr/ diff --git a/sources/tech/20180103 How To Find The Installed Proprietary Packages In Arch Linux.md b/sources/tech/20180103 How To Find The Installed Proprietary Packages In Arch Linux.md index 69b523426c..8dc2e92513 100644 --- a/sources/tech/20180103 How To Find The Installed Proprietary Packages In Arch Linux.md +++ b/sources/tech/20180103 How To Find The Installed Proprietary Packages In Arch Linux.md @@ -1,3 +1,4 @@ +Translating by stevenzdg988 How To Find The Installed Proprietary Packages In Arch Linux ====== ![](https://www.ostechnix.com/wp-content/uploads/2018/01/Absolutely-Proprietary-720x340.jpg) diff --git a/sources/tech/20180103 How to preconfigure LXD containers with cloud-init.md b/sources/tech/20180103 How to preconfigure LXD containers with cloud-init.md deleted file mode 100644 index ed6eacd2fb..0000000000 --- a/sources/tech/20180103 How to preconfigure LXD containers with cloud-init.md +++ /dev/null @@ -1,197 +0,0 @@ -How to preconfigure LXD containers with cloud-init -====== -You are creating containers and you want them to be somewhat preconfigured. For example, you want them to run automatically **apt update** as soon as they are launched. Or, get some packages pre-installed, or run a few commands. Here is how to perform this early initialization with [**cloud-init**][1] through [LXD to container images that support **cloud-init**][2]. - -In the following, we are creating a separate LXD profile with some cloud-init instructions, then launch a container using that profile. - -### How to create a new LXD profile - -Let's see the existing profiles. -``` -$ **lxc profile list** -+---------|---------+ -| NAME | USED BY | -+---------|---------+ -| default | 11 | -+---------|---------+ -``` - -There is one profile, **default**. We copy it to a new name, so that we can start adding our instructions on that profile. -``` -$ **lxc profile copy default devprofile** - -$ **lxc profile list** -+------------|---------+ -| NAME | USED BY | -+------------|---------+ -| default | 11 | -+------------|---------+ -| devprofile | 0 | -+------------|---------+ -``` - -We have a new profile to work on, **devprofile**. Here is how it looks, -``` -$ **lxc profile show devprofile** -config: - environment.TZ: "" -description: Default LXD profile -devices: - eth0: - nictype: bridged - parent: lxdbr0 - type: nic - root: - path: / - pool: default - type: disk -name: devprofile -used_by: [] -``` - -Note the main sections, **config:** , **description:** , **devices:** , **name:** , and **used_by:**. There is careful indentation in the profile, and when you make edits, you need to take care of the indentation. - -### How to add cloud-init to an LXD profile - -In the **config:** section of a LXD profile, we can insert [cloud-init][1] instructions. Those[ cloud-init][1] instructions will be passed to the container and will be used when it is first launched. - -Here are those that we are going to use in the example, -``` - package_upgrade: true - packages: - - build-essential - locale: es_ES.UTF-8 - timezone: Europe/Madrid - runcmd: - - [touch, /tmp/simos_was_here] -``` - -**package_upgrade: true** means that we want **cloud-init** to run **sudo apt upgrade** when the container is first launched. Under **packages:** we list the packages that we want to get automatically installed. Then we set the **locale** and **timezone**. In the Ubuntu container images, the default locale for **root** is **C.UTF-8** , for the **ubuntu** account it 's **en_US.UTF-8**. The timezone is **Etc/UTC**. Finally, we show [how to run a Unix command with **runcmd**][3]. - -The part that needs a bit of attention is how to insert the **cloud-init** instructions into the LXD profile. My preferred way is -``` -$ **lxc profile edit devprofile** -``` - -This opens up a text editor and allows to paste the instructions. Here is [how the result should look like][4], -``` -$ **lxc profile show devprofile** -config: - environment.TZ: "" - - - user.user-data: | - #cloud-config - package_upgrade: true - packages: - - build-essential - locale: es_ES.UTF-8 - timezone: Europe/Madrid - runcmd: - - [touch, /tmp/simos_was_here] - - -description: Default LXD profile -devices: - eth0: - nictype: bridged - parent: lxdbr0 - type: nic - root: - path: / - pool: default - type: disk -name: devprofile -used_by: [] -``` - -WordPress can get a bit messed with indentation when you copy/paste, therefore, you may use [this pastebin][4] instead. - -### How to launch a container using a profile - -Let's launch a new container using the profile **devprofile**. -``` -$ **lxc launch --profile devprofile ubuntu:x mydev** -``` - -Let's get into the container and figure out whether our instructions took effect. -``` -$ **lxc exec mydev bash** -root@mydev:~# **ps ax** - PID TTY STAT TIME COMMAND - 1 ? Ss 0:00 /sbin/init - ... - 427 ? Ss 0:00 /usr/bin/python3 /usr/bin/cloud-init modules --mode=f - 430 ? S 0:00 /bin/sh -c tee -a /var/log/cloud-init-output.log - 431 ? S 0:00 tee -a /var/log/cloud-init-output.log - 432 ? S 0:00 /usr/bin/apt-get --option=Dpkg::Options::=--force-con - 437 ? S 0:00 /usr/lib/apt/methods/http - 438 ? S 0:00 /usr/lib/apt/methods/http - 440 ? S 0:00 /usr/lib/apt/methods/gpgv - 570 ? Ss 0:00 bash - 624 ? S 0:00 /usr/lib/apt/methods/store - 625 ? R+ 0:00 ps ax -root@mydev:~# -``` - -We connected quite quickly, and **ps ax** shows that the package update is indeed taking place! We can get the full output at /var/log/cloud-init-output.log and in there, -``` -Generating locales (this might take a while)... - es_ES.UTF-8... done -Generation complete. -``` - -The locale got set. The **root** user keeps having the **C.UTF-8** default locale. It is only the non-root account **ubuntu** that gets the new locale. -``` -Hit:1 http://archive.ubuntu.com/ubuntu xenial InRelease -Get:2 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB] -Get:3 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB] -``` - -Here is **apt update** that is required before installing packages. -``` -The following packages will be upgraded: - libdrm2 libseccomp2 squashfs-tools unattended-upgrades -4 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. -Need to get 211 kB of archives. -``` - -Here is runs **package_upgrade: true** and installs any available packages. -``` -The following NEW packages will be installed: - binutils build-essential cpp cpp-5 dpkg-dev fakeroot g++ g++-5 gcc gcc-5 - libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl -``` - -This is from our instruction to install the **build-essential** meta-package. - -What about the **runcmd** instruction? -``` -root@mydev:~# **ls -l /tmp/** -total 1 --rw-r--r-- 1 root root 0 Jan 3 15:23 simos_was_here -root@mydev:~# -``` - -It worked as well! - -### Conclusion - -When we launch LXD containers, we often need some configuration to be enabled by default and avoid repeated actions. The way to solve this, is to create LXD profiles. Each profile captures those configurations. Finally, when we launch the new container, we specify which LXD profile to use. - - --------------------------------------------------------------------------------- - -via: https://blog.simos.info/how-to-preconfigure-lxd-containers-with-cloud-init/ - -作者：[Simos Xenitellis][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://blog.simos.info/author/simos/ -[1]:http://cloudinit.readthedocs.io/en/latest/index.html -[2]:https://github.com/lxc/lxd/blob/master/doc/cloud-init.md -[3]:http://cloudinit.readthedocs.io/en/latest/topics/modules.html#runcmd -[4]:https://paste.ubuntu.com/26313399/ diff --git a/sources/tech/20180104 4 Tools for Network Snooping on Linux.md b/sources/tech/20180104 4 Tools for Network Snooping on Linux.md new file mode 100644 index 0000000000..0ba60006ee --- /dev/null +++ b/sources/tech/20180104 4 Tools for Network Snooping on Linux.md @@ -0,0 +1,197 @@ +4 Tools for Network Snooping on Linux +====== +Computer networking data has to be exposed, because packets can't travel blindfolded, so join us as we use `whois`, `dig`, `nmcli`, and `nmap` to snoop networks. + +Do be polite and don't run `nmap` on any network but your own, because probing other people's networks can be interpreted as a hostile act. + +### Thin and Thick whois + +You may have noticed that our beloved old `whois` command doesn't seem to give the level of detail that it used to. Check out this example for Linux.com: +``` +$ whois linux.com +Domain Name: LINUX.COM +Registry Domain ID: 4245540_DOMAIN_COM-VRSN +Registrar WHOIS Server: whois.namecheap.com +Registrar URL: http://www.namecheap.com +Updated Date: 2018-01-10T12:26:50Z +Creation Date: 1994-06-02T04:00:00Z +Registry Expiry Date: 2018-06-01T04:00:00Z +Registrar: NameCheap Inc. +Registrar IANA ID: 1068 +Registrar Abuse Contact Email: abuse@namecheap.com +Registrar Abuse Contact Phone: +1.6613102107 +Domain Status: ok https://icann.org/epp#ok +Name Server: NS5.DNSMADEEASY.COM +Name Server: NS6.DNSMADEEASY.COM +Name Server: NS7.DNSMADEEASY.COM +DNSSEC: unsigned +[...] + +``` + +There is quite a bit more, mainly annoying legalese. But where is the contact information? It is sitting on whois.namecheap.com (see the third line of output above): +``` +$ whois -h whois.namecheap.com linux.com + +``` + +I won't print the output here, as it is very long, containing the Registrant, Admin, and Tech contact information. So what's the deal, Lucille? Some registries, such as .com and .net are "thin" registries, storing a limited subset of domain data. To get complete information use the `-h`, or `--host` option, to get the complete dump from the domain's `Registrar WHOIS Server`. + +Most of the other top-level domains are thick registries, such as .info. Try `whois blockchain.info` to see an example. + +Want to get rid of the obnoxious legalese? Use the `-H` option. + +### Digging DNS + +Use the `dig` command to compare the results from different name servers to check for stale entries. DNS records are cached all over the place, and different servers have different refresh intervals. This is the simplest usage: +``` +$ dig linux.com +<<>> DiG 9.10.3-P4-Ubuntu <<>> linux.com +;; global options: +cmd +;; Got answer: +;; ->>HEADER<<<- opcode: QUERY, status: NOERROR, id: 13694 +;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1 + +;; OPT PSEUDOSECTION: +; EDNS: version: 0, flags:; udp: 1440 +;; QUESTION SECTION: +;linux.com. IN A + +;; ANSWER SECTION: +linux.com. 10800 IN A 151.101.129.5 +linux.com. 10800 IN A 151.101.65.5 +linux.com. 10800 IN A 151.101.1.5 +linux.com. 10800 IN A 151.101.193.5 + +;; Query time: 92 msec +;; SERVER: 127.0.1.1#53(127.0.1.1) +;; WHEN: Tue Jan 16 15:17:04 PST 2018 +;; MSG SIZE rcvd: 102 + +``` + +Take notice of the SERVER: 127.0.1.1#53(127.0.1.1) line near the end of the output. This is your default caching resolver. When the address is localhost, that means there is a DNS server installed on your machine. In my case that is Dnsmasq, which is being used by Network Manager: +``` +$ ps ax|grep dnsmasq +2842 ? S 0:00 /usr/sbin/dnsmasq --no-resolv --keep-in-foreground +--no-hosts --bind-interfaces --pid-file=/var/run/NetworkManager/dnsmasq.pid +--listen-address=127.0.1.1 + +``` + +The `dig` default is to return A records, which define the domain name. IPv6 has AAAA records: +``` +$ $ dig linux.com AAAA +[...] +;; ANSWER SECTION: +linux.com. 60 IN AAAA 64:ff9b::9765:105 +linux.com. 60 IN AAAA 64:ff9b::9765:4105 +linux.com. 60 IN AAAA 64:ff9b::9765:8105 +linux.com. 60 IN AAAA 64:ff9b::9765:c105 +[...] + +``` + +Checkitout, Linux.com has IPv6 addresses. Very good! If your Internet service provider supports IPv6 then you can connect over IPv6. (Sadly, my overpriced mobile broadband does not.) + +Suppose you make some DNS changes to your domain, or you're seeing `dig` results that don't look right. Try querying with a public DNS service, like OpenNIC: +``` +$ dig @69.195.152.204 linux.com +[...] +;; Query time: 231 msec +;; SERVER: 69.195.152.204#53(69.195.152.204) + +``` + +`dig` confirms that you're getting your lookup from 69.195.152.204. You can query all kinds of servers and compare results. + +### Upstream Name Servers + +I want to know what my upstream name servers are. To find this, I first look in `/etc/resolv/conf`: +``` +$ cat /etc/resolv.conf +# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8) +# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN +nameserver 127.0.1.1 + +``` + +Thanks, but I already knew that. Your Linux distribution may be configured differently, and you'll see your upstream servers. Let's try `nmcli`, the Network Manager command-line tool: +``` +$ nmcli dev show | grep DNS +IP4.DNS[1]: 192.168.1.1 + +``` + +Now we're getting somewhere, as that is the address of my mobile hotspot, and I should have thought of that myself. I can log in to its weird little Web admin panel to see its upstream servers. A lot of consumer Internet gateways don't let you view or change these settings, so try an external service such as [What's my DNS server?][1] + +### List IPv4 Addresses on your Network + +Which IPv4 addresses are up and in use on your network? +``` +$ nmap -sn 192.168.1.0/24 +Starting Nmap 7.01 ( https://nmap.org ) at 2018-01-14 14:03 PST +Nmap scan report for Mobile.Hotspot (192.168.1.1) +Host is up (0.011s latency). +Nmap scan report for studio (192.168.1.2) +Host is up (0.000071s latency). +Nmap scan report for nellybly (192.168.1.3) +Host is up (0.015s latency) +Nmap done: 256 IP addresses (2 hosts up) scanned in 2.23 seconds + +``` + +Everyone wants to scan their network for open ports. This example looks for services and their versions: +``` +$ nmap -sV 192.168.1.1/24 + +Starting Nmap 7.01 ( https://nmap.org ) at 2018-01-14 16:46 PST +Nmap scan report for Mobile.Hotspot (192.168.1.1) +Host is up (0.0071s latency). +Not shown: 997 closed ports +PORT STATE SERVICE VERSION +22/tcp filtered ssh +53/tcp open domain dnsmasq 2.55 +80/tcp open http GoAhead WebServer 2.5.0 + +Nmap scan report for studio (192.168.1.102) +Host is up (0.000087s latency). +Not shown: 998 closed ports +PORT STATE SERVICE VERSION +22/tcp open ssh OpenSSH 7.2p2 Ubuntu 4ubuntu2.2 (Ubuntu Linux; protocol 2.0) +631/tcp open ipp CUPS 2.1 +Service Info: OS: Linux; CPE: cpe:/o:linux:linux_kernel + +Service detection performed. Please report any incorrect results at https://nmap.org/submit/ . +Nmap done: 256 IP addresses (2 hosts up) scanned in 11.65 seconds + +``` + +These are interesting results. Let's try the same run from a different Internet account, to see if any of these services are exposed to big bad Internet. You have a second network if you have a smartphone. There are probably apps you can download, or use your phone as a hotspot to your faithful Linux computer. Fetch the WAN IP address from the hotspot control panel and try again: +``` +$ nmap -sV 12.34.56.78 + +Starting Nmap 7.01 ( https://nmap.org ) at 2018-01-14 17:05 PST +Nmap scan report for 12.34.56.78 +Host is up (0.0061s latency). +All 1000 scanned ports on 12.34.56.78 are closed + +``` + +That's what I like to see. Consult the fine man pages for these commands to learn more fun snooping techniques. + +Learn more about Linux through the free ["Introduction to Linux" ][2]course from The Linux Foundation and edX. + +-------------------------------------------------------------------------------- + +via: https://www.linux.com/learn/intro-to-linux/2018/1/4-tools-network-snooping-linux + +作者：[Carla Schroder][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.linux.com/users/cschroder +[1]:http://www.whatsmydnsserver.com/ +[2]:https://training.linuxfoundation.org/linux-courses/system-administration-training/introduction-to-linux diff --git a/sources/tech/20180104 How does gdb call functions.md b/sources/tech/20180104 How does gdb call functions.md index a62b30ea31..c88fae999e 100644 --- a/sources/tech/20180104 How does gdb call functions.md +++ b/sources/tech/20180104 How does gdb call functions.md @@ -1,3 +1,5 @@ +translating by ucasFL + How does gdb call functions? ============================================================ diff --git a/sources/tech/20180106 Meltdown and Spectre Linux Kernel Status.md b/sources/tech/20180106 Meltdown and Spectre Linux Kernel Status.md deleted file mode 100644 index d98fddad78..0000000000 --- a/sources/tech/20180106 Meltdown and Spectre Linux Kernel Status.md +++ /dev/null @@ -1,103 +0,0 @@ -translated by hopefully2333 - -Meltdown and Spectre Linux Kernel Status -============================================================ - - -By now, everyone knows that something “big” just got announced regarding computer security. Heck, when the [Daily Mail does a report on it][1] , you know something is bad… - -Anyway, I’m not going to go into the details about the problems being reported, other than to point you at the wonderfully written [Project Zero paper on the issues involved here][2]. They should just give out the 2018 [Pwnie][3] award right now, it’s that amazingly good. - -If you do want technical details for how we are resolving those issues in the kernel, see the always awesome [lwn.net writeup for the details][4]. - -Also, here’s a good summary of [lots of other postings][5] that includes announcements from various vendors. - -As for how this was all handled by the companies involved, well this could be described as a textbook example of how  _NOT_  to interact with the Linux kernel community properly. The people and companies involved know what happened, and I’m sure it will all come out eventually, but right now we need to focus on fixing the issues involved, and not pointing blame, no matter how much we want to. - -### What you can do right now - -If your Linux systems are running a normal Linux distribution, go update your kernel. They should all have the updates in them already. And then keep updating them over the next few weeks, we are still working out lots of corner case bugs given that the testing involved here is complex given the huge variety of systems and workloads this affects. If your distro does not have kernel updates, then I strongly suggest changing distros right now. - -However there are lots of systems out there that are not running “normal” Linux distributions for various reasons (rumor has it that it is way more than the “traditional” corporate distros). They rely on the LTS kernel updates, or the normal stable kernel updates, or they are in-house franken-kernels. For those people here’s the status of what is going on regarding all of this mess in the upstream kernels you can use. - -### Meltdown – x86 - -Right now, Linus’s kernel tree contains all of the fixes we currently know about to handle the Meltdown vulnerability for the x86 architecture. Go enable the CONFIG_PAGE_TABLE_ISOLATION kernel build option, and rebuild and reboot and all should be fine. - -However, Linus’s tree is currently at 4.15-rc6 + some outstanding patches. 4.15-rc7 should be out tomorrow, with those outstanding patches to resolve some issues, but most people do not run a -rc kernel in a “normal” environment. - -Because of this, the x86 kernel developers have done a wonderful job in their development of the page table isolation code, so much so that the backport to the latest stable kernel, 4.14, has been almost trivial for me to do. This means that the latest 4.14 release (4.14.12 at this moment in time), is what you should be running. 4.14.13 will be out in a few more days, with some additional fixes in it that are needed for some systems that have boot-time problems with 4.14.12 (it’s an obvious problem, if it does not boot, just add the patches now queued up.) - -I would personally like to thank Andy Lutomirski, Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, Peter Zijlstra, Josh Poimboeuf, Juergen Gross, and Linus Torvalds for all of the work they have done in getting these fixes developed and merged upstream in a form that was so easy for me to consume to allow the stable releases to work properly. Without that effort, I don’t even want to think about what would have happened. - -For the older long term stable (LTS) kernels, I have leaned heavily on the wonderful work of Hugh Dickins, Dave Hansen, Jiri Kosina and Borislav Petkov to bring the same functionality to the 4.4 and 4.9 stable kernel trees. I had also had immense help from Guenter Roeck, Kees Cook, Jamie Iles, and many others in tracking down nasty bugs and missing patches. I want to also call out David Woodhouse, Eduardo Valentin, Laura Abbott, and Rik van Riel for their help with the backporting and integration as well, their help was essential in numerous tricky places. - -These LTS kernels also have the CONFIG_PAGE_TABLE_ISOLATION build option that should be enabled to get complete protection. - -As this backport is very different from the mainline version that is in 4.14 and 4.15, there are different bugs happening, right now we know of some VDSO issues that are getting worked on, and some odd virtual machine setups are reporting strange errors, but those are the minority at the moment, and should not stop you from upgrading at all right now. If you do run into problems with these releases, please let us know on the stable kernel mailing list. - -If you rely on any other kernel tree other than 4.4, 4.9, or 4.14 right now, and you do not have a distribution supporting you, you are out of luck. The lack of patches to resolve the Meltdown problem is so minor compared to the hundreds of other known exploits and bugs that your kernel version currently contains. You need to worry about that more than anything else at this moment, and get your systems up to date first. - -Also, go yell at the people who forced you to run an obsoleted and insecure kernel version, they are the ones that need to learn that doing so is a totally reckless act. - -### Meltdown – ARM64 - -Right now the ARM64 set of patches for the Meltdown issue are not merged into Linus’s tree. They are [staged and ready to be merged][6] into 4.16-rc1 once 4.15 is released in a few weeks. Because these patches are not in a released kernel from Linus yet, I can not backport them into the stable kernel releases (hey, we have [rules][7] for a reason…) - -Due to them not being in a released kernel, if you rely on ARM64 for your systems (i.e. Android), I point you at the [Android Common Kernel tree][8] All of the ARM64 fixes have been merged into the [3.18,][9] [4.4,][10] and [4.9 branches][11] as of this point in time. - -I would strongly recommend just tracking those branches as more fixes get added over time due to testing and things catch up with what gets merged into the upstream kernel releases over time, especially as I do not know when these patches will land in the stable and LTS kernel releases at this point in time. - -For the 4.4 and 4.9 LTS kernels, odds are these patches will never get merged into them, due to the large number of prerequisite patches required. All of those prerequisite patches have been long merged and tested in the android-common kernels, so I think it is a better idea to just rely on those kernel branches instead of the LTS release for ARM systems at this point in time. - -Also note, I merge all of the LTS kernel updates into those branches usually within a day or so of being released, so you should be following those branches no matter what, to ensure your ARM systems are up to date and secure. - -### Spectre - -Now things get “interesting”… - -Again, if you are running a distro kernel, you  _might_  be covered as some of the distros have merged various patches into them that they claim mitigate most of the problems here. I suggest updating and testing for yourself to see if you are worried about this attack vector - -For upstream, well, the status is there is no fixes merged into any upstream tree for these types of issues yet. There are numerous patches floating around on the different mailing lists that are proposing solutions for how to resolve them, but they are under heavy development, some of the patch series do not even build or apply to any known trees, the series conflict with each other, and it’s a general mess. - -This is due to the fact that the Spectre issues were the last to be addressed by the kernel developers. All of us were working on the Meltdown issue, and we had no real information on exactly what the Spectre problem was at all, and what patches were floating around were in even worse shape than what have been publicly posted. - -Because of all of this, it is going to take us in the kernel community a few weeks to resolve these issues and get them merged upstream. The fixes are coming in to various subsystems all over the kernel, and will be collected and released in the stable kernel updates as they are merged, so again, you are best off just staying up to date with either your distribution’s kernel releases, or the LTS and stable kernel releases. - -It’s not the best news, I know, but it’s reality. If it’s any consolation, it does not seem that any other operating system has full solutions for these issues either, the whole industry is in the same boat right now, and we just need to wait and let the developers solve the problem as quickly as they can. - -The proposed solutions are not trivial, but some of them are amazingly good. The [Retpoline][12] post from Paul Turner is an example of some of the new concepts being created to help resolve these issues. This is going to be an area of lots of research over the next years to come up with ways to mitigate the potential problems involved in hardware that wants to try to predict the future before it happens. - -### Other arches - -Right now, I have not seen patches for any other architectures than x86 and arm64\. There are rumors of patches floating around in some of the enterprise distributions for some of the other processor types, and hopefully they will surface in the weeks to come to get merged properly upstream. I have no idea when that will happen, if you are dependant on a specific architecture, I suggest asking on the arch-specific mailing list about this to get a straight answer. - -### Conclusion - -Again, update your kernels, don’t delay, and don’t stop. The updates to resolve these problems will be continuing to come for a long period of time. Also, there are still lots of other bugs and security issues being resolved in the stable and LTS kernel releases that are totally independent of these types of issues, so keeping up to date is always a good idea. - -Right now, there are a lot of very overworked, grumpy, sleepless, and just generally pissed off kernel developers working as hard as they can to resolve these issues that they themselves did not cause at all. Please be considerate of their situation right now. They need all the love and support and free supply of their favorite beverage that we can provide them to ensure that we all end up with fixed systems as soon as possible. - --------------------------------------------------------------------------------- - -via: http://kroah.com/log/blog/2018/01/06/meltdown-status/ - -作者：[Greg Kroah-Hartman ][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:http://kroah.com -[1]:http://www.dailymail.co.uk/sciencetech/article-5238789/Intel-says-security-updates-fix-Meltdown-Spectre.html -[2]:https://googleprojectzero.blogspot.fr/2018/01/reading-privileged-memory-with-side.html -[3]:https://pwnies.com/ -[4]:https://lwn.net/Articles/743265/ -[5]:https://lwn.net/Articles/742999/ -[6]:https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/log/?h=kpti -[7]:https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html -[8]:https://android.googlesource.com/kernel/common/ -[9]:https://android.googlesource.com/kernel/common/+/android-3.18 -[10]:https://android.googlesource.com/kernel/common/+/android-4.4 -[11]:https://android.googlesource.com/kernel/common/+/android-4.9 -[12]:https://support.google.com/faqs/answer/7625886 diff --git a/sources/talk/20180108 You GNOME it- Windows and Apple devs get a compelling reason to turn to Linux.md b/sources/tech/20180108 You GNOME it- Windows and Apple devs get a compelling reason to turn to Linux.md similarity index 100% rename from sources/talk/20180108 You GNOME it- Windows and Apple devs get a compelling reason to turn to Linux.md rename to sources/tech/20180108 You GNOME it- Windows and Apple devs get a compelling reason to turn to Linux.md diff --git a/sources/tech/20180110 Best Linux Screenshot and Screencasting Tools.md b/sources/tech/20180110 Best Linux Screenshot and Screencasting Tools.md deleted file mode 100644 index fbd10d2194..0000000000 --- a/sources/tech/20180110 Best Linux Screenshot and Screencasting Tools.md +++ /dev/null @@ -1,147 +0,0 @@ -Best Linux Screenshot and Screencasting Tools -====== -![](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/best-linux-screenshot-and-screencasting-tools_orig.jpg) - -There comes a time you want to capture an error on your screen and send it to the developers or want help from _Stack Overflow,_ you need the right tools to take that screenshot and save it or send it. There are tools in the form of programs and others as shell extensions for GNOME. Not to worry, here are the best Linux Screenshot taking tools that you can use to take those screenshots or make a screencast. - -## Best Linux Screenshot Or Screencasting Tools - -### 1\. Shutter - - [![shutter linux screenshot taking tools](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/shutter-linux-screenshot-taking-tools_orig.jpg)][2] - -[Shutter][3] is one of the best Linux screenshot taking tools. It has the advantage of taking different screenshots depending on what you want to take on your screen. After you take the screenshot, it allows you to see the screenshot before saving it after you take the screenshot. It also includes an extension menu that shows up on your top panel for GNOME. That makes accessing the app much easier and much convenient for anyone to use. - -​You can take screenshots of a selection, a window, desktop, window under cursor, section, menu, tooltip or web. Shutter allows you to upload the screenshots directly to the cloud using the preferred cloud services provider. This Linux tool also allows you to edit your screenshots before you save them. It also comes with plugins that you can add or remove. - -To install it, you will have to type the following in the terminal: - -``` -sudo add-apt-repository -y ppa:shutter/ppa -sudo apt-get update && sudo apt-get install shutter -``` - -### 2. Vokoscreen - - [![vokoscreen screencasting tool for linux](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/vokoscreen-screencasting-tool-for-linux_orig.jpg)][4] - - -[Vokoscreen][5] is an app that allows you to record your screen as you show around and narrate what you are doing on the screen. It is easy to use, has a simple interface and includes a top panel menu for easy access when you are recording your screen. - -​ - -You can choose to record the whole screen, a window or just a selection of an area. Customizing the recording is easy to get the type of screen recording you want to achieve. Vokoscreen even allows you to create a gif as a screen recording. You can also record yourself using the webcam in case you were narrating as tutorials so that you can engage the learners. Once you are done, you can playback the recording right from the application so that you don’t have to keep navigating to find the recording. - - [![vokoscreen preferences](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/vokoscreen-preferences_orig.jpg)][6] - -You can install Vocoscreen from your distro repository. Or download the package from [pkgs.org][7] , select the Linux distro you are using. - -``` -sudo dpkg -i vokoscreen_2.5.0-1_amd64.deb -``` - -### 3. OBS - - [![obs linux screencasting tool](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/obs-linux-screencasting-tool_orig.jpg)][8] - -[OBS][9] can be used to record your screen as well as record streams from the internet. It allows you to see whatever you are recording as you stream or as you narrate your screen recording. It allows you to choose the quality of your recording according to your preferences. It also allows you to choose the type of file you want your recording to save to. In addition to the feature of recording, you can switch to Studio mode allowing you to edit your recording to make a complete video without having to use any other external editing software. To install OBS in your Linux distribution, you must have FFmpeg installed on your machine. To install FFmpeg type the following in the terminal for ubuntu 14.04 and earlier: - -``` -sudo add-apt-repository ppa:kirillshkrogalev/ffmpeg-next - -sudo apt-get update && sudo apt-get install ffmpeg -``` - -​For ubuntu 15.04 and later you can just type the following in the terminal to install FFmpeg: - -``` -sudo apt-get install ffmpeg -``` - -​If you have already installed FFmpeg, type the following in the terminal to install OBS: - -``` -sudo add-apt-repository ppa:obsproject/obs-studio - -sudo apt-get update - -sudo apt-get install obs-studio -``` - -### 4. Green Recorder - - [![green recording linux tool](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/green-recording-linux-tool_orig.jpg)][10] - -[Green recorder][11] is a simple interface based program that allows you to record the screen. You can choose what to record including video or just audio and allow you to show the mouse pointer and even follow it as you record your screen. You can record a window or just a selected area on your screen so that only what you want to record shows up in your recording. You can customize the number of frames to record in your final video. In case you want to start recording after a delay, you have the option to configure the delay you wish to set. You have the option to run a command after the recording is done that will run on your machine immediately after you stop recording. - -​ - -To install green recorder, type the following in the terminal: - -``` -sudo add-apt-repository ppa:fossproject/ppa - -sudo apt update && sudo apt install green-recorder -``` - -### 5. Kazam - - [![kazam screencasting tool for linux](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/kazam-screencasting-tool-for-linux_orig.jpg)][12] - -[Kazam][13] Linux screenshot tool is very popular amongst Linux users. It is an intuitive simple to use app that allows you to take a screencast or a screenshot allowing you to customise the delay before taking a screencast or screenshot. It allows you to select the area, window or fullscreen you want to capture. Kazam’s interface is well laid out and not as complicated as other apps. Its features will leave you happy about taking your screenshots. Kazam also includes a system tray icon and menu that allows you to take the screenshot without going to the application itself. - -​​ - -To install Kazam, type the following in the terminal: - -``` -sudo apt-get install kazam -``` - -​If the PPA is not found, you can install it manually using the following commands: - -``` -sudo add-apt-repository ppa:kazam-team/stable-series - -sudo apt-get update && sudo apt-get install kazam -``` - -### 6. Screenshot tool GNOME extension - - [![gnome screenshot extension](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/gnome-screenshot-extension-compressed_orig.jpg)][1] - -There is a GNOME extension just named screenshot tool that always shows up on the system panel until you disable it. It is convenient since it just sits on the system panel until you will trigger it to take a screenshot. The main advantage of this tool is that it is the quickest to access since it is always in your system panel unless you deactivate it in the tweak utility tool. The tool also has a preferences window allowing you to tweak it to your preferences. To install it on your GNOME desktop, head to extensions.gnome.org and search for “_Screenshot Tool”._ - -You must have the gnome extensions chrome extension installed as well as GNOME tweaks tool installed to use the tool. - - [![gnome screenshot extension preferences](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/gnome-screenshot-extension-preferences_orig.jpg)][14] - -The **Linux screenshot tools** are quite helpful especially when you don’t know what to do when you come across a problem and want to share the error with [the Linux community][15] or the developers of a program that you are using. Learning developers or programmers or anyone else need it will find these tools useful to share your screenshots. Youtubers and tutorial makers will find the screencasting tools even more useful when they use them to record their tutorials and post them.​ - - --------------------------------------------------------------------------------- - -via: http://www.linuxandubuntu.com/home/best-linux-screenshot-screencasting-tools - -作者：[linuxandubuntu][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:http://www.linuxandubuntu.com -[1]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/gnome-screenshot-extension-compressed_orig.jpg -[2]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/shutter-linux-screenshot-taking-tools_orig.jpg -[3]:http://shutter-project.org/ -[4]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/vokoscreen-screencasting-tool-for-linux_orig.jpg -[5]:https://github.com/vkohaupt/vokoscreen -[6]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/vokoscreen-preferences_orig.jpg -[7]:https://pkgs.org/download/vokoscreen -[8]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/obs-linux-screencasting-tool_orig.jpg -[9]:https://obsproject.com/ -[10]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/green-recording-linux-tool_orig.jpg -[11]:https://github.com/foss-project/green-recorder -[12]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/kazam-screencasting-tool-for-linux_orig.jpg -[13]:https://launchpad.net/kazam -[14]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/gnome-screenshot-extension-preferences_orig.jpg -[15]:http://www.linuxandubuntu.com/home/top-10-communities-to-help-you-learn-linux diff --git a/sources/tech/20180111 Multimedia Apps for the Linux Console.md b/sources/tech/20180111 Multimedia Apps for the Linux Console.md deleted file mode 100644 index 6cdd3ef857..0000000000 --- a/sources/tech/20180111 Multimedia Apps for the Linux Console.md +++ /dev/null @@ -1,112 +0,0 @@ -Translating by Yinr - -Multimedia Apps for the Linux Console -====== - -![](https://www.linux.com/sites/lcom/files/styles/rendered_file/public/multimedia.jpg?itok=v-XrnKRB) -The Linux console supports multimedia, so you can enjoy music, movies, photos, and even read PDF files. - -When last we met, we learned that the Linux console supports multimedia. Yes, really! You can enjoy music, movies, photos, and even read PDF files without being in an X session with MPlayer, fbi, and fbgs. And, as a bonus, you can enjoy a Matrix-style screensaver for the console, CMatrix. - -You will probably have make some tweaks to your system to make this work. The examples used here are for Ubuntu Linux 16.04. - -### MPlayer - -You're probably familiar with the amazing and versatile MPlayer, which supports almost every video and audio format, and runs on nearly everything, including Linux, Android, Windows, Mac, Kindle, OS/2, and AmigaOS. Using MPLayer in your console will probably require some tweaking, depending on your Linux distribution. To start, try playing a video: -``` -$ mplayer [video name] - -``` - -If it works, then hurrah, and you can invest your time in learning useful MPlayer options, such as controlling the size of the video screen. However, some Linux distributions are managing the framebuffer differently than in the olden days, and you may have to adjust some settings to make it work. This is how to make it work on recent Ubuntu releases. - -First, add yourself to the video group. - -Second, verify that `/etc/modprobe.d/blacklist-framebuffer.conf` has this line: `#blacklist vesafb`. It should already be commented out, and if it isn't then comment it. All the other module lines should be un-commented, which prevents them from loading. Side note: if you want to dig more deeply into managing your framebuffer, the module for your video card may give better performance. - -Add these two modules to the end of `/etc/initramfs-tools/modules`, `vesafb` and `fbcon`, then rebuild the initramfs image: -``` -$ sudo nano /etc/initramfs-tools/modules - # List of modules that you want to include in your initramfs. - # They will be loaded at boot time in the order below. - fbcon - vesafb - -$ sudo update-initramfs -u - -``` - -[fbcon][1] is the Linux framebuffer console. It runs on top of the framebuffer and adds graphical features. It requires a framebuffer device, which is supplied by the `vesafb` module. - -Now you must edit your GRUB2 configuration. In `/etc/default/grub` you should see a line like this: -``` -GRUB_CMDLINE_LINUX_DEFAULT="quiet splash" - -``` - -It may have some other options, but it should be there. Add `vga=789`: -``` -GRUB_CMDLINE_LINUX_DEFAULT="quiet splash vga=789" - -``` - -Reboot and enter your console (Ctrl+Alt+F1), and try playing a video. This command selects the `fbdev2` video device; I haven't learned yet how to know which one to use, but I had to use it to play the video. The default screen size is 320x240, so I scaled it to 960: -``` -$ mplayer -vo fbdev2 -vf scale -zoom -xy 960 AlienSong_mp4.mov -``` - -And behold Figure 1. It's grainy because I have a low-fi copy of this video, not because MPlayer is making it grainy. - -MPLayer plays CDs, DVDs, network streams, and has a giant batch of playback options, which I shall leave as your homework to explore. - -### fbi Image Viewer - -`fbi`, the framebuffer image viewer, comes in the [fbida][2] package on most Linuxes. It has native support for the common image file formats, and uses `convert` (from Image Magick), if it is installed, for other formats. Its simplest use is to view a single image file: -``` -$ fbi filename - -``` - -Use the arrow keys to scroll a large image, + and - to zoom, and r and l to rotate 90 degress right and left. Press the Escape key to close the image. You can play a slideshow by giving `fbi` a list of files: -``` -$ fbi --list file-list.txt - -``` - -`fbi` supports autozoom. With `-a` `fbi` controls the zoom factor. `--autoup` and `--autodown` tell `fbi` to only zoom up or down. Control the blend time between images with `--blend [time]`, in milliseconds. Press the k and j keys to jump behind and ahead in your file list. - -`fbi` has commands for creating file lists from images you have viewed, and for exporting your commands to a file, and a host of other cool options. Check out `man fbi` for complete options. - -### CMatrix Console Screensaver - -The Matrix screensaver is still my favorite (Figure 2), second only to the bouncing cow. [CMatrix][3] runs on the console. Simply type `cmatrix` to start it, and Ctrl+C stops it. Run `cmatrix -s` to launch it in screensaver mode, which exits on any keypress. `-C` changes the color. Your choices are green, red, blue, yellow, white, magenta, cyan, and black. - -CMatrix supports asynchronous key presses, which means you can change options while it's running. - -`-B` is all bold text, and `-B` is partially bold. - -### fbgs PDF Viewer - -It seems that the addiction to PDF documents is pandemic and incurable, though PDFs are better than they used to be, with live hyperlinks, copy-paste, and good text search. The `fbgs` console PDF viewer is part of the `fbida` package. Options include page size, resolution, page selections, and most `fbi` options, with the exceptions listed in `man fbgs`. The main option I use is page size; you get `-l`, `xl`, and `xxl` to choose from: -``` -$ fbgs -xl annoyingpdf.pdf - -``` - -Learn more about Linux through the free ["Introduction to Linux" ][4]course from The Linux Foundation and edX. - --------------------------------------------------------------------------------- - -via: https://www.linux.com/learn/intro-to-linux/2018/1/multimedia-apps-linux-console - -作者：[Carla Schroder][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://www.linux.com/users/cschroder -[1]:https://www.mjmwired.net/kernel/Documentation/fb/fbcon.txt -[2]:https://www.kraxel.org/blog/linux/fbida/ -[3]:http://www.asty.org/cmatrix/ -[4]:https://training.linuxfoundation.org/linux-courses/system-administration-training/introduction-to-linux diff --git a/sources/tech/20180111 What is the deal with GraphQL.md b/sources/tech/20180111 What is the deal with GraphQL.md new file mode 100644 index 0000000000..98edcf18e0 --- /dev/null +++ b/sources/tech/20180111 What is the deal with GraphQL.md @@ -0,0 +1,41 @@ +What is the deal with GraphQL? +====== + +![](https://ryanmccue.ca/content/images/2018/01/Copy-of-Copy-of-Copy-of-Copy-of-Copy-of-Copy-of-Electric-Love.png) + +There has been lots of talks lately about this thing called [GraphQL][1]. It is a relatively new technology coming out of Facebook and is starting to be widely adopted by large companies like [Github][2], Facebook, Twitter, Yelp, and many others. Basically, GraphQL is an alternative to REST, it replaces many dumb endpoints, `/user/1`, `/user/1/comments` with `/graphql` and you use the post body or query string to request the data you need, like, `/graphql?query={user(id:1){id,username,comments{text}}}`. You pick the pieces of data you need and can nest down to relations to avoid multiple calls. This is a different way of thinking about a backend, but in some situations, it makes practical sense. + +### My Experience with GraphQL + +Originally when I heard about it I was very skeptical, after dabbling in [Apollo Server][3] I was not convinced. Why would you use some silly new technology when you can simply build REST endpoints! But after digging deeper and learning more about its use cases, I came around. I still think REST has a place and will be important for the foreseeable future, but with how bad many APIs and their documentation are, this can be a breath of fresh air... + +### Why Use GraphQL Over REST? + +Although I have used GraphQL, and think it is a compelling and exciting technology, I believe it does not replace REST. That being said there are compelling reasons to pick GraphQL over REST in some situations. When you are building mobile apps or web apps which are made with high mobile traffic in mind GraphQL really shines. The reason for this is mobile data. REST uses many calls and often returns unused data whereas, with GraphQL, you can define precisely what you want to be returned for minimal data usage. + +You can get do all the above with REST by making multiple endpoints available, but that also adds complexity to the project. It also means there will be back and forth between the front and backend teams. + +### What Should You Use? + +GraphQL is a new technology which is now mainstream. But many developers are not aware of it or choose not to learn it because they think it's a fad. I feel like for most projects you can get away using either REST or GraphQL. Developing using GraphQL has great benefits like enforcing documentation, which helps teams work better together, and provides clear expectations for each query. This will likely speed up development after the initial hurdle of wrapping your head around GraphQL. + +Although I have been comparing GraphQL and REST, I think in most cases a mixture of the two will produce the best results. Combine the strengths of both instead of seeing it strightly as just using GraphQL or just using REST. + +### Final Thoughts + +Both technologies are here to stay. And done right both technologies can make fast and efficient backends. GraphQL has an edge up because it allows the client to query only the data they need by default, but that is at a potential sacrifice of endpoint speed. Ultimately, if I were starting a new project, I would go with a mix of both GraphQL and REST. + +-------------------------------------------------------------------------------- + +via: https://ryanmccue.ca/what-is-the-deal-with-graphql/ + +作者：[Ryan McCue][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://ryanmccue.ca/author/ryan/ +[1]:http://graphql.org/ +[2]:https://developer.github.com/v4/ +[3]:https://github.com/apollographql/apollo-server diff --git a/sources/tech/20180112 Top 5 Firefox extensions to install now.md b/sources/tech/20180112 Top 5 Firefox extensions to install now.md deleted file mode 100644 index 3717b7c96d..0000000000 --- a/sources/tech/20180112 Top 5 Firefox extensions to install now.md +++ /dev/null @@ -1,85 +0,0 @@ -translating by ypingcn - -Top 5 Firefox extensions to install now -====== - -The right extensions can greatly enhance your browser's capabilities, but it's important to choose carefully. Here are five that are worth a look. - -![](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/firefox_blue_lead.jpg?itok=gYaubJUv) - -The web browser has become a critical component of the computing experience for many users. Modern browsers have evolved into powerful and extensible platforms. As part of this, _extensions_ can add or modify their functionality. Extensions for Firefox are built using the WebExtensions API, a cross-browser development system. - -Which extensions should you install? Generally, that decision comes down to how you use your browser, your views on privacy, how much you trust extension developers, and other personal preferences. - -First, I'd like to point out that browser extensions often require the ability to read and/or change everything on the web pages you visit. You should consider the ramifications of this _very_ carefully. If an extension has modify access to all the web pages you visit, it could act as a key logger, intercept credit card information, track you online, insert advertisements, and perform a variety of other nefarious activities. - -That doesn't mean every extension will surreptitiously do these things, but you should carefully consider the installation source, the permissions involved, your risk profile, and other factors before you install any extension. Keep in mind you can use profiles to manage how an extension impacts your attack surface--for example, using a dedicated profile with no extensions to perform tasks such as online banking. - -With that in mind, here are five Firefox extensions that you may want to consider. - -### uBlock Origin - -![ublock origin ad blocker screenshot][2] - - -Ublock Origin blocks ads and malware while enabling users to define their own content filters. - -[uBlock Origin][3] is a fast, low-memory, wide-spectrum blocker that not only blocks ads but also lets you enforce your own content filtering. The default behavior of uBlock Origin is to block ads, trackers, and malware sites using multiple predefined filter lists. From there it allows you to arbitrarily add lists and rules, or even lock down to a default-deny mode. In addition to being powerful, this extension has proven to be efficient and performant. - -### Privacy Badger - -![privacy badger ad blocker][5] - - -Privacy Badger uses algorithms to seamlessly block ads and trackers that violate the principles of user consent. - -As its name indicates, [Privacy Badger][6] is a privacy-focused extension that blocks ads and third-party trackers. From the EFF: "Privacy Badger was born out of our desire to be able to recommend a single extension that would automatically analyze and block any tracker or ad that violated the principle of user consent; which could function well without any settings, knowledge, or configuration by the user; which is produced by an organization that is unambiguously working for its users rather than for advertisers; and which uses algorithmic methods to decide what is and isn't tracking." - -Why is Privacy Badger on this list when it may seem so similar to uBlock Origin? One reason is that it fundamentally works differently than uBlock Origin. Another is that a practice of defense in depth is a sound policy to follow. - -### LastPass - -![lastpass password manager screenshot][8] - - -LastPass is a user-friendly password manager plugin that supports two-factor authorization. - -This is likely a controversial addition for many. Whether you should use a password manager at all--and if you do, whether you should choose one that has a browser plugin--is a hotly debated topic, and the answer very much depends on your personal risk profile. I'd assert that most casual computer users should use one, because it's much better than the most common alternative: using the same weak password everywhere. - -[LastPass][9] is user-friendly, supports two-factor authentication, and is reasonably secure. The company has had a few security incidents in the past, but it responded well and is well-funded moving forward. Keep in mind that using a password manager isn't an all-or-nothing proposition. Many users choose to use it for the majority of their passwords, while keeping a few complicated, well-constructed passwords for important sites such as banking and multi-factor authentication in their head. - -### Xmarks Sync - -[Xmarks Sync][10] is a convenient extension that will sync your bookmarks, open tabs, profiles, and browser history across instances. If you have multiple machines, want to sync across desktop and mobile, or use multiple different browsers on the same machine, take a look at Xmarks Sync. (Note that this extension was recently acquired by LastPass.) - -### Awesome Screenshot Plus - -[Awesome Screenshot Plus][11] allows you to easily capture all or part of any web page, as well as add annotations and comments, blur sensitive information, and more. You can also share images using an optional online service. I've found this tool great for capturing parts of sites for debugging issues, discussing design, and sharing information. It's one of those tools you'll find yourself using more than you might have expected. - -I've found all five of these extensions useful, and I recommend them to others. That said, there are many browser extensions out there. I'm curious about which ones other Opensource.com community members currently use and recommend. Let me know in the comments. - -![Awesome Screenshot Plus screenshot][13] - - -Awesome Screenshot Plus allows you to easily capture all or part of any web page. - --------------------------------------------------------------------------------- - -via: https://opensource.com/article/18/1/top-5-firefox-extensions - -作者：[Jeremy Garcia][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://opensource.com/users/jeremy-garcia -[2]:https://opensource.com/sites/default/files/ublock.png (ublock origin ad blocker screenshot) -[3]:https://addons.mozilla.org/en-US/firefox/addon/ublock-origin/ -[5]:https://opensource.com/sites/default/files/images/life-uploads/privacy_badger_1.0.1.png (privacy badger ad blocker screenshot) -[6]:https://www.eff.org/privacybadger -[8]:https://opensource.com/sites/default/files/images/life-uploads/lastpass4.jpg (lastpass password manager screenshot) -[9]:https://addons.mozilla.org/en-US/firefox/addon/lastpass-password-manager/ -[10]:https://addons.mozilla.org/en-US/firefox/addon/xmarks-sync/ -[11]:https://addons.mozilla.org/en-US/firefox/addon/screenshot-capture-annotate/ -[13]:https://opensource.com/sites/default/files/screenshot_from_2018-01-04_17-11-32.png (Awesome Screenshot Plus screenshot) diff --git a/sources/tech/20180115 How To Boot Into Linux Command Line.md b/sources/tech/20180115 How To Boot Into Linux Command Line.md deleted file mode 100644 index 7a63f47f90..0000000000 --- a/sources/tech/20180115 How To Boot Into Linux Command Line.md +++ /dev/null @@ -1,61 +0,0 @@ -How To Boot Into Linux Command Line -====== -![](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/how-to-boot-into-linux-command-line_orig.jpg) - -There may be times where you need or want to boot up a [Linux][1] system without using a GUI, that is with no X, but rather opt for the command line. Whatever the reason, fortunately, booting straight into the Linux **command-line** is very simple. It requires a simple change to the boot parameter after the other kernel options. This change specifies the runlevel to boot the system into. - -### ​Why Do This? - -If your system does not run Xorg because the configuration is invalid, or if the display manager is broken, or whatever may prevent the GUI from starting properly, booting into the command-line will allow you to troubleshoot by logging into a terminal (assuming you know what you’re doing to start with) and do whatever you need to do. Booting into the command-line is also a great way to become more familiar with the terminal, otherwise, you can do it just for fun. - -### ​Accessing GRUB Menu - -On startup, you will need access to the GRUB boot menu. You may need to hold the SHIFT key down before the system boots if the menu isn’t set to display every time the computer is started. In the menu, the [Linux distribution][2] entry must be selected. Once highlighted, press ‘e’ to edit the boot parameters. - - [![zorin os grub menu](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/gnu-grub_orig.png)][3] - - Older GRUB versions follow a similar mechanism. The boot manager should provide instructions on how to edit the boot parameters. - -### ​​Specify the Runlevel - -​An editor will appear and you will see the options that GRUB parses to the kernel. Navigate to the line that starts with ‘linux’ (older GRUB versions may be ‘kernel’; select that and follow the instructions). This specifies parameters to parse into the kernel. At the end of that line (may appear to span multiple lines, depending on resolution), you simply specify the runlevel to boot into, which is 3 (multi-user mode, text-only). - - [![customize grub menu](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/runlevel_orig.png)][4] - -Pressing Ctrl-X or F10 will boot the system using those parameters. Boot-up will continue as normal. The only thing that has changed is the runlevel to boot into. - -​ - -This is what was started up: - - [![boot linux in command line](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/runlevel_1_orig.png)][5] - -### Runlevels - -You can specify different runlevels to boot into with runlevel 5 being the default one. 1 boots into “single-user” mode, which boots into a root shell. 3 provides a multi-user, command-line only system. - -### Switch From Command-Line - -At some point, you may want to run the display manager again to use a GUI, and the quickest way to do that is running this: -``` -$ sudo init 5 -``` - -And it is as simple as that. Personally, I find the command-line much more exciting and hands-on than using GUI tools; however, that’s just my preference. - --------------------------------------------------------------------------------- - -via: http://www.linuxandubuntu.com/home/how-to-boot-into-linux-command-line - -作者：[LinuxAndUbuntu][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:http://www.linuxandubuntu.com -[1]:http://www.linuxandubuntu.com/home/category/linux -[2]:http://www.linuxandubuntu.com/home/category/distros -[3]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/gnu-grub_orig.png -[4]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/runlevel_orig.png -[5]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/runlevel_1_orig.png diff --git a/sources/tech/20180116 SPARTA - Network Penetration Testing GUI Toolkit.md b/sources/tech/20180116 SPARTA - Network Penetration Testing GUI Toolkit.md new file mode 100644 index 0000000000..06427c101d --- /dev/null +++ b/sources/tech/20180116 SPARTA - Network Penetration Testing GUI Toolkit.md @@ -0,0 +1,107 @@ +SPARTA – Network Penetration Testing GUI Toolkit +====== + +![](https://i0.wp.com/gbhackers.com/wp-content/uploads/2018/01/GjWDZ1516079830.png?resize=696%2C379&ssl=1) + +SPARTA is GUI application developed with python and inbuild Network Penetration Testing Kali Linux tool. It simplifies scanning and enumeration phase with faster results. + +Best thing of SPARTA GUI Toolkit it scans detects the service running on the target port. + +Also, it provides Bruteforce attack for scanned open ports and services as a part of enumeration phase. + + +Also Read: Network Pentesting Checklist][1] + +## Installation + +Please clone the latest version of SPARTA from github: + +``` +git clone https://github.com/secforce/sparta.git +``` + +Alternatively, download the latest zip file [here][2]. +``` +cd /usr/share/ +git clone https://github.com/secforce/sparta.git +``` +Place the "sparta" file in /usr/bin/ and make it executable. +Type 'sparta' in any terminal to launch the application. + + +## The scope of Network Penetration Testing Work: + + * Organizations security weaknesses in their network infrastructures are identified by a list of host or targeted host and add them to the scope. + * Select menu bar - File > Add host(s) to scope + + + +[![Network Penetration Testing][3]][4] + +[![Network Penetration Testing][5]][6] + + * Above figures show target Ip is added to the scope.According to your network can add the range of IPs to scan. + * After adding Nmap scan will begin and results will be very faster.now scanning phase is done. + + + +## Open Ports & Services: + + * Nmap results will provide target open ports and services. + + + +[![Network Penetration Testing][7]][8] + + * Above figure shows that target operating system, Open ports and services are discovered as scan results. + + + +## Brute Force Attack on Open ports: + + * Let us Brute force Server Message Block (SMB) via port 445 to enumerate the list of users and their valid passwords. + + + +[![Network Penetration Testing][9]][10] + + * Right-click and Select option Send to Brute.Also, select discovered Open ports and service on target. + * Browse and add dictionary files for Username and password fields. + + + +[![Network Penetration Testing][11]][12] + + * Click Run to start the Brute force attack on the target.Above Figure shows Brute force attack is successfully completed on the target IP and the valid password is Found! + * Always think failed login attempts will be logged as Event logs in Windows. + * Password changing policy should be 15 to 30 days will be a good practice. + * Always recommended to use a strong password as per policy.Password lockout policy is a good one to stop brute force attacks (After 5 failure attempts account will be locked) + * The integration of business-critical asset to SIEM( security incident & Event Management) will detect these kinds of attacks as soon as possible. + + + +SPARTA is timing saving GUI Toolkit for pentesters for scanning and enumeration phase.SPARTA Scans and Bruteforce various protocols.It has many more features! Happy Hacking. + +-------------------------------------------------------------------------------- + +via: https://gbhackers.com/sparta-network-penetration-testing-gui-toolkit/ + +作者：[Balaganesh][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://gbhackers.com/author/balaganesh/ +[1]:https://gbhackers.com/network-penetration-testing-checklist-examples/ +[2]:https://github.com/SECFORCE/sparta/archive/master.zip +[3]:https://i0.wp.com/gbhackers.com/wp-content/uploads/2018/01/Screenshot-526.png?resize=696%2C495&ssl=1 +[4]:https://i0.wp.com/gbhackers.com/wp-content/uploads/2018/01/Screenshot-526.png?ssl=1 +[5]:https://i2.wp.com/gbhackers.com/wp-content/uploads/2018/01/Screenshot-527.png?resize=696%2C516&ssl=1 +[6]:https://i2.wp.com/gbhackers.com/wp-content/uploads/2018/01/Screenshot-527.png?ssl=1 +[7]:https://i2.wp.com/gbhackers.com/wp-content/uploads/2018/01/Screenshot-528.png?resize=696%2C519&ssl=1 +[8]:https://i2.wp.com/gbhackers.com/wp-content/uploads/2018/01/Screenshot-528.png?ssl=1 +[9]:https://i1.wp.com/gbhackers.com/wp-content/uploads/2018/01/Screenshot-529.png?resize=696%2C525&ssl=1 +[10]:https://i1.wp.com/gbhackers.com/wp-content/uploads/2018/01/Screenshot-529.png?ssl=1 +[11]:https://i2.wp.com/gbhackers.com/wp-content/uploads/2018/01/Screenshot-531.png?resize=696%2C523&ssl=1 +[12]:https://i2.wp.com/gbhackers.com/wp-content/uploads/2018/01/Screenshot-531.png?ssl=1 diff --git a/sources/tech/20180117 Linux tee Command Explained for Beginners (6 Examples).md b/sources/tech/20180117 Linux tee Command Explained for Beginners (6 Examples).md deleted file mode 100644 index e1be9e3da2..0000000000 --- a/sources/tech/20180117 Linux tee Command Explained for Beginners (6 Examples).md +++ /dev/null @@ -1,130 +0,0 @@ -Linux tee Command Explained for Beginners (6 Examples) -====== - -There are times when you want to manually track output of a command and also simultaneously make sure the output is being written to a file so that you can refer to it later. If you are looking for a Linux tool which can do this for you, you'll be glad to know there exists a command **tee** that's built for this purpose. - -In this tutorial, we will discuss the basics of the tee command using some easy to understand examples. But before we do that, it's worth mentioning that all examples used in this article have been tested on Ubuntu 16.04 LTS. - -### Linux tee command - -The tee command basically reads from the standard input and writes to standard output and files. Following is the syntax of the command: - -``` -tee [OPTION]... [FILE]... -``` - -And here's how the man page explains it: -``` -Copy standard input to each FILE, and also to standard output. -``` - -The following Q&A-styled examples should give you a better idea on how the command works. - -### Q1. How to use tee command in Linux? - -Suppose you are using the ping command for some reason. - -ping google.com - -[![How to use tee command in Linux][1]][2] - -And what you want, is that the output should also get written to a file in parallel. Then here's where you can use the tee command. - -``` -ping google.com | tee output.txt -``` - -The following screenshot shows the output was written to the 'output.txt' file along with being written on stdout. - -[![tee command output][3]][4] - -So that should clear the basic usage of tee. - -### Q2. How to make sure tee appends information in files? - -By default, the tee command overwrites information in a file when used again. However, if you want, you can change this behavior by using the -a command line option. - -``` -[command] | tee -a [file] -``` - -So basically, the -a option forces tee to append information to the file. - -### Q3. How to make tee write to multiple files? - -That's pretty easy. You just have to mention their names. - -``` -[command] | tee [file1] [file2] [file3] -``` - -For example: - -``` -ping google.com | tee output1.txt output2.txt output3.txt -``` - -[![How to make tee write to multiple files][5]][6] - -### Q4. How to make tee redirect output of one command to another? - -You can not only use tee to simultaneously write output to files, but also to pass on the output as input to other commands. For example, the following command will not only store the filenames in 'output.txt' but also let you know - through wc - the number of entries in the output.txt file. - -``` -ls file* | tee output.txt | wc -l -``` - -[![How to make tee redirect output of one command to another][7]][8] - -### Q5. How to write to a file with elevated privileges using tee? - -Suppose you opened a file in the [Vim editor][9], made a lot of changes, and then when you tried saving those changes, you got an error that made you realize that it's a root-owned file, meaning you need to have sudo privileges to save these changes. - -[![How to write to a file with elevated privileges using tee][10]][11] - -In scenarios like these, you can use tee to elevate privileges on the go. - -``` -:w !sudo tee % -``` - -The aforementioned command will ask you for root password, and then let you save the changes. - -### Q6. How to make tee ignore interrupt? - -The -i command line option enables tee to ignore the interrupt signal (`SIGINT`), which is usually issued when you press the crl+c key combination. - -``` -[command] | tee -i [file] -``` - -This is useful when you want to kill the command with ctrl+c but want tee to exit gracefully. - -### Conclusion - -You'll likely agree now that tee is an extremely useful command. We've discussed it's basic usage as well as majority of its command line options here. The tool doesn't have a steep learning curve, so just practice all these examples, and you should be good to go. For more information, head to the tool's [man page][12]. - - --------------------------------------------------------------------------------- - -via: https://www.howtoforge.com/linux-tee-command/ - -作者：[Himanshu Arora][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://www.howtoforge.com -[1]:https://www.howtoforge.com/images/command-tutorial/ping-example.png -[2]:https://www.howtoforge.com/images/command-tutorial/big/ping-example.png -[3]:https://www.howtoforge.com/images/command-tutorial/ping-with-tee.png -[4]:https://www.howtoforge.com/images/command-tutorial/big/ping-with-tee.png -[5]:https://www.howtoforge.com/images/command-tutorial/tee-mult-files1.png -[6]:https://www.howtoforge.com/images/command-tutorial/big/tee-mult-files1.png -[7]:https://www.howtoforge.com/images/command-tutorial/tee-redirect-output.png -[8]:https://www.howtoforge.com/images/command-tutorial/big/tee-redirect-output.png -[9]:https://www.howtoforge.com/vim-basics -[10]:https://www.howtoforge.com/images/command-tutorial/vim-write-error.png -[11]:https://www.howtoforge.com/images/command-tutorial/big/vim-write-error.png -[12]:https://linux.die.net/man/1/tee diff --git a/sources/tech/20180118 Getting Started with ncurses.md b/sources/tech/20180118 Getting Started with ncurses.md new file mode 100644 index 0000000000..78e53efa79 --- /dev/null +++ b/sources/tech/20180118 Getting Started with ncurses.md @@ -0,0 +1,214 @@ +leemeans translating +Getting Started with ncurses +====== +How to use curses to draw to the terminal screen. + +While graphical user interfaces are very cool, not every program needs to run with a point-and-click interface. For example, the venerable vi editor ran in plain-text terminals long before the first GUI. + +The vi editor is one example of a screen-oriented program that draws in "text" mode, using a library called curses, which provides a set of programming interfaces to manipulate the terminal screen. The curses library originated in BSD UNIX, but Linux systems provide this functionality through the ncurses library. + +[For a "blast from the past" on ncurses, see ["ncurses: Portable Screen-Handling for Linux"][1], September 1, 1995, by Eric S. Raymond.] + +Creating programs that use curses is actually quite simple. In this article, I show an example program that leverages curses to draw to the terminal screen. + +### Sierpinski's Triangle + +One simple way to demonstrate a few curses functions is by generating Sierpinski's Triangle. If you aren't familiar with this method to generate Sierpinski's Triangle, here are the rules: + +1. Set three points that define a triangle. + +2. Randomly select a point anywhere (x,y). + +Then: + +1. Randomly select one of the triangle's points. + +2. Set the new x,y to be the midpoint between the previous x,y and the triangle point. + +3. Repeat. + +So with those instructions, I wrote this program to draw Sierpinski's Triangle to the terminal screen using the curses functions: + +``` + + 1 /* triangle.c */ + 2 + 3 #include + 4 #include + 5 + 6 #include "getrandom_int.h" + 7 + 8 #define ITERMAX 10000 + 9 + 10 int main(void) + 11 { + 12 long iter; + 13 int yi, xi; + 14 int y[3], x[3]; + 15 int index; + 16 int maxlines, maxcols; + 17 + 18 /* initialize curses */ + 19 + 20 initscr(); + 21 cbreak(); + 22 noecho(); + 23 + 24 clear(); + 25 + 26 /* initialize triangle */ + 27 + 28 maxlines = LINES - 1; + 29 maxcols = COLS - 1; + 30 + 31 y[0] = 0; + 32 x[0] = 0; + 33 + 34 y[1] = maxlines; + 35 x[1] = maxcols / 2; + 36 + 37 y[2] = 0; + 38 x[2] = maxcols; + 39 + 40 mvaddch(y[0], x[0], '0'); + 41 mvaddch(y[1], x[1], '1'); + 42 mvaddch(y[2], x[2], '2'); + 43 + 44 /* initialize yi,xi with random values */ + 45 + 46 yi = getrandom_int() % maxlines; + 47 xi = getrandom_int() % maxcols; + 48 + 49 mvaddch(yi, xi, '.'); + 50 + 51 /* iterate the triangle */ + 52 + 53 for (iter = 0; iter < ITERMAX; iter++) { + 54 index = getrandom_int() % 3; + 55 + 56 yi = (yi + y[index]) / 2; + 57 xi = (xi + x[index]) / 2; + 58 + 59 mvaddch(yi, xi, '*'); + 60 refresh(); + 61 } + 62 + 63 /* done */ + 64 + 65 mvaddstr(maxlines, 0, "Press any key to quit"); + 66 + 67 refresh(); + 68 + 69 getch(); + 70 endwin(); + 71 + 72 exit(0); + 73 } + +``` + +Let me walk through that program by way of explanation. First, the getrandom_int() is my own wrapper to the Linux getrandom() system call, but it's guaranteed to return a positive integer value. Otherwise, you should be able to identify the code lines that initialize and then iterate Sierpinski's Triangle, based on the above rules. Aside from that, let's look at the curses functions I used to draw the triangle on a terminal. + +Most curses programs will start with these four instructions. 1) The initscr() function determines the terminal type, including its size and features, and sets up the curses environment based on what the terminal can support. The cbreak() function disables line buffering and sets curses to take one character at a time. The noecho() function tells curses not to echo the input back to the screen, and the clear() function clears the screen: + +``` + + 20 initscr(); + 21 cbreak(); + 22 noecho(); + 23 + 24 clear(); + +``` + +The program then sets a few variables to define the three points that define a triangle. Note the use of LINES and COLS here, which were set by initscr(). These values tell the program how many lines and columns exist on the terminal. Screen coordinates start at zero, so the top-left of the screen is row 0, column 0\. The bottom-right of the screen is row LINES - 1, column COLS - 1\. To make this easy to remember, my program sets these values in the variables maxlines and maxcols, respectively. + +Two simple methods to draw text on the screen are the addch() and addstr() functions. To put text at a specific screen location, use the related mvaddch() and mvaddstr() functions. My program uses these functions in several places. First, the program draws the three points that define the triangle, labeled "0", "1" and "2": + +``` + + 40 mvaddch(y[0], x[0], '0'); + 41 mvaddch(y[1], x[1], '1'); + 42 mvaddch(y[2], x[2], '2'); + +``` + +To draw the random starting point, the program makes a similar call: + +``` + + 49 mvaddch(yi, xi, '.'); + +``` + +And to draw each successive point in Sierpinski's Triangle iteration: + +``` + + 59 mvaddch(yi, xi, '*'); + +``` + +When the program is done, it displays a helpful message at the lower-left corner of the screen (at row maxlines, column 0): + +``` + + 65 mvaddstr(maxlines, 0, "Press any key to quit"); + +``` + +It's important to note that curses maintains a version of the screen in memory and updates the screen only when you ask it to. This provides greater performance, especially if you want to display a lot of text to the screen. This is because curses can update only those parts of the screen that changed since the last update. To cause curses to update the terminal screen, use the refresh() function. + +In my example program, I've chosen to update the screen after "drawing" each successive point in Sierpinski's Triangle. By doing so, users should be able to observe each iteration in the triangle. + +Before exiting, I use the getch() function to wait for the user to press a key. Then I call endwin() to exit the curses environment and return the terminal screen to normal control: + +``` + + 69 getch(); + 70 endwin(); + +``` + +### Compiling and Sample Output + +Now that you have your first sample curses program, it's time to compile and run it. Remember that Linux systems implement the curses functionality via the ncurses library, so you need to link with -lncurses when you compile—for example: + +``` + +$ ls +getrandom_int.c getrandom_int.h triangle.c + +$ gcc -Wall -lncurses -o triangle triangle.c getrandom_int.c + +``` + +Running the triangle program on a standard 80x24 terminal is not very interesting. You just can't see much detail in Sierpinski's Triangle at that resolution. If you run a terminal window and set a very small font size, you can see the fractal nature of Sierpinski's Triangle more easily. On my system, the output looks like Figure 1. + +![](http://www.linuxjournal.com/files/linuxjournal.com/ufiles/imagecache/large-550px-centered/u1000009/triangle.png) + +Figure 1. Output of the triangle Program + +Despite the random nature of the iteration, every run of Sierpinski's Triangle will look pretty much the same. The only difference will be where the first few points are drawn to the screen. In this example, you can see the single dot that starts the triangle, near point 1\. It looks like the program picked point 2 next, and you can see the asterisk halfway between the dot and the "2". And it looks like the program randomly picked point 2 for the next random number, because you can see the asterisk halfway between the first asterisk and the "2". From there, it's impossible to tell how the triangle was drawn, because all of the successive dots fall within the triangle area. + +### Starting to Learn ncurses + +This program is a simple example of how to use the curses functions to draw characters to the screen. You can do so much more with curses, depending on what you need your program to do. In a follow up article, I will show how to use curses to allow the user to interact with the screen. If you are interested in getting a head start with curses, I encourage you to read Pradeep Padala's ["NCURSES Programming HOWTO"][2], at the Linux Documentation Project. + +### About the author + +Jim Hall is an advocate for free and open-source software, best known for his work on the FreeDOS Project, and he also focuses on the usability of open-source software. Jim is the Chief Information Officer at Ramsey County, Minn. + +-------------------------------------------------------------------------------- + +via: http://www.linuxjournal.com/content/getting-started-ncurses + +作者：[Jim Hall][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://www.linuxjournal.com/users/jim-hall +[1]:http://www.linuxjournal.com/article/1124 +[2]:http://tldp.org/HOWTO/NCURSES-Programming-HOWTO diff --git a/sources/tech/20180118 How To List and Delete iptables Firewall Rules.md b/sources/tech/20180118 How To List and Delete iptables Firewall Rules.md new file mode 100644 index 0000000000..b6b875ad11 --- /dev/null +++ b/sources/tech/20180118 How To List and Delete iptables Firewall Rules.md @@ -0,0 +1,106 @@ +How To List and Delete iptables Firewall Rules +====== +![How To List and Delete iptables Firewall Rules][1] + +We'll show you, how to list and delete iptables firewall rules. Iptables is a command line utility that allows system administrators to configure the packet filtering rule set on Linux. iptables requires elevated privileges to operate and must be executed by user root, otherwise it fails to function. + +### How to List iptables Firewall Rules + +Iptables allows you to list all the rules which are already added to the packet filtering rule set. In order to be able to check this you need to have SSH access to the server. [Connect to your Linux VPS via SSH][2] and run the following command: +``` +sudo iptables -nvL +``` + +To run the command above your user need to have `sudo` privileges. Otherwise, you need to [add sudo user on your Linux VPS][3] or use the root user. + +If there are no rules added to the packet filtering ruleset the output should be similar to the one below: +``` +Chain INPUT (policy ACCEPT 0 packets, 0 bytes) + pkts bytes target prot opt in out source destination + +Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) + pkts bytes target prot opt in out source destination + +Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) + pkts bytes target prot opt in out source destination + +``` + +Since NAT (Network Address Translation) can also be configured via iptables, you can use iptables to list the NAT rules: +``` +sudo iptables -t nat -n -L -v +``` + +The output will be similar to the one below if there are no rules added: +``` +Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) + pkts bytes target prot opt in out source destination + +Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes) + pkts bytes target prot opt in out source destination + +Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) + pkts bytes target prot opt in out source destination + +``` + +If this is the case we recommend you to check our tutorial on How to [Set Up a Firewall with iptables on Ubuntu and CentOS][4] to make your server more secure. + +### How to Delete iptables Firewall Rules + +At some point, you may need to remove a specific iptables firewall rule on your server. For that purpose you need to use the following syntax: +``` +iptables [-t table] -D chain rulenum +``` + +For example, if you have a firewall rule to block all connections from 111.111.111.111 to your server on port 22 and you want to remove that rule, you can use the following command: +``` +sudo iptables -D INPUT -s 111.111.111.111 -p tcp --dport 22 -j DROP +``` + +Now that you removed the iptables firewall rule you need to save the changes to make them persistent. + +In case you are using [Ubuntu VPS][5] you need to install additional package for that purpose. To install the required package use the following command: +``` +sudo apt-get install iptables-persistent +``` + +On **Ubutnu 14.04** you can save and reload the firewall rules using the commands below: +``` +sudo /etc/init.d/iptables-persistent save +sudo /etc/init.d/iptables-persistent reload +``` + +On **Ubuntu 16.04** use the following commands instead: +``` +sudo netfilter-persistent save +sudo netfilter-persistent reload +``` + +If you are using [CentOS VPS][6] you can save the changes using the command below: +``` +service iptables save +``` + +Of course, you don't have to list and delete iptables firewall rules if you use one of our [Managed VPS Hosting][7] services, in which case you can simply ask our expert Linux admins to help you list and delete iptables firewall rules on your server. They are available 24×7 and will take care of your request immediately. + +**PS**. If you liked this post, on how to list and delete iptables firewall rules, please share it with your friends on the social networks using the buttons on the left or simply leave a reply below. Thanks. + +-------------------------------------------------------------------------------- + +via: https://www.rosehosting.com/blog/how-to-list-and-delete-iptables-firewall-rules/ + +作者：[RoseHosting][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.rosehosting.com +[1]:https://www.rosehosting.com/blog/wp-content/uploads/2018/01/How-To-List-and-Delete-iptables-Firewall-Rules.jpg +[2]:https://www.rosehosting.com/blog/connect-to-your-linux-vps-via-ssh/ +[3]:https://www.rosehosting.com/blog/how-to-create-a-sudo-user-on-ubuntu/ +[4]:https://www.rosehosting.com/blog/how-to-set-up-a-firewall-with-iptables-on-ubuntu-and-centos/ +[5]:https://www.rosehosting.com/ubuntu-vps.html +[6]:https://www.rosehosting.com/centos-vps.html +[7]:https://www.rosehosting.com/managed-vps-hosting.html diff --git a/sources/tech/20180118 How to Play Sound Through Two or More Output Devices in Linux.md b/sources/tech/20180118 How to Play Sound Through Two or More Output Devices in Linux.md new file mode 100644 index 0000000000..2f35b15ac7 --- /dev/null +++ b/sources/tech/20180118 How to Play Sound Through Two or More Output Devices in Linux.md @@ -0,0 +1,62 @@ +translating by lujun9972 +How to Play Sound Through Two or More Output Devices in Linux +====== + +![](https://www.maketecheasier.com/assets/uploads/2018/01/output-audio-multiple-devices-featured.jpg) + +Handling audio in Linux can be a pain. Pulseaudio has made it both better and worse. While some things work better than they did before, other things have become more complicated. Handling audio output is one of those things. + +If you want to enable multiple audio outputs from your Linux PC, you can use a simple utility to enable your other sound devices on a virtual interface. It's a lot easier than it sounds. + +In case you're wondering why you'd want to do this, a pretty common instance is playing video from your computer on a TV and using both the PC and TV speakers. + +### Install Paprefs + +The easiest way to enable audio playback from multiple sources is to use a simple graphical utility called "paprefs." It's short for PulseAudio Preferences. + +It's available through the Ubuntu repositories, so just install it with Apt. +``` +sudo apt install paprefs +``` + +When the install finishes, you can just launch the program. + +### Enable Dual Audio Playback + +Even though the utility is graphical, it's still probably easier to launch it by typing `paprefs` in the command line as a regular user. + +The window that opens has a few tabs with settings that you can tweak. The tab that you're looking for is the last one, "Simultaneous Output." + +![Paprefs on Ubuntu][1] + +There isn't a whole lot on the tab, just a checkbox to enable the setting. + +Next, open up the regular sound preferences. It's in different places on different distributions. On Ubuntu it'll be under the GNOME system settings. + +![Enable Simultaneous Audio][2] + +Once you have your sound preferences open, select the "Output" tab. Select the "Simultaneous output" radio button. It's now your default output. + +### Test It + +To test it, you can use anything you like, but music always works. If you are using a video, like suggested earlier, you can certainly test it with that as well. + +If everything is working well, you should hear audio out of all connected devices. + +That's all there really is to do. This works best when there are multiple devices, like the HDMI port and the standard analog output. You can certainly try it with other configurations, too. You should also keep in mind that there will only be a single volume control, so adjust the physical output devices accordingly. + + +-------------------------------------------------------------------------------- + +via: https://www.maketecheasier.com/play-sound-through-multiple-devices-linux/ + +作者：[Nick Congleton][a] +译者：[lujun9972](https://github.com/lujun9972) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.maketecheasier.com/author/nickcongleton/ +[1]:https://www.maketecheasier.com/assets/uploads/2018/01/sa-paprefs.jpg (Paprefs on Ubuntu) +[2]:https://www.maketecheasier.com/assets/uploads/2018/01/sa-enable.jpg (Enable Simultaneous Audio) +[3]:https://depositphotos.com/89314442/stock-photo-headphones-on-speakers.html diff --git a/sources/tech/20180118 Rediscovering make- the power behind rules.md b/sources/tech/20180118 Rediscovering make- the power behind rules.md new file mode 100644 index 0000000000..ea500a2689 --- /dev/null +++ b/sources/tech/20180118 Rediscovering make- the power behind rules.md @@ -0,0 +1,102 @@ +Translating by cncuckoo + +Rediscovering make: the power behind rules +====== + +![](https://user-images.githubusercontent.com/4419992/35015638-0529f1c0-faf4-11e7-9801-4995fc4b54f0.jpg) + +I used to think makefiles were just a convenient way to list groups of shell commands; over time I've learned how powerful, flexible, and full-featured they are. This post brings to light over some of those features related to rules. + +### Rules + +Rules are instructions that indicate `make` how and when a file called the target should be built. The target can depend on other files called prerequisites. + +You instruct `make` how to build the target in the recipe, which is no more than a set of shell commands to be executed, one at a time, in the order they appear. The syntax looks like this: +``` +target_name : prerequisites + recipe +``` + +Once you have defined a rule, you can build the target from the command line by executing: +``` +$ make target_name +``` + +Once the target is built, `make` is smart enough to not run the recipe ever again unless at least one of the prerequisites has changed. + +### More on prerequisites + +Prerequisites indicate two things: + + * When the target should be built: if a prerequisite is newer than the target, `make` assumes that the target should be built. + * An order of execution: since prerequisites can, in turn, be built by another rule on the makefile, they also implicitly set an order on which rules are executed. + + + +If you want to define an order, but you don't want to rebuild the target if the prerequisite changes, you can use a special kind of prerequisite called order only, which can be placed after the normal prerequisites, separated by a pipe (`|`) + +### Patterns + +For convenience, `make` accepts patterns for targets and prerequisites. A pattern is defined by including the `%` character, a wildcard that matches any number of literal characters or an empty string. Here are some examples: + + * `%`: match any file + * `%.md`: match all files with the `.md` extension + * `prefix%.go`: match all files that start with `prefix` that have the `.go` extension + + + +### Special targets + +There's a set of target names that have special meaning for `make` called special targets. + +You can find the full list of special targets in the [documentation][1]. As a rule of thumb, special targets start with a dot followed by uppercase letters. + +Here are a few useful ones: + +**.PHONY** : Indicates `make` that the prerequisites of this target are considered to be phony targets, which means that `make` will always run it's recipe regardless of whether a file with that name exists or what its last-modification time is. + +**.DEFAULT** : Used for any target for which no rules are found. + +**.IGNORE** : If you specify prerequisites for `.IGNORE`, `make` will ignore errors in execution of their recipes. + +### Substitutions + +Substitutions are useful when you need to modify the value of a variable with alterations that you specify. + +A substitution has the form `$(var:a=b)` and its meaning is to take the value of the variable `var`, replace every `a` at the end of a word with `b` in that value, and substitute the resulting string. For example: +``` +foo := a.o +bar : = $(foo:.o=.c) # sets bar to a.c +``` + +note: special thanks to [Luis Lavena][2] for letting me know about the existence of substitutions. + +### Archive Files + +Archive files are used to collect multiple data files together into a single file (same concept as a zip file), they are built with the `ar` Unix utility. `ar` can be used to create archives for any purpose, but has been largely replaced by `tar` for any other purposes than [static libraries][3]. + +In `make`, you can use an individual member of an archive file as a target or prerequisite as follows: +``` +archive(member) : prerequisite + recipe +``` + +### Final Thoughts + +There's a lot more to discover about make, but at least this counts as a start, I strongly encourage you to check the [documentation][4], create a dumb makefile, and just play with it. + +-------------------------------------------------------------------------------- + +via: https://monades.roperzh.com/rediscovering-make-power-behind-rules/ + +作者：[Roberto Dip][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://monades.roperzh.com +[1]:https://www.gnu.org/software/make/manual/make.html#Special-Targets +[2]:https://twitter.com/luislavena/ +[3]:http://tldp.org/HOWTO/Program-Library-HOWTO/static-libraries.html +[4]:https://www.gnu.org/software/make/manual/make.html diff --git a/sources/tech/20180118 Securing the Linux filesystem with Tripwire.md b/sources/tech/20180118 Securing the Linux filesystem with Tripwire.md new file mode 100644 index 0000000000..a359e3a422 --- /dev/null +++ b/sources/tech/20180118 Securing the Linux filesystem with Tripwire.md @@ -0,0 +1,112 @@ +Securing the Linux filesystem with Tripwire +====== + +![](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/file_system.jpg?itok=pzCrX1Kc) + +While Linux is considered to be the most secure operating system (ahead of Windows and MacOS), it is still vulnerable to rootkits and other variants of malware. Thus, Linux users need to know how to protect their servers or personal computers from destruction, and the first step they need to take is to protect the filesystem. + +In this article, we'll look at [Tripwire][1], an excellent tool for protecting Linux filesystems. Tripwire is an integrity checking tool that enables system administrators, security engineers, and others to detect alterations to system files. Although it's not the only option available ([AIDE][2] and [Samhain][3] offer similar features), Tripwire is arguably the most commonly used integrity checker for Linux system files, and it is available as open source under GPLv2. + +### How Tripwire works + +It's helpful to know how Tripwire operates in order to understand what it does once it's installed. Tripwire is made up of two major components: policy and database. Policy lists all the files and directories that the integrity checker should take a snapshot of, in addition to creating rules for identifying violations of changes to directories and files. Database consists of the snapshot taken by Tripwire. + +Tripwire also has a configuration file, which specifies the locations of the database, policy file, and Tripwire executable. It also provides two cryptographic keys--site key and local key--to protect important files against tampering. The site key protects the policy and configuration files, while the local key protects the database and generated reports. + +Tripwire works by periodically comparing the directories and files against the snapshot in the database and reporting any changes. + +### Installing Tripwire + +In order to use Tripwire, we need to download and install it first. Tripwire works on almost all Linux distributions; you can download an open source version from [Sourceforge][4] and install it as follows, depending on your version of Linux. + +Debian and Ubuntu users can install Tripwire directly from the repository using `apt-get`. Non-root users should type the `sudo` command to install Tripwire via `apt-get`. +``` + + +sudo apt-get update + +sudo  apt-get install tripwire   +``` + +CentOS and other rpm-based distributions use a similar process. For the sake of best practice, update your repository before installing a new package such as Tripwire. The command `yum install epel-release` simply means we want to install extra repositories. (`epel` stands for Extra Packages for Enterprise Linux.) +``` + + +yum update + +yum install epel-release + +yum install tripwire   +``` + +This command causes the installation to run a configuration of packages that are required for Tripwire to function effectively. In addition, it will ask if you want to select passphrases during installation. You can select "Yes" to both prompts. + +Also, select or choose "Yes" if it's required to build the configuration file. Choose and confirm a passphrase for a site key and for a local key. (A complex passphrase such as `Il0ve0pens0urce` is recommended.) + +### Build and initialize Tripwire's database + +Next, initialize the Tripwire database as follows: +``` + + +tripwire --init +``` + +You'll need to provide your local key passphrase to run the commands. + +### Basic integrity checking using Tripwire + +You can use the following command to instruct Tripwire to check whether your files or directories have been modified. Tripwire's ability to compare files and directories against the initial snapshot in the database is based on the rules you created in the active policy. +``` + + +tripwire  --check   +``` + +You can also limit the `-check` command to specific files or directories, such as in this example: +``` + + +tripwire   --check   /usr/tmp   +``` + +In addition, if you need extended help on using Tripwire's `-check` command, this command allows you to consult Tripwire's manual: +``` + + +tripwire  --check  --help   +``` + +### Generating reports using Tripwire + +To easily generate a daily system integrity report, create a `crontab` with this command: +``` + + +crontab -e +``` + +Afterward, you can edit this file (with the text editor of your choice) to introduce tasks to be run by cron. For instance, you can set up a cron job to send Tripwire reports to your email daily at 5:40 a.m. by using this command: +``` + + +40 5  *  *  *  usr/sbin/tripwire   --check +``` + +Whether you decide to use Tripwire or another integrity checker with similar features, the key issue is making sure you have a solution to protect the security of your Linux filesystem. + +-------------------------------------------------------------------------------- + +via: https://opensource.com/article/18/1/securing-linux-filesystem-tripwire + +作者：[Michael Kwaku Aboagye][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://opensource.com/users/revoks +[1]:https://www.tripwire.com/ +[2]:http://aide.sourceforge.net/ +[3]:http://www.la-samhna.de/samhain/ +[4]:http://sourceforge.net/projects/tripwire diff --git a/sources/tech/20180119 5 of the Best Linux Dark Themes that Are Easy on the Eyes.md b/sources/tech/20180119 5 of the Best Linux Dark Themes that Are Easy on the Eyes.md new file mode 100644 index 0000000000..db70cd8732 --- /dev/null +++ b/sources/tech/20180119 5 of the Best Linux Dark Themes that Are Easy on the Eyes.md @@ -0,0 +1,73 @@ +5 of the Best Linux Dark Themes that Are Easy on the Eyes +====== + +![](https://www.maketecheasier.com/assets/uploads/2017/12/linux-themes.png) + +There are several reasons people opt for dark themes on their computers. Some find them easy on the eye while others prefer them because of their medical condition. Programmers, especially, like dark themes because they reduce glare on the eyes. + +If you are a Linux user and a dark theme lover, you are in luck. Here are five of the best dark themes for Linux. Check them out! + +### 1. OSX-Arc-Shadow + +![OSX-Arc-Shadow Theme][1] + +As its name implies, this theme is inspired by OS X. It is a flat theme based on Arc. The theme supports GTK 3 and GTK 2 desktop environments, so Gnome, Cinnamon, Unity, Manjaro, Mate, and XFCE users can install and use the theme. [OSX-Arc-Shadow][2] is part of the OSX-Arc theme collection. The collection has several other themes (dark and light) included. You can download the whole collection and just use the dark variants. + +Debian- and Ubuntu-based distro users have the option of installing the stable release using the .deb files found on this [page][3]. The compressed source files are also on the same page. Arch Linux users, check out this [AUR link][4]. Finally, to install the theme manually, extract the zip content to the "~/.themes" folder and set it as your current theme, controls, and window borders. + +### 2. Kiss-Kool-Red version 2 + +![Kiss-Kool-Red version 2 ][5] + +The theme is only a few days old. It has a darker look compared to OSX-Arc-Shadow and red selection outlines. It is especially appealing to those who want more contrast and less glare from the computer screen. Hence, It reduces distraction when used at night or in places with low lights. It supports GTK 3 and GTK2. + +Head to [gnome-looks][6] to download the theme under the "Files" menu. The installation procedure is simple: extract the theme into the "~/.themes" folder and set it as your current theme, controls, and window borders. + +### 3. Equilux + +![Equilux][7] + +Equilux is another simple dark theme based on Materia Theme. It has a neutral dark color tone and is not overly fancy. The contrast between the selection outlines is also minimal and not as sharp as the red color in Kiss-Kool-Red. The theme is truly made with reduction of eye strain in mind. + +[Download the compressed file][8] and unzip it into your "~/.themes" folder. Then, you can set it as your theme. You can check [its GitHub page][9] for the latest additions. + +### 4. Deepin Dark + +![Deepin Dark][10] + +Deepin Dark is a completely dark theme. For those who like a little more darkness, this theme is definitely one to consider. Moreover, it also reduces the amount of glare from the computer screen. Additionally, it supports Unity. [Download Deepin Dark here][11]. + +### 5. Ambiance DS BlueSB12 + +![Ambiance DS BlueSB12 ][12] + +Ambiance DS BlueSB12 is a simple dark theme, so it makes the important details stand out. It helps with focus as is not unnecessarily fancy. It is very similar to Deepin Dark. Especially relevant to Ubuntu users, it is compatible with Ubuntu 17.04. You can download and try it from [here][13]. + +### Conclusion + +If you use a computer for a very long time, dark themes are a great way to reduce the strain on your eyes. Even if you don't, dark themes can help you in many other ways like improving your focus. Let us know which is your favorite. + +-------------------------------------------------------------------------------- + +via: https://www.maketecheasier.com/best-linux-dark-themes/ + +作者：[Bruno Edoh][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.maketecheasier.com +[1]:https://www.maketecheasier.com/assets/uploads/2017/12/osx-arc-shadow.png (OSX-Arc-Shadow Theme) +[2]:https://github.com/LinxGem33/OSX-Arc-Shadow/ +[3]:https://github.com/LinxGem33/OSX-Arc-Shadow/releases +[4]:https://aur.archlinux.org/packages/osx-arc-shadow/ +[5]:https://www.maketecheasier.com/assets/uploads/2017/12/Kiss-Kool-Red.png (Kiss-Kool-Red version 2 ) +[6]:https://www.gnome-look.org/p/1207964/ +[7]:https://www.maketecheasier.com/assets/uploads/2017/12/equilux.png (Equilux) +[8]:https://www.gnome-look.org/p/1182169/ +[9]:https://github.com/ddnexus/equilux-theme +[10]:https://www.maketecheasier.com/assets/uploads/2017/12/deepin-dark.png (Deepin Dark ) +[11]:https://www.gnome-look.org/p/1190867/ +[12]:https://www.maketecheasier.com/assets/uploads/2017/12/ambience.png (Ambiance DS BlueSB12 ) +[13]:https://www.gnome-look.org/p/1013664/ diff --git a/sources/tech/20180119 How to Install Tripwire IDS Intrusion Detection System on Linux.md b/sources/tech/20180119 How to Install Tripwire IDS Intrusion Detection System on Linux.md new file mode 100644 index 0000000000..fb994b7f54 --- /dev/null +++ b/sources/tech/20180119 How to Install Tripwire IDS Intrusion Detection System on Linux.md @@ -0,0 +1,102 @@ +How to Install Tripwire IDS (Intrusion Detection System) on Linux +============================================================ + + +Tripwire is a popular Linux Intrusion Detection System (IDS) that runs on systems in order to detect if unauthorized filesystem changes occurred over time. + +In CentOS and RHEL distributions, tripwire is not a part of official repositories. However, the tripwire package can be installed via [Epel repositories][1]. + +To begin, first install Epel repositories in CentOS and RHEL system, by issuing the below command. + +``` +# yum install epel-release +``` + +After you’ve installed Epel repositories, make sure you update the system with the following command. + +``` +# yum update +``` + +After the update process finishes, install Tripwire IDS software by executing the below command. + +``` +# yum install tripwire +``` + +Fortunately, tripwire is a part of Ubuntu and Debian default repositories and can be installed with following commands. + +``` +$ sudo apt update +$ sudo apt install tripwire +``` + +On Ubuntu and Debian, the tripwire installation will be asked to choose and confirm a site key and local key passphrase. These keys are used by tripwire to secure its configuration files. + + [![Create Tripwire Site and Local Key](https://www.tecmint.com/wp-content/uploads/2018/01/Create-Site-and-Local-key.png)][2] + +Create Tripwire Site and Local Key + +On CentOS and RHEL, you need to create tripwire keys with the below command and supply a passphrase for site key and local key. + +``` +# tripwire-setup-keyfiles +``` + [![Create Tripwire Keys](https://www.tecmint.com/wp-content/uploads/2018/01/Create-Tripwire-Keys.png)][3] + +Create Tripwire Keys + +In order to validate your system, you need to initialize Tripwire database with the following command. Due to the fact that the database hasn’t been initialized yet, tripwire will display a lot of false-positive warnings. + +``` +# tripwire --init +``` + [![Initialize Tripwire Database](https://www.tecmint.com/wp-content/uploads/2018/01/Initialize-Tripwire-Database.png)][4] + +Initialize Tripwire Database + +Finally, generate a tripwire system report in order to check the configurations by issuing the below command. Use `--help` switch to list all tripwire check command options. + +``` +# tripwire --check --help +# tripwire --check +``` + +After tripwire check command completes, review the report by opening the file with the extension `.twr` from /var/lib/tripwire/report/ directory with your favorite text editor command, but before that you need to convert to text file. + +``` +# twprint --print-report --twrfile /var/lib/tripwire/report/tecmint-20170727-235255.twr > report.txt +# vi report.txt +``` + [![Tripwire System Report](https://www.tecmint.com/wp-content/uploads/2018/01/Tripwire-System-Report.png)][5] + +Tripwire System Report + +That’s It! you have successfully installed Tripwire on Linux server. I hope you can now easily configure your [Tripwire IDS][6]. + +-------------------------------------------------------------------------------- + +作者简介: + +I'am a computer addicted guy, a fan of open source and linux based system software, have about 4 years experience with Linux distributions desktop, servers and bash scripting. + +------- + +via: https://www.tecmint.com/install-tripwire-ids-intrusion-detection-system-on-linux/ + +作者：[ Matei Cezar][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.tecmint.com/author/cezarmatei/ +[1]:https://www.tecmint.com/how-to-enable-epel-repository-for-rhel-centos-6-5/ +[2]:https://www.tecmint.com/wp-content/uploads/2018/01/Create-Site-and-Local-key.png +[3]:https://www.tecmint.com/wp-content/uploads/2018/01/Create-Tripwire-Keys.png +[4]:https://www.tecmint.com/wp-content/uploads/2018/01/Initialize-Tripwire-Database.png +[5]:https://www.tecmint.com/wp-content/uploads/2018/01/Tripwire-System-Report.png +[6]:https://www.tripwire.com/ +[7]:https://www.tecmint.com/author/cezarmatei/ +[8]:https://www.tecmint.com/10-useful-free-linux-ebooks-for-newbies-and-administrators/ +[9]:https://www.tecmint.com/free-linux-shell-scripting-books/ \ No newline at end of file diff --git a/sources/tech/20180119 Linux mv Command Explained for Beginners (8 Examples).md b/sources/tech/20180119 Linux mv Command Explained for Beginners (8 Examples).md new file mode 100644 index 0000000000..78cf02f4a9 --- /dev/null +++ b/sources/tech/20180119 Linux mv Command Explained for Beginners (8 Examples).md @@ -0,0 +1,188 @@ +translating by cncuckoo + +Linux mv Command Explained for Beginners (8 Examples) +====== + +Just like [cp][1] for copying and rm for deleting, Linux also offers an in-built command for moving and renaming files. It's called **mv**. In this article, we will discuss the basics of this command line tool using easy to understand examples. Please note that all examples used in this tutorial have been tested on Ubuntu 16.04 LTS. + +#### Linux mv command + +As already mentioned, the mv command in Linux is used to move or rename files. Following is the syntax of the command: + +``` +mv [OPTION]... [-T] SOURCE DEST +mv [OPTION]... SOURCE... DIRECTORY +mv [OPTION]... -t DIRECTORY SOURCE... +``` + +And here's what the man page says about it: +``` +Rename SOURCE to DEST, or move SOURCE(s) to DIRECTORY. +``` + +The following Q&A-styled examples will give you a better idea on how this tool works. + +#### Q1. How to use mv command in Linux? + +If you want to just rename a file, you can use the mv command in the following way: + +``` +mv [filename] [new_filename] +``` + +For example: + +``` +mv names.txt fullnames.txt +``` + +[![How to use mv command in Linux][2]][3] + +Similarly, if the requirement is to move a file to a new location, use the mv command in the following way: + +``` +mv [filename] [dest-dir] +``` + +For example: + +``` +mv fullnames.txt /home/himanshu/Downloads +``` + +[![Linux mv command][4]][5] + +#### Q2. How to make sure mv prompts before overwriting? + +By default, the mv command doesn't prompt when the operation involves overwriting an existing file. For example, the following screenshot shows the existing full_names.txt was overwritten by mv without any warning or notification. + +[![How to make sure mv prompts before overwriting][6]][7] + +However, if you want, you can force mv to prompt by using the **-i** command line option. + +``` +mv -i [file_name] [new_file_name] +``` + +[![the -i command option][8]][9] + +So the above screenshots clearly shows that **-i** leads to mv asking for user permission before overwriting an existing file. Please note that in case you want to explicitly specify that you don't want mv to prompt before overwriting, then use the **-f** command line option. + +#### Q3. How to make mv not overwrite an existing file? + +For this, you need to use the **-n** command line option. + +``` +mv -n [filename] [new_filename] +``` + +The following screenshot shows the mv operation wasn't successful as a file with name 'full_names.txt' already existed and the command had -n option in it. + +[![How to make mv not overwrite an existing file][10]][11] + +Note: +``` +If you specify more than one of -i, -f, -n, only the final one takes effect. +``` + +#### Q4. How to make mv remove trailing slashes (if any) from source argument? + +To remove any trailing slashes from source arguments, use the **\--strip-trailing-slashes** command line option. + +``` +mv --strip-trailing-slashes [source] [dest] +``` + +Here's how the official documentation explains the usefulness of this option: +``` +This is useful when a + +source + + argument may have a trailing slash and specify a symbolic link to a directory. This scenario is in fact rather common because some shells can automatically append a trailing slash when performing file name completion on such symbolic links. Without this option, + +mv + +, for example, (via the system's rename function) must interpret a trailing slash as a request to dereference the symbolic link and so must rename the indirectly referenced + +directory + + and not the symbolic link. Although it may seem surprising that such behavior be the default, it is required by POSIX and is consistent with other parts of that standard. +``` + +#### Q5. How to make mv treat destination as normal file? + +To be absolutely sure that the destination entity is treated as a normal file (and not a directory), use the **-T** command line option. + +``` +mv -T [source] [dest] +``` + +Here's why this command line option exists: +``` +This can help avoid race conditions in programs that operate in a shared area. For example, when the command 'mv /tmp/source /tmp/dest' succeeds, there is no guarantee that /tmp/source was renamed to /tmp/dest: it could have been renamed to/tmp/dest/source instead, if some other process created /tmp/dest as a directory. However, if mv -T /tmp/source /tmp/dest succeeds, there is no question that/tmp/source was renamed to /tmp/dest. +``` +``` +In the opposite situation, where you want the last operand to be treated as a directory and want a diagnostic otherwise, you can use the --target-directory (-t) option. +``` + +#### Q6. How to make mv move file only when its newer than destination file? + +Suppose there exists a file named fullnames.txt in Downloads directory of your system, and there's a file with same name in your home directory. Now, you want to update ~/Downloads/fullnames.txt with ~/fullnames.txt, but only when the latter is newer. Then in this case, you'll have to use the **-u** command line option. + +``` +mv -u ~/fullnames.txt ~/Downloads/fullnames.txt +``` + +This option is particularly useful in cases when you need to take such decisions from within a shell script. + +#### Q7. How make mv emit details of what all it is doing? + +If you want mv to output information explaining what exactly it's doing, then use the **-v** command line option. + +``` +mv -v [filename] [new_filename] +``` + +For example, the following screenshots shows mv emitting some helpful details of what exactly it did. + +[![How make mv emit details of what all it is doing][12]][13] + +#### Q8. How to force mv to create backup of existing destination files? + +This you can do using the **-b** command line option. The backup file created this way will have the same name as the destination file, but with a tilde (~) appended to it. Here's an example: + +[![How to force mv to create backup of existing destination files][14]][15] + +#### Conclusion + +As you'd have guessed by now, mv is as important as cp and rm for the functionality it offers - renaming/moving files around is also one of the basic operations after all. We've discussed a majority of command line options this tool offers. So you can just practice them and start using the command. To know more about mv, head to its [man page][16]. + + +-------------------------------------------------------------------------------- + +via: https://www.howtoforge.com/linux-mv-command/ + +作者：[Himanshu Arora][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.howtoforge.com +[1]:https://www.howtoforge.com/linux-cp-command/ +[2]:https://www.howtoforge.com/images/command-tutorial/mv-rename-ex.png +[3]:https://www.howtoforge.com/images/command-tutorial/big/mv-rename-ex.png +[4]:https://www.howtoforge.com/images/command-tutorial/mv-transfer-file.png +[5]:https://www.howtoforge.com/images/command-tutorial/big/mv-transfer-file.png +[6]:https://www.howtoforge.com/images/command-tutorial/mv-overwrite.png +[7]:https://www.howtoforge.com/images/command-tutorial/big/mv-overwrite.png +[8]:https://www.howtoforge.com/images/command-tutorial/mv-prompt-overwrite.png +[9]:https://www.howtoforge.com/images/command-tutorial/big/mv-prompt-overwrite.png +[10]:https://www.howtoforge.com/images/command-tutorial/mv-n-option.png +[11]:https://www.howtoforge.com/images/command-tutorial/big/mv-n-option.png +[12]:https://www.howtoforge.com/images/command-tutorial/mv-v-option.png +[13]:https://www.howtoforge.com/images/command-tutorial/big/mv-v-option.png +[14]:https://www.howtoforge.com/images/command-tutorial/mv-b-option.png +[15]:https://www.howtoforge.com/images/command-tutorial/big/mv-b-option.png +[16]:https://linux.die.net/man/1/mv diff --git a/sources/tech/20180119 PlayOnLinux For Easier Use Of Wine.md b/sources/tech/20180119 PlayOnLinux For Easier Use Of Wine.md new file mode 100644 index 0000000000..2af3433920 --- /dev/null +++ b/sources/tech/20180119 PlayOnLinux For Easier Use Of Wine.md @@ -0,0 +1,153 @@ +PlayOnLinux For Easier Use Of Wine +====== + +![](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux-for-easier-use-of-wine_orig.jpg) + +[PlayOnLinux][1] is a free program that helps to install, run, and manage Windows software on Linux. It can also manage virtual C: drives (known as Wine prefixes), and download and install certain Windows libraries for getting some software to run on Wine properly. Creating different drives using different Wine versions is also possible. It is very handy because what runs well in one version may not run as well (if at all) on a newer version. There is [PlayOnMac][2] for macOS and PlayOnBSD for FreeBSD. + +[Wine][3] is the compatibility layer that allows many programs developed for Windows to run under operating systems such as Linux, FreeBSD, macOS and other UNIX systems. The app database ([AppDB][4]) gives users an overview of a multitude of programs that will function on Wine, however successfully. + +Both programs can be obtained using your distribution’s software center or package manager for convenience. + +### Installing Programs Using PlayOnLinux + +Installing software is easy. PlayOnLinux has hundreds of scripts to aid in installing different software with which to run the setup. In the sidebar, select “Install Software”. You will find several categories to choose from. + +​ + +Hundreds of games can be installed this way. + + [![install games playonlinux](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_orig.png)][5] + +​Office software can be installed as well, including Microsoft Office as shown here. + + [![microsoft office in linux playonlinux](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_1_orig.png)][6] + +​Let’s install Notepad++ using the script. You can select the script to read the compatibility rating according to PlayOnLinux, and an overview of the program. To get a better idea of compatibility, refer to the WineHQ App Database and find “Browse Apps” to find a program like Notepad++. + + [![install notepad++ in linux](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_2_orig.png)][7] + +​Once you press “Install”, if you are using PlayOnLinux for the first time, you will encounter two popups: one to give you tips when installing programs with a script, and the other to not submit bug reports to WineHQ because PlayOnLinux has nothing to do with them. + +​ + +​During the installation, I was given the choice to either download the setup executable, or select one on the computer. I downloaded the file but received a File Mismatch error; however, I continued and it was successful. It’s not perfect, but it is functional. (It is possible to submit bug reports to PlayOnLinux if the option is given.) + +[![bug report on playonlinux](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_3_orig.png)][8] + +Nevertheless, I was able to install Notepad++ successfully, run it, and update it to the latest version (at the time of writing 7.5.3) from version 7.4.2. + +​ + +Also during installation, it created a virtual C: drive specifically for Notepad++. As there are no other Wine versions available for PlayOnLinux to use, it defaults to using the version installed on the system. In this case, it is more than adequate for Notepad++ to run smoothly. + +### Installing Non-Listed Programs + +You can also install a program that is not on the list by pressing “Install Non-Listed Program” on the bottom-left corner of the install menu. Bear in mind that there is no script to install certain libraries to make things work properly. You will need to do this yourself. Look at the Wine AppDB for information for your program. Also, if the app isn’t listed, it doesn’t mean that it won’t work with Wine. It just means no one has given any information about it. + +​ + +I’ve installed Graphmatica, a graph plotting program, using this method. First I selected the option to install it on a new virtual drive. + + [![install non listed programs on linux](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_4_orig.png)][9] + +​Then I selected the option to install additional libraries after creating the drive and select a Wine version to use in doing so. + + [![playonlinux setup wizard](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_5_orig.png)][10] + +​I then proceeded to select Gecko (which encountered an error for some reason), and Mono 2.10 to install. + + [![playonlinux wizard POL_install](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_6_orig.png)][11] + +​Finally, I installed Graphmatica. It’s as simple as that. + + [![software installation done playonlinux](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_7_orig.png)][12] + +A launcher can be created after installation. A list of executables found in the drive will appear. Search for the app executable (may not always be obvious) which may have its icon, select it and give it a display name. The icon will appear on the desktop. + + [![install graphmatica in linux playonlinux](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_8_orig.png)][13] + [![playonlinux install windows software](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_9_orig.png)][14] + +### Multiple “C:” Drives + +Now that we have easily installed a program, let’s have a look at the drive configuration. In the main window, press “Configure” in the toolbar and this window will show. + + [![multiple c: drives in linux](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/editor/playonlinux_10.png?1516170517)][15] + +On the left are the drives that are found within PlayOnLinux. To the right, the “General” tab allows you to create shortcuts of programs installed on that virtual drive. + +​ + +The “Wine” tab has 8 buttons, including those to launch the Wine configuration program (winecfg), control panel, registry editor, command prompt, etc. + + [![playonlinux configuration wine](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_11_orig.png)][16] + +​“Install Components” allows you to select different Windows libraries like DirectX 9, .NET Framework versions 2 – 4.5, Visual C++ runtime, etc., like [winetricks][17]. + + [![install playonlinux components](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_12_orig.png)][18] + +“Display” allows the user to control advanced graphics settings like GLSL support, video memory size, and more. And “Miscellaneous” is for other actions like running an executable found anywhere on the computer to be run under the selected virtual drive. + +### Creating Virtual Drives Without Installing Programs + +To create a drive without installing software, simply press “New” below the list of drives to launch the virtual drive creator. Drives are created using the same method used in installing programs not found in the install menu. Follow the prompts, select either a 32-bit or 64-bit installation (in this case we only have 32-bit versions so select 32-bit), choose the Wine version, and give the drive a name. Once completed, it will appear in the drive list. + + [![playonlinux sandbox](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_13_orig.png)][19] + +### Managing Wine Versions + +Entire Wine versions can be downloaded using the manager. To access this through the menu bar, press “Tools” and select “Manage Wine versions”. Sometimes different software can behave differently between Wine versions. A Wine update can break something that made your application work in the previous version; thus rendering the application broken or completely unusable. Therefore, this feature is one of the highlights of PlayOnLinux. + +​ + +If you’re still on the configuration window, in the “General” tab, you can also access the version manager by pressing the “+” button next to the Wine version field. + + [![playonlinux select wine version](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_14_orig.png)][20] + +To install a version of Wine (32-bit or 64-bit), simply select the version, and press the “>” button to download and install it. After installation, if setup executables for Mono, and/or the Gecko HTML engine have not yet been downloaded by PlayOnLinux, they will be downloaded. + +​ + +I went ahead and installed the 2.21-staging version of Wine afterward. + + [![select wine version playonlinux](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_15_orig.png)][21] + +​To remove a version, press the “<” button. + +### Conclusion + +​This article demonstrated how to use PlayOnLinux to easily install Windows software into separate virtual C: drives, create and manage virtual drives, and manage several Wine versions. The software isn’t perfect, but it is still functional and useful. Managing different drives with different Wine versions is one of the key features of PlayOnLinux. It is a lot easier to use a front-end for Wine such as PlayOnLinux than pure Wine. + + +-------------------------------------------------------------------------------- + +via: http://www.linuxandubuntu.com/home/playonlinux-for-easier-use-of-wine + +作者：[LinuxAndUbuntu][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://www.linuxandubuntu.com +[1]:https://www.playonlinux.com/en/ +[2]:https://www.playonmac.com +[3]:https://www.winehq.org/ +[4]:http://appdb.winehq.org/ +[5]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_orig.png +[6]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_1_orig.png +[7]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_2_orig.png +[8]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_3_orig.png +[9]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_4_orig.png +[10]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_5_orig.png +[11]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_6_orig.png +[12]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_7_orig.png +[13]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_8_orig.png +[14]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_9_orig.png +[15]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_10_orig.png +[16]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_11_orig.png +[17]:https://github.com/Winetricks/winetricks +[18]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_12_orig.png +[19]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_13_orig.png +[20]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_14_orig.png +[21]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/playonlinux_15_orig.png diff --git a/sources/tech/20180119 Two great uses for the cp command Bash shortcuts.md b/sources/tech/20180119 Two great uses for the cp command Bash shortcuts.md new file mode 100644 index 0000000000..9a45c26e7a --- /dev/null +++ b/sources/tech/20180119 Two great uses for the cp command Bash shortcuts.md @@ -0,0 +1,154 @@ +Translating by cncuckoo + +Two great uses for the cp command: Bash shortcuts +============================================================ + +### Here's how to streamline the backup and synchronize functions of the cp command. + +![Two great uses for the cp command: Bash shortcuts ](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/yearbook-haff-rx-linux-file-lead_0.png?itok=-i0NNfDC) + +>Image by : [Internet Archive Book Images][6]. Modified by Opensource.com. CC BY-SA 4.0 + +Last July, I wrote about [two great uses for the cp command][7]: making a backup of a file, and synchronizing a secondary copy of a folder. + +Having discovered these great utilities, I find that they are more verbose than necessary, so I created shortcuts to them in my Bash shell startup script. I thought I’d share these shortcuts in case they are useful to others or could offer inspiration to Bash users who haven’t quite taken on aliases or shell functions. + +### Updating a second copy of a folder – Bash alias + +The general pattern for updating a second copy of a folder with cp is: + +``` +cp -r -u -v SOURCE-FOLDER DESTINATION-DIRECTORY +``` + +I can easily remember the -r option because I use it often when copying folders around. I can probably, with some more effort, remember -v, and with even more effort, -u (is it “update” or “synchronize” or…). + +Or I can just use the [alias capability in Bash][8] to convert the cp command and options to something more memorable, like this: + +``` +alias sync='cp -r -u -v' +``` + +``` +sync Pictures /media/me/4388-E5FE +``` + +Not sure if you already have a sync alias defined? You can list all your currently defined aliases by typing the word alias at the command prompt in your terminal window. + +Like this so much you just want to start using it right away? Open a terminal window and type: + +``` +echo "alias sync='cp -r -u -v'" >> ~/.bash_aliases +``` + +``` +me@mymachine~$ alias + +alias alert='notify-send --urgency=low -i "$([ $? = 0 ] && echo terminal || echo error)" "$(history|tail -n1|sed -e '\''s/^\s*[0-9]\+\s*//;s/[;&|]\s*alert$//'\'')"' + +alias egrep='egrep --color=auto' + +alias fgrep='fgrep --color=auto' + +alias grep='grep --color=auto' + +alias gvm='sdk' + +alias l='ls -CF' + +alias la='ls -A' + +alias ll='ls -alF' + +alias ls='ls --color=auto' + +alias sync='cp -r -u -v' + +me@mymachine:~$ +``` + +### Making versioned backups – Bash function + +The general pattern for making a backup of a file with cp is: + +``` +cp --force --backup=numbered WORKING-FILE BACKED-UP-FILE +``` + +Besides remembering the options to the cp command, we also need to remember to repeat the WORKING-FILE name a second time. But why repeat ourselves when [a Bash function][9] can take care of that overhead for us, like this: + +Again, you can save this to your .bash_aliases file in your home directory. + +``` +function backup { + +    if [ $# -ne 1 ]; then + +        echo "Usage: $0 filename" + +    elif [ -f $1 ] ; then + +        echo "cp --force --backup=numbered $1 $1" + +        cp --force --backup=numbered $1 $1 + +    else + +        echo "$0: $1 is not a file" + +    fi + +} +``` + +The first if statement checks to make sure that only one argument is provided to the function, otherwise printing the correct usage with the echo command. + +The elif statement checks to make sure the argument provided is a file, and if so, it (verbosely) uses the second echo to print the cp command to be used and then executes it. + +If the single argument is not a file, the third echo prints an error message to that effect. + +In my home directory, if I execute the backup command so defined on the file checkCounts.sql, I see that backup creates a file called checkCounts.sql.~1~. If I execute it once more, I see a new file checkCounts.sql.~2~. + +Success! As planned, I can go on editing checkCounts.sql, but if I take a snapshot of it every so often with backup, I can return to the most recent snapshot should I run into trouble. + +At some point, it’s better to start using git for version control, but backup as defined above is a nice cheap tool when you need to create snapshots but you’re not ready for git. + +### Conclusion + +In my last article, I promised you that repetitive tasks can often be easily streamlined through the use of shell scripts, shell functions, and shell aliases. + +Here I’ve shown concrete examples of the use of shell aliases and shell functions to streamline the synchronize and backup functionality of the cp command. If you’d like to learn more about this, check out the two articles cited above: [How to save keystrokes at the command line with alias][10] and [Shell scripting: An introduction to the shift method and custom functions][11], written by my colleagues Greg and Seth, respectively. + + +### About the author + + [![](https://opensource.com/sites/default/files/styles/profile_pictures/public/clh_portrait2.jpg?itok=V1V-YAtY)][13] Chris Hermansen  + + + Engaged in computing since graduating from the University of British Columbia in 1978, I have been a full-time Linux user since 2005 and a full-time Solaris, SunOS and UNIX System V user before that. On the technical side of things, I have spent a great deal of my career doing data analysis; especially spatial data analysis. I have a substantial amount of programming experience in relation to data analysis, using awk, Python, PostgreSQL, PostGIS and lately Groovy. I have also built a few... [more about Chris Hermansen][14] + +-------------------------------------------------------------------------------- + +via: https://opensource.com/article/18/1/two-great-uses-cp-command-update + +作者：[Chris Hermansen][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://opensource.com/users/clhermansen +[1]:https://opensource.com/users/clhermansen +[2]:https://opensource.com/users/clhermansen +[3]:https://opensource.com/user/37806/feed +[4]:https://opensource.com/article/18/1/two-great-uses-cp-command-update?rate=J_7R7wSPbukG9y8jrqZt3EqANfYtVAwZzzpopYiH3C8 +[5]:https://opensource.com/article/18/1/two-great-uses-cp-command-update#comments +[6]:https://www.flickr.com/photos/internetarchivebookimages/14803082483/in/photolist-oy6EG4-pZR3NZ-i6r3NW-e1tJSX-boBtf7-oeYc7U-o6jFKK-9jNtc3-idt2G9-i7NG1m-ouKjXe-owqviF-92xFBg-ow9e4s-gVVXJN-i1K8Pw-4jybMo-i1rsBr-ouo58Y-ouPRzz-8cGJHK-85Evdk-cru4Ly-rcDWiP-gnaC5B-pAFsuf-hRFPcZ-odvBMz-hRCE7b-mZN3Kt-odHU5a-73dpPp-hUaaAi-owvUMK-otbp7Q-ouySkB-hYAgmJ-owo4UZ-giHgqu-giHpNc-idd9uQ-osAhcf-7vxk63-7vwN65-fQejmk-pTcLgA-otZcmj-fj1aSX-hRzHQk-oyeZfR +[7]:https://opensource.com/article/17/7/two-great-uses-cp-command +[8]:https://opensource.com/article/17/5/introduction-alias-command-line-tool +[9]:https://opensource.com/article/17/1/shell-scripting-shift-method-custom-functions +[10]:https://opensource.com/article/17/5/introduction-alias-command-line-tool +[11]:https://opensource.com/article/17/1/shell-scripting-shift-method-custom-functions +[12]:https://opensource.com/tags/linux +[13]:https://opensource.com/users/clhermansen +[14]:https://opensource.com/users/clhermansen diff --git a/sources/tech/20180120 socat as a handler for multiple reverse shells - System Overlord.md b/sources/tech/20180120 socat as a handler for multiple reverse shells - System Overlord.md new file mode 100644 index 0000000000..b57a1e0140 --- /dev/null +++ b/sources/tech/20180120 socat as a handler for multiple reverse shells - System Overlord.md @@ -0,0 +1,66 @@ +socat as a handler for multiple reverse shells · System Overlord +====== + +I was looking for a new way to handle multiple incoming reverse shells. My shells needed to be encrypted and I preferred not to use Metasploit in this case. Because of the way I was deploying my implants, I wasn't able to use separate incoming port numbers or other ways of directing the traffic to multiple listeners. + +Obviously, it's important to keep each reverse shell separated, so I couldn't just have a listener redirecting all the connections to STDIN/STDOUT. I also didn't want to wait for sessions serially - obviously I wanted to be connected to all of my implants simultaneously. (And allow them to disconnect/reconnect as needed due to loss of network connectivity.) + +As I was thinking about the problem, I realized that I basically wanted `tmux` for reverse shells. So I began to wonder if there was some way to connect `openssl s_server` or something similar to `tmux`. Given the limitations of `s_server`, I started looking at `socat`. Despite it's versatility, I've actually only used it once or twice before this, so I spent a fair bit of time reading the man page and the examples. + +I couldn't find a way to get `socat` to talk directly to `tmux` in a way that would spawn each connection as a new window (file descriptors are not passed to the newly-started process in `tmux new-window`), so I ended up with a strange workaround. I feel a little bit like Rube Goldberg inventing C2 software (and I need to get something more permanent and featureful eventually, but this was a quick and dirty PoC), but I've put together a chain of `socat` to get a working solution. + +My implementation works by having a single `socat` process receive the incoming connections (forking on incoming connection), and executing a script that first starts a `socat` instance within tmux, and then another `socat` process to copy from the first to the second over a UNIX domain socket. + +Yes, this is 3 socat processes. It's a little ridiculous, but I couldn't find a better approach. Roughly speaking, the communications flow looks a little like this: +``` +TLS data <--> socat listener <--> script stdio <--> socat <--> unix socket <--> socat in tmux <--> terminal window + +``` + +Getting it started is fairly simple. Begin by generating your SSL certificate. In this case, I'm using a self-signed certificate, but obviously you could go through a commercial CA, Let's Encrypt, etc. +``` +openssl req -newkey rsa:2048 -nodes -keyout server.key -x509 -days 30 -out server.crt +cat server.key server.crt > server.pem + +``` + +Now we will create the script that is run on each incoming connection. This script needs to launch a `tmux` window running a `socat` process copying from a UNIX domain socket to `stdio` (in tmux), and then connecting another `socat` between the `stdio` coming in to the UNIX domain socket. +``` +#!/bin/bash + +SOCKDIR=$(mktemp -d) +SOCKF=${SOCKDIR}/usock + +# Start tmux, if needed +tmux start +# Create window +tmux new-window "socat UNIX-LISTEN:${SOCKF},umask=0077 STDIO" +# Wait for socket +while test ! -e ${SOCKF} ; do sleep 1 ; done +# Use socat to ship data between the unix socket and STDIO. +exec socat STDIO UNIX-CONNECT:${SOCKF} +``` + +The while loop is necessary to make sure that the last `socat` process does not attempt to open the UNIX domain socket before it has been created by the new `tmux` child process. + +Finally, we can launch the `socat` process that will accept the incoming requests (handling all the TLS steps) and execute our per-connection script: +``` +socat OPENSSL-LISTEN:8443,cert=server.pem,reuseaddr,verify=0,fork EXEC:./socatscript.sh + +``` + +This listens on port 8443, using the certificate and private key contained in `server.pem`, performs a `fork()` on accepting each incoming connection (so they do not block each other) and disables certificate verification (since we're not expecting our clients to provide a certificate). On the other side, it launches our script, providing the data from the TLS connection via STDIO. + +At this point, an incoming TLS connection connects, and is passed through our processes to eventually arrive on the `STDIO` of a new window in the running `tmux` server. Each connection gets its own window, allowing us to easily see and manage the connections for our implants. + +-------------------------------------------------------------------------------- + +via: https://systemoverlord.com/2018/01/20/socat-as-a-handler-for-multiple-reverse-shells.html + +作者：[David][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://systemoverlord.com/about diff --git a/sources/tech/20180122 A Simple Command-line Snippet Manager.md b/sources/tech/20180122 A Simple Command-line Snippet Manager.md new file mode 100644 index 0000000000..1c8ef14fb6 --- /dev/null +++ b/sources/tech/20180122 A Simple Command-line Snippet Manager.md @@ -0,0 +1,319 @@ +A Simple Command-line Snippet Manager +====== + +![](https://www.ostechnix.com/wp-content/uploads/2018/01/pet-6-720x340.png) + +We can't remember all the commands, right? Yes. Except the frequently used commands, it is nearly impossible to remember some long commands that we rarely use. That's why we need to some external tools to help us to find the commands when we need them. In the past, we have reviewed two useful utilities named [**" Bashpast"**][1] and [**" Keep"**][2]. Using Bashpast, we can easily bookmark the Linux commands for easier repeated invocation. And, the Keep utility can be used to keep the some important and lengthy commands in your Terminal, so you can use them on demand. Today, we are going to see yet another tool in the series to help you remembering commands. Say hello to **" Pet"**, a simple command-line snippet manager written in **Go** language. + +Using Pet, you can; + + * Register/add your important, long and complex command snippets. + * Search the saved command snippets interactively. + * Run snippets directly without having to type over and over. + * Edit the saved command snippets easily. + * Sync the snippets via Gist. + * Use variables in snippets. + * And more yet to come. + + + +#### Installing Pet CLI Snippet Manager + +Since it is written in Go language, make sure you have installed Go in your system. + +After Go language, grab the latest binaries from [**the releases page**][3]. +``` +wget https://github.com/knqyf263/pet/releases/download/v0.2.4/pet_0.2.4_linux_amd64.zip +``` + +For 32 bit: +``` +wget https://github.com/knqyf263/pet/releases/download/v0.2.4/pet_0.2.4_linux_386.zip +``` + +Extract the downloaded archive: +``` +unzip pet_0.2.4_linux_amd64.zip +``` + +32 bit: +``` +unzip pet_0.2.4_linux_386.zip +``` + +Copy the pet binary file to your PATH (i.e **/usr/local/bin** or the like). +``` +sudo cp pet /usr/local/bin/ +``` + +Finally, make it executable: +``` +sudo chmod +x /usr/local/bin/pet +``` + +If you're using Arch based systems, then you can install it from AUR using any AUR helper tools. + +Using [**Pacaur**][4]: +``` +pacaur -S pet-git +``` + +Using [**Packer**][5]: +``` +packer -S pet-git +``` + +Using [**Yaourt**][6]: +``` +yaourt -S pet-git +``` + +Using [**Yay** :][7] +``` +yay -S pet-git +``` + +Also, you need to install **[fzf][8]** or [**peco**][9] tools to enable interactive search. Refer the official GitHub links to know how to install these tools. + +#### Usage + +Run 'pet' without any arguments to view the list of available commands and general options. +``` +$ pet +pet - Simple command-line snippet manager. + +Usage: + pet [command] + +Available Commands: + configure Edit config file + edit Edit snippet file + exec Run the selected commands + help Help about any command + list Show all snippets + new Create a new snippet + search Search snippets + sync Sync snippets + version Print the version number + +Flags: + --config string config file (default is $HOME/.config/pet/config.toml) + --debug debug mode + -h, --help help for pet + +Use "pet [command] --help" for more information about a command. +``` + +To view the help section of a specific command, run: +``` +$ pet [command] --help +``` + +**Configure Pet** + +It just works fine with default values. However, you can change the default directory to save snippets, choose the selector (fzf or peco) to use, the default text editor to edit snippets, add GIST id details etc. + +To configure Pet, run: +``` +$ pet configure +``` + +This command will open the default configuration in the default text editor (for example **vim** in my case). Change/edit the values as per your requirements. +``` +[General] + snippetfile = "/home/sk/.config/pet/snippet.toml" + editor = "vim" + column = 40 + selectcmd = "fzf" + +[Gist] + file_name = "pet-snippet.toml" + access_token = "" + gist_id = "" + public = false +~ +``` + +**Creating Snippets** + +To create a new snippet, run: +``` +$ pet new +``` + +Add the command and the description and hit ENTER to save it. +``` +Command> echo 'Hell1o, Welcome1 2to OSTechNix4' | tr -d '1-9' +Description> Remove numbers from output. +``` + +[![][10]][11] + +This is a simple command to remove all numbers from the echo command output. You can easily remember it. But, if you rarely use it, you may forgot it completely after few days. Of course we can search the history using "CTRL+r", but "Pet" is much easier. Also, Pet can help you to add any number of entries. + +Another cool feature is we can easily add the previous command. To do so, add the following lines in your **.bashrc** or **.zshrc** file. +``` +function prev() { + PREV=$(fc -lrn | head -n 1) + sh -c "pet new `printf %q "$PREV"`" +} +``` + +Do the following command to take effect the saved changes. +``` +source .bashrc +``` + +Or, +``` +source .zshrc +``` + +Now, run any command, for example: +``` +$ cat Documents/ostechnix.txt | tr '|' '\n' | sort | tr '\n' '|' | sed "s/.$/\\n/g" +``` + +To add the above command, you don't have to use "pet new" command. just do: +``` +$ prev +``` + +Add the description to the command snippet and hit ENTER to save. + +[![][10]][12] + +**List snippets** + +To view the saved snippets, run: +``` +$ pet list +``` + +[![][10]][13] + +**Edit Snippets** + +If you want to edit the description or the command of a snippet, run: +``` +$ pet edit +``` + +This will open all saved snippets in your default text editor. You can edit or change the snippets as you wish. +``` +[[snippets]] + description = "Remove numbers from output." + command = "echo 'Hell1o, Welcome1 2to OSTechNix4' | tr -d '1-9'" + output = "" + +[[snippets]] + description = "Alphabetically sort one line of text" + command = "\t prev" + output = "" +``` + +**Use Tags in snippets** + +To use tags to a snippet, use **-t** flag like below. +``` +$ pet new -t +Command> echo 'Hell1o, Welcome1 2to OSTechNix4' | tr -d '1-9 +Description> Remove numbers from output. +Tag> tr command examples + +``` + +**Execute Snippets** + +To execute a saved snippet, run: +``` +$ pet exec +``` + +Choose the snippet you want to run from the list and hit ENTER to run it. + +[![][10]][14] + +Remember you need to install fzf or peco to use this feature. + +**Search Snippets** + +If you have plenty of saved snippets, you can easily search them using a string or key word like below. +``` +$ pet search +``` + +Enter the search term or keyword to narrow down the search results. + +[![][10]][15] + +**Sync Snippets** + +First, you need to obtain the access token. Go to this link and create access token (only need "gist" scope). + +Configure Pet using command: +``` +$ pet configure +``` + +Set that token to **access_token** in **[Gist]** field. + +After setting, you can upload snippets to Gist like below. +``` +$ pet sync -u +Gist ID: 2dfeeeg5f17e1170bf0c5612fb31a869 +Upload success + +``` + +You can also download snippets on another PC. To do so, edit configuration file and set **Gist ID** to **gist_id** in **[Gist]**. + +Then, download the snippets using command: +``` +$ pet sync +Download success + +``` + +For more details, refer the help section: +``` +pet -h +``` + +Or, +``` +pet [command] -h +``` + +And, that's all. Hope this helps. As you can see, Pet usage is fairly simple and easy to use! If you're having hard time remembering lengthy commands, Pet utility can definitely be useful. + +Cheers! + + + +-------------------------------------------------------------------------------- + +via: https://www.ostechnix.com/pet-simple-command-line-snippet-manager/ + +作者：[SK][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.ostechnix.com/author/sk/ +[1]:https://www.ostechnix.com/bookmark-linux-commands-easier-repeated-invocation/ +[2]:https://www.ostechnix.com/save-commands-terminal-use-demand/ +[3]:https://github.com/knqyf263/pet/releases +[4]:https://www.ostechnix.com/install-pacaur-arch-linux/ +[5]:https://www.ostechnix.com/install-packer-arch-linux-2/ +[6]:https://www.ostechnix.com/install-yaourt-arch-linux/ +[7]:https://www.ostechnix.com/yay-found-yet-another-reliable-aur-helper/ +[8]:https://github.com/junegunn/fzf +[9]:https://github.com/peco/peco +[10]:data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7 +[11]:http://www.ostechnix.com/wp-content/uploads/2018/01/pet-1.png () +[12]:http://www.ostechnix.com/wp-content/uploads/2018/01/pet-2.png () +[13]:http://www.ostechnix.com/wp-content/uploads/2018/01/pet-3.png () +[14]:http://www.ostechnix.com/wp-content/uploads/2018/01/pet-4.png () +[15]:http://www.ostechnix.com/wp-content/uploads/2018/01/pet-5.png () diff --git a/sources/tech/20180122 An overview of the Perl 5 engine.md b/sources/tech/20180122 An overview of the Perl 5 engine.md new file mode 100644 index 0000000000..a26266a39a --- /dev/null +++ b/sources/tech/20180122 An overview of the Perl 5 engine.md @@ -0,0 +1,130 @@ +An overview of the Perl 5 engine +====== + +![](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/camel-perl-lead.png?itok=VyEv-C5o) + +As I described in "[My DeLorean runs Perl][1]," switching to Perl has vastly improved my development speed and possibilities. Here I'll dive deeper into the design of Perl 5 to discuss aspects important to systems programming. + +Some years ago, I wrote "OpenGL bindings for Bash" as sort of a joke. The implementation was simply an X11 program written in C that read OpenGL calls on [stdin][2] (yes, as text) and emitted user input on [stdout][3] . Then I had a littlefile that would declare all the OpenGL functions as Bash functions, which echoed the name of the function into a pipe, starting the GL interpreter process if it wasn't already running. The point of the exercise was to show that OpenGL (the 1.4 API, not the newer shader stuff) could render a lot of graphics with just a few calls per frame by using GL display lists. The OpenGL library did all the heavy lifting, and Bash just printed a few dozen lines of text per frame. + +In the end though, Bash is a really horrible [glue language][4], both from high overhead and limited available operations and syntax. [Perl][5], on the other hand, is a great glue language. + +### Syntax aside... + +If you're not a regular Perl user, the first thing you probably notice is the syntax. + +Perl 5 is built on a long legacy of awkward syntax, but more recent versions have removed the need for much of the punctuation. The remaining warts can mostly be avoided by choosing modules that give you domain-specific "syntactic sugar," which even alter the Perl syntax as it is parsed. This is in stark contrast to most other languages, where you are stuck with the syntax you're given, and infinitely more flexible than C's macros. Combined with Perl's powerful sparse-syntax operators, like `map`, `grep`, `sort`, and similar user-defined operators, I can almost always write complex algorithms more legibly and with less typing using Perl than with JavaScript, PHP, or any compiled language. + +So, because syntax is what you make of it, I think the underlying machine is the most important aspect of the language to consider. Perl 5 has a very capable engine, and it differs in interesting and useful ways from other languages. + +### A layer above C + +I don't recommend anyone start working with Perl by looking at the interpreter's internal API, but a quick description is useful. One of the main problems we deal with in the world of C is acquiring and releasing memory while also supporting control flow through a chain of function calls. C has a rough ability to throw exceptions using `longjmp`, but it doesn't do any cleanup for you, so it is almost useless without a framework to manage resources. The Perl interpreter is exactly this sort of framework. + +Perl provides a stack of variables independent from C's stack of function calls on which you can mark the logical boundaries of a Perl scope. There are also API calls you can use to allocate memory, Perl variables, etc., and tell Perl to automatically free them at the end of the Perl scope. Now you can make whatever C calls you like, "die" out of the middle of them, and let Perl clean everything up for you. + +Although this is a really unconventional perspective, I bring it up to emphasize that Perl sits on top of C and allows you to use as much or as little interpreted overhead as you like. Perl's internal API is certainly not as nice as C++ for general programming, but C++ doesn't give you an interpreted language on top of your work when you're done. I've lost track of the number of times that I wanted reflective capability to inspect or alter my C++ objects, and following that rabbit hole has derailed more than one of my personal projects. + +### Lisp-like functions + +Perl functions take a list of arguments. The downside is that you have to do argument count and type checking at runtime. The upside is you don't end up doing that much, because you can just let the interpreter's own runtime check catch those mistakes. You can also create the effect of C++'s overloaded functions by inspecting the arguments you were given and behaving accordingly. + +Because arguments are a list, and return values are a list, this encourages [Lisp-style programming][6], where you use a series of functions to filter a list of data elements. This "piping" or "streaming" effect can result in some really complicated loops turning into a single line of code. + +Every function is available to the language as a `coderef` that can be passed around in variables, including anonymous closure functions. Also, I find `sub {}` more convenient to type than JavaScript's `function(){}` or C++11's `[&](){}`. + +### Generic data structures + +The variables in Perl are either "scalars," references, arrays, or "hashes" ... or some other stuff that I'll skip. + +Scalars act as a string/integer/float hybrid and are automatically typecast as needed for the purpose you are using them. In other words, instead of determining the operation by the type of variable, the type of operator determines how the variable should be interpreted. This is less efficient than if the language knows the type in advance, but not as inefficient as, for example, shell scripting because Perl caches the type conversions. + +Perl scalars may contain null characters, so they are fully usable as buffers for binary data. The scalars are mutable and copied by value, but optimized with copy-on-write, and substring operations are also optimized. Strings support unicode characters but are stored efficiently as normal bytes until you append a codepoint above 255. + +References (which are considered scalars as well) hold a reference to any other variable; `hashrefs` and `arrayrefs` are most common, along with the `coderefs` described above. + +Arrays are simply a dynamic-length array of scalars (or references). + +Hashes (i.e., dictionaries, maps, or whatever you want to call them) are a performance-tuned hash table implementation where every key is a string and every value is a scalar (or reference). Hashes are used in Perl in the same way structs are used in C. Clearly a hash is less efficient than a struct, but it keeps things generic so tasks that require dozens of lines of code in other languages can become one-liners in Perl. For instance, you can dump the contents of a hash into a list of (key, value) pairs or reconstruct a hash from such a list as a natural part of the Perl syntax. + +### Object model + +Any reference can be "blessed" to make it into an object, granting it a multiple-inheritance method-dispatch table. The blessing is simply the name of a package (namespace), and any function in that namespace becomes an available method of the object. The inheritance tree is defined by variables in the package. As a result, you can make modifications to classes or class hierarchies or create new classes on the fly with simple data edits, rather than special keywords or built-in reflection APIs. By combining this with Perl's `local` keyword (where changes to a global are automatically undone at the end of the current scope), you can even make temporary changes to class methods or inheritance! + +Perl objects only have methods, so attributes are accessed via accessors like the canonical Java `get_` and `set_` methods. Perl authors usually combine them into a single method of just the attribute name and differentiate `get` from `set` by whether a parameter was given. + +You can also "re-bless" objects from one class to another, which enables interesting tricks not available in most other languages. Consider state machines, where each method would normally start by checking the object's current state; you can avoid that in Perl by swapping the method table to one that matches the object's state. + +### Visibility + +While other languages spend a bunch of effort on access rules between classes, Perl adopted a simple "if the name begins with underscore, don't touch it unless it's yours" convention. Although I can see how this could be a problem with an undisciplined software team, it has worked great in my experience. The only thing C++'s `private` keyword ever did for me was impair my debugging efforts, yet it felt dirty to make everything `public`. Perl removes my guilt. + +Likewise, an object provides methods, but you can ignore them and just access the underlying Perl data structure. This is another huge boost for debugging. + +### Garbage collection via reference counting + +Although [reference counting][7] is a rather leak-prone form of memory management (it doesn't detect cycles), it has a few upsides. It gives you deterministic destruction of your objects, like in C++, and never interrupts your program with a surprise garbage collection. It strongly encourages module authors to use a tree-of-objects pattern, which I much prefer vs. the tangle-of-objects pattern often seen in Java and JavaScript. (I've found trees to be much more easily tested with unit tests.) But, if you need a tangle of objects, Perl does offer "weak" references, which won't be considered when deciding if it's time to garbage-collect something. + +On the whole, the only time this ever bites me is when making heavy use of closures for event-driven callbacks. It's easy to have an object hold a reference to an event handle holding a reference to a callback that references the containing object. Again, weak references solve this, but it's an extra thing to be aware of that JavaScript or Python don't make you worry about. + +### Parallelism + +The Perl interpreter is a single thread, although modules written in C can use threads of their own internally, and Perl often includes support for multiple interpreters within the same process. + +Although this is a large limitation, knowing that a data structure will only ever be touched by one thread is nice, and it means you don't need locks when accessing them from C code. Even in Java, where locking is built into the syntax in convenient ways, it can be a real time sink to reason through all the ways that threads can interact (and especially annoying that they force you to deal with that in every GUI program you write). + +There are several event libraries available to assist in writing event-driven callback programs in the style of Node.js to avoid the need for threads. + +### Access to C libraries + +Aside from directly writing your own C extensions via Perl's [XS][8] system, there are already lots of common C libraries wrapped for you and available on Perl's [CPAN][9] repository. There is also a great module, [Inline::C][10], that takes most of the pain out of bridging between Perl and C, to the point where you just paste C code into the middle of a Perl module. (It compiles the first time you run it and caches the .so shared object file for subsequent runs.) You still need to learn some of the Perl interpreter API if you want to manipulate the Perl stack or pack/unpack Perl's variables other than your C function arguments and return value. + +### Memory usage + +Perl can use a surprising amount of memory, especially if you make use of heavyweight libraries and create thousands of objects, but with the size of today's systems it usually doesn't matter. It also isn't much worse than other interpreted systems. My personal preference is to only use lightweight libraries, which also generally improve performance. + +### Startup speed + +The Perl interpreter starts in under five milliseconds on modern hardware. If you take care to use only lightweight modules, you can use Perl for anything you might have used Bash for, like `hotplug` scripts. + +### Regex implementation + +Perl provides the mother of all regex implementations... but you probably already knew that. Regular expressions are built into Perl's syntax rather than being an object-oriented or function-based API; this helps encourage their use for any text processing you might need to do. + +### Ubiquity and stability + +Perl 5 is installed on just about every modern Unix system, and the CPAN module collection is extensive and easy to install. There's a production-quality module for almost any task, with solid test coverage and good documentation. + +Perl 5 has nearly complete backward compatibility across two decades of releases. The community has embraced this as well, so most of CPAN is pretty stable. There's even a crew of testers who run unit tests on all of CPAN on a regular basis to help detect breakage. + +The toolchain is also pretty solid. The documentation syntax (POD) is a little more verbose than I'd like, but it yields much more useful results than [doxygen][11] or [Javadoc][12]. You can run `perldoc FILENAME` to instantly see the documentation of the module you're writing. `perldoc Module::Name` shows you the specific documentation for the version of the module that you would load from your `include` path and can likewise show you the source code of that module without needing to browse deep into your filesystem. + +The testcase system (the `prove` command and Test Anything Protocol, or TAP) isn't specific to Perl and is extremely simple to work with (as opposed to unit testing based around language-specific object-oriented structure, or XML). Modules like `Test::More` make writing the test cases so easy that you can write a test suite in about the same time it would take to test your module once by hand. The testing effort barrier is so low that I've started using TAP and the POD documentation style for my non-Perl projects as well. + +### In summary + +Perl 5 still has a lot to offer despite the large number of newer languages competing with it. The frontend syntax hasn't stopped evolving, and you can improve it however you like with custom modules. The Perl 5 engine is capable of handling most programming problems you can throw at it, and it is even suitable for low-level work as a "glue" layer on top of C libraries. Once you get really familiar with it, it can even be an environment for developing C code. + +-------------------------------------------------------------------------------- + +via: https://opensource.com/article/18/1/why-i-love-perl-5 + +作者：[Michael Conrad][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://opensource.com/users/nerdvana +[1]:https://opensource.com/article/17/12/my-delorean-runs-perl +[2]:https://en.wikipedia.org/wiki/Standard_streams#Standard_input_(stdin) +[3]:https://en.wikipedia.org/wiki/Standard_streams#Standard_output_(stdout) +[4]:https://www.techopedia.com/definition/19608/glue-language +[5]:https://www.perl.org/ +[6]:https://en.wikipedia.org/wiki/Lisp_(programming_language) +[7]:https://en.wikipedia.org/wiki/Reference_counting +[8]:https://en.wikipedia.org/wiki/XS_(Perl) +[9]:https://www.cpan.org/ +[10]:https://metacpan.org/pod/distribution/Inline-C/lib/Inline/C.pod +[11]:http://www.stack.nl/~dimitri/doxygen/ +[12]:http://www.oracle.com/technetwork/java/javase/documentation/index-jsp-135444.html diff --git a/sources/tech/20180122 Ick- a continuous integration system.md b/sources/tech/20180122 Ick- a continuous integration system.md new file mode 100644 index 0000000000..4620e2c036 --- /dev/null +++ b/sources/tech/20180122 Ick- a continuous integration system.md @@ -0,0 +1,75 @@ +Ick: a continuous integration system +====== +**TL;DR:** Ick is a continuous integration or CI system. See for more information. + +More verbose version follows. + +### First public version released + +The world may not need yet another continuous integration system (CI), but I do. I've been unsatisfied with the ones I've tried or looked at. More importantly, I am interested in a few things that are more powerful than what I've ever even heard of. So I've started writing my own. + +My new personal hobby project is called ick. It is a CI system, which means it can run automated steps for building and testing software. The home page is at , and the [download][1] page has links to the source code and .deb packages and an Ansible playbook for installing it. + +I have now made the first publicly advertised release, dubbed ALPHA-1, version number 0.23. It is of alpha quality, and that means it doesn't have all the intended features and if any of the features it does have work, you should consider yourself lucky. + +### Invitation to contribute + +Ick has so far been my personal project. I am hoping to make it more than that, and invite contributions. See the [governance][2] page for the constitution, the [getting started][3] page for tips on how to start contributing, and the [contact][4] page for how to get in touch. + +### Architecture + +Ick has an architecture consisting of several components that communicate over HTTPS using RESTful APIs and JSON for structured data. See the [architecture][5] page for details. + +### Manifesto + +Continuous integration (CI) is a powerful tool for software development. It should not be tedious, fragile, or annoying. It should be quick and simple to set up, and work quietly in the background unless there's a problem in the code being built and tested. + +A CI system should be simple, easy, clear, clean, scalable, fast, comprehensible, transparent, reliable, and boost your productivity to get things done. It should not be a lot of effort to set up, require a lot of hardware just for the CI, need frequent attention for it to keep working, and developers should never have to wonder why something isn't working. + +A CI system should be flexible to suit your build and test needs. It should support multiple types of workers, as far as CPU architecture and operating system version are concerned. + +Also, like all software, CI should be fully and completely free software and your instance should be under your control. + +(Ick is little of this yet, but it will try to become all of it. In the best possible taste.) + +### Dreams of the future + +In the long run, I would ick to have features like ones described below. It may take a while to get all of them implemented. + + * A build may be triggered by a variety of events. Time is an obvious event, as is source code repository for the project changing. More powerfully, any build dependency changing, regardless of whether the dependency comes from another project built by ick, or a package from, say, Debian: ick should keep track of all the packages that get installed into the build environment of a project, and if any of their versions change, it should trigger the project build and tests again. + + * Ick should support building in (or against) any reasonable target, including any Linux distribution, any free operating system, and any non-free operating system that isn't brain-dead. + + * Ick should manage the build environment itself, and be able to do builds that are isolated from the build host or the network. This partially works: one can ask ick to build a container and run a build in the container. The container is implemented using systemd-nspawn. This can be improved upon, however. (If you think Docker is the only way to go, please contribute support for that.) + + * Ick should support any workers that it can control over ssh or a serial port or other such neutral communication channel, without having to install an agent of any kind on them. Ick won't assume that it can have, say, a full Java run time, so that the worker can be, say, a micro controller. + + * Ick should be able to effortlessly handle very large numbers of projects. I'm thinking here that it should be able to keep up with building everything in Debian, whenever a new Debian source package is uploaded. (Obviously whether that is feasible depends on whether there are enough resources to actually build things, but ick itself should not be the bottleneck.) + + * Ick should optionally provision workers as needed. If all workers of a certain type are busy, and ick's been configured to allow using more resources, it should do so. This seems like it would be easy to do with virtual machines, containers, cloud providers, etc. + + * Ick should be flexible in how it can notify interested parties, particularly about failures. It should allow an interested party to ask to be notified over IRC, Matrix, Mastodon, Twitter, email, SMS, or even by a phone call and speech syntethiser. "Hello, interested party. It is 04:00 and you wanted to be told when the hello package has been built for RISC-V." + + + + +### Please give feedback + +If you try ick, or even if you've just read this far, please share your thoughts on it. See the [contact][4] page for where to send it. Public feedback is preferred over private, but if you prefer private, that's OK too. + +-------------------------------------------------------------------------------- + +via: https://blog.liw.fi/posts/2018/01/22/ick_a_continuous_integration_system/ + +作者：[Lars Wirzenius][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://blog.liw.fi/ +[1]:http://ick.liw.fi/download/ +[2]:http://ick.liw.fi/governance/ +[3]:http://ick.liw.fi/getting-started/ +[4]:http://ick.liw.fi/contact/ +[5]:http://ick.liw.fi/architecture/ diff --git a/sources/tech/20180122 Linux rm Command Explained for Beginners (8 Examples).md b/sources/tech/20180122 Linux rm Command Explained for Beginners (8 Examples).md new file mode 100644 index 0000000000..4e5e002754 --- /dev/null +++ b/sources/tech/20180122 Linux rm Command Explained for Beginners (8 Examples).md @@ -0,0 +1,174 @@ +Translating by yizhuoyan + +Linux rm Command Explained for Beginners (8 Examples) +====== + +Deleting files is a fundamental operation, just like copying files or renaming/moving them. In Linux, there's a dedicated command - dubbed **rm** \- that lets you perform all deletion-related operations. In this tutorial, we will discuss the basics of this tool along with some easy to understand examples. + +But before we do that, it's worth mentioning that all examples mentioned in the article have been tested on Ubuntu 16.04 LTS. + +#### Linux rm command + +So in layman's terms, we can simply say the rm command is used for removing/deleting files and directories. Following is the syntax of the command: + +``` +rm [OPTION]... [FILE]... +``` + +And here's how the tool's man page describes it: +``` +This manual page documents the GNU version of rm. rm removes each specified file. By default, it +does not remove directories. + +If  the  -I or --interactive=once option is given, and there are more than three files or the -r, +-R, or --recursive are given, then rm prompts the user for whether to proceed with the entire +operation. If the response is not affirmative, the entire command is aborted. + +Otherwise, if a file is unwritable, standard input is a terminal, and the -f or --force option is +not given, or the -i or --interactive=always option is given, rm prompts the user for whether to +remove the file. If the response is not affirmative, the file is skipped. +``` + +The following Q&A-styled examples will give you a better idea on how the tool works. + +#### Q1. How to remove files using rm command? + +That's pretty easy and straightforward. All you have to do is to pass the name of the files (along with paths if they are not in the current working directory) as input to the rm command. + +``` +rm [filename] +``` + +For example: + +``` +rm testfile.txt +``` + +[![How to remove files using rm command][1]][2] + +#### Q2. How to remove directories using rm command? + +If you are trying to remove a directory, then you need to use the **-r** command line option. Otherwise, rm will throw an error saying what you are trying to delete is a directory. + +``` +rm -r [dir name] +``` + +For example: + +``` +rm -r testdir +``` + +[![How to remove directories using rm command][3]][4] + +#### Q3. How to make rm prompt before every removal? + +If you want rm to prompt before each delete action it performs, then use the **-i** command line option. + +``` +rm -i [file or dir] +``` + +For example, suppose you want to delete a directory 'testdir' and all its contents, but want rm to prompt before every deletion, then here's how you can do that: + +``` +rm -r -i testdir +``` + +[![How to make rm prompt before every removal][5]][6] + +#### Q4. How to force rm to ignore nonexistent files? + +The rm command lets you know through an error message if you try deleting a non-existent file or directory. + +[![Linux rm command example][7]][8] + +However, if you want, you can make rm suppress such error/notifications - all you have to do is to use the **-f** command line option. + +``` +rm -f [filename] +``` + +[![How to force rm to ignore nonexistent files][9]][10] + +#### Q5. How to make rm prompt only in some scenarios? + +There exists a command line option **-I** , which when used, makes sure the command only prompts once before removing more than three files, or when removing recursively. + +For example, the following screenshot shows this option in action - there was no prompt when two files were deleted, but the command prompted when more than three files were deleted. + +[![How to make rm prompt only in some scenarios][11]][12] + +#### Q6. How rm works when dealing with root directory? + +Of course, deleting root directory is the last thing a Linux user would want. That's why, the rm command doesn't let you perform a recursive delete operation on this directory by default. + +[![How rm works when dealing with root directory][13]][14] + +However, if you want to go ahead with this operation for whatever reason, then you need to tell this to rm by using the **\--no-preserve-root** option. When this option is enabled, rm doesn't treat the root directory (/) specially. + +In case you want to know the scenarios in which a user might want to delete the root directory of their system, head [here][15]. + +#### Q7. How to make rm only remove empty directories? + +In case you want to restrict rm's directory deletion ability to only empty directories, then you can use the -d command line option. + +``` +rm -d [dir] +``` + +The following screenshot shows the -d command line option in action - only empty directory got deleted. + +[![How to make rm only remove empty directories][16]][17] + +#### Q8. How to force rm to emit details of operation it is performing? + +If you want rm to display detailed information of the operation being performed, then this can be done by using the **-v** command line option. + +``` +rm -v [file or directory name] +``` + +For example: + +[![How to force rm to emit details of operation it is performing][18]][19] + +#### Conclusion + +Given the kind of functionality it offers, rm is one of the most frequently used commands in Linux (like [cp][20] and mv). Here, in this tutorial, we have covered almost all major command line options this tool provides. rm has a bit of learning curve associated with, so you'll have to spent some time practicing its options before you start using the tool in your day to day work. For more information, head to the command's [man page][21]. + + +-------------------------------------------------------------------------------- + +via: https://www.howtoforge.com/linux-rm-command/ + +作者：[Himanshu Arora][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.howtoforge.com +[1]:https://www.howtoforge.com/images/command-tutorial/rm-basic-usage.png +[2]:https://www.howtoforge.com/images/command-tutorial/big/rm-basic-usage.png +[3]:https://www.howtoforge.com/images/command-tutorial/rm-r.png +[4]:https://www.howtoforge.com/images/command-tutorial/big/rm-r.png +[5]:https://www.howtoforge.com/images/command-tutorial/rm-i-option.png +[6]:https://www.howtoforge.com/images/command-tutorial/big/rm-i-option.png +[7]:https://www.howtoforge.com/images/command-tutorial/rm-non-ext-error.png +[8]:https://www.howtoforge.com/images/command-tutorial/big/rm-non-ext-error.png +[9]:https://www.howtoforge.com/images/command-tutorial/rm-f-option.png +[10]:https://www.howtoforge.com/images/command-tutorial/big/rm-f-option.png +[11]:https://www.howtoforge.com/images/command-tutorial/rm-I-option.png +[12]:https://www.howtoforge.com/images/command-tutorial/big/rm-I-option.png +[13]:https://www.howtoforge.com/images/command-tutorial/rm-root-default.png +[14]:https://www.howtoforge.com/images/command-tutorial/big/rm-root-default.png +[15]:https://superuser.com/questions/742334/is-there-a-scenario-where-rm-rf-no-preserve-root-is-needed +[16]:https://www.howtoforge.com/images/command-tutorial/rm-d-option.png +[17]:https://www.howtoforge.com/images/command-tutorial/big/rm-d-option.png +[18]:https://www.howtoforge.com/images/command-tutorial/rm-v-option.png +[19]:https://www.howtoforge.com/images/command-tutorial/big/rm-v-option.png +[20]:https://www.howtoforge.com/linux-cp-command/ +[21]:https://linux.die.net/man/1/rm diff --git a/sources/tech/20180123 Installing Awstat for analyzing Apache logs.md b/sources/tech/20180123 Installing Awstat for analyzing Apache logs.md new file mode 100644 index 0000000000..b635417a47 --- /dev/null +++ b/sources/tech/20180123 Installing Awstat for analyzing Apache logs.md @@ -0,0 +1,117 @@ +Installing Awstat for analyzing Apache logs +====== +AWSTAT is free an very powerful log analyser tool for apache log files. After analyzing logs from apache, it present them in easy to understand graphical format. Awstat is short for Advanced Web statistics & it works on command line interface or on CGI. + +In this tutorial, we will be installing AWSTAT on our Centos 7 machine for analyzing apache logs. + +( **Recommended read** :[ **Scheduling important jobs with crontab**][1]) + +### Pre-requisites + + **1-** A website hosted on apache web server, to create one read below mentioned tutorials on apache web servers, + +( **Recommended reads** - [**installing Apache**][2], [**Securing apache with SSL cert**][3] & **hardening tips for apache** ) + + **2-** Epel repository enabled on the system, as Awstat packages are not available on default repositories. To enable epel-repo , run + +``` +$ rpm -Uvh https://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-10.noarch.rpm +``` + +### Installing Awstat + +Once the epel-repository has been enabled on the system, awstat can be installed by running, + +``` + $ yum install awstat +``` + +When awstat is installed, it creates a file for apache at '/etc/httpd/conf.d/awstat.conf' with some configurations. These configurations are good to be used incase web server &awstat are configured on the same machine but if awstat is on different machine than the webserver, then some changes are to be made to the file. + +#### Configuring Apache for Awstat + +To configure awstat for a remote web server, open /etc/httpd/conf.d/awstat.conf, & update the parameter 'Allow from' with the IP address of the web server + +``` +$ vi /etc/httpd/conf.d/awstat.conf + + +Options None +AllowOverride None + +# Apache 2.4 +Require local + + +# Apache 2.2 +Order allow,deny +Allow from 127.0.0.1 +Allow from 192.168.1.100 + + +``` + +Save the file & restart the apache services to implement the changes, + +``` + $ systemctl restart httpd +``` + +#### Configuring AWSTAT + +For every website that we add to awstat, a different configuration file needs to be created with the website information . An example file is created in folder '/etc/awstats' by the name 'awstats.localhost.localdomain.conf', we can make copies of it & configure our website with this, + +``` +$ cd /etc/awstats +$ cp awstats.localhost.localdomain.conf awstats.linuxtechlab.com.conf +``` + +Now open the file & edit the following three parameters to match your website, + +``` +$ vi awstats.linuxtechlab.com.conf + +LogFile="/var/log/httpd/access.log" +SiteDomain="linuxtechlab.com" +HostAliases=www.linuxtechlab.com localhost 127.0.0.1 +``` + +Last step is to update the configuration file, which can be done executing the command below, + +``` +/usr/share/awstats/wwwroot/cgi-bin/awstats.pl -config=linuxtechlab.com -update +``` + +#### Checking the awstat page + +To test/check the awstat page, open web-browser & enter the following URL in the address bar, +**https://linuxtechlab.com/awstats/awstats.pl?config=linuxtechlab.com** + +![awstat][5] + +**Note-** we can also schedule a cron job to update the awstat on regular basis. An example for the crontab + +``` +$ crontab -e +0 1 * * * /usr/share/awstats/wwwroot/cgi-bin/awstats.pl -config=linuxtechlab.com–update +``` + +We now end our tutorial on installing Awstat for analyzing apache logs, please leave your comments/queries in the comment box below. + + +-------------------------------------------------------------------------------- + +via: http://linuxtechlab.com/installing-awstat-analyzing-apache-logs/ + +作者：[SHUSAIN][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://linuxtechlab.com/author/shsuain/ +[1]:http://linuxtechlab.com/scheduling-important-jobs-crontab/ +[2]:http://linuxtechlab.com/beginner-guide-configure-apache/ +[3]:http://linuxtechlab.com/create-ssl-certificate-apache-server/ +[4]:https://i1.wp.com/linuxtechlab.com/wp-content/plugins/a3-lazy-load/assets/images/lazy_placeholder.gif?resize=602%2C312 +[5]:https://i0.wp.com/linuxtechlab.com/wp-content/uploads/2017/04/awstat.jpg?resize=602%2C312 diff --git a/sources/tech/20180123 Never miss a Magazine-s article, build your own RSS notification system.md b/sources/tech/20180123 Never miss a Magazine-s article, build your own RSS notification system.md new file mode 100644 index 0000000000..8794ca611a --- /dev/null +++ b/sources/tech/20180123 Never miss a Magazine-s article, build your own RSS notification system.md @@ -0,0 +1,170 @@ +Never miss a Magazine's article, build your own RSS notification system +====== + +![](https://fedoramagazine.org/wp-content/uploads/2018/01/learn-python-rss-notifier.png-945x400.jpg) + +Python is a great programming language to quickly build applications that make our life easier. In this article we will learn how to use Python to build a RSS notification system, the goal being to have fun learning Python using Fedora. If you are looking for a complete RSS notifier application, there are a few already packaged in Fedora. + +### Fedora and Python - getting started + +Python 3.6 is available by default in Fedora, that includes Python's extensive standard library. The standard library provides a collection of modules which make some tasks simpler for us. For example, in our case we will use the [**sqlite3**][1] module to create, add and read data from a database. In the case where a particular problem we are trying to solve is not covered by the standard library, the chance is that someone has already developed a module for everyone to use. The best place to search for such modules is the Python Package Index known as [PyPI][2]. In our example we are going to use the [**feedparser**][3] to parse an RSS feed. + +Since **feedparser** is not in the standard library, we have to install it in our system. Luckily for us there is an rpm package in Fedora, so the installation of **feedparser** is as simple as: +``` +$ sudo dnf install python3-feedparser +``` + +We now have everything we need to start coding our application. + +### Storing the feed data + +We need to store data from the articles that have already been published so that we send a notification only for new articles. The data we want to store will give us a unique way to identify an article. Therefore we will store the **title** and the **publication date** of the article. + +So let's create our database using python **sqlite3** module and a simple SQL query. We are also adding the modules we are going to use later ( **feedparser** , **smtplib** and **email** ). + +#### Creating the Database +``` +#!/usr/bin/python3 +import sqlite3 +import smtplib +from email.mime.text import MIMEText + +import feedparser + +db_connection = sqlite3.connect('/var/tmp/magazine_rss.sqlite') +db = db_connection.cursor() +db.execute(' CREATE TABLE IF NOT EXISTS magazine (title TEXT, date TEXT)') + +``` + +These few lines of code create a new sqlite database stored in a file called 'magazine_rss.sqlite', and then create a new table within the database called 'magazine'. This table has two columns - 'title' and 'date' - that can store data of the type TEXT, which means that the value of each column will be a text string. + +#### Checking the Database for old articles + +Since we only want to add new articles to our database we need a function that will check if the article we get from the RSS feed is already in our database or not. We will use it to decide if we should send an email notification (new article) or not (old article). Ok let's code this function. +``` +def article_is_not_db(article_title, article_date): + """ Check if a given pair of article title and date + is in the database. + Args: + article_title (str): The title of an article + article_date (str): The publication date of an article + Return: + True if the article is not in the database + False if the article is already present in the database + """ + db.execute("SELECT * from magazine WHERE title=? AND date=?", (article_title, article_date)) + if not db.fetchall(): + return True + else: + return False +``` + +The main part of this function is the SQL query we execute to search through the database. We are using a SELECT instruction to define which column of our magazine table we will run the query on. We are using the 0_sync_master.sh 1_add_new_article_manual.sh 1_add_new_article_newspaper.sh 2_start_translating.sh 3_continue_the_work.sh 4_finish.sh 5_pause.sh base.sh env format.test lctt.cfg parse_url_by_manual.sh parse_url_by_newspaper.py parse_url_by_newspaper.sh README.org reformat.sh symbol to select all columns ( title and date). Then we ask to select only the rows of the table WHERE the article_title and article_date string are equal to the value of the title and date column. + +To finish, we have a simple logic that will return True if the query did not return any results and False if the query found an article in database matching our title, date pair. + +#### Adding a new article to the Database + +Now we can code the function to add a new article to the database. +``` +def add_article_to_db(article_title, article_date): + """ Add a new article title and date to the database + Args: + article_title (str): The title of an article + article_date (str): The publication date of an article + """ + db.execute("INSERT INTO magazine VALUES (?,?)", (article_title, article_date)) + db_connection.commit() +``` + +This function is straight forward, we are using a SQL query to INSERT a new row INTO the magazine table with the VALUES of the article_title and article_date. Then we commit the change to make it persistent. + +That's all we need from the database's point of view, let's look at the notification system and how we can use python to send emails. + +### Sending an email notification + +Let's create a function to send an email using the python standard library module **smtplib.** We are also using the **email** module from the standard library to format our email message. +``` +def send_notification(article_title, article_url): + """ Add a new article title and date to the database + + Args: + article_title (str): The title of an article + article_url (str): The url to access the article + """ + + smtp_server = smtplib.SMTP('smtp.gmail.com', 587) + smtp_server.ehlo() + smtp_server.starttls() + smtp_server.login('your_email@gmail.com', '123your_password') + msg = MIMEText(f'\nHi there is a new Fedora Magazine article : {article_title}. \nYou can read it here {article_url}') + msg['Subject'] = 'New Fedora Magazine Article Available' + msg['From'] = 'your_email@gmail.com' + msg['To'] = 'destination_email@gmail.com' + smtp_server.send_message(msg) + smtp_server.quit() +``` + +In this example I am using the Google mail smtp server to send an email, but this will work with any email services that provides you with a SMTP server. Most of this function is boilerplate needed to configure the access to the smtp server. You will need to update the code with your email address and credentials. + +If you are using 2 Factor Authentication with your gmail account you can setup a password app that will give you a unique password to use for this application. Check out this help [page][4]. + +### Reading Fedora Magazine RSS feed + +We now have functions to store an article in the database and send an email notification, let's create a function that parses the Fedora Magazine RSS feed and extract the articles' data. +``` +def read_article_feed(): + """ Get articles from RSS feed """ + feed = feedparser.parse('https://fedoramagazine.org/feed/') + for article in feed['entries']: + if article_is_not_db(article['title'], article['published']): + send_notification(article['title'], article['link']) + add_article_to_db(article['title'], article['published']) + +if __name__ == '__main__': + read_article_feed() + db_connection.close() +``` + +Here we are making use of the **feedparser.parse** function. The function returns a dictionary representation of the RSS feed, for the full reference of the representation you can consult **feedparser** 's [documentation][5]. + +The RSS feed parser will return the last 10 articles as entries and then we extract the following information: the title, the link and the date the article was published. As a result, we can now use the functions we have previously defined to check if the article is not in the database, then send a notification email and finally, add the article to our database. + +The last if statement is used to execute our read_article_feed function and then close the database connection when we execute our script. + +### Running our script + +Finally, to run our script we need to give the correct permission to the file. Next, we make use of the **cron** utility to automatically execute our script every hour (1 minute past the hour). **cron** is a job scheduler that we can use to run a task at a fixed time. +``` +$ chmod a+x my_rss_notifier.py +$ sudo cp my_rss_notifier.py /etc/cron.hourly +``` + +To keep this tutorial simple, we are using the cron.hourly directory to execute the script every hours, I you wish to learn more about **cron** and how to configure the **crontab,** please read **cron 's** wikipedia [page][6]. + +### Conclusion + +In this tutorial we have learned how to use Python to create a simple sqlite database, parse an RSS feed and send emails. I hope that this showed you how you can easily build your own application using Python and Fedora. + +The script is available on github [here][7]. + + +-------------------------------------------------------------------------------- + +via: https://fedoramagazine.org/never-miss-magazines-article-build-rss-notification-system/ + +作者：[Clément Verna][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://fedoramagazine.org +[1]:https://docs.python.org/3/library/sqlite3.html +[2]:https://pypi.python.org/pypi +[3]:https://pypi.python.org/pypi/feedparser/5.2.1 +[4]:https://support.google.com/accounts/answer/185833?hl=en +[5]:https://pythonhosted.org/feedparser/reference.html +[6]:https://en.wikipedia.org/wiki/Cron +[7]:https://github.com/cverna/rss_feed_notifier diff --git a/sources/tech/20180125 BUILDING A FULL-TEXT SEARCH APP USING DOCKER AND ELASTICSEARCH.md b/sources/tech/20180125 BUILDING A FULL-TEXT SEARCH APP USING DOCKER AND ELASTICSEARCH.md new file mode 100644 index 0000000000..d064544e1f --- /dev/null +++ b/sources/tech/20180125 BUILDING A FULL-TEXT SEARCH APP USING DOCKER AND ELASTICSEARCH.md @@ -0,0 +1,1382 @@ +BUILDING A FULL-TEXT SEARCH APP USING DOCKER AND ELASTICSEARCH +============================================================ + + _How does Wikipedia sort though 5+ million articles to find the most relevant one for your research?_ + + _How does Facebook find the friend who you're looking for (and whose name you've misspelled), across a userbase of 2+ billion people?_ + + _How does Google search the entire internet for webpages relevant to your vague, typo-filled search query?_ + +In this tutorial, we'll walk through setting up our own full-text search application (of an admittedly lesser complexity than the systems in the questions above). Our example app will provide a UI and API to search the complete texts of 100 literary classics such as  _Peter Pan_ ,  _Frankenstein_ , and  _Treasure Island_ . + +You can preview a completed version of the tutorial app here - [https://search.patricktriest.com][6] + +![preview webapp](https://cdn.patricktriest.com/blog/images/posts/elastic-library/sample_4_0.png) + +The source code for the application is 100% open-source and can be found at the GitHub repository here - [https://github.com/triestpa/guttenberg-search][7] + +Adding fast, flexible full-text search to apps can be a challenge. Most mainstream databases, such as [PostgreSQL][8] and [MongoDB][9], offer very basic text searching capabilities due to limitations on their existing query and index structures. In order to implement high quality full-text search, a separate datastore is often the best option. [Elasticsearch][10] is a leading open-source datastore that is optimized to perform incredibly flexible and fast full-text search. + +We'll be using [Docker][11] to setup our project environment and dependencies. Docker is a containerization engine used by the likes of [Uber][12], [Spotify][13], [ADP][14], and [Paypal][15]. A major advantage of building a containerized app is that the project setup is virtually the same on Windows, macOS, and Linux - which makes writing this tutorial quite a bit simpler for me. Don't worry if you've never used Docker, we'll go through the full project configuration further down. + +We'll also be using [Node.js][16] (with the [Koa][17] framework), and [Vue.js][18] to build our search API and frontend web app respectively. + +### 1 - WHAT IS ELASTICSEARCH? + +Full-text search is a heavily requested feature in modern applications. Search can also be one of the most difficult features to implement competently - many popular websites have subpar search functionality that returns results slowly and has trouble finding non-exact matches. Often, this is due to limitations in the underlying database: most standard relational databases are limited to basic `CONTAINS` or `LIKE`SQL queries, which provide only the most basic string matching functionality. + +We'd like our search app to be : + +1. **Fast** - Search results should be returned almost instantly, in order to provide a responsive user experience. + +2. **Flexible** - We'll want to be able to modify how the search is performed, in order to optimize for different datasets and use cases. + +3. **Forgiving** - If a search contains a typo, we'd still like to return relevant results for what the user might have been trying to search for. + +4. **Full-Text** - We don't want to limit our search to specific matching keywords or tags - we want to search  _everything_  in our datastore (including large text fields) for a match. + +![Elastic Search Logo](https://storage.googleapis.com/cdn.patricktriest.com/blog/images/posts/elastic-library/Elasticsearch-Logo.png) + +In order to build a super-powered search feature, it’s often most ideal to use a datastore that is optimized for the task of full-text search. This is where [Elasticsearch][19]comes into play; Elasticsearch is an open-source in-memory datastore written in Java and originally built on the [Apache Lucene][20] library. + +Here are some examples of real-world Elasticsearch use cases from the official [Elastic website][21]. + +* Wikipedia uses Elasticsearch to provide full-text search with highlighted search snippets, and search-as-you-type and did-you-mean suggestions. + +* The Guardian uses Elasticsearch to combine visitor logs with social -network data to provide real-time feedback to its editors about the public’s response to new articles. + +* Stack Overflow combines full-text search with geolocation queries and uses more-like-this to find related questions and answers. + +* GitHub uses Elasticsearch to query 130 billion lines of code. + +### What makes Elasticsearch different from a "normal" database? + +At its core, Elasticsearch is able to provide fast and flexible full-text search through the use of  _inverted indices_ . + +An "index" is a data structure to allow for ultra-fast data query and retrieval operations in databases. Databases generally index entries by storing an association of fields with the matching table rows. By storing the index in a searchable data structure (often a [B-Tree][22]), databases can achieve sub-linear time on optimized queries (such as “Find the row with ID = 5”). + +![Relational Index](https://cdn.patricktriest.com/blog/images/posts/elastic-library/db_index.png) + +We can think of a database index like an old-school library card catalog - it tells you precisely where the entry that you're searching for is located, as long as you already know the title and author of the book. Database tables generally have multiple indices in order to speed up queries on specific fields (i.e. an index on the `name`column would greatly speed up queries for rows with a specific name). + +Inverted indexes work in a substantially different manner. The content of each row (or document) is split up, and each individual entry (in this case each word) points back to any documents that it was found within. + +![Inverted Index](https://cdn.patricktriest.com/blog/images/posts/elastic-library/invertedIndex.jpg) + +This inverted-index data structure allows us to very quickly find, say, all of the documents where “football” was mentioned. Through the use of a heavily optimized in-memory inverted index, Elasticsearch enables us to perform some very powerful and customizable full-text searches on our stored data. + +### 2 - PROJECT SETUP + +### 2.0 - Docker + +We'll be using [Docker][23] to manage the environments and dependencies for this project. Docker is a containerization engine that allows applications to be run in isolated environments, unaffected by the host operating system and local development environment. Many web-scale companies run a majority of their server infrastructure in containers now, due to the increased flexibility and composability of containerized application components. + +![Docker Logo](https://storage.googleapis.com/cdn.patricktriest.com/blog/images/posts/elastic-library/docker.png) + +The advantage of using Docker for me, as the friendly author of this tutorial, is that the local environment setup is minimal and consistent across Windows, macOS, and Linux systems. Instead of going through divergent installation instructions for Node.js, Elasticsearch, and Nginx, we can instead just define these dependencies in Docker configuration files, and then run our app anywhere using this configuration. Furthermore, since each application component will run in it's own isolated container, there is much less potential for existing junk on our local machines to interfere, so "But it works on my machine!" types of scenarios will be much more rare when debugging issues. + +### 2.1 - Install Docker & Docker-Compose + +The only dependencies for this project are [Docker][24] and [docker-compose][25], the later of which is an officially supported tool for defining multiple container configurations to  _compose_  into a single application stack. + +Install Docker - [https://docs.docker.com/engine/installation/][26] +Install Docker Compose - [https://docs.docker.com/compose/install/][27] + +### 2.2 - Setup Project Directories + +Create a base directory (say `guttenberg_search`) for the project. To organize our project we'll work within two main subdirectories. + +* `/public` - Store files for the frontend Vue.js webapp. + +* `/server` - Server-side Node.js source code + +### 2.3 - Add Docker-Compose Config + +Next, we'll create a `docker-compose.yml` file to define each container in our application stack. + +1. `gs-api` - The Node.js container for the backend application logic. + +2. `gs-frontend` - An Ngnix container for serving the frontend webapp files. + +3. `gs-search` - An Elasticsearch container for storing and searching data. + +``` +version: '3' + +services: + api: # Node.js App + container_name: gs-api + build: . + ports: + - "3000:3000" # Expose API port + - "9229:9229" # Expose Node process debug port (disable in production) + environment: # Set ENV vars + - NODE_ENV=local + - ES_HOST=elasticsearch + - PORT=3000 + volumes: # Attach local book data directory + - ./books:/usr/src/app/books + + frontend: # Nginx Server For Frontend App + container_name: gs-frontend + image: nginx + volumes: # Serve local "public" dir + - ./public:/usr/share/nginx/html + ports: + - "8080:80" # Forward site to localhost:8080 + + elasticsearch: # Elasticsearch Instance + container_name: gs-search + image: docker.elastic.co/elasticsearch/elasticsearch:6.1.1 + volumes: # Persist ES data in seperate "esdata" volume + - esdata:/usr/share/elasticsearch/data + environment: + - bootstrap.memory_lock=true + - "ES_JAVA_OPTS=-Xms512m -Xmx512m" + - discovery.type=single-node + ports: # Expose Elasticsearch ports + - "9300:9300" + - "9200:9200" + +volumes: # Define seperate volume for Elasticsearch data + esdata: + +``` + +This file defines our entire application stack - no need to install Elasticsearch, Node, or Nginx on your local system. Each container is forwarding ports to the host system (`localhost`), in order for us to access and debug the Node API, Elasticsearch instance, and fronted web app from our host machine. + +### 2.4 - Add Dockerfile + +We are using official prebuilt images for Nginx and Elasticsearch, but we'll need to build our own image for the Node.js app. + +Define a simple `Dockerfile` configuration in the application root directory. + +``` +# Use Node v8.9.0 LTS +FROM node:carbon + +# Setup app working directory +WORKDIR /usr/src/app + +# Copy package.json and package-lock.json +COPY package*.json ./ + +# Install app dependencies +RUN npm install + +# Copy sourcecode +COPY . . + +# Start app +CMD [ "npm", "start" ] + +``` + +This Docker configuration extends the official Node.js image, copies our application source code, and installs the NPM dependencies within the container. + +We'll also add a `.dockerignore` file to avoid copying unneeded files into the container. + +``` +node_modules/ +npm-debug.log +books/ +public/ + +``` + +> Note that we're not copying the `node_modules` directory into our container - this is because we'll be running `npm install` from within the container build process. Attempting to copy the `node_modules` from the host system into a container can cause errors since some packages need to be specifically built for certain operating systems. For instance, installing the `bcrypt` package on macOS and attempting to copy that module directly to an Ubuntu container will not work because `bcyrpt`relies on a binary that needs to be built specifically for each operating system. + +### 2.5 - Add Base Files + +In order to test out the configuration, we'll need to add some placeholder files to the app directories. + +Add this base HTML file at `public/index.html` + +``` +Hello World From The Frontend Container + +``` + +Next, add the placeholder Node.js app file at `server/app.js`. + +``` +const Koa = require('koa') +const app = new Koa() + +app.use(async (ctx, next) => { + ctx.body = 'Hello World From the Backend Container' +}) + +const port = process.env.PORT || 3000 + +app.listen(port, err => { + if (err) console.error(err) + console.log(`App Listening on Port ${port}`) +}) + +``` + +Finally, add our `package.json` Node app configuration. + +``` +{ + "name": "guttenberg-search", + "version": "0.0.1", + "description": "Source code for Elasticsearch tutorial using 100 classic open source books.", + "scripts": { + "start": "node --inspect=0.0.0.0:9229 server/app.js" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/triestpa/guttenberg-search.git" + }, + "author": "patrick.triest@gmail.com", + "license": "MIT", + "bugs": { + "url": "https://github.com/triestpa/guttenberg-search/issues" + }, + "homepage": "https://github.com/triestpa/guttenberg-search#readme", + "dependencies": { + "elasticsearch": "13.3.1", + "joi": "13.0.1", + "koa": "2.4.1", + "koa-joi-validate": "0.5.1", + "koa-router": "7.2.1" + } +} + +``` + +This file defines the application start command and the Node.js package dependencies. + +> Note - You don't have to run `npm install` - the dependencies will be installed inside the container when it is built. + +### 2.6 - Try it Out + +Everything is in place now to test out each component of the app. From the base directory, run `docker-compose build`, which will build our Node.js application container. + +![docker build output](https://cdn.patricktriest.com/blog/images/posts/elastic-library/sample_0_3.png) + +Next, run `docker-compose up` to launch our entire application stack. + +![docker compose output](https://cdn.patricktriest.com/blog/images/posts/elastic-library/sample_0_2.png) + +> This step might take a few minutes since Docker has to download the base images for each container. In subsequent runs, starting the app should be nearly instantaneous, since the required images will have already been downloaded. + +Try visiting `localhost:8080` in your browser - you should see a simple "Hello World" webpage. + +![frontend sample output](https://cdn.patricktriest.com/blog/images/posts/elastic-library/sample_0_0.png) + +Visit `localhost:3000` to verify that our Node server returns it's own "Hello World" message. + +![backend sample output](https://cdn.patricktriest.com/blog/images/posts/elastic-library/sample_0_1.png) + +Finally, visit `localhost:9200` to check that Elasticsearch is running. It should return information similar to this. + +``` +{ + "name" : "SLTcfpI", + "cluster_name" : "docker-cluster", + "cluster_uuid" : "iId8e0ZeS_mgh9ALlWQ7-w", + "version" : { + "number" : "6.1.1", + "build_hash" : "bd92e7f", + "build_date" : "2017-12-17T20:23:25.338Z", + "build_snapshot" : false, + "lucene_version" : "7.1.0", + "minimum_wire_compatibility_version" : "5.6.0", + "minimum_index_compatibility_version" : "5.0.0" + }, + "tagline" : "You Know, for Search" +} + +``` + +If all three URLs display data successfully, congrats! The entire containerized stack is running, so now we can move on to the fun part. + +### 3 - CONNECT TO ELASTICSEARCH + +The first thing that we'll need to do in our app is connect to our local Elasticsearch instance. + +### 3.0 - Add ES Connection Module + +Add the following Elasticsearch initialization code to a new file `server/connection.js`. + +``` +const elasticsearch = require('elasticsearch') + +// Core ES variables for this project +const index = 'library' +const type = 'novel' +const port = 9200 +const host = process.env.ES_HOST || 'localhost' +const client = new elasticsearch.Client({ host: { host, port } }) + +/** Check the ES connection status */ +async function checkConnection () { + let isConnected = false + while (!isConnected) { + console.log('Connecting to ES') + try { + const health = await client.cluster.health({}) + console.log(health) + isConnected = true + } catch (err) { + console.log('Connection Failed, Retrying...', err) + } + } +} + +checkConnection() + +``` + +Let's rebuild our Node app now that we've made changes, using `docker-compose build`. Next, run `docker-compose up -d` to start the application stack as a background daemon process. + +With the app started, run `docker exec gs-api "node" "server/connection.js"` on the command line in order to run our script within the container. You should see some system output similar to the following. + +``` +{ cluster_name: 'docker-cluster', + status: 'yellow', + timed_out: false, + number_of_nodes: 1, + number_of_data_nodes: 1, + active_primary_shards: 1, + active_shards: 1, + relocating_shards: 0, + initializing_shards: 0, + unassigned_shards: 1, + delayed_unassigned_shards: 0, + number_of_pending_tasks: 0, + number_of_in_flight_fetch: 0, + task_max_waiting_in_queue_millis: 0, + active_shards_percent_as_number: 50 } + +``` + +Go ahead and remove the `checkConnection()` call at the bottom before moving on, since in our final app we'll be making that call from outside the connection module. + +### 3.1 - Add Helper Function To Reset Index + +In `server/connection.js` add the following function below `checkConnection`, in order to provide an easy way to reset our Elasticsearch index. + +``` +/** Clear the index, recreate it, and add mappings */ +async function resetIndex (index) { + if (await client.indices.exists({ index })) { + await client.indices.delete({ index }) + } + + await client.indices.create({ index }) + await putBookMapping() +} + +``` + +### 3.2 - Add Book Schema + +Next, we'll want to add a "mapping" for the book data schema. Add the following function below `resetIndex` in `server/connection.js`. + +``` +/** Add book section schema mapping to ES */ +async function putBookMapping () { + const schema = { + title: { type: 'keyword' }, + author: { type: 'keyword' }, + location: { type: 'integer' }, + text: { type: 'text' } + } + + return client.indices.putMapping({ index, type, body: { properties: schema } }) +} + +``` + +Here we are defining a mapping for the `book` index. An Elasticsearch `index` is roughly analogous to a SQL `table` or a MongoDB `collection`. Adding a mapping allows us to specify each field and datatype for the stored documents. Elasticsearch is schema-less, so we don't technically need to add a mapping, but doing so will give us more control over how the data is handled. + +For instance - we're assigning the `keyword` type to the "title" and "author" fields, and the `text` type to the "text" field. Doing so will cause the search engine to treat these string fields differently - During a search, the engine will search  _within_  the `text` field for potential matches, whereas `keyword` fields will be matched based on their full content. This might seem like a minor distinction, but it can have a huge impact on the behavior and speed of different searches. + +Export the exposed properties and functions at the bottom of the file, so that they can be accessed by other modules in our app. + +``` +module.exports = { + client, index, type, checkConnection, resetIndex +} + +``` + +### 4 - LOAD THE RAW DATA + +We'll be using data from [Project Gutenberg][28] - an online effort dedicated to providing free, digital copies of books within the public domain. For this project, we'll be populating our library with 100 classic books, including texts such as  _The Adventures of Sherlock Holmes_ ,  _Treasure Island_ ,  _The Count of Monte Cristo_ ,  _Around the World in 80 Days_ ,  _Romeo and Juliet_ , and  _The Odyssey_ . + +![Book Covers](https://storage.googleapis.com/cdn.patricktriest.com/blog/images/posts/elastic-library/books.jpg) + +### 4.1 - Download Book Files + +I've zipped the 100 books into a file that you can download here - +[https://cdn.patricktriest.com/data/books.zip][29] + +Extract this file into a `books/` directory in your project. + +If you want, you can do this by using the following commands (requires [wget][30] and ["The Unarchiver" CLI][31]). + +``` +wget https://cdn.patricktriest.com/data/books.zip +unar books.zip + +``` + +### 4.2 - Preview A Book + +Try opening one of the book files, say `219-0.txt`. You'll notice that it starts with an open access license, followed by some lines identifying the book title, author, release dates, language and character encoding. + +``` +Title: Heart of Darkness + +Author: Joseph Conrad + +Release Date: February 1995 [EBook #219] +Last Updated: September 7, 2016 + +Language: English + +Character set encoding: UTF-8 + +``` + +After these lines comes `*** START OF THIS PROJECT GUTENBERG EBOOK HEART OF DARKNESS ***`, after which the book content actually starts. + +If you scroll to the end of the book you'll see the matching message `*** END OF THIS PROJECT GUTENBERG EBOOK HEART OF DARKNESS ***`, which is followed by a much more detailed version of the book's license. + +In the next steps, we'll programmatically parse the book metadata from this header and extract the book content from between the `*** START OF` and `***END OF` place markers. + +### 4.3 - Read Data Dir + +Let's write a script to read the content of each book and to add that data to Elasticsearch. We'll define a new Javascript file `server/load_data.js` in order to perform these operations. + +First, we'll obtain a list of every file within the `books/` data directory. + +Add the following content to `server/load_data.js`. + +``` +const fs = require('fs') +const path = require('path') +const esConnection = require('./connection') + +/** Clear ES index, parse and index all files from the books directory */ +async function readAndInsertBooks () { + try { + // Clear previous ES index + await esConnection.resetIndex() + + // Read books directory + let files = fs.readdirSync('./books').filter(file => file.slice(-4) === '.txt') + console.log(`Found ${files.length} Files`) + + // Read each book file, and index each paragraph in elasticsearch + for (let file of files) { + console.log(`Reading File - ${file}`) + const filePath = path.join('./books', file) + const { title, author, paragraphs } = parseBookFile(filePath) + await insertBookData(title, author, paragraphs) + } + } catch (err) { + console.error(err) + } +} + +readAndInsertBooks() + +``` + +We'll use a shortcut command to rebuild our Node.js app and update the running container. + +Run `docker-compose up -d --build` to update the application. This is a shortcut for running `docker-compose build` and `docker-compose up -d`. + +![docker build output](https://cdn.patricktriest.com/blog/images/posts/elastic-library/sample_1_0.png) + +Run`docker exec gs-api "node" "server/load_data.js"` in order to run our `load_data` script within the container. You should see the Elasticsearch status output, followed by `Found 100 Books`. + +After this, the script will exit due to an error because we're calling a helper function (`parseBookFile`) that we have not yet defined. + +![docker exec output](https://cdn.patricktriest.com/blog/images/posts/elastic-library/sample_1_1.png) + +### 4.4 - Read Data File + +Next, we'll read the metadata and content for each book. + +Define a new function in `server/load_data.js`. + +``` +/** Read an individual book text file, and extract the title, author, and paragraphs */ +function parseBookFile (filePath) { + // Read text file + const book = fs.readFileSync(filePath, 'utf8') + + // Find book title and author + const title = book.match(/^Title:\s(.+)$/m)[1] + const authorMatch = book.match(/^Author:\s(.+)$/m) + const author = (!authorMatch || authorMatch[1].trim() === '') ? 'Unknown Author' : authorMatch[1] + + console.log(`Reading Book - ${title} By ${author}`) + + // Find Guttenberg metadata header and footer + const startOfBookMatch = book.match(/^\*{3}\s*START OF (THIS|THE) PROJECT GUTENBERG EBOOK.+\*{3}$/m) + const startOfBookIndex = startOfBookMatch.index + startOfBookMatch[0].length + const endOfBookIndex = book.match(/^\*{3}\s*END OF (THIS|THE) PROJECT GUTENBERG EBOOK.+\*{3}$/m).index + + // Clean book text and split into array of paragraphs + const paragraphs = book + .slice(startOfBookIndex, endOfBookIndex) // Remove Guttenberg header and footer + .split(/\n\s+\n/g) // Split each paragraph into it's own array entry + .map(line => line.replace(/\r\n/g, ' ').trim()) // Remove paragraph line breaks and whitespace + .map(line => line.replace(/_/g, '')) // Guttenberg uses "_" to signify italics. We'll remove it, since it makes the raw text look messy. + .filter((line) => (line && line.length !== '')) // Remove empty lines + + console.log(`Parsed ${paragraphs.length} Paragraphs\n`) + return { title, author, paragraphs } +} + +``` + +This function performs a few important tasks. + +1. Read book text from the file system. + +2. Use regular expressions (check out [this post][1] for a primer on using regex) to parse the book title and author. + +3. Identify the start and end of the book content, by matching on the all-caps "Project Guttenberg" header and footer. + +4. Extract the book text content. + +5. Split each paragraph into its own array. + +6. Clean up the text and remove blank lines. + +As a return value, we'll form an object containing the book's title, author, and an array of paragraphs within the book. + +Run `docker-compose up -d --build` and `docker exec gs-api "node" "server/load_data.js"` again, and you should see the same output as before, this time with three extra lines at the end of the output. + +![docker exec output](https://cdn.patricktriest.com/blog/images/posts/elastic-library/sample_2_0.png) + +Success! Our script successfully parsed the title and author from the text file. The script will again end with an error since we still have to define one more helper function. + +### 4.5 - Index Datafile in ES + +As a final step, we'll bulk-upload each array of paragraphs into the Elasticsearch index. + +Add a new `insertBookData` function to `load_data.js`. + +``` +/** Bulk index the book data in Elasticsearch */ +async function insertBookData (title, author, paragraphs) { + let bulkOps = [] // Array to store bulk operations + + // Add an index operation for each section in the book + for (let i = 0; i < paragraphs.length; i++) { + // Describe action + bulkOps.push({ index: { _index: esConnection.index, _type: esConnection.type } }) + + // Add document + bulkOps.push({ + author, + title, + location: i, + text: paragraphs[i] + }) + + if (i > 0 && i % 500 === 0) { // Do bulk insert in 500 paragraph batches + await esConnection.client.bulk({ body: bulkOps }) + bulkOps = [] + console.log(`Indexed Paragraphs ${i - 499} - ${i}`) + } + } + + // Insert remainder of bulk ops array + await esConnection.client.bulk({ body: bulkOps }) + console.log(`Indexed Paragraphs ${paragraphs.length - (bulkOps.length / 2)} - ${paragraphs.length}\n\n\n`) +} + +``` + +This function will index each paragraph of the book, with author, title, and paragraph location metadata attached. We are inserting the paragraphs using a bulk operation, which is much faster than indexing each paragraph individually. + +> We're bulk indexing the paragraphs in batches, instead of inserting all of them at once. This was a last minute optimization which I added in order for the app to run on the low-ish memory (1.7 GB) host machine that serves `search.patricktriest.com`. If you have a reasonable amount of RAM (4+ GB), you probably don't need to worry about batching each bulk upload, + +Run `docker-compose up -d --build` and `docker exec gs-api "node" "server/load_data.js"` one more time - you should now see a full output of 100 books being parsed and inserted in Elasticsearch. This might take a minute or so. + +![data loading output](https://cdn.patricktriest.com/blog/images/posts/elastic-library/sample_3_0.png) + +### 5 - SEARCH + +Now that Elasticsearch has been populated with one hundred books (amounting to roughly 230,000 paragraphs), let's try out some search queries. + +### 5.0 - Simple HTTP Query + +First, let's just query Elasticsearch directly using it's HTTP API. + +Visit this URL in your browser - `http://localhost:9200/library/_search?q=text:Java&pretty` + +Here, we are performing a bare-bones full-text search to find the word "Java" within our library of books. + +You should see a JSON response similar to the following. + +``` +{ + "took" : 11, + "timed_out" : false, + "_shards" : { + "total" : 5, + "successful" : 5, + "skipped" : 0, + "failed" : 0 + }, + "hits" : { + "total" : 13, + "max_score" : 14.259304, + "hits" : [ + { + "_index" : "library", + "_type" : "novel", + "_id" : "p_GwFWEBaZvLlaAUdQgV", + "_score" : 14.259304, + "_source" : { + "author" : "Charles Darwin", + "title" : "On the Origin of Species", + "location" : 1080, + "text" : "Java, plants of, 375." + } + }, + { + "_index" : "library", + "_type" : "novel", + "_id" : "wfKwFWEBaZvLlaAUkjfk", + "_score" : 10.186235, + "_source" : { + "author" : "Edgar Allan Poe", + "title" : "The Works of Edgar Allan Poe", + "location" : 827, + "text" : "After many years spent in foreign travel, I sailed in the year 18-- , from the port of Batavia, in the rich and populous island of Java, on a voyage to the Archipelago of the Sunda islands. I went as passenger--having no other inducement than a kind of nervous restlessness which haunted me as a fiend." + } + }, + ... + ] + } +} + +``` + +The Elasticseach HTTP interface is useful for testing that our data is inserted successfully, but exposing this API directly to the web app would be a huge security risk. The API exposes administrative functionality (such as directly adding and deleting documents), and should ideally not ever be exposed publicly. Instead, we'll write a simple Node.js API to receive requests from the client, and make the appropriate query (within our private local network) to Elasticsearch. + +### 5.1 - Query Script + +Let's now try querying Elasticsearch from our Node.js application. + +Create a new file, `server/search.js`. + +``` +const { client, index, type } = require('./connection') + +module.exports = { + /** Query ES index for the provided term */ + queryTerm (term, offset = 0) { + const body = { + from: offset, + query: { match: { + text: { + query: term, + operator: 'and', + fuzziness: 'auto' + } } }, + highlight: { fields: { text: {} } } + } + + return client.search({ index, type, body }) + } +} + +``` + +Our search module defines a simple `search` function, which will perform a `match`query using the input term. + +Here are query fields broken down - + +* `from` - Allows us to paginate the results. Each query returns 10 results by default, so specifying `from: 10` would allow us to retrieve results 10-20. + +* `query` - Where we specify the actual term that we are searching for. + +* `operator` - We can modify the search behavior; in this case, we're using the "and" operator to prioritize results that contain all of the tokens (words) in the query. + +* `fuzziness` - Adjusts tolerance for spelling mistakes, `auto` defaults to `fuzziness: 2`. A higher fuzziness will allow for more corrections in result hits. For instance, `fuzziness: 1` would allow `Patricc` to return `Patrick` as a match. + +* `highlights` - Returns an extra field with the result, containing HTML to display the exact text subset and terms that were matched with the query. + +Feel free to play around with these parameters, and to customize the search query further by exploring the [Elastic Full-Text Query DSL][32]. + +### 6 - API + +Let's write a quick HTTP API in order to access our search functionality from a frontend app. + +### 6.0 - API Server + +Replace our existing `server/app.js` file with the following contents. + +``` +const Koa = require('koa') +const Router = require('koa-router') +const joi = require('joi') +const validate = require('koa-joi-validate') +const search = require('./search') + +const app = new Koa() +const router = new Router() + +// Log each request to the console +app.use(async (ctx, next) => { + const start = Date.now() + await next() + const ms = Date.now() - start + console.log(`${ctx.method} ${ctx.url} - ${ms}`) +}) + +// Log percolated errors to the console +app.on('error', err => { + console.error('Server Error', err) +}) + +// Set permissive CORS header +app.use(async (ctx, next) => { + ctx.set('Access-Control-Allow-Origin', '*') + return next() +}) + +// ADD ENDPOINTS HERE + +const port = process.env.PORT || 3000 + +app + .use(router.routes()) + .use(router.allowedMethods()) + .listen(port, err => { + if (err) throw err + console.log(`App Listening on Port ${port}`) + }) + +``` + +This code will import our server dependencies and set up simple logging and error handling for a [Koa.js][33] Node API server. + +### 6.1 - Link endpoint with queries + +Next, we'll add an endpoint to our server in order to expose our Elasticsearch query function. + +Insert the following code below the `// ADD ENDPOINTS HERE` comment in `server/app.js`. + +``` +/** + * GET /search + * Search for a term in the library + */ +router.get('/search', async (ctx, next) => { + const { term, offset } = ctx.request.query + ctx.body = await search.queryTerm(term, offset) + } +) + +``` + +Restart the app using `docker-compose up -d --build`. In your browser, try calling the search endpoint. For example, this request would search the entire library for passages mentioning "Java" - `http://localhost:3000/search?term=java` + +The result will look quite similar to the response from earlier when we called the Elasticsearch HTTP interface directly. + +``` +{ + "took": 242, + "timed_out": false, + "_shards": { + "total": 5, + "successful": 5, + "skipped": 0, + "failed": 0 + }, + "hits": { + "total": 93, + "max_score": 13.356944, + "hits": [{ + "_index": "library", + "_type": "novel", + "_id": "eHYHJmEBpQg9B4622421", + "_score": 13.356944, + "_source": { + "author": "Charles Darwin", + "title": "On the Origin of Species", + "location": 1080, + "text": "Java, plants of, 375." + }, + "highlight": { + "text": ["Java, plants of, 375."] + } + }, { + "_index": "library", + "_type": "novel", + "_id": "2HUHJmEBpQg9B462xdNg", + "_score": 9.030668, + "_source": { + "author": "Unknown Author", + "title": "The King James Bible", + "location": 186, + "text": "10:4 And the sons of Javan; Elishah, and Tarshish, Kittim, and Dodanim." + }, + "highlight": { + "text": ["10:4 And the sons of Javan; Elishah, and Tarshish, Kittim, and Dodanim."] + } + } + ... + ] + } +} + +``` + +### 6.2 - Input validation + +This endpoint is still brittle - we are not doing any checks on the request parameters, so invalid or missing values would result in a server error. + +We'll add some middleware to the endpoint in order to validate input parameters using [Joi][34] and the [Koa-Joi-Validate][35] library. + +``` +/** + * GET /search + * Search for a term in the library + * Query Params - + * term: string under 60 characters + * offset: positive integer + */ +router.get('/search', + validate({ + query: { + term: joi.string().max(60).required(), + offset: joi.number().integer().min(0).default(0) + } + }), + async (ctx, next) => { + const { term, offset } = ctx.request.query + ctx.body = await search.queryTerm(term, offset) + } +) + +``` + +Now, if you restart the server and make a request with a missing term(`http://localhost:3000/search`), you will get back an HTTP 400 error with a relevant message, such as `Invalid URL Query - child "term" fails because ["term" is required]`. + +To view live logs from the Node app, you can run `docker-compose logs -f api`. + +### 7 - FRONT-END APPLICATION + +Now that our `/search` endpoint is in place, let's wire up a simple web app to test out the API. + +### 7.0 - Vue.js App + +We'll be using Vue.js to coordinate our frontend. + +Add a new file, `/public/app.js`, to hold our Vue.js application code. + +``` +const vm = new Vue ({ + el: '#vue-instance', + data () { + return { + baseUrl: 'http://localhost:3000', // API url + searchTerm: 'Hello World', // Default search term + searchDebounce: null, // Timeout for search bar debounce + searchResults: [], // Displayed search results + numHits: null, // Total search results found + searchOffset: 0, // Search result pagination offset + + selectedParagraph: null, // Selected paragraph object + bookOffset: 0, // Offset for book paragraphs being displayed + paragraphs: [] // Paragraphs being displayed in book preview window + } + }, + async created () { + this.searchResults = await this.search() // Search for default term + }, + methods: { + /** Debounce search input by 100 ms */ + onSearchInput () { + clearTimeout(this.searchDebounce) + this.searchDebounce = setTimeout(async () => { + this.searchOffset = 0 + this.searchResults = await this.search() + }, 100) + }, + /** Call API to search for inputted term */ + async search () { + const response = await axios.get(`${this.baseUrl}/search`, { params: { term: this.searchTerm, offset: this.searchOffset } }) + this.numHits = response.data.hits.total + return response.data.hits.hits + }, + /** Get next page of search results */ + async nextResultsPage () { + if (this.numHits > 10) { + this.searchOffset += 10 + if (this.searchOffset + 10 > this.numHits) { this.searchOffset = this.numHits - 10} + this.searchResults = await this.search() + document.documentElement.scrollTop = 0 + } + }, + /** Get previous page of search results */ + async prevResultsPage () { + this.searchOffset -= 10 + if (this.searchOffset < 0) { this.searchOffset = 0 } + this.searchResults = await this.search() + document.documentElement.scrollTop = 0 + } + } +}) + +``` + +The app is pretty simple - we're just defining some shared data properties, and adding methods to retrieve and paginate through search results. The search input is debounced by 100ms, to prevent the API from being called with every keystroke. + +Explaining how Vue.js works is outside the scope of this tutorial, but this probably won't look too crazy if you've used Angular or React. If you're completely unfamiliar with Vue, and if you want something quick to get started with, I would recommend the official quick-start guide - [https://vuejs.org/v2/guide/][36] + +### 7.1 - HTML + +Replace our placeholder `/public/index.html` file with the following contents, in order to load our Vue.js app and to layout a basic search interface. + +``` + + + + + Elastic Library + + + + + + + + +

+ +
+
+ + +
+
+ + +
+
{{ numHits }} Hits
+
Displaying Results {{ searchOffset }} - {{ searchOffset + 9 }}
+
+ + +
+ + +
+ + +
+
+
+
+
{{ hit._source.title }} - {{ hit._source.author }}
+
Location {{ hit._source.location }}
+
+
+ + +
+ + +
+ + +
+ + + + + + + +``` + +### 7.2 - CSS + +Add a new file, `/public/styles.css`, with some custom UI styling. + +``` +body { font-family: 'EB Garamond', serif; } + +.mui-textfield > input, .mui-btn, .mui--text-subhead, .mui-panel > .mui--text-headline { + font-family: 'Open Sans', sans-serif; +} + +.all-caps { text-transform: uppercase; } +.app-container { padding: 16px; } +.search-results em { font-weight: bold; } +.book-modal > button { width: 100%; } +.search-results .mui-divider { margin: 14px 0; } + +.search-results { + display: flex; + flex-direction: row; + flex-wrap: wrap; + justify-content: space-around; +} + +.search-results > div { + flex-basis: 45%; + box-sizing: border-box; + cursor: pointer; +} + +@media (max-width: 600px) { + .search-results > div { flex-basis: 100%; } +} + +.paragraphs-container { + max-width: 800px; + margin: 0 auto; + margin-bottom: 48px; +} + +.paragraphs-container .mui--text-body1, .paragraphs-container .mui--text-body2 { + font-size: 1.8rem; + line-height: 35px; +} + +.book-modal { + width: 100%; + height: 100%; + padding: 40px 10%; + box-sizing: border-box; + margin: 0 auto; + background-color: white; + overflow-y: scroll; + position: fixed; + top: 0; + left: 0; +} + +.pagination-panel { + display: flex; + justify-content: space-between; +} + +.title-row { + display: flex; + justify-content: space-between; + align-items: flex-end; +} + +@media (max-width: 600px) { + .title-row{ + flex-direction: column; + text-align: center; + align-items: center + } +} + +.locations-label { + text-align: center; + margin: 8px; +} + +.modal-footer { + position: fixed; + bottom: 0; + left: 0; + width: 100%; + display: flex; + justify-content: space-around; + background: white; +} + +``` + +### 7.3 - Try it out + +Open `localhost:8080` in your web browser, you should see a simple search interface with paginated results. Try typing in the top search bar to find matches from different terms. + +![preview webapp](https://cdn.patricktriest.com/blog/images/posts/elastic-library/sample_4_0.png) + +> You  _do not_  have to re-run the `docker-compose up` command for the changes to take effect. The local `public` directory is mounted to our Nginx fileserver container, so frontend changes on the local system will be automatically reflected in the containerized app. + +If you try clicking on any result, nothing happens - we still have one more feature to add to the app. + +### 8 - PAGE PREVIEWS + +It would be nice to be able to click on each search result and view it in the context of the book that it's from. + +### 8.0 - Add Elasticsearch Query + +First, we'll need to define a simple query to get a range of paragraphs from a given book. + +Add the following function to the `module.exports` block in `server/search.js`. + +``` +/** Get the specified range of paragraphs from a book */ +getParagraphs (bookTitle, startLocation, endLocation) { + const filter = [ + { term: { title: bookTitle } }, + { range: { location: { gte: startLocation, lte: endLocation } } } + ] + + const body = { + size: endLocation - startLocation, + sort: { location: 'asc' }, + query: { bool: { filter } } + } + + return client.search({ index, type, body }) +} + +``` + +This new function will return an ordered array of paragraphs between the start and end locations of a given book. + +### 8.1 - Add API Endpoint + +Now, let's link this function to an API endpoint. + +Add the following to `server/app.js`, below the original `/search` endpoint. + +``` +/** + * GET /paragraphs + * Get a range of paragraphs from the specified book + * Query Params - + * bookTitle: string under 256 characters + * start: positive integer + * end: positive integer greater than start + */ +router.get('/paragraphs', + validate({ + query: { + bookTitle: joi.string().max(256).required(), + start: joi.number().integer().min(0).default(0), + end: joi.number().integer().greater(joi.ref('start')).default(10) + } + }), + async (ctx, next) => { + const { bookTitle, start, end } = ctx.request.query + ctx.body = await search.getParagraphs(bookTitle, start, end) + } +) + +``` + +### 8.2 - Add UI functionality + +Now that our new endpoint is in place, let's add some frontend functionality to query and display full pages from the book. + +Add the following functions to the `methods` block of `/public/app.js`. + +``` + /** Call the API to get current page of paragraphs */ + async getParagraphs (bookTitle, offset) { + try { + this.bookOffset = offset + const start = this.bookOffset + const end = this.bookOffset + 10 + const response = await axios.get(`${this.baseUrl}/paragraphs`, { params: { bookTitle, start, end } }) + return response.data.hits.hits + } catch (err) { + console.error(err) + } + }, + /** Get next page (next 10 paragraphs) of selected book */ + async nextBookPage () { + this.$refs.bookModal.scrollTop = 0 + this.paragraphs = await this.getParagraphs(this.selectedParagraph._source.title, this.bookOffset + 10) + }, + /** Get previous page (previous 10 paragraphs) of selected book */ + async prevBookPage () { + this.$refs.bookModal.scrollTop = 0 + this.paragraphs = await this.getParagraphs(this.selectedParagraph._source.title, this.bookOffset - 10) + }, + /** Display paragraphs from selected book in modal window */ + async showBookModal (searchHit) { + try { + document.body.style.overflow = 'hidden' + this.selectedParagraph = searchHit + this.paragraphs = await this.getParagraphs(searchHit._source.title, searchHit._source.location - 5) + } catch (err) { + console.error(err) + } + }, + /** Close the book detail modal */ + closeBookModal () { + document.body.style.overflow = 'auto' + this.selectedParagraph = null + } + +``` + +These five functions provide the logic for downloading and paginating through pages (ten paragraphs each) in a book. + +Now we just need to add a UI to display the book pages. Add this markup below the `` comment in `/public/index.html`. + +``` + +
+
+ +
+
{{ selectedParagraph._source.title }}
+
{{ selectedParagraph._source.author }}
+
+
+
+
Locations {{ bookOffset - 5 }} to {{ bookOffset + 5 }}
+
+
+ + +
+
+ {{ paragraph._source.text }} +
+
+ {{ paragraph._source.text }} +
+
+
+
+ + +
+ + + +
+
+ +``` + +Restart the app server (`docker-compose up -d --build`) again and open up `localhost:8080`. When you click on a search result, you are now able to view the surrounding paragraphs. You can now even read the rest of the book to completion if you're entertained by what you find. + +![preview webapp book page](https://cdn.patricktriest.com/blog/images/posts/elastic-library/sample_5_0.png) + +Congrats, you've completed the tutorial application! + +Feel free to compare your local result against the completed sample hosted here - [https://search.patricktriest.com/][37] + +### 9 - DISADVANTAGES OF ELASTICSEARCH + +### 9.0 - Resource Hog + +Elasticsearch is computationally demanding. The [official recommendation][38] is to run ES on a machine with 64 GB of RAM, and they strongly discourage running it on anything with under 8 GB of RAM. Elasticsearch is an  _in-memory_  datastore, which allows it to return results extremely quickly, but also results in a very significant system memory footprint. In production, [it is strongly recommended to run multiple Elasticsearch nodes in a cluster][39] to allow for high server availability, automatic sharding, and data redundancy in case of a node failure. + +I've got our tutorial application running on a $15/month GCP compute instance (at [search.patricktriest.com][40]) with 1.7 GB of RAM, and it  _just barely_  is able to run the Elasticsearch node; sometimes the entire machine freezes up during the initial data-loading step. Elasticsearch is, in my experience, much more of a resource hog than more traditional databases such as PostgreSQL and MongoDB, and can be significantly more expensive to host as a result. + +### 9.1 - Syncing with Databases + +In most applications, storing all of the data in Elasticsearch is not an ideal option. It is possible to use ES as the primary transactional database for an app, but this is generally not recommended due to the lack of ACID compliance in Elasticsearch, which can lead to lost write operations when ingesting data at scale. In many cases, ES serves a more specialized role, such as powering the text searching features of the app. This specialized use requires that some of the data from the primary database is replicated to the Elasticsearch instance. + +For instance, let's imagine that we're storing our users in a PostgreSQL table, but using Elasticsearch to power our user-search functionality. If a user, "Albert", decides to change his name to "Al", we'll need this change to be reflected in both our primary PostgreSQL database and in our auxiliary Elasticsearch cluster. + +This can be a tricky integration to get right, and the best answer will depend on your existing stack. There are a multitude of open-source options available, from [a process to watch a MongoDB operation log][41] and automatically sync detected changes to ES, to a [PostgresSQL plugin][42] to create a custom PSQL-based index that communicates automatically with Elasticsearch. + +If none of the available pre-built options work, you could always just add some hooks into your server code to update the Elasticsearch index manually based on database changes. I would consider this final option to be a last resort, since keeping ES in sync using custom business logic can be complex, and is likely to introduce numerous bugs to the application. + +The need to sync Elasticsearch with a primary database is more of an architectural complexity than it is a specific weakness of ES, but it's certainly worth keeping in mind when considering the tradeoffs of adding a dedicated search engine to your app. + +### CONCLUSION + +Full-text search is one of the most important features in many modern applications - and is one of the most difficult to implement well. Elasticsearch is a fantastic option for adding fast and customizable text search to your application, but there are alternatives. [Apache Solr][43] is a similar open source search platform that is built on Apache Lucene - the same library at the core of Elasticsearch. [Algolia][44] is a search-as-a-service web platform which is growing quickly in popularity and is likely to be easier to get started with for beginners (but as a tradeoff is less customizable and can get quite expensive). + +"Search-bar" style features are far from the only use-case for Elasticsearch. ES is also a very common tool for log storage and analysis, commonly used in an ELK (Elasticsearch, Logstash, Kibana) stack configuration. The flexible full-text search allowed by Elasticsearch can also be very useful for a wide variety of data science tasks - such as correcting/standardizing the spellings of entities within a dataset or searching a large text dataset for similar phrases. + +Here are some ideas for your own projects. + +* Add more of your favorite books to our tutorial app and create your own private library search engine. + +* Create an academic plagiarism detection engine by indexing papers from [Google Scholar][2]. + +* Build a spell checking application by indexing every word in the dictionary to Elasticsearch. + +* Build your own Google-competitor internet search engine by loading the [Common Crawl Corpus][3] into Elasticsearch (caution - with over 5 billion pages, this can be a very expensive dataset play with). + +* Use Elasticsearch for journalism: search for specific names and terms in recent large-scale document leaks such as the [Panama Papers][4] and [Paradise Papers][5]. + +The source code for this tutorial application is 100% open-source and can be found at the GitHub repository here - [https://github.com/triestpa/guttenberg-search][45] + +I hope you enjoyed the tutorial! Please feel free to post any thoughts, questions, or criticisms in the comments below. + + +-------------------------------------------------------------------------------- + +作者简介： + +Full-stack engineer, data enthusiast, insatiable learner, obsessive builder. You can find me wandering on a mountain trail, pretending not to be lost. + +------------- + + +via: https://blog.patricktriest.com/text-search-docker-elasticsearch/ + +作者：[Patrick Triest][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://blog.patricktriest.com/author/patrick/ +[1]:https://blog.patricktriest.com/you-should-learn-regex/ +[2]:https://scholar.google.com/ +[3]:https://aws.amazon.com/public-datasets/common-crawl/ +[4]:https://en.wikipedia.org/wiki/Panama_Papers +[5]:https://en.wikipedia.org/wiki/Paradise_Papers +[6]:https://search.patricktriest.com/ +[7]:https://github.com/triestpa/guttenberg-search +[8]:https://www.postgresql.org/ +[9]:https://www.mongodb.com/ +[10]:https://www.elastic.co/ +[11]:https://www.docker.com/ +[12]:https://www.uber.com/ +[13]:https://www.spotify.com/us/ +[14]:https://www.adp.com/ +[15]:https://www.paypal.com/us/home +[16]:https://nodejs.org/en/ +[17]:http://koajs.com/ +[18]:https://vuejs.org/ +[19]:https://www.elastic.co/ +[20]:https://lucene.apache.org/core/ +[21]:https://www.elastic.co/guide/en/elasticsearch/guide/2.x/getting-started.html +[22]:https://en.wikipedia.org/wiki/B-tree +[23]:https://www.docker.com/ +[24]:https://www.docker.com/ +[25]:https://docs.docker.com/compose/ +[26]:https://docs.docker.com/engine/installation/ +[27]:https://docs.docker.com/compose/install/ +[28]:https://www.gutenberg.org/ +[29]:https://cdn.patricktriest.com/data/books.zip +[30]:https://www.gnu.org/software/wget/ +[31]:https://theunarchiver.com/command-line +[32]:https://www.elastic.co/guide/en/elasticsearch/reference/current/full-text-queries.html +[33]:http://koajs.com/ +[34]:https://github.com/hapijs/joi +[35]:https://github.com/triestpa/koa-joi-validate +[36]:https://vuejs.org/v2/guide/ +[37]:https://search.patricktriest.com/ +[38]:https://www.elastic.co/guide/en/elasticsearch/guide/current/hardware.html +[39]:https://www.elastic.co/guide/en/elasticsearch/guide/2.x/distributed-cluster.html +[40]:https://search.patricktriest.com/ +[41]:https://github.com/mongodb-labs/mongo-connector +[42]:https://github.com/zombodb/zombodb +[43]:https://lucene.apache.org/solr/ +[44]:https://www.algolia.com/ +[45]:https://github.com/triestpa/guttenberg-search +[46]:https://blog.patricktriest.com/tag/guides/ +[47]:https://blog.patricktriest.com/tag/javascript/ +[48]:https://blog.patricktriest.com/tag/nodejs/ +[49]:https://blog.patricktriest.com/tag/web-development/ +[50]:https://blog.patricktriest.com/tag/devops/ \ No newline at end of file diff --git a/sources/tech/20180125 Building a Linux-based HPC system on the Raspberry.md b/sources/tech/20180125 Building a Linux-based HPC system on the Raspberry.md new file mode 100644 index 0000000000..eab5ac90b3 --- /dev/null +++ b/sources/tech/20180125 Building a Linux-based HPC system on the Raspberry.md @@ -0,0 +1,153 @@ +Building a Linux-based HPC system on the Raspberry Pi with Ansible +============================================================ + +### Create a high-performance computing cluster with low-cost hardware and open source software. + +![Building a Linux-based HPC system on the Raspberry Pi with Ansible](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/code_computer_development_programming.png?itok=4OM29-82 "Building a Linux-based HPC system on the Raspberry Pi with Ansible") +Image by : opensource.com + +In my [previous article for Opensource.com][14], I introduced the [OpenHPC][15] project, which aims to accelerate innovation in high-performance computing (HPC). This article goes a step further by using OpenHPC's capabilities to build a small HPC system. To call it an  _HPC system_  might sound bigger than it is, so maybe it is better to say this is a system based on the [Cluster Building Recipes][16] published by the OpenHPC project. + +The resulting cluster consists of two Raspberry Pi 3 systems acting as compute nodes and one virtual machine acting as the master node: + + +![Map of HPC cluster](https://opensource.com/sites/default/files/u128651/hpc_with_pi-1.png "Map of HPC cluster") + +My master node is running CentOS on x86_64 and my compute nodes are running a slightly modified CentOS on aarch64. + +This is what the setup looks in real life: + + +![HPC hardware setup](https://opensource.com/sites/default/files/u128651/hpc_with_pi-2.jpg "HPC hardware setup") + +To set up my system like an HPC system, I followed some of the steps from OpenHPC's Cluster Building Recipes [install guide for CentOS 7.4/aarch64 + Warewulf + Slurm][17] (PDF). This recipe includes provisioning instructions using [Warewulf][18]; because I manually installed my three systems, I skipped the Warewulf parts and created an [Ansible playbook][19] for the steps I took. + + +Once my cluster was set up by the [Ansible][26] playbooks, I could start to submit jobs to my resource manager. The resource manager, [Slurm][27] in my case, is the instance in the cluster that decides where and when my jobs are executed. One possibility to start a simple job on the cluster is: +``` +[ohpc@centos01 ~]$ srun hostname +calvin +``` + +If I need more resources, I can tell Slurm that I want to run my command on eight CPUs: + +``` +[ohpc@centos01 ~]$ srun -n 8 hostname +hobbes +hobbes +hobbes +hobbes +calvin +calvin +calvin +calvin +``` + +In the first example, Slurm ran the specified command (`hostname`) on a single CPU, and in the second example Slurm ran the command on eight CPUs. One of my compute nodes is named `calvin` and the other is named `hobbes`; that can be seen in the output of the above commands. Each of the compute nodes is a Raspberry Pi 3 with four CPU cores. + +Another way to submit jobs to my cluster is the command `sbatch`, which can be used to execute scripts with the output written to a file instead of my terminal. + +``` +[ohpc@centos01 ~]$ cat script1.sh +#!/bin/sh +date +hostname +sleep 10 +date +[ohpc@centos01 ~]$ sbatch script1.sh +Submitted batch job 101 +``` + +This will create an output file called `slurm-101.out` with the following content: + +``` +Mon 11 Dec 16:42:31 UTC 2017 +calvin +Mon 11 Dec 16:42:41 UTC 2017 +``` + +To demonstrate the basic functionality of the resource manager, simple and serial command line tools are suitable—but a bit boring after doing all the work to set up an HPC-like system. + +A more interesting application is running an [Open MPI][20] parallelized job on all available CPUs on the cluster. I'm using an application based on [Game of Life][21], which was used in a [video][22] called "Running Game of Life across multiple architectures with Red Hat Enterprise Linux." In addition to the previously used MPI-based Game of Life implementation, the version now running on my cluster colors the cells for each involved host differently. The following script starts the application interactively with a graphical output: + +``` +$ cat life.mpi +#!/bin/bash + +module load gnu6 openmpi3 + +if [[ "$SLURM_PROCID" != "0" ]]; then +    exit +fi + +mpirun ./mpi_life -a -p -b +``` + +I start the job with the following command, which tells Slurm to allocate eight CPUs for the job: + +``` +$ srun -n 8 --x11 life.mpi +``` + +For demonstration purposes, the job has a graphical interface that shows the current result of the calculation: + + +![](https://opensource.com/sites/default/files/u128651/hpc_with_pi-3.png) + +The position of the red cells is calculated on one of the compute nodes, and the green cells are calculated on the other compute node. I can also tell the Game of Life program to color the cell for each used CPU (there are four per compute node) differently, which leads to the following output: + + +![](https://opensource.com/sites/default/files/u128651/hpc_with_pi-4.png) + +Thanks to the installation recipes and the software packages provided by OpenHPC, I was able to set up two compute nodes and a master node in an HPC-type configuration. I can submit jobs to my resource manager, and I can use the software provided by OpenHPC to start MPI applications utilizing all my Raspberry Pis' CPUs. + +* * * + + _To learn more about using OpenHPC to build a Raspberry Pi cluster, please attend Adrian Reber's talks at [DevConf.cz 2018][10], January 26-28, in Brno, Czech Republic, and at the [CentOS Dojo 2018][11], on February 2, in Brussels._ + +### About the author + + [![](https://opensource.com/sites/default/files/styles/profile_pictures/public/pictures/gotchi-square.png?itok=PJKu7LHn)][23] Adrian Reber - Adrian is a Senior Software Engineer at Red Hat and is migrating processes at least since 2010\. He started to migrate processes in a high performance computing environment and at some point he migrated so many processes that he got a PhD for that and since he joined Red Hat he started to migrate containers. Occasionally he still migrates single processes and is still interested in high performance computing topics.[More about me][12] + +-------------------------------------------------------------------------------- + +via: https://opensource.com/article/18/1/how-build-hpc-system-raspberry-pi-and-openhpc + +作者：[Adrian Reber ][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://opensource.com/users/adrianreber +[1]:https://opensource.com/resources/what-are-linux-containers?utm_campaign=containers&intcmp=70160000000h1s6AAA +[2]:https://opensource.com/resources/what-docker?utm_campaign=containers&intcmp=70160000000h1s6AAA +[3]:https://opensource.com/resources/what-is-kubernetes?utm_campaign=containers&intcmp=70160000000h1s6AAA +[4]:https://developers.redhat.com/blog/2016/01/13/a-practical-introduction-to-docker-container-terminology/?utm_campaign=containers&intcmp=70160000000h1s6AAA +[5]:https://opensource.com/file/384031 +[6]:https://opensource.com/file/384016 +[7]:https://opensource.com/file/384021 +[8]:https://opensource.com/file/384026 +[9]:https://opensource.com/article/18/1/how-build-hpc-system-raspberry-pi-and-openhpc?rate=l9n6B6qRcR20LJyXEoUoWEZ4mb2nDc9sFZ1YSPc60vE +[10]:https://devconfcz2018.sched.com/event/DJYi/openhpc-introduction +[11]:https://wiki.centos.org/Events/Dojo/Brussels2018 +[12]:https://opensource.com/users/adrianreber +[13]:https://opensource.com/user/188446/feed +[14]:https://opensource.com/article/17/11/openhpc +[15]:https://openhpc.community/ +[16]:https://openhpc.community/downloads/ +[17]:https://github.com/openhpc/ohpc/releases/download/v1.3.3.GA/Install_guide-CentOS7-Warewulf-SLURM-1.3.3-aarch64.pdf +[18]:https://en.wikipedia.org/wiki/Warewulf +[19]:http://people.redhat.com/areber/openhpc/ansible/ +[20]:https://www.open-mpi.org/ +[21]:https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life +[22]:https://www.youtube.com/watch?v=n8DvxMcOMXk +[23]:https://opensource.com/users/adrianreber +[24]:https://opensource.com/users/adrianreber +[25]:https://opensource.com/users/adrianreber +[26]:https://www.ansible.com/ +[27]:https://slurm.schedmd.com/ +[28]:https://opensource.com/tags/raspberry-pi +[29]:https://opensource.com/tags/programming +[30]:https://opensource.com/tags/linux +[31]:https://opensource.com/tags/ansible \ No newline at end of file diff --git a/sources/tech/20180125 Keep Accurate Time on Linux with NTP.md b/sources/tech/20180125 Keep Accurate Time on Linux with NTP.md new file mode 100644 index 0000000000..817931c2a4 --- /dev/null +++ b/sources/tech/20180125 Keep Accurate Time on Linux with NTP.md @@ -0,0 +1,146 @@ +Keep Accurate Time on Linux with NTP +====== + +![](https://www.linux.com/sites/lcom/files/styles/rendered_file/public/usno-amc.jpg?itok=KA8HwI02) + +How to keep the correct time and keep your computers synchronized without abusing time servers, using NTP and systemd. + +### What Time is It? + +Linux is funky when it comes to telling the time. You might think that the `time` tells the time, but it doesn't because it is a timer that measures how long a process runs. To get the time, you run the `date` command, and to view more than one date, you use `cal`. Timestamps on files are also a source of confusion as they are typically displayed in two different ways, depending on your distro defaults. This example is from Ubuntu 16.04 LTS: +``` +$ ls -l +drwxrwxr-x 5 carla carla 4096 Mar 27 2017 stuff +drwxrwxr-x 2 carla carla 4096 Dec 8 11:32 things +-rw-rw-r-- 1 carla carla 626052 Nov 21 12:07 fatpdf.pdf +-rw-rw-r-- 1 carla carla 2781 Apr 18 2017 oddlots.txt + +``` + +Some display the year, some display the time, which makes ordering your files rather a mess. The GNU default is files dated within the last six months display the time instead of the year. I suppose there is a reason for this. If your Linux does this, try `ls -l --time-style=long-iso` to display the timestamps all the same way, sorted alphabetically. See [How to Change the Linux Date and Time: Simple Commands][1] to learn all manner of fascinating ways to manage the time on Linux. + +### Check Current Settings + +NTP, the network time protocol, is the old-fashioned way of keeping correct time on computers. `ntpd`, the NTP daemon, periodically queries a public time server and adjusts your system time as needed. It's a simple lightweight protocol that is easy to set up for basic use. Systemd has barged into NTP territory with the `systemd-timesyncd.service`, which acts as a client to `ntpd`. + +Before messing with NTP, let's take a minute to check that current time settings are correct. + +There are (at least) two timekeepers on your system: system time, which is managed by the Linux kernel, and the hardware clock on your motherboard, which is also called the real-time clock (RTC). When you enter your system BIOS, you see the hardware clock time and you can change its settings. When you install a new Linux, and in some graphical time managers, you are asked if you want your RTC set to the UTC (Coordinated Universal Time) zone. It should be set to UTC, because all time zone and daylight savings time calculations are based on UTC. Use the `hwclock` command to check: +``` +$ sudo hwclock --debug +hwclock from util-linux 2.27.1 +Using the /dev interface to the clock. +Hardware clock is on UTC time +Assuming hardware clock is kept in UTC time. +Waiting for clock tick... +...got clock tick +Time read from Hardware Clock: 2018/01/22 22:14:31 +Hw clock time : 2018/01/22 22:14:31 = 1516659271 seconds since 1969 +Time since last adjustment is 1516659271 seconds +Calculated Hardware Clock drift is 0.000000 seconds +Mon 22 Jan 2018 02:14:30 PM PST .202760 seconds + +``` + +"Hardware clock is kept in UTC time" confirms that your RTC is on UTC, even though it translates the time to your local time. If it were set to local time it would report "Hardware clock is kept in local time." + +You should have a `/etc/adjtime` file. If you don't, sync your RTC to system time: +``` +$ sudo hwclock -w + +``` + +This should generate the file, and the contents should look like this example: +``` +$ cat /etc/adjtime +0.000000 1516661953 0.000000 +1516661953 +UTC + +``` + +The new-fangled systemd way is to run `timedatectl`, which does not need root permissions: +``` +$ timedatectl + Local time: Mon 2018-01-22 14:17:51 PST + Universal time: Mon 2018-01-22 22:17:51 UTC + RTC time: Mon 2018-01-22 22:17:51 + Time zone: America/Los_Angeles (PST, -0800) + Network time on: yes +NTP synchronized: yes + RTC in local TZ: no + +``` + +"RTC in local TZ: no" confirms that it is on UTC time. What if it is on local time? There are, as always, multiple ways to change it. The easy way is with a nice graphical configuration tool, like YaST in openSUSE. You can use `timedatectl`: +``` +$ timedatectl set-local-rtc 0 +``` + +Or edit `/etc/adjtime`, replacing UTC with LOCAL. + +### systemd-timesyncd Client + +Now I'm tired, and we've just gotten to the good part. Who knew timekeeping was so complex? We haven't even scratched the surface; read `man 8 hwclock` to get an idea of how time is kept on computers. + +Systemd provides the `systemd-timesyncd.service` client, which queries remote time servers and adjusts your system time. Configure your servers in `/etc/systemd/timesyncd.conf`. Most Linux distributions provide a default configuration that points to time servers that they maintain, like Fedora: +``` +[Time] +#NTP= +#FallbackNTP=0.fedora.pool.ntp.org 1.fedora.pool.ntp.org + +``` + +You may enter any other servers you desire, such as your own local NTP server, on the `NTP=` line in a space-delimited list. (Remember to uncomment this line.) Anything you put on the `NTP=` line overrides the fallback. + +What if you are not using systemd? Then you need only NTP. + +### Setting up NTP Server and Client + +It is a good practice to set up your own LAN NTP server, so that you are not pummeling public NTP servers from all of your computers. On most Linuxes NTP comes in the `ntp` package, and most of them provide `/etc/ntp.conf` to configure the service. Consult [NTP Pool Time Servers][2] to find the NTP server pool that is appropriate for your region. Then enter 4-5 servers in your `/etc/ntp.conf` file, with each server on its own line: +``` +driftfile /var/ntp.drift +logfile /var/log/ntp.log +server 0.europe.pool.ntp.org +server 1.europe.pool.ntp.org +server 2.europe.pool.ntp.org +server 3.europe.pool.ntp.org + +``` + +The `driftfile` tells `ntpd` where to store the information it needs to quickly synchronize your system clock with the time servers at startup, and your logs should have their own home instead of getting dumped into the syslog. Use your Linux distribution defaults for these files if it provides them. + +Now start the daemon; on most Linuxes this is `sudo systemctl start ntpd`. Let it run for a few minutes, then check its status: +``` +$ ntpq -p + remote refid st t when poll reach delay offset jitter +============================================================== ++dev.smatwebdesi 192.168.194.89 3 u 25 64 37 92.456 -6.395 18.530 +*chl.la 127.67.113.92 2 u 23 64 37 75.175 8.820 8.230 ++four0.fairy.mat 35.73.197.144 2 u 22 64 37 116.272 -10.033 40.151 +-195.21.152.161 195.66.241.2 2 u 27 64 37 107.559 1.822 27.346 + +``` + +I have no idea what any of that means, other than your daemon is talking to the remote time servers, and that is what you want. To permanently enable it, run `sudo systemctl enable ntpd`. If your Linux doesn't use systemd then it is your homework to figure out how to run `ntpd`. + +Now you can set up `systemd-timesyncd` on your other LAN hosts to use your local NTP server, or install NTP on them and enter your local server in their `/etc/ntp.conf` files. + +NTP servers take a beating, and demand continually increases. You can help by running your own public NTP server. Come back next week to learn how. + +Learn more about Linux through the free ["Introduction to Linux" ][3]course from The Linux Foundation and edX. + +-------------------------------------------------------------------------------- + +via: https://www.linux.com/learn/intro-to-linux/2018/1/keep-accurate-time-linux-ntp + +作者：[CARLA SCHRODER][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.linux.com/users/cschroder +[1]:https://www.linux.com/learn/how-change-linux-date-and-time-simple-commands +[2]:http://support.ntp.org/bin/view/Servers/NTPPoolServers +[3]:https://training.linuxfoundation.org/linux-courses/system-administration-training/introduction-to-linux diff --git a/sources/tech/20180126 An introduction to the Web Simple Perl module a minimalist web framework.md b/sources/tech/20180126 An introduction to the Web Simple Perl module a minimalist web framework.md new file mode 100644 index 0000000000..ab8c29b2b6 --- /dev/null +++ b/sources/tech/20180126 An introduction to the Web Simple Perl module a minimalist web framework.md @@ -0,0 +1,106 @@ +An introduction to the Web::Simple Perl module, a minimalist web framework +============================================================ + +### Perl module Web::Simple is easy to learn and packs a big enough punch for a variety of one-offs and smaller services. + + +![An introduction to the Web::Simple Perl module, a minimalist web framework](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/openweb-osdc-lead.png?itok=yjU4KliG "An introduction to the Web::Simple Perl module, a minimalist web framework") +Image credits : [You as a Machine][10]. Modified by Rikki Endsley. [CC BY-SA 2.0][11]. + +One of the more-prominent members of the Perl community is [Matt Trout][12], technical director at [Shadowcat Systems][13]. He's been building core tools for Perl applications for years, including being a co-maintaner of the [Catalyst][14] MVC (Model, View, Controller) web framework, creator of the [DBIx::Class][15] object-management system, and much more. In person, he's energetic, interesting, brilliant, and sometimes hard to keep up with. When Matt writes code…well, think of a runaway chainsaw, with the trigger taped down and the safety features disabled. He's off and running, and you never quite know what will come out. Two things are almost certain: the module will precisely fit the purpose Matt has in mind, and it will show up on CPAN for others to use. + + +One of Matt's special-purpose modules is [Web::Simple][23]. Touted as "a quick and easy way to build simple web applications," it is a stripped-down, minimalist web framework, with an easy to learn interface. Web::Simple is not at all designed for a large-scale application; however, it may be ideal for a small tool that does one or two things in a lower-traffic environment. I can also envision it being used for rapid prototyping if you wanted to create quick wireframes of a new application for demonstrations. + +### Installation, and a quick "Howdy!" + +You can install the module using `cpan` or `cpanm`. Once you've got it installed, you're ready to write simple web apps without having to hassle with managing the connections or any of that—just your functionality. Here's a quick example: + +``` +#!/usr/bin/perl +package HelloReader; +use Web::Simple; + +sub dispatch_request { +  GET => sub { +    [ 200, [ 'Content-type', 'text/plain' ], [ 'Howdy, Opensource.com reader!' ] ] +  }, +  '' => sub { +    [ 405, [ 'Content-type', 'text/plain' ], [ 'You cannot do that, friend. Sorry.' ] ] +  } +} + +HelloReader->run_if_script; +``` + +There are a couple of things to notice right off. For one, I didn't `use strict` and `use warnings` like I usually would. Web::Simple imports those for you, so you don't have to. It also imports [Moo][16], a minimalist OO framework, so if you know Moo and want to use it here, you can! The heart of the system lies in the `dispatch_request`method, which you must define in your application. Each entry in the method is a match string, followed by a subroutine to respond if that string matches. The subroutine must return an array reference containing status, headers, and content of the reply to the request. + +### Matching + +The matching system in Web::Simple is powerful, allowing for complicated matches, passing parameters in a URL, query parameters, and extension matches, in pretty much any combination you want. As you can see in the example above, starting with a capital letter will match on the request method, and you can combine that with a path match easily: + +``` +'GET + /person/*' => sub { +  my ($self, $person) = @_; +  # write some code to retrieve and display a person +  }, +'POST + /person/* + %*' => sub { +  my ($self, $person, $params) = @_; +  # write some code to modify a person, perhaps +  } +``` + +In the latter case, the third part of the match indicates that we should pick up all the POST parameters and put them in a hashref called `$params` for use by the subroutine. Using `?` instead of `%` in that part of the match would pick up query parameters, as normally used in a GET request. There's also a useful exported subroutine called `redispatch_to`. This tool lets you redirect, without using a 3xx redirect; it's handled internally, invisible to the user. So: + +``` +'GET + /some/url' => sub { +  redispatch_to '/some/other/url'; +} +``` + +A GET request to `/some/url` would get handled as if it was sent to `/some/other/url`, without a redirect, and the user won't see a redirect in their browser. + +I've just scratched the surface with this module. If you're looking for something production-ready for larger projects, you'll be better off with [Dancer][17] or [Catalyst][18]. But with its light weight and built-in Moo integration, Web::Simple packs a big enough punch for a variety of one-offs and smaller services. + +### About the author + + [![](https://opensource.com/sites/default/files/styles/profile_pictures/public/pictures/dsc_0028.jpg?itok=RS0GBh25)][19] Ruth Holloway - Ruth Holloway has been a system administrator and software developer for a long, long time, getting her professional start on a VAX 11/780, way back when. She spent a lot of her career (so far) serving the technology needs of libraries, and has been a contributor since 2008 to the Koha open source library automation suite.Ruth is currently a Perl Developer at cPanel in Houston, and also serves as chief of staff for an obnoxious cat. In her copious free time, she occasionally reviews old romance... [more about Ruth Holloway][7][More about me][8] + +-------------------------------------------------------------------------------- + +via: https://opensource.com/article/18/1/introduction-websimple-perl-module-minimalist-web-framework + +作者：[Ruth Holloway ][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://opensource.com/users/druthb +[1]:https://opensource.com/tags/python?src=programming_resource_menu1 +[2]:https://opensource.com/tags/javascript?src=programming_resource_menu2 +[3]:https://opensource.com/tags/perl?src=programming_resource_menu3 +[4]:https://developers.redhat.com/?intcmp=7016000000127cYAAQ&src=programming_resource_menu4 +[5]:http://perldoc.perl.org/functions/package.html +[6]:https://opensource.com/article/18/1/introduction-websimple-perl-module-minimalist-web-framework?rate=ICN35y076ElpInDKoMqp-sN6f4UVF-n2Qt6dL6lb3kM +[7]:https://opensource.com/users/druthb +[8]:https://opensource.com/users/druthb +[9]:https://opensource.com/user/36051/feed +[10]:https://www.flickr.com/photos/youasamachine/8025582590/in/photolist-decd6C-7pkccp-aBfN9m-8NEffu-3JDbWb-aqf5Tx-7Z9MTZ-rnYTRu-3MeuPx-3yYwA9-6bSLvd-irmvxW-5Asr4h-hdkfCA-gkjaSQ-azcgct-gdV5i4-8yWxCA-9G1qDn-5tousu-71V8U2-73D4PA-iWcrTB-dDrya8-7GPuxe-5pNb1C-qmnLwy-oTxwDW-3bFhjL-f5Zn5u-8Fjrua-bxcdE4-ddug5N-d78G4W-gsYrFA-ocrBbw-pbJJ5d-682rVJ-7q8CbF-7n7gDU-pdfgkJ-92QMx2-aAmM2y-9bAGK1-dcakkn-8rfyTz-aKuYvX-hqWSNP-9FKMkg-dyRPkY +[11]:https://creativecommons.org/licenses/by/2.0/ +[12]:https://shadow.cat/resources/bios/matt_short/ +[13]:https://shadow.cat/ +[14]:https://metacpan.org/pod/Catalyst +[15]:https://metacpan.org/pod/DBIx::Class +[16]:https://metacpan.org/pod/Moo +[17]:http://perldancer.org/ +[18]:http://www.catalystframework.org/ +[19]:https://opensource.com/users/druthb +[20]:https://opensource.com/users/druthb +[21]:https://opensource.com/users/druthb +[22]:https://opensource.com/article/18/1/introduction-websimple-perl-module-minimalist-web-framework#comments +[23]:https://metacpan.org/pod/Web::Simple +[24]:https://opensource.com/tags/perl +[25]:https://opensource.com/tags/programming +[26]:https://opensource.com/tags/perl-column +[27]:https://opensource.com/tags/web-development \ No newline at end of file diff --git a/sources/tech/20180126 Creating an Adventure Game in the Terminal with ncurses.md b/sources/tech/20180126 Creating an Adventure Game in the Terminal with ncurses.md new file mode 100644 index 0000000000..221c53a4ed --- /dev/null +++ b/sources/tech/20180126 Creating an Adventure Game in the Terminal with ncurses.md @@ -0,0 +1,324 @@ +Creating an Adventure Game in the Terminal with ncurses +====== +How to use curses functions to read the keyboard and manipulate the screen. + +My [previous article][1] introduced the ncurses library and provided a simple program that demonstrated a few curses functions to put text on the screen. In this follow-up article, I illustrate how to use a few other curses functions. + +### An Adventure + +When I was growing up, my family had an Apple II computer. It was on this machine that my brother and I taught ourselves how to write programs in AppleSoft BASIC. After writing a few math puzzles, I moved on to creating games. Having grown up in the 1980s, I already was a fan of the Dungeons and Dragons tabletop games, where you role-played as a fighter or wizard on some quest to defeat monsters and plunder loot in strange lands. So it shouldn't be surprising that I also created a rudimentary adventure game. + +The AppleSoft BASIC programming environment supported a neat feature: in standard resolution graphics mode (GR mode), you could probe the color of a particular pixel on the screen. This allowed a shortcut to create an adventure game. Rather than create and update an in-memory map that was transferred to the screen periodically, I could rely on GR mode to maintain the map for me, and my program could query the screen as the player's character moved around the screen. Using this method, I let the computer do most of the hard work. Thus, my top-down adventure game used blocky GR mode graphics to represent my game map. + +My adventure game used a simple map that represented a large field with a mountain range running down the middle and a large lake on the upper-left side. I might crudely draw this map for a tabletop gaming campaign to include a narrow path through the mountains, allowing the player to pass to the far side. + +![](http://www.linuxjournal.com/files/linuxjournal.com/ufiles/imagecache/large-550px-centered/u1000009/quest-map.jpg) + +Figure 1. A simple Tabletop Game Map with a Lake and Mountains + +You can draw this map in cursesusing characters to represent grass, mountains and water. Next, I describe how to do just that using curses functions and how to create and play a similar adventure game in the Linux terminal. + +### Constructing the Program + +In my last article, I mentioned that most curses programs start with the same set of instructions to determine the terminal type and set up the curses environment: + +``` +initscr(); +cbreak(); +noecho(); + +``` + +For this program, I add another statement: + +``` +keypad(stdscr, TRUE); + +``` + +The TRUE flag allows curses to read the keypad and function keys from the user's terminal. If you want to use the up, down, left and right arrow keys in your program, you need to use keypad(stdscr, TRUE) here. + +Having done that, you now can start drawing to the terminal screen. The curses functions include several ways to draw text on the screen. In my previous article, I demonstrated the addch() and addstr() functions and their associated mvaddch() and mvaddstr() counterparts that first moved to a specific location on the screen before adding text. To create the adventure game map on the terminal, you can use another set of functions: vline() and hline(), and their partner functions mvvline() and mvhline(). These mv functions accept screen coordinates, a character to draw and how many times to repeat that character. For example, mvhline(1, 2, '-', 20) will draw a line of 20 dashes starting at line 1, column 2. + +To draw the map to the terminal screen programmatically, let's define this draw_map() function: + +``` +#define GRASS ' ' +#define EMPTY '.' +#define WATER '~' +#define MOUNTAIN '^' +#define PLAYER '*' + +void draw_map(void) +{ + int y, x; + + /* draw the quest map */ + + /* background */ + + for (y = 0; y < LINES; y++) { + mvhline(y, 0, GRASS, COLS); + } + + /* mountains, and mountain path */ + + for (x = COLS / 2; x < COLS * 3 / 4; x++) { + mvvline(0, x, MOUNTAIN, LINES); + } + + mvhline(LINES / 4, 0, GRASS, COLS); + + /* lake */ + + for (y = 1; y < LINES / 2; y++) { + mvhline(y, 1, WATER, COLS / 3); + } +} + +``` + +In drawing this map, note the use of mvvline() and mvhline() to fill large chunks of characters on the screen. I created the fields of grass by drawing horizontal lines (mvhline) of characters starting at column 0, for the entire height and width of the screen. I added the mountains on top of that by drawing vertical lines (mvvline), starting at row 0, and a mountain path by drawing a single horizontal line (mvhline). And, I created the lake by drawing a series of short horizontal lines (mvhline). It may seem inefficient to draw overlapping rectangles in this way, but remember that curses doesn't actually update the screen until I call the refresh() function later. + +Having drawn the map, all that remains to create the game is to enter a loop where the program waits for the user to press one of the up, down, left or right direction keys and then moves a player icon appropriately. If the space the player wants to move into is unoccupied, it allows the player to go there. + +You can use curses as a shortcut. Rather than having to instantiate a version of the map in the program and replicate this map to the screen, you can let the screen keep track of everything for you. The inch() function, and associated mvinch() function, allow you to probe the contents of the screen. This allows you to query curses to find out whether the space the player wants to move into is already filled with water or blocked by mountains. To do this, you'll need a helper function that you'll use later: + +``` +int is_move_okay(int y, int x) +{ + int testch; + + /* return true if the space is okay to move into */ + + testch = mvinch(y, x); + return ((testch == GRASS) || (testch == EMPTY)); +} + +``` + +As you can see, this function probes the location at column y, row x and returns true if the space is suitably unoccupied, or false if not. + +That makes it really easy to write a navigation loop: get a key from the keyboard and move the user's character around depending on the up, down, left and right arrow keys. Here's a simplified version of that loop: + +``` + + do { + ch = getch(); + + /* test inputted key and determine direction */ + + switch (ch) { + case KEY_UP: + if ((y > 0) && is_move_okay(y - 1, x)) { + y = y - 1; + } + break; + case KEY_DOWN: + if ((y < LINES - 1) && is_move_okay(y + 1, x)) { + y = y + 1; + } + break; + case KEY_LEFT: + if ((x > 0) && is_move_okay(y, x - 1)) { + x = x - 1; + } + break; + case KEY_RIGHT + if ((x < COLS - 1) && is_move_okay(y, x + 1)) { + x = x + 1; + } + break; + } + } + while (1); + +``` + +To use this in a game, you'll need to add some code inside the loop to allow other keys (for example, the traditional WASD movement keys), provide a method for the user to quit the game and move the player's character around the screen. Here's the program in full: + +``` + +/* quest.c */ + +#include +#include + +#define GRASS ' ' +#define EMPTY '.' +#define WATER '~' +#define MOUNTAIN '^' +#define PLAYER '*' + +int is_move_okay(int y, int x); +void draw_map(void); + +int main(void) +{ + int y, x; + int ch; + + /* initialize curses */ + + initscr(); + keypad(stdscr, TRUE); + cbreak(); + noecho(); + + clear(); + + /* initialize the quest map */ + + draw_map(); + + /* start player at lower-left */ + + y = LINES - 1; + x = 0; + + do { + /* by default, you get a blinking cursor - use it to indicate player */ + + mvaddch(y, x, PLAYER); + move(y, x); + refresh(); + + ch = getch(); + + /* test inputted key and determine direction */ + + switch (ch) { + case KEY_UP: + case 'w': + case 'W': + if ((y > 0) && is_move_okay(y - 1, x)) { + mvaddch(y, x, EMPTY); + y = y - 1; + } + break; + case KEY_DOWN: + case 's': + case 'S': + if ((y < LINES - 1) && is_move_okay(y + 1, x)) { + mvaddch(y, x, EMPTY); + y = y + 1; + } + break; + case KEY_LEFT: + case 'a': + case 'A': + if ((x > 0) && is_move_okay(y, x - 1)) { + mvaddch(y, x, EMPTY); + x = x - 1; + } + break; + case KEY_RIGHT: + case 'd': + case 'D': + if ((x < COLS - 1) && is_move_okay(y, x + 1)) { + mvaddch(y, x, EMPTY); + x = x + 1; + } + break; + } + } + while ((ch != 'q') && (ch != 'Q')); + + endwin(); + + exit(0); +} + +int is_move_okay(int y, int x) +{ + int testch; + + /* return true if the space is okay to move into */ + + testch = mvinch(y, x); + return ((testch == GRASS) || (testch == EMPTY)); +} + +void draw_map(void) +{ + int y, x; + + /* draw the quest map */ + + /* background */ + + for (y = 0; y < LINES; y++) { + mvhline(y, 0, GRASS, COLS); + } + + /* mountains, and mountain path */ + + for (x = COLS / 2; x < COLS * 3 / 4; x++) { + mvvline(0, x, MOUNTAIN, LINES); + } + + mvhline(LINES / 4, 0, GRASS, COLS); + + /* lake */ + + for (y = 1; y < LINES / 2; y++) { + mvhline(y, 1, WATER, COLS / 3); + } +} + +``` + +In the full program listing, you can see the complete arrangement of curses functions to create the game: + +1) Initialize the curses environment. + +2) Draw the map. + +3) Initialize the player coordinates (lower-left). + +4) Loop: + +* Draw the player's character. + +* Get a key from the keyboard. + +* Adjust the player's coordinates up, down, left or right, accordingly. + +* Repeat. + +5) When done, close the curses environment and exit. + +### Let's Play + +When you run the game, the player's character starts in the lower-left corner. As the player moves around the play area, the program creates a "trail" of dots. This helps show where the player has been before, so the player can avoid crossing the path unnecessarily. + +![](http://www.linuxjournal.com/files/linuxjournal.com/ufiles/imagecache/large-550px-centered/u1000009/quest-start.png) + +Figure 2\. The player starts the game in the lower-left corner. + +![](http://www.linuxjournal.com/files/linuxjournal.com/ufiles/imagecache/large-550px-centered/u1000009/quest-1.png) + +Figure 3\. The player can move around the play area, such as around the lake and through the mountain pass. + +To create a complete adventure game on top of this, you might add random encounters with various monsters as the player navigates his or her character around the play area. You also could include special items the player could discover or loot after defeating enemies, which would enhance the player's abilities further. + +But to start, this is a good program for demonstrating how to use the curses functions to read the keyboard and manipulate the screen. + +### Next Steps + +This program is a simple example of how to use the curses functions to update and read the screen and keyboard. You can do so much more with curses, depending on what you need your program to do. In a follow up article, I plan to show how to update this sample program to use colors. In the meantime, if you are interested in learning more about curses, I encourage you to read Pradeep Padala's [NCURSES Programming HOWTO][2] at the Linux Documentation Project. + + +-------------------------------------------------------------------------------- + +via: http://www.linuxjournal.com/content/creating-adventure-game-terminal-ncurses + +作者：[Jim Hall][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://www.linuxjournal.com/users/jim-hall +[1]:http://www.linuxjournal.com/content/getting-started-ncurses +[2]:http://tldp.org/HOWTO/NCURSES-Programming-HOWTO diff --git a/sources/tech/20180126 How To Manage NodeJS Packages Using Npm.md b/sources/tech/20180126 How To Manage NodeJS Packages Using Npm.md new file mode 100644 index 0000000000..ac27816a7b --- /dev/null +++ b/sources/tech/20180126 How To Manage NodeJS Packages Using Npm.md @@ -0,0 +1,372 @@ +How To Manage NodeJS Packages Using Npm +====== + +![](https://www.ostechnix.com/wp-content/uploads/2018/01/npm-720x340.png) + +A while ago, we have published a guide to [**manage Python packages using PIP**][1]. Today, we are going to discuss how to manage NodeJS packages using Npm. NPM is the largest software registry that contains over 600,000 packages. Everyday, developers across the world shares and downloads packages through npm. In this guide, I will explain the the basics of working with npm, such as installing packages(locally and globally), installing certain version of a package, updating, removing and managing NodeJS packages and so on. + +### Manage NodeJS Packages Using Npm + +##### Installing NPM + +Since npm is written in NodeJS, we need to install NodeJS in order to use npm. To install NodeJS on different Linux distributions, refer the following link. + +Once installed, ensure that NodeJS and NPM have been properly installed. There are couple ways to do this. + +To check where node has been installed: +``` +$ which node +/home/sk/.nvm/versions/node/v9.4.0/bin/node +``` + +Check its version: +``` +$ node -v +v9.4.0 +``` + +Log in to Node REPL session: +``` +$ node +> .help +.break Sometimes you get stuck, this gets you out +.clear Alias for .break +.editor Enter editor mode +.exit Exit the repl +.help Print this help message +.load Load JS from a file into the REPL session +.save Save all evaluated commands in this REPL session to a file +> .exit +``` + +Check where npm installed: +``` +$ which npm +/home/sk/.nvm/versions/node/v9.4.0/bin/npm +``` + +And the version: +``` +$ npm -v +5.6.0 +``` + +Great! Node and NPM have been installed and are working! As you may have noticed, I have installed NodeJS and NPM in my $HOME directory to avoid permission issues while installing modules globally. This is the recommended method by NodeJS team. + +Well, let us go ahead to see managing NodeJS modules (or packages) using npm. + +##### Installing NodeJS modules + +NodeJS modules can either be installed locally or globally(system wide). Now I am going to show how to install a package locally. + +**Install packages locally** + +To manage packages locally, we normally use **package.json** file. + +First, let us create our project directory. +``` +$ mkdir demo +``` +``` +$ cd demo +``` + +Create a package.json file inside your project's directory. To do so, run: +``` +$ npm init +``` + +Enter the details of your package such as name, version, author, github page etc., or just hit ENTER key to accept the default values and type **YES** to confirm. +``` +This utility will walk you through creating a package.json file. +It only covers the most common items, and tries to guess sensible defaults. + +See `npm help json` for definitive documentation on these fields +and exactly what they do. + +Use `npm install ` afterwards to install a package and +save it as a dependency in the package.json file. + +Press ^C at any time to quit. +package name: (demo) +version: (1.0.0) +description: demo nodejs app +entry point: (index.js) +test command: +git repository: +keywords: +author: +license: (ISC) +About to write to /home/sk/demo/package.json: + +{ + "name": "demo", + "version": "1.0.0", + "description": "demo nodejs app", + "main": "index.js", + "scripts": { + "test": "echo \"Error: no test specified\" && exit 1" + }, + "author": "", + "license": "ISC" +} + +Is this ok? (yes) yes +``` + +The above command initializes your project and create package.json file. + +You can also do this non-interactively using command: +``` +npm init --y +``` + +This will create a package.json file quickly with default values without the user interaction. + +Now let us install package named [**commander**][2]. +``` +$ npm install commander +``` + +Sample output: +``` +npm notice created a lockfile as package-lock.json. You should commit this file. +npm WARN demo@1.0.0 No repository field. + ++ commander@2.13.0 +added 1 package in 2.519s +``` + +This will create a directory named **" node_modules"** (if it doesn't exist already) in the project's root directory and download the packages in it. + +Let us check the package.json file. +``` +$ cat package.json +{ + "name": "demo", + "version": "1.0.0", + "description": "demo nodejs app", + "main": "index.js", + "scripts": { + "test": "echo \"Error: no test specified\" && exit 1" + }, + "author": "", + "license": "ISC", + **"dependencies": {** +**"commander": "^2.13.0"** + } +} +``` + +You will see the dependencies have been added. The caret ( **^** ) at the front of the version number indicates that when installing, npm will pull the highest version of the package it can find. +``` +$ ls node_modules/ +commander +``` + +The advantage of package.json file is if you had the package.json file in your project's directory, you can just type "npm install", then npm will look into the dependencies that listed in the file and download all of them. You can even share it with other developers or push into your GitHub repository, so when they type "npm install", they will get all the same packages that you have. + +You may also noticed another json file named **package-lock.json**. This file ensures that the dependencies remain the same on all systems the project is installed on. + +To use the installed package in your program, create a file **index.js** (or any name of you choice) in the project's directory with the actual code, and then run it using command: +``` +$ node index.js +``` + +**Install packages globally** + +If you want to use a package as a command line tool, then it is better to install it globally. This way, it works no matter which directory is your current directory. +``` +$ npm install async -g ++ async@2.6.0 +added 2 packages in 4.695s +``` + +Or, +``` +$ npm install async --global +``` + +To install a specific version of a package, we do: +``` +$ npm install async@2.6.0 --global +``` + +##### Updating NodeJS modules + +To update the local packages, go the the project's directory where the package.json is located and run: +``` +$ npm update +``` + +Then, run the following command to ensure all packages were updated. +``` +$ npm outdated +``` + +If there is no update, then it returns nothing. + +To find out which global packages need to be updated, run: +``` +$ npm outdated -g --depth=0 +``` + +If there is no output, then all packages are updated. + +To update the a single global package, run: +``` +$ npm update -g +``` + +To update all global packages, run: +``` +$ npm update -g +``` + +##### Listing NodeJS modules + +To list the local packages, go the project's directory and run: +``` +$ npm list +demo@1.0.0 /home/sk/demo +└── commander@2.13.0 +``` + +As you see, I have installed "commander" package in local mode. + +To list global packages, run this command from any location: +``` +$ npm list -g +``` + +Sample output: +``` +/home/sk/.nvm/versions/node/v9.4.0/lib +├─┬ async@2.6.0 +│ └── lodash@4.17.4 +└─┬ npm@5.6.0 + ├── abbrev@1.1.1 + ├── ansi-regex@3.0.0 + ├── ansicolors@0.3.2 + ├── ansistyles@0.1.3 + ├── aproba@1.2.0 + ├── archy@1.0.0 +[...] +``` + +This command will list all modules and their dependencies. + +To list only the top level modules, use -depth=0 option: +``` +$ npm list -g --depth=0 +/home/sk/.nvm/versions/node/v9.4.0/lib +├── async@2.6.0 +└── npm@5.6.0 +``` + +##### Searching NodeJS modules + +To search for a module, use "npm search" command: +``` +npm search +``` + +Example: +``` +$ npm search request +``` + +This command will display all modules that contains the search string "request". + +##### Removing NodeJS modules + +To remove a local package, go to the project's directory and run following command to remove the package from your **node_modules** directory: +``` +$ npm uninstall +``` + +To remove it from the dependencies in **package.json** file, use the **save** flag like below: +``` +$ npm uninstall --save + +``` + +To remove the globally installed packages, run: +``` +$ npm uninstall -g +``` + +##### Cleaning NPM cache + +By default, NPM keeps the copy of a installed package in the cache folder named npm in your $HOME directory when installing it. So, you can install it next time without having to download again. + +To view the cached modules: +``` +$ ls ~/.npm +``` + +The cache folder gets flooded with all old packages over time. It is better to clean the cache from time to time. + +As of npm@5, the npm cache self-heals from corruption issues and data extracted from the cache is guaranteed to be valid. If you want to make sure everything is consistent, run: +``` +$ npm cache verify +``` + +To clear the entire cache, run: +``` +$ npm cache clean --force +``` + +##### Viewing NPM configuration + +To view the npm configuration, type: +``` +$ npm config list +``` + +Or, +``` +$ npm config ls +``` + +Sample output: +``` +; cli configs +metrics-registry = "https://registry.npmjs.org/" +scope = "" +user-agent = "npm/5.6.0 node/v9.4.0 linux x64" + +; node bin location = /home/sk/.nvm/versions/node/v9.4.0/bin/node +; cwd = /home/sk +; HOME = /home/sk +; "npm config ls -l" to show all defaults. +``` + +To display the current global location: +``` +$ npm config get prefix +/home/sk/.nvm/versions/node/v9.4.0 +``` + +And, that's all for now. What we have just covered here is just the basics. NPM is a vast topic. For more details, head over to the the [**NPM Getting Started**][3] guide. + +Hope this was useful. More good stuffs to come. Stay tuned! + +Cheers! + + + +-------------------------------------------------------------------------------- + +via: https://www.ostechnix.com/manage-nodejs-packages-using-npm/ + +作者：[SK][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.ostechnix.com/author/sk/ +[1]:https://www.ostechnix.com/manage-python-packages-using-pip/ +[2]:https://www.npmjs.com/package/commander +[3]:https://docs.npmjs.com/getting-started/ diff --git a/sources/tech/20180126 How to Make a Minecraft Server - ThisHosting.Rocks.md b/sources/tech/20180126 How to Make a Minecraft Server - ThisHosting.Rocks.md new file mode 100644 index 0000000000..30c6ccc54e --- /dev/null +++ b/sources/tech/20180126 How to Make a Minecraft Server - ThisHosting.Rocks.md @@ -0,0 +1,418 @@ +translating by heart4lor + +How to Make a Minecraft Server – ThisHosting.Rocks +====== +We’ll show you how to make a Minecraft server with beginner-friendly step-by-step instructions. It will be a persistent multiplayer server that you can play on with your friends from all around the world. You don’t have to be in a LAN. + +### How to Make a Minecraft Server – Quick Guide + +This is our “Table of contents” if you’re in a hurry and want to go straight to the point. We recommend reading everything though. + +* [Learn stuff][1] (optional) + +* [Learn more stuff][2] (optional) + +* [Requirements][3] (required) + +* [Install and start the Minecraft server][4] (required) + +* [Run the server even after you log out of your VPS][5] (optional) + +* [Make the server automatically start at boot][6] (optional) + +* [Configure your Minecraft server][7] (required) + +* [FAQs][8] (optional) + +Before going into the actual instructions, a few things you should know: + +#### Reasons why you would NOT use a specialized Minecraft server hosting provider + +Since you’re here, you’re obviously interested in hosting your own Minecraft server. There are more reasons why you would not use a specialized Minecraft hosting provider, but here are a few: + +* They’re slow most of the time. This is because you actually share the resources with multiple users. It becomes overloaded at some point. Most of them oversell their servers too. + +* You don’t have full control over the Minecraft server or the actual server. You cannot customize anything you want to. + +* You’re limited. Those kinds of hosting plans are always limited in one way or another. + +Of course, there are positives to using a Minecraft hosting provider. The best upside is that you don’t actually have to do all the stuff we’ll write about below. But where’s the fun in that? +![🙂](https://s.w.org/images/core/emoji/2.3/svg/1f642.svg) + +#### Why you should NOT use your personal computer to make a Minecraft server + +We noticed lots of tutorials showing you how to host a server on your own computer. There are downsides to doing that, like: + +* Your home internet is not secured enough to handle DDoS attacks. Game servers are often prone to DDoS attacks, and your home network setup is most probably not secured enough to handle them. It’s most likely not powerful enough to handle a small attack. + +* You’ll need to handle port forwarding. If you’ve tried making a Minecraft server on your home network, you’ve surely stumbled upon port forwarding and had issues with it. + +* You’ll need to keep your computer on at all times. Your electricity bill will sky-rocket and you’ll add unnecessary load to your hardware. The hardware most servers use is enterprise-grade and designed to handle loads, with improved stability and longevity. + +* Your home internet is not fast enough. Home networks are not designed to handle multiplayer games. You’ll need a much larger internet plan to even consider making a small server. Luckily, data centers have multiple high-speed, enterprise-grade internet connections making sure they have (or strive to have) 100% uptime. + +* Your hardware is most likely not good enough. Again, servers use enterprise-grade hardware, latest and fastest CPUs, SSDs, and much more. Your personal computer most likely does not. + +* You probably use Windows/MacOS on your personal computer. Though this is debatable, we believe that Linux is much better for game hosting. Don’t worry, you don’t really need to know everything about Linux to make a Minecraft server (though it’s recommended). We’ll show you everything you need to know. + +Our tip is not to use your personal computer, though technically you can. It’s not expensive to buy a cloud server. We’ll show you how to make a Minecraft server on cloud hosting below. It’s easy if you carefully follow the steps. + +### Making a Minecraft Server – Requirements + +There are a few requirements. You should have and know all of this before continuing to the tutorial: + +* You’ll need a [Linux cloud server][9]. We recommend [Vultr][10]. Their prices are cheap, services are high-quality, customer support is great, all server hardware is high-end. Check the [Minecraft server requirements][11] to find out what kind of server you should get (resources like RAM and Disk space). We recommend getting the $20 per month server. They support hourly pricing so if you only need the server temporary for playing with friends, you’ll pay less. Choose the Ubuntu 16.04 distro during signup. Choose the closest server location to where your players live during the signup process. Keep in mind that you’ll be responsible for your server. So you’ll have to secure it and manage it. If you don’t want to do that, you can get a [managed server][12], in which case the hosting provider will likely make a Minecraft server for you. + +* You’ll need an SSH client to connect to the Linux cloud server. [PuTTy][13] is often recommended for beginners, but we also recommend [MobaXTerm][14]. There are many other SSH clients to choose from, so pick your favorite. + +* You’ll need to setup your server (basic security setup at least). Google it and you’ll find many tutorials. You can use [Linode’s Security Guide][15] and follow the exact steps on your [Vultr][16] server. + +* We’ll handle the software requirements like Java below. + +And finally, onto our actual tutorial: + +### How to Make a Minecraft Server on Ubuntu (Linux) + +These instructions are written for and tested on an Ubuntu 16.04 server from [Vultr][17]. Though they’ll also work on Ubuntu 14.04, [Ubuntu 18.04][18], and any other Ubuntu-based distro, and any other server provider. + +We’re using the default Vanilla server from Minecraft. You can use alternatives like CraftBukkit or Spigot that allow more customizations and plugins. Though if you use too many plugins you’ll essentially ruin the server. There are pros and cons to each one. Nevertheless, the instructions below are for the default Vanilla server to keep things simple and beginner-friendly. We may publish a tutorial for CraftBukkit soon if there’s an interest. + +#### 1. Login to your server + +We’ll use the root user. If you use a limited-user, you’ll have to execute most commands with ‘sudo’. You’ll get a warning if you’re doing something you don’t have enough permissions for. + +You can login to your server via your SSH client. Use your server IP and your port (most likely 22). + +After you log in, make sure you [secure your server][19]. + +#### 2. Update Ubuntu + +You should always first update your Ubuntu before you do anything else. You can update it with the following commands: + +``` +apt-get update && apt-get upgrade +``` + +Hit “enter” and/or “y” when prompted. + +#### 3. Install necessary tools + +You’ll need a few packages and tools for various things in this tutorial like text editing, making your server persistent etc. Install them with the following command: + +``` +apt-get install nano wget screen bash default-jdk ufw +``` + +Some of them may already be installed. + +#### 4. Download Minecraft Server + +First, create a directory where you’ll store your Minecraft server and all other files: + +``` +mkdir /opt/minecraft +``` + +And navigate to the new directory: + +``` +cd /opt/minecraft +``` + +Now you can download the Minecraft Server file. Go to the [download page][20] and get the link there. Download the file with wget: + +``` +wget https://s3.amazonaws.com/Minecraft.Download/versions/1.12.2/minecraft_server.1.12.2.jar +``` + +#### 5. Install the Minecraft server + +Once you’ve downloaded the server .jar file, you need to run it once and it will generate some files, including an eula.txt license file. The first time you run it, it will return an error and exit. That’s supposed to happen. Run in with the following command: + +``` +java -Xms2048M -Xmx3472M -jar minecraft_server.1.12.2.jar nogui +``` + +“-Xms2048M” is the minimum RAM that your Minecraft server can use and “-Xmx3472M” is the maximum. [Adjust][21] this based on your server’s resources. If you got the 4GB RAM server from [Vultr][22] you can leave them as-is, if you don’t use the server for anything else other than Minecraft. + +After that command ends and returns an error, a new eula.txt file will be generated. You need to accept the license in that file. You can do that by adding “eula=true” to the file with the following command: + +``` +sed -i.orig 's/eula=false/eula=true/g' eula.txt +``` + +You can now start the server again and access the Minecraft server console with that same java command from before: + +``` +java -Xms2048M -Xmx3472M -jar minecraft_server.1.12.2.jar nogui +``` + +Make sure you’re in the /opt/minecraft directory, or the directory where you installed your MC server. + +You’re free to stop here if you’re just testing this and need it for the short-term. If you’re having trouble loggin into the server, you’ll need to [configure your firewall][23]. + +The first time you successfully start the server it will take a bit longer to generate + +We’ll show you how to create a script so you can start the server with it. + +#### 6. Start the Minecraft server with a script, make it persistent, and enable it at boot + +To make things easier, we’ll create a bash script that will start the server automatically. + +So first, create a bash script with nano: + +``` +nano /opt/minecraft/startminecraft.sh +``` + +A new (blank) file will open. Paste the following: + +``` +#!/bin/bash +cd /opt/minecraft/ && java -Xms2048M -Xmx3472M -jar minecraft_server.1.12.2.jar nogui +``` + +If you’re new to nano – you can save and close the file with “CTRL + X”, then “Y”, and hitting enter. This script navigates to your Minecraft server directory you created previously and runs the java command for starting the server. You need to make it executable with the following command: + +``` +chmod +x startminecraft.sh +``` + +Then, you can start the server anytime with the following command: + +``` +/opt/minecraft/startminecraft.sh +``` + +But, if/when you log out of the SSH session the server will turn off. To keep the server up without being logged in all the time, you can use a screen session. A screen session basically means that it will keep running until the actual server reboots or turns off. + +Start a screen session with this command: + +``` +screen -S minecraft +``` + +Once you’re in the screen session (looks like you would start a new ssh session), you can use the bash script from earlier to start the server: + +``` +/opt/minecraft/startminecraft.sh +``` + +To get out of the screen session, you should press CTRL + A-D. Even after you get out of the screen session (detach), the server will keep running. You can safely log off your Ubuntu server now, and the Minecraft server you created will keep running. + +But, if the Ubuntu server reboots or shuts off, the screen session won’t work anymore. So **to do everything we did before automatically at boot** , do the following: + +Open the /etc/rc.local file: + +``` +nano /etc/rc.local +``` + +and add the following line above the “exit 0” line: + +``` +screen -dm -S minecraft /opt/minecraft/startminecraft.sh +exit 0 +``` + +Save and close the file. + +To access the Minecraft server console, just run the following command to attach to the screen session: + +``` +screen -r minecraft +``` + +That’s it for now. Congrats and have fun! You can now connect to your Minecraft server or configure/modify it. + +### Configure your Ubuntu Server + +You’ll, of course, need to set up your Ubuntu server and secure it if you haven’t already done so. Follow the [guide we mentioned earlier][24] and google it for more info. The configurations you need to do for your Minecraft server on your Ubuntu server are: + +#### Enable and configure the firewall + +First, if it’s not already enabled, you should enable UFW that you previously installed: + +``` +ufw enable +``` + +You should allow the default Minecraft server port: + +``` +ufw allow 25565/tcp +``` + +You should allow and deny other rules depending on how you use your server. You should deny ports like 80 and 443 if you don’t use the server for hosting websites. Google a UFW/Firewall guide for Ubuntu and you’ll get recommendations. Be careful when setting up your firewall, you may lock yourself out of your server if you block the SSH port. + +Since this is the default port, it often gets automatically scanned and attacked. You can prevent attacks by blocking access to anyone that’s not of your whitelist. + +First, you need to enable the whitelist mode in your [server.properties][25] file. To do that, open the file: + +``` +nano /opt/minecraft/server.properties +``` + +And change “white-list” line to “true”: + +``` +white-list=true +``` + +Save and close the file. + +Then restart your server (either by restarting your Ubuntu server or by running the start bash script again): + +``` +/opt/minecraft/startminecraft.sh +``` + +Access the Minecraft server console: + +``` +screen -r minecraft +``` + +And if you want someone to be able to join your server, you need to add them to the whitelist with the following command: + +``` +whitelist add PlayerUsername +``` + +To remove them from the whitelist, use: + +``` +whitelist remove PlayerUsername +``` + +Exit the screen session (server console) with CTRL + A-D. It’s worth noting that this will deny access to everyone but the whitelisted usernames. + + [![how to create a minecraft server](https://thishosting.rocks/wp-content/uploads/2018/01/create-a-minecraft-server.jpg)][26] + +### How to Make a Minecraft Server – FAQs + +We’ll answer some frequently asked questions about Minecraft Servers and our guide. + +#### How do I restart the Minecraft server? + +If you followed every step from our tutorial, including enabling the server to start on boot, you can just reboot your Ubuntu server. If you didn’t set it up to start at boot, you can just run the start script again which will restart the Minecraft server: + +``` +/opt/minecraft/startminecraft.sh +``` + +#### How do I configure my Minecraft server? + +You can configure your server using the [server.properties][27] file. Check the Minecraft Wiki for more info, though you can leave everything as-is and it will work perfectly fine. + +If you want to change the game mode, difficulty and stuff like that, you can use the server console. Access the server console by running: + +``` +screen -r minecraft +``` + +And execute [commands][28] there. Commands like: + +``` +difficulty hard +``` + +``` +gamemode survival @a +``` + +You may need to restart the server depending on what command you used. There are many more commands you can use, check the [wiki][29] for more. + +#### How do I upgrade my Minecraft server? + +If there’s a new release, you need to do this: + +Navigate to the minecraft directory: + +``` +cd /opt/minecraft +``` + +Download the latest version, example 1.12.3 with wget: + +``` +wget https://s3.amazonaws.com/Minecraft.Download/versions/1.12.3/minecraft_server.1.12.3.jar +``` + +Next, run and build the new server: + +``` +java -Xms2048M -Xmx3472M -jar minecraft_server.1.12.3.jar nogui +``` + +Finally, update your start script: + +``` +nano /opt/minecraft/startminecraft.sh +``` + +And update the version number accordingly: + +``` +#!/bin/bash +cd /opt/minecraft/ && java -Xms2048M -Xmx3472M -jar minecraft_server.1.12.3.jar nogui +``` + +Now you can restart the server and everything should go well. + +#### Why is your Minecraft server tutorial so long, and yet others are only 2 lines long?! + +We tried to make this beginner-friendly and be as detailed as possible. We also showed you how to make the Minecraft server persistent and start it automatically at boot, we showed you how to configure your server and everything. I mean, sure, you can start a Minecraft server with a couple of lines, but it would definitely suck, for more than one reason. + +#### I don’t know Linux or anything you wrote about here, how do I make a Minecraft server? + +Just read all of our article and copy and paste the commands. If you really don’t know how to do it all, [we can do it for you][30], or just get a [managed][31] server [provider][32] and let them do it for you. + +#### How do I install mods on my server? How do I install plugins? + +Our article is intended to be a starting guide. You should check the [Minecraft wiki][33] for more info, or just google it. There are plenty of tutorials online + +-------------------------------------------------------------------------------- + +via: https://thishosting.rocks/how-to-make-a-minecraft-server/ + +作者：[ThisHosting.Rocks][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://thishosting.rocks +[1]:https://thishosting.rocks/how-to-make-a-minecraft-server/#reasons +[2]:https://thishosting.rocks/how-to-make-a-minecraft-server/#not-pc +[3]:https://thishosting.rocks/how-to-make-a-minecraft-server/#requirements +[4]:https://thishosting.rocks/how-to-make-a-minecraft-server/#make-minecraft-server +[5]:https://thishosting.rocks/how-to-make-a-minecraft-server/#persistent +[6]:https://thishosting.rocks/how-to-make-a-minecraft-server/#boot +[7]:https://thishosting.rocks/how-to-make-a-minecraft-server/#configure-minecraft-server +[8]:https://thishosting.rocks/how-to-make-a-minecraft-server/#faqs +[9]:https://thishosting.rocks/cheap-cloud-hosting-providers-comparison/ +[10]:https://thishosting.rocks/go/vultr/ +[11]:https://minecraft.gamepedia.com/Server/Requirements/Dedicated +[12]:https://thishosting.rocks/best-cheap-managed-vps/ +[13]:https://www.chiark.greenend.org.uk/~sgtatham/putty/ +[14]:https://mobaxterm.mobatek.net/ +[15]:https://www.linode.com/docs/security/securing-your-server/ +[16]:https://thishosting.rocks/go/vultr/ +[17]:https://thishosting.rocks/go/vultr/ +[18]:https://thishosting.rocks/ubuntu-18-04-new-features-release-date/ +[19]:https://www.linode.com/docs/security/securing-your-server/ +[20]:https://minecraft.net/en-us/download/server +[21]:https://minecraft.gamepedia.com/Commands +[22]:https://thishosting.rocks/go/vultr/ +[23]:https://thishosting.rocks/how-to-make-a-minecraft-server/#configure-minecraft-server +[24]:https://www.linode.com/docs/security/securing-your-server/ +[25]:https://minecraft.gamepedia.com/Server.properties +[26]:https://thishosting.rocks/wp-content/uploads/2018/01/create-a-minecraft-server.jpg +[27]:https://minecraft.gamepedia.com/Server.properties +[28]:https://minecraft.gamepedia.com/Commands +[29]:https://minecraft.gamepedia.com/Commands +[30]:https://thishosting.rocks/support/ +[31]:https://thishosting.rocks/best-cheap-managed-vps/ +[32]:https://thishosting.rocks/best-cheap-managed-vps/ +[33]:https://minecraft.gamepedia.com/Minecraft_Wiki diff --git a/sources/tech/20180126 Linux kill Command Tutorial for Beginners (5 Examples).md b/sources/tech/20180126 Linux kill Command Tutorial for Beginners (5 Examples).md new file mode 100644 index 0000000000..8fcdedef0e --- /dev/null +++ b/sources/tech/20180126 Linux kill Command Tutorial for Beginners (5 Examples).md @@ -0,0 +1,113 @@ +Linux kill Command Tutorial for Beginners (5 Examples) +====== + +Sometimes, while working on a Linux machine, you'll see that an application or a command line process gets stuck (becomes unresponsive). Then in those cases, terminating it is the only way out. Linux command line offers a utility that you can use in these scenarios. It's called **kill**. + +In this tutorial, we will discuss the basics of kill using some easy to understand examples. But before we do that, it's worth mentioning that all examples in the article have been tested on an Ubuntu 16.04 machine. + +#### Linux kill command + +The kill command is usually used to kill a process. Internally it sends a signal, and depending on what you want to do, there are different signals that you can send using this tool. Following is the command's syntax: + +``` +kill [options] [...] +``` + +And here's how the tool's man page describes it: +``` +The default signal for kill is TERM. Use -l or -L to list available signals. Particularly useful +signals include HUP, INT, KILL, STOP, CONT, and 0. Alternate signals may be specified in three ways: +-9, -SIGKILL or -KILL. Negative PID values may be used to choose whole process groups; see the PGID +column in ps command output.  A PID of -1 is special; it indicates all processes except the kill +process  itself and init. +``` + +The following Q&A-styled examples should give you a better idea of how the kill command works. + +#### Q1. How to terminate a process using kill command? + +This is very easy - all you need to do is to get the pid of the process you want to kill, and then pass it to the kill command. + +``` +kill [pid] +``` + +For example, I wanted to kill the 'gthumb' process on my system. So i first used the ps command to fetch the application's pid, and then passed it to the kill command to terminate it. Here's the screenshot showing all this: + +[![How to terminate a process using kill command][1]][2] + +#### Q2. How to send a custom signal? + +As already mentioned in the introduction section, TERM is the default signal that kill sends to the application/process in question. However, if you want, you can send any other signal that kill supports using the **-s** command line option. + +``` +kill -s [signal] [pid] +``` + +For example, if a process isn't responding to the TERM signal (which allows the process to do final cleanup before quitting), you can go for the KILL signal (which doesn't let process do any cleanup). Following is the command you need to run in that case. + +``` +kill -s KILL [pid] +``` + +#### Q3. What all signals you can send using kill? + +Of course, the next logical question that'll come to your mind is how to know which all signals you can send using kill. Well, thankfully, there exists a command line option **-l** that lists all supported signals. + +``` +kill -l +``` + +Following is the output the above command produced on our system: + +[![What all signals you can send using kill][3]][4] + +#### Q4. What are the other ways in which signal can be sent? + +In one of the previous examples, we told you if you want to send the KILL signal, you can do it in the following way: + +``` +kill -s KILL [pid] +``` + +However, there are a couple of other alternatives as well: + +``` +kill -s SIGKILL [pid] + +kill -s 9 [pid] +``` + +The corresponding number can be known using the -l option we've already discussed in the previous example. + +#### Q5. How to kill all running process in one go? + +In case a user wants to kill all processes that they can (this depends on their privilege level), then instead of specifying a large number of process IDs, they can simply pass the -1 option to kill. + +For example: + +``` +kill -s KILL -1 +``` + +#### Conclusion + +The kill command is pretty straightforward to understand and use. There's a slight learning curve in terms of the list of signal options it offers, but as we explained in here, there's an option to take a quick look at that list as well. Just practice whatever we've discussed and you should be good to go. For more information, head to the tool's [man page][5]. + + +-------------------------------------------------------------------------------- + +via: https://www.howtoforge.com/linux-kill-command/ + +作者：[Himanshu Arora][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.howtoforge.com +[1]:https://www.howtoforge.com/images/usage_of_pfsense_to_block_dos_attack_/kill-default.png +[2]:https://www.howtoforge.com/images/usage_of_pfsense_to_block_dos_attack_/big/kill-default.png +[3]:https://www.howtoforge.com/images/usage_of_pfsense_to_block_dos_attack_/kill-l-option.png +[4]:https://www.howtoforge.com/images/usage_of_pfsense_to_block_dos_attack_/big/kill-l-option.png +[5]:https://linux.die.net/man/1/kill diff --git a/sources/tech/20180126 Running a Python application on Kubernetes.md b/sources/tech/20180126 Running a Python application on Kubernetes.md new file mode 100644 index 0000000000..4ce9f38726 --- /dev/null +++ b/sources/tech/20180126 Running a Python application on Kubernetes.md @@ -0,0 +1,280 @@ +Running a Python application on Kubernetes +============================================================ + +### This step-by-step tutorial takes you through the process of deploying a simple Python application on Kubernetes. + +![](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/build_structure_tech_program_code_construction.png?itok=nVsiLuag) +Image by : opensource.com + +Kubernetes is an open source platform that offers deployment, maintenance, and scaling features. It simplifies management of containerized Python applications while providing portability, extensibility, and self-healing capabilities. + +Whether your Python applications are simple or more complex, Kubernetes lets you efficiently deploy and scale them, seamlessly rolling out new features while limiting resources to only those required. + +In this article, I will describe the process of deploying a simple Python application to Kubernetes, including: + +* Creating Python container images + +* Publishing the container images to an image registry + +* Working with persistent volume + +* Deploying the Python application to Kubernetes + +### Requirements + +You will need Docker, kubectl, and this [source code][10]. + +Docker is an open platform to build and ship distributed applications. To install Docker, follow the [official documentation][11]. To verify that Docker runs your system: + +``` +$ docker info +Containers: 0 +Images: 289 +Storage Driver: aufs + Root Dir: /var/lib/docker/aufs + Dirs: 289 +Execution Driver: native-0.2 +Kernel Version: 3.16.0-4-amd64 +Operating System: Debian GNU/Linux 8 (jessie) +WARNING: No memory limit support +WARNING: No swap limit support +``` + +kubectl is a command-line interface for executing commands against a Kubernetes cluster. Run the shell script below to install kubectl: + +``` +curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl +``` + +Deploying to Kubernetes requires a containerized application. Let's review containerizing Python applications. + +### Containerization at a glance + +Containerization involves enclosing an application in a container with its own operating system. This full machine virtualization option has the advantage of being able to run an application on any machine without concerns about dependencies. + +Roman Gaponov's [article][12] serves as a reference. Let's start by creating a container image for our Python code. + +### Create a Python container image + +To create these images, we will use Docker, which enables us to deploy applications inside isolated Linux software containers. Docker is able to automatically build images using instructions from a Docker file. + +This is a Docker file for our Python application: + +``` +FROM python:3.6 +MAINTAINER XenonStack + +# Creating Application Source Code Directory +RUN mkdir -p /k8s_python_sample_code/src + +# Setting Home Directory for containers +WORKDIR /k8s_python_sample_code/src + +# Installing python dependencies +COPY requirements.txt /k8s_python_sample_code/src +RUN pip install --no-cache-dir -r requirements.txt + +# Copying src code to Container +COPY . /k8s_python_sample_code/src/app + +# Application Environment variables +ENV APP_ENV development + +# Exposing Ports +EXPOSE 5035 + +# Setting Persistent data +VOLUME ["/app-data"] + +# Running Python Application +CMD ["python", "app.py"] +``` + +This Docker file contains instructions to run our sample Python code. It uses the Python 3.5 development environment. + +### Build a Python Docker image + +We can now build the Docker image from these instructions using this command: + +``` +docker build -t k8s_python_sample_code . +``` + +This command creates a Docker image for our Python application. + +### Publish the container images + +We can publish our Python container image to different private/public cloud repositories, like Docker Hub, AWS ECR, Google Container Registry, etc. For this tutorial, we'll use Docker Hub. + +Before publishing the image, we need to tag it to a version: + +``` +docker tag k8s_python_sample_code:latest k8s_python_sample_code:0.1 +``` + +### Push the image to a cloud repository + +Using a Docker registry other than Docker Hub to store images requires you to add that container registry to the local Docker daemon and Kubernetes Docker daemons. You can look up this information for the different cloud registries. We'll use Docker Hub in this example. + +Execute this Docker command to push the image: + +``` +docker push k8s_python_sample_code +``` + +### Working with CephFS persistent storage + +Kubernetes supports many persistent storage providers, including AWS EBS, CephFS, GlusterFS, Azure Disk, NFS, etc. I will cover Kubernetes persistence storage with CephFS. + +To use CephFS for persistent data to Kubernetes containers, we will create two files: + +persistent-volume.yml + +``` +apiVersion: v1 +kind: PersistentVolume +metadata: +  name: app-disk1 +  namespace: k8s_python_sample_code +spec: +  capacity: +  storage: 50Gi +  accessModes: +  - ReadWriteMany +  cephfs: +  monitors: +    - "172.17.0.1:6789" +  user: admin +  secretRef: +    name: ceph-secret +  readOnly: false +``` + +persistent_volume_claim.yaml + +``` +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: +  name: appclaim1 +  namespace: k8s_python_sample_code +spec: +  accessModes: +  - ReadWriteMany +  resources: +  requests: +    storage: 10Gi +``` + +We can now use kubectl to add the persistent volume and claim to the Kubernetes cluster: + +``` +$ kubectl create -f persistent-volume.yml +$ kubectl create -f persistent-volume-claim.yml +``` + +We are now ready to deploy to Kubernetes. + +### Deploy the application to Kubernetes + +To manage the last mile of deploying the application to Kubernetes, we will create two important files: a service file and a deployment file. + +Create a file and name it `k8s_python_sample_code.service.yml` with the following content: + +``` +apiVersion: v1 +kind: Service +metadata: +  labels: +  k8s-app: k8s_python_sample_code +  name: k8s_python_sample_code +  namespace: k8s_python_sample_code +spec: +  type: NodePort +  ports: +  - port: 5035 +  selector: +  k8s-app: k8s_python_sample_code +``` + +Create a file and name it `k8s_python_sample_code.deployment.yml` with the following content: + +``` +apiVersion: extensions/v1beta1 +kind: Deployment +metadata: +  name: k8s_python_sample_code +  namespace: k8s_python_sample_code +spec: +  replicas: 1 +  template: +  metadata: +    labels: +    k8s-app: k8s_python_sample_code +  spec: +    containers: +    - name: k8s_python_sample_code +      image: k8s_python_sample_code:0.1 +      imagePullPolicy: "IfNotPresent" +      ports: +      - containerPort: 5035 +      volumeMounts: +        - mountPath: /app-data +          name: k8s_python_sample_code +     volumes:  +         - name: +           persistentVolumeClaim: +             claimName: appclaim1 +``` + +Finally, use kubectl to deploy the application to Kubernetes: + +``` +$ kubectl create -f k8s_python_sample_code.deployment.yml $ kubectl create -f k8s_python_sample_code.service.yml +``` + +Your application was successfully deployed to Kubernetes. + +You can verify whether your application is running by inspecting the running services: + +``` +kubectl get services +``` + +May Kubernetes free you from future deployment hassles! + + _Want to learn more about Python? Nanjekye's book, [Python 2 and 3 Compatibility][7]offers clean ways to write code that will run on both Python 2 and 3, including detailed examples of how to convert existing Python 2-compatible code to code that will run reliably on both Python 2 and 3._ + + +### About the author + + [![](https://opensource.com/sites/default/files/styles/profile_pictures/public/pictures/joannah-nanjekye.jpg?itok=F4RqEjoA)][13] Joannah Nanjekye - Straight Outta 256 , I choose Results over Reasons, Passionate Aviator, Show me the code.[More about me][8] + +-------------------------------------------------------------------------------- + +via: https://opensource.com/article/18/1/running-python-application-kubernetes + +作者：[Joannah Nanjekye ][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://opensource.com/users/nanjekyejoannah +[1]:https://opensource.com/resources/python?intcmp=7016000000127cYAAQ +[2]:https://opensource.com/resources/python/ides?intcmp=7016000000127cYAAQ +[3]:https://opensource.com/resources/python/gui-frameworks?intcmp=7016000000127cYAAQ +[4]:https://opensource.com/tags/python?intcmp=7016000000127cYAAQ +[5]:https://developers.redhat.com/?intcmp=7016000000127cYAAQ +[6]:https://opensource.com/article/18/1/running-python-application-kubernetes?rate=D9iKksKbd9q9vOVb92Mg-v0Iyqn0QVO5fbIERTbSHz4 +[7]:https://www.apress.com/gp/book/9781484229545 +[8]:https://opensource.com/users/nanjekyejoannah +[9]:https://opensource.com/user/196386/feed +[10]:https://github.com/jnanjekye/k8s_python_sample_code/tree/master +[11]:https://docs.docker.com/engine/installation/ +[12]:https://hackernoon.com/docker-tutorial-getting-started-with-python-redis-and-nginx-81a9d740d091 +[13]:https://opensource.com/users/nanjekyejoannah +[14]:https://opensource.com/users/nanjekyejoannah +[15]:https://opensource.com/users/nanjekyejoannah +[16]:https://opensource.com/tags/python +[17]:https://opensource.com/tags/kubernetes \ No newline at end of file diff --git a/sources/tech/20180127 How to install KVM on CentOS 7 - RHEL 7 Headless Server.md b/sources/tech/20180127 How to install KVM on CentOS 7 - RHEL 7 Headless Server.md new file mode 100644 index 0000000000..6dce30d6dc --- /dev/null +++ b/sources/tech/20180127 How to install KVM on CentOS 7 - RHEL 7 Headless Server.md @@ -0,0 +1,342 @@ +How to install KVM on CentOS 7 / RHEL 7 Headless Server +====== + + +How do I install and configure KVM (Kernel-based Virtual Machine) on a CentOS 7 or RHEL (Red Hat Enterprise Linux) 7 server? How can I setup KMV on a CentOS 7 and use cloud images/cloud-init for installing guest VM? + + +Kernel-based Virtual Machine (KVM) is virtualization software for CentOS or RHEL 7. KVM turn your server into a hypervisor. This page shows how to setup and manage a virtualized environment with KVM in CentOS 7 or RHEL 7. It also described how to install and administer Virtual Machines (VMs) on a physical server using the CLI. Make sure that **Virtualization Technology (VT)** is enabled in your server 's BIOS. You can also run the following command [to test if CPU Support Intel VT and AMD-V Virtualization tech][1] +``` +$ lscpu | grep Virtualization +Virtualization: VT-x +``` + + + +### Follow installation steps of KVM on CentOS 7/RHEL 7 headless sever + +#### Step 1: Install kvm + +Type the following [yum command][2]: +`# yum install qemu-kvm libvirt libvirt-python libguestfs-tools virt-install` +[![How to install KVM on CentOS 7 RHEL 7 Headless Server][3]][3] +Start the libvirtd service: +``` +# systemctl enable libvirtd +# systemctl start libvirtd +``` + +#### Step 2: Verify kvm installation + +Make sure KVM module loaded using lsmod command and [grep command][4]: +`# lsmod | grep -i kvm` + +#### Step 3: Configure bridged networking + +By default dhcpd based network bridge configured by libvirtd. You can verify that with the following commands: +``` +# brctl show +# virsh net-list +``` +[![KVM default networking][5]][5] +All VMs (guest machine) only have network access to other VMs on the same server. A private network 192.168.122.0/24 created for you. Verify it: +`# virsh net-dumpxml default` +If you want your VMs avilable to other servers on your LAN, setup a a network bridge on the server that connected to the your LAN. Update your nic config file such as ifcfg-enp3s0 or em1: +`# vi /etc/sysconfig/network-scripts/enp3s0 ` +Add line: +``` +BRIDGE=br0 +``` + +[Save and close the file in vi][6]. Edit /etc/sysconfig/network-scripts/ifcfg-br0 and add: +`# vi /etc/sysconfig/network-scripts/ifcfg-br0` +Append the following: +``` +DEVICE="br0" +# I am getting ip from DHCP server # +BOOTPROTO="dhcp" +IPV6INIT="yes" +IPV6_AUTOCONF="yes" +ONBOOT="yes" +TYPE="Bridge" +DELAY="0" +``` + +Restart the networking service (warning ssh command will disconnect, it is better to reboot the box): +`# systemctl restart NetworkManager` +Verify it with brctl command: +`# brctl show` + +#### Step 4: Create your first virtual machine + +I am going to create a CentOS 7.x VM. First, grab CentOS 7.x latest ISO image using the wget command: +``` +# cd /var/lib/libvirt/boot/ +# wget https://mirrors.kernel.org/centos/7.4.1708/isos/x86_64/CentOS-7-x86_64-Minimal-1708.iso +``` +Verify ISO images: +``` +# wget https://mirrors.kernel.org/centos/7.4.1708/isos/x86_64/sha256sum.txt +# sha256sum -c sha256sum.txt +``` + +##### Create CentOS 7.x VM + +In this example, I'm creating CentOS 7.x VM with 2GB RAM, 2 CPU core, 1 nics and 40GB disk space, enter: +``` +# virt-install \ +--virt-type=kvm \ +--name centos7 \ +--ram 2048 \ +--vcpus=1 \ +--os-variant=centos7.0 \ +--cdrom=/var/lib/libvirt/boot/CentOS-7-x86_64-Minimal-1708.iso \ +--network=bridge=br0,model=virtio \ +--graphics vnc \ +--disk path=/var/lib/libvirt/images/centos7.qcow2,size=40,bus=virtio,format=qcow2 +``` +To configure vnc login from another terminal over ssh and type: +``` +# virsh dumpxml centos7 | grep vnc + +``` +Please note down the port value (i.e. 5901). You need to use an SSH client to setup tunnel and a VNC client to access the remote vnc server. Type the following SSH port forwarding command from your client/desktop/macbook pro system: +`$ ssh vivek@server1.cyberciti.biz -L 5901:127.0.0.1:5901` +Once you have ssh tunnel established, you can point your VNC client at your own 127.0.0.1 (localhost) address and port 5901 as follows: +[![][7]][7] +You should see CentOS Linux 7 guest installation screen as follows: +[![][8]][8] +Now just follow on screen instructions and install CentOS 7. Once installed, go ahead and click the reboot button. The remote server closed the connection to our VNC client. You can reconnect via KVM client to configure the rest of the server including SSH based session or firewall. + +#### Step 5: Using cloud images + +The above installation method is okay for learning purpose or a single VM. Do you need to deploy lots of VMs? Try cloud images. You can modify pre built cloud images as per your needs. For example, add users, ssh keys, setup time zone, and more using [Cloud-init][9] which is the defacto multi-distribution package that handles early initialization of a cloud instance. Let us see how to create CentOS 7 vm with 1024MB ram, 20GB disk space, and 1 vCPU. + +##### Grab CentOS 7 cloud image + +``` +# cd /var/lib/libvirt/boot +# wget http://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2 +``` + +##### Create required directories + +``` +# D=/var/lib/libvirt/images +# VM=centos7-vm1 ## vm name ## +# mkdir -vp $D/$VM +mkdir: created directory '/var/lib/libvirt/images/centos7-vm1' +``` + +##### Create meta-data file + +``` +# cd $D/$VM +# vi meta-data +``` +Append the following: +``` +instance-id: centos7-vm1 +local-hostname: centos7-vm1 +``` + +##### Crete user-data file + +I am going to login into VM using ssh keys. So make sure you have ssh-keys in place: +`# ssh-keygen -t ed25519 -C "VM Login ssh key"` +[![ssh-keygen command][10]][11] +See "[How To Setup SSH Keys on a Linux / Unix System][12]" for more info. Edit user-data as follows: +``` +# cd $D/$VM +# vi user-data +``` +Add as follows (replace hostname, users, ssh-authorized-keys as per your setup): +``` +#cloud-config + +# Hostname management +preserve_hostname: False +hostname: centos7-vm1 +fqdn: centos7-vm1.nixcraft.com + +# Users +users: + - default + - name: vivek + groups: ['wheel'] + shell: /bin/bash + sudo: ALL=(ALL) NOPASSWD:ALL + ssh-authorized-keys: + - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIMP3MOF2ot8MOdNXCpHem0e2Wemg4nNmL2Tio4Ik1JY VM Login ssh key + +# Configure where output will go +output: + all: ">> /var/log/cloud-init.log" + +# configure interaction with ssh server +ssh_genkeytypes: ['ed25519', 'rsa'] + +# Install my public ssh key to the first user-defined user configured +# in cloud.cfg in the template (which is centos for CentOS cloud images) +ssh_authorized_keys: + - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIMP3MOF2ot8MOdNXCpHem0e2Wemg4nNmL2Tio4Ik1JY VM Login ssh key + +# set timezone for VM +timezone: Asia/Kolkata + +# Remove cloud-init +runcmd: + - systemctl stop network && systemctl start network + - yum -y remove cloud-init +``` + +##### Copy cloud image + +``` +# cd $D/$VM +# cp /var/lib/libvirt/boot/CentOS-7-x86_64-GenericCloud.qcow2 $VM.qcow2 +``` + +##### Create 20GB disk image + +``` +# cd $D/$VM +# export LIBGUESTFS_BACKEND=direct +# qemu-img create -f qcow2 -o preallocation=metadata $VM.new.image 20G +# virt-resize --quiet --expand /dev/sda1 $VM.qcow2 $VM.new.image +``` +[![Set VM image disk size][13]][13] +Overwrite it resized image: +``` +# cd $D/$VM +# mv $VM.new.image $VM.qcow2 +``` + +##### Creating a cloud-init ISO + +`# mkisofs -o $VM-cidata.iso -V cidata -J -r user-data meta-data` +[![Creating a cloud-init ISO][14]][14] + +##### Creating a pool + +``` +# virsh pool-create-as --name $VM --type dir --target $D/$VM +Pool centos7-vm1 created +``` + +##### Installing a CentOS 7 VM + +``` +# cd $D/$VM +# virt-install --import --name $VM \ +--memory 1024 --vcpus 1 --cpu host \ +--disk $VM.qcow2,format=qcow2,bus=virtio \ +--disk $VM-cidata.iso,device=cdrom \ +--network bridge=virbr0,model=virtio \ +--os-type=linux \ +--os-variant=centos7.0 \ +--graphics spice \ +--noautoconsole +``` +Delete unwanted files: +``` +# cd $D/$VM +# virsh change-media $VM hda --eject --config +# rm meta-data user-data centos7-vm1-cidata.iso +``` + +##### Find out IP address of VM + +`# virsh net-dhcp-leases default` +[![CentOS7-VM1- Created][15]][15] + +##### Log in to your VM + +Use ssh command: +`# ssh vivek@192.168.122.85` +[![Sample VM session][16]][16] + +### Useful commands + +Let us see some useful commands for managing VMs. + +#### List all VMs + +`# virsh list --all` + +#### Get VM info + +``` +# virsh dominfo vmName +# virsh dominfo centos7-vm1 +``` + +#### Stop/shutdown a VM + +`# virsh shutdown centos7-vm1` + +#### Start VM + +`# virsh start centos7-vm1` + +#### Mark VM for autostart at boot time + +`# virsh autostart centos7-vm1` + +#### Reboot (soft & safe reboot) VM + +`# virsh reboot centos7-vm1` +Reset (hard reset/not safe) VM +`# virsh reset centos7-vm1` + +#### Delete VM + +``` +# virsh shutdown centos7-vm1 +# virsh undefine centos7-vm1 +# virsh pool-destroy centos7-vm1 +# D=/var/lib/libvirt/images +# VM=centos7-vm1 +# rm -ri $D/$VM +``` +To see a complete list of virsh command type +``` +# virsh help | less +# virsh help | grep reboot +``` + + +### About the author + +The author is the creator of nixCraft and a seasoned sysadmin and a trainer for the Linux operating system/Unix shell scripting. He has worked with global clients and in various industries, including IT, education, defense and space research, and the nonprofit sector. Follow him on [Twitter][17], [Facebook][18], [Google+][19]. + +-------------------------------------------------------------------------------- + +via: https://www.cyberciti.biz/faq/how-to-install-kvm-on-centos-7-rhel-7-headless-server/ + +作者：[Vivek Gite][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.cyberciti.biz +[1]:https://www.cyberciti.biz/faq/linux-xen-vmware-kvm-intel-vt-amd-v-support/ +[2]:https://www.cyberciti.biz/faq/rhel-centos-fedora-linux-yum-command-howto/ (See Linux/Unix yum command examples for more info) +[3]:https://www.cyberciti.biz/media/new/faq/2018/01/How-to-install-KVM-on-CentOS-7-RHEL-7-Headless-Server.jpg +[4]:https://www.cyberciti.biz/faq/howto-use-grep-command-in-linux-unix/ (See Linux/Unix grep command examples for more info) +[5]:https://www.cyberciti.biz/media/new/faq/2018/01/KVM-default-networking.jpg +[6]:https://www.cyberciti.biz/faq/linux-unix-vim-save-and-quit-command/ +[7]:https://www.cyberciti.biz/media/new/faq/2016/01/vnc-client.jpg +[8]:https://www.cyberciti.biz/media/new/faq/2016/01/centos7-guest-vnc.jpg +[9]:https://cloudinit.readthedocs.io/en/latest/index.html +[10]:https://www.cyberciti.biz/media/new/faq/2018/01/ssh-keygen-pub-key.jpg +[11]:https://www.cyberciti.biz/faq/linux-unix-generating-ssh-keys/ +[12]:https://www.cyberciti.biz/faq/how-to-set-up-ssh-keys-on-linux-unix/ +[13]:https://www.cyberciti.biz/media/new/faq/2018/01/Set-VM-image-disk-size.jpg +[14]:https://www.cyberciti.biz/media/new/faq/2018/01/Creating-a-cloud-init-ISO.jpg +[15]:https://www.cyberciti.biz/media/new/faq/2018/01/CentOS7-VM1-Created.jpg +[16]:https://www.cyberciti.biz/media/new/faq/2018/01/Sample-VM-session.jpg +[17]:https://twitter.com/nixcraft +[18]:https://facebook.com/nixcraft +[19]:https://plus.google.com/+CybercitiBiz diff --git a/sources/tech/20180127 Your instant Kubernetes cluster.md b/sources/tech/20180127 Your instant Kubernetes cluster.md new file mode 100644 index 0000000000..b17619762a --- /dev/null +++ b/sources/tech/20180127 Your instant Kubernetes cluster.md @@ -0,0 +1,171 @@ +Your instant Kubernetes cluster +============================================================ + + +This is a condensed and updated version of my previous tutorial [Kubernetes in 10 minutes][10]. I've removed just about everything I can so this guide still makes sense. Use it when you want to create a cluster on the cloud or on-premises as fast as possible. + +### 1.0 Pick a host + +We will be using Ubuntu 16.04 for this guide so that you can copy/paste all the instructions. Here are several environments where I've tested this guide. Just pick where you want to run your hosts. + +* [DigitalOcean][1] - developer cloud + +* [Civo][2] - UK developer cloud + +* [Packet][3] - bare metal cloud + +* 2x Dell Intel i7 boxes - at home + +> Civo is a relatively new developer cloud and one thing that I really liked was how quickly they can bring up hosts - in about 25 seconds. I'm based in the UK so I also get very low latency. + +### 1.1 Provision the machines + +You can get away with a single host for testing but I'd recommend at least three so we have a single master and two worker nodes. + +Here are some other guidelines: + +* Pick dual-core hosts with ideally at least 2GB RAM + +* If you can pick a custom username when provisioning the host then do that rather than root. For example Civo offers an option of `ubuntu`, `civo` or `root`. + +Now run through the following steps on each machine. It should take you less than 5-10 minutes. If that's too slow for you then you can use my utility script [kept in a Gist][11]: + +``` +$ curl -sL https://gist.githubusercontent.com/alexellis/e8bbec45c75ea38da5547746c0ca4b0c/raw/23fc4cd13910eac646b13c4f8812bab3eeebab4c/configure.sh | sh + +``` + +### 1.2 Login and install Docker + +Install Docker from the Ubuntu apt repository. This will be an older version of Docker but as Kubernetes is tested with old versions of Docker it will work in our favour. + +``` +$ sudo apt-get update \ + && sudo apt-get install -qy docker.io + +``` + +### 1.3 Disable the swap file + +This is now a mandatory step for Kubernetes. The easiest way to do this is to edit `/etc/fstab` and to comment out the line referring to swap. + +To save a reboot then type in `sudo swapoff -a`. + +> Disabling swap memory may appear like a strange requirement at first. If you are curious about this step then [read more here][4]. + +### 1.4 Install Kubernetes packages + +``` +$ sudo apt-get update \ + && sudo apt-get install -y apt-transport-https \ + && curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - + +$ echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" \ + | sudo tee -a /etc/apt/sources.list.d/kubernetes.list \ + && sudo apt-get update + +$ sudo apt-get update \ + && sudo apt-get install -y \ + kubelet \ + kubeadm \ + kubernetes-cni + +``` + +### 1.5 Create the cluster + +At this point we create the cluster by initiating the master with `kubeadm`. Only do this on the master node. + +> Despite any warnings I have been assured by [Weaveworks][5] and Lucas (the maintainer) that `kubeadm` is suitable for production use. + +``` +$ sudo kubeadm init + +``` + +If you missed a step or there's a problem then `kubeadm` will let you know at this point. + +Take a copy of the Kube config: + +``` +mkdir -p $HOME/.kube +sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config +sudo chown $(id -u):$(id -g) $HOME/.kube/config + +``` + +Make sure you note down the join token command i.e. + +``` +$ sudo kubeadm join --token c30633.d178035db2b4bb9a 10.0.0.5:6443 --discovery-token-ca-cert-hash sha256: + +``` + +### 2.0 Install networking + +Many networking providers are available for Kubernetes, but none are included by default, so let's use Weave Net from [Weaveworks][12] which is one of the most popular options in the Kubernetes community. It tends to work out of the box without additional configuration. + +``` +$ kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')" + +``` + +If you have private networking enabled on your host then you may need to alter the private subnet that Weavenet uses for allocating IP addresses to Pods (containers). Here's an example of how to do that: + +``` +$ curl -SL "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')&env.IPALLOC_RANGE=172.16.6.64/27" \ +| kubectl apply -f - + +``` + +> Weave also have a very cool visualisation tool called Weave Cloud. It's free and will show you the path traffic is taking between your Pods. [See here for an example with the OpenFaaS project][6]. + +### 2.2 Join the worker nodes to the cluster + +Now you can switch to each of your workers and use the `kubeadm join` command from 1.5\. Once you run that log out of the workers. + +### 3.0 Profit + +That's it - we're done. You have a cluster up and running and can deploy your applications. If you need to setup a dashboard UI then consult the [Kubernetes documentation][13]. + +``` +$ kubectl get nodes +NAME STATUS ROLES AGE VERSION +openfaas1 Ready master 20m v1.9.2 +openfaas2 Ready 19m v1.9.2 +openfaas3 Ready 19m v1.9.2 + +``` + +If you want to see my running through creating a cluster step-by-step and showing you how `kubectl` works then checkout my video below and make sure you subscribe + + +You can also get an "instant" Kubernetes cluster on your Mac for development using Minikube or Docker for Mac Edge edition. [Read my review and first impressions here][14]. + + +-------------------------------------------------------------------------------- + +via: https://blog.alexellis.io/your-instant-kubernetes-cluster/ + +作者：[Alex Ellis ][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://blog.alexellis.io/author/alex/ +[1]:https://www.digitalocean.com/ +[2]:https://www.civo.com/ +[3]:https://packet.net/ +[4]:https://github.com/kubernetes/kubernetes/issues/53533 +[5]:https://weave.works/ +[6]:https://www.weave.works/blog/openfaas-gke +[7]:https://blog.alexellis.io/tag/kubernetes/ +[8]:https://blog.alexellis.io/tag/k8s/ +[9]:https://blog.alexellis.io/tag/cloud-native/ +[10]:https://www.youtube.com/watch?v=6xJwQgDnMFE +[11]:https://gist.github.com/alexellis/e8bbec45c75ea38da5547746c0ca4b0c +[12]:https://weave.works/ +[13]:https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/ +[14]:https://blog.alexellis.io/docker-for-mac-with-kubernetes/ +[15]:https://blog.alexellis.io/your-instant-kubernetes-cluster/# \ No newline at end of file diff --git a/sources/tech/20180128 Getting Linux Jobs.md b/sources/tech/20180128 Getting Linux Jobs.md new file mode 100644 index 0000000000..a7f1a075a5 --- /dev/null +++ b/sources/tech/20180128 Getting Linux Jobs.md @@ -0,0 +1,98 @@ +Getting Linux Jobs +====== + +In a qualitative review of job posting websites, even highly skilled Linux administrators would be hamstrung to succeed in getting to the stage of an interview. + +All of this results in hundreds of decent and skilled people being snubbed without cause simply because today's job market requires a few extra tools to increase the odds. + +I have two colleagues and a cousin who have all received certifications with RedHat, managed quite extensive server rooms, and received earnest recommendations from former employers. + +All of these skills, certifications and experience come to naught as they apply to employer ads that are crudely constructed by someone hurriedly cutting and pasting snippets of "skill words" from a list of technical terms. + +Not surprisingly, today's politeness has gone the way of the bird, and a **non-response** from companies posting ads seems to be the new way of communicating. + +Unfortunately, it also means that these recruiters/HR personnel probably did **not** get the best candidate. + +The reason I can say this with such conviction is because of the type of buffoonery that takes place so often when creating job ads in the first place. + +Walter, another [Reallylinux.com][3] guest writer, presented how [**Job Want Ads Have Gone Mad**][4]. + +Perhaps he's right. However, I believe every Linux job seeker can avoid pitfalls of a job hunt by keeping in mind **three key facts** about job ads. + +First, few advertisements for Linux administrators are exclusively about Linux. + +Bear in mind the occasional Linux system administrator job, where you would actually be using Linux on servers. Instead, many jobs that rise up on a "Linux administrator" search are actually referring to a plethora of 'NX operating systems. + +For example, here is a quote from a **"Linux Administrator"** job posting: +This role will provide support for build system integration, especially operating system installation support for BSD applications... + +Or another ad declares in the bowels of its content: +Windows administration experience required. + +Ironically, if you show up to interview for any of these types of jobs and focus on Linux, they probably will not choose you. + +Even more importantly, if you simply include Linux as your expertise, they may not even bother with your resume, because they can't tell the difference between UNIX, BSD, Linux, etc. + +As a result, if you are conscientious and only include Linux on your resume, you are automatically out. But change that Linux to UNIX/Linux and you end up getting a bit farther in the human resources bureaucracy. + +I had two colleagues that ended up changing this on their resumes and getting a much better hit ratio for interviews, which were still slim pickings because most job ads are tailored with some particular person already in mind. The main intent behind such job ads being a cover for the ass of the department making the claim of having an open job. + +Second, the only person at the company who cares at all about the system administrator position is the technical lead/manager hiring for the slot. Others at the company, including the HR contact or the management could not care less. + +I remember sitting in a board room as a fly on the wall, hearing one executive vice president refer to server administrators as "dime a dozen geeks." How wrong they are to suggest this. + +Ironically, one day should the mail system fail, or the PBX connectivity hiccup, or perhaps core business files disappear from the intranet, these same executives are the first to get on the phone and threaten to fire the system admins. + +Perhaps if they would stop leaving so many hot air telephone messages, or filling their emails with 35MB photographs of another vice president's fishing trip and wife, the servers wouldn't be so problematic. + +Be aware that a Linux administrator ad, or any job posting for server administrator is placed because someone at the TECHNICAL level sees an urgent need for staffing. You're not going to get any empathy talking to HR or any leader of the company. Instead, take the time to find out who the hiring technical manager is and try to telephone them. + +You can always call them directly because you have some "specific technical questions" you know the HR person could not answer. This opens the dialogue with the person who actually cares that the position is filled and ensures you get a foot in because you took the time for personal contact, even if it was a 60 second phone call. + +What if the HR beauracracy won't let you through? + +Start asking as many tech questions as possible direct to the HR hiring contact, such as how their Linux clusters are setup and do they run VMs exclusively? Anything relatively technical will send these HR people in a tizzy and allow you the question: "may I contact the technical manager of the team?" + +If the response is a fluffy "maybe" or "I'll get back to you on that" they already filled the slot in their mind with someone else two weeks earlier, such as the HR staff member's fiance. They simply wanted it to look less like nepotism and more like indeterminism with a dash of egoism. + +``` +"They simply wanted it to look less like nepotism and more like indeterminism with a dash of egoism." +``` + +So take the time to find out who is the direct TECHNICAL leader hiring for the position and talk to them. It can make a difference and get you past some of the baloney. + +Third, few job ads today include any semblance of reality. + +I've seen enough ads requiring a junior system administrator with expertise that senior level experts don't have, to know the plan is to list the blue sky wish list and then find out who applies. + +In this situation, the Linux administrator ad you apply for, should include some key phrases for which you already have experience or certifications. + +The trick is to so overload your resume with the key phrases that MATCH their ad, it becomes almost impossible for them to determine which phrases you left out. + +This doesn't necessarily translate to a job, but it often adds enough intrigue to get you an interview, which now a days is a major step. + +By understanding and applying these three techniques, hopefully those seeking Linux administrator jobs have a head start on those who have only a slim chance in hell. + +Even if these tips don't get you interviews right away, you can use the experience and awareness when you go to the next trade show, or company sponsored technical conference. + +I strongly recommend you regularly attend these as well, especially if they are reasonably close, as they always provide a kick start to networking. + +Remember that job networking now a days is a pseudonym for "getting the gossip on which companies are actually hiring and which ones are just lying about jobs to give the appearance of growth for shareholders." + + + +-------------------------------------------------------------------------------- + +via: http://reallylinux.com/docs/gettinglinuxjobs.shtml + +作者：[Andrea W.Codingly][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://reallylinux.com +[1]:http://www.reallylinux.com +[2]:http://reallylinux.com/docs/linuxrecessionproof.shtml +[3]:http://reallylinux.com +[4]:http://reallylinux.com/docs/wantadsmad.shtml diff --git a/sources/tech/20180128 How to add network bridge with nmcli (NetworkManager) on Linux.md b/sources/tech/20180128 How to add network bridge with nmcli (NetworkManager) on Linux.md new file mode 100644 index 0000000000..bf7772ef1a --- /dev/null +++ b/sources/tech/20180128 How to add network bridge with nmcli (NetworkManager) on Linux.md @@ -0,0 +1,146 @@ +How to add network bridge with nmcli (NetworkManager) on Linux +====== + +I am using Debian Linux 9 "stretch" on the desktop. I would like to create network bridge with NetworkManager. But, I am unable to find the option to add br0. How can I create or add network bridge with nmcli for NetworkManager on Linux? + +A bridge is nothing but a device which joins two local networks into one network. It works at the data link layer, i.e., layer 2 of the OSI model. Network bridge often used with virtualization and other software. Disabling NetworkManager for a simple bridge especially on Linux Laptop/desktop doesn't make any sense. The nmcli tool can create Persistent bridge configuration without editing any files. **This page shows how to create a bridge interface using the Network Manager command line tool called nmcli**. + + + +### How to create/add network bridge with nmcli + +The procedure to add a bridge interface on Linux is as follows when you want to use Network Manager: + +1. Open the Terminal app +2. Get info about the current connection: +``` +nmcli con show +``` +3. Add a new bridge: +``` +nmcli con add type bridge ifname br0 +``` +4. Create a slave interface: +``` +nmcli con add type bridge-slave ifname eno1 master br0 +``` +5. Turn on br0: +``` +nmcli con up br0 +``` + +Let us see how to create a bridge, named br0 in details. + +### Get current network config + +You can view connection from the Network Manager GUI in settings: +[![Getting Network Info on Linux][1]][1] +Another option is to type the following command: +``` +$ nmcli con show +$ nmcli connection show --active +``` +[![View the connections with nmcli][2]][2] +I have a "Wired connection 1" which uses the eno1 Ethernet interface. My system has a VPN interface too. I am going to setup a bridge interface named br0 and add, (or enslave) an interface to eno1. + +### How to create a bridge, named br0 + +``` +$ sudo nmcli con add ifname br0 type bridge con-name br0 +$ sudo nmcli con add type bridge-slave ifname eno1 master br0 +$ nmcli connection show +``` +[![Create bridge interface using nmcli on Linux][3]][3] +You can disable STP too: +``` +$ sudo nmcli con modify br0 bridge.stp no +$ nmcli con show +$ nmcli -f bridge con show br0 +``` +The last command shows the bridge settings including disabled STP: +``` +bridge.mac-address: -- +bridge.stp: no +bridge.priority: 32768 +bridge.forward-delay: 15 +bridge.hello-time: 2 +bridge.max-age: 20 +bridge.ageing-time: 300 +bridge.multicast-snooping: yes +``` + + +### How to turn on bridge interface + +You must turn off "Wired connection 1" and turn on br0: +``` +$ sudo nmcli con down "Wired connection 1" +$ sudo nmcli con up br0 +$ nmcli con show +``` +Use [ip command][4] to view the IP settings: +``` +$ ip a s +$ ip a s br0 +``` +[![Build a network bridge with nmcli on Linux][5]][5] + +### Optional: How to use br0 with KVM + +Now you can connect VMs (virtual machine) created with KVM/VirtualBox/VMware workstation to a network directly without using NAT. Create a file named br0.xml for KVM using vi command or [cat command][6]: +`$ cat /tmp/br0.xml` +Append the following code: +``` + + br0 + + + +``` + +Run virsh command as follows: +``` +# virsh net-define /tmp/br0.xml +# virsh net-start br0 +# virsh net-autostart br0 +# virsh net-list --all +``` +Sample outputs: +``` + Name State Autostart Persistent +---------------------------------------------------------- + br0 active yes yes + default inactive no yes +``` + + +For more info read the following man page: +``` +$ man ip +$ man nmcli +``` + +### about the author + +The author is the creator of nixCraft and a seasoned sysadmin and a trainer for the Linux operating system/Unix shell scripting. He has worked with global clients and in various industries, including IT, education, defense and space research, and the nonprofit sector. Follow him on [Twitter][7], [Facebook][8], [Google+][9]. + +-------------------------------------------------------------------------------- + +via: https://www.cyberciti.biz/faq/how-to-add-network-bridge-with-nmcli-networkmanager-on-linux/ + +作者：[Vivek Gite][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.cyberciti.biz +[1]:https://www.cyberciti.biz/media/new/faq/2018/01/Getting-Network-Info-on-Linux.jpg +[2]:https://www.cyberciti.biz/media/new/faq/2018/01/View-the-connections-with-nmcli.jpg +[3]:https://www.cyberciti.biz/media/new/faq/2018/01/Create-bridge-interface-using-nmcli-on-Linux.jpg +[4]:https://www.cyberciti.biz/faq/linux-ip-command-examples-usage-syntax/ (See Linux/Unix ip command examples for more info) +[5]:https://www.cyberciti.biz/media/new/faq/2018/01/Build-a-network-bridge-with-nmcli-on-Linux.jpg +[6]:https://www.cyberciti.biz/faq/linux-unix-appleosx-bsd-cat-command-examples/ (See Linux/Unix cat command examples for more info) +[7]:https://twitter.com/nixcraft +[8]:https://facebook.com/nixcraft +[9]:https://plus.google.com/+CybercitiBiz diff --git a/sources/tech/20180129 5 Real World Uses for Redis.md b/sources/tech/20180129 5 Real World Uses for Redis.md new file mode 100644 index 0000000000..61f7c09b3b --- /dev/null +++ b/sources/tech/20180129 5 Real World Uses for Redis.md @@ -0,0 +1,109 @@ +5 Real World Uses for Redis +============================================================ + + +Redis is a powerful in-memory data structure store which has many uses including a database, a cache, and a message broker. Most people often think of it a simple key-value store, but it has so much more power. I will be going over some real world examples of some of the many things Redis can do for you. + +### 1\. Full Page Cache + +The first thing is full page caching. If you are using server-side rendered content, you do not want to re-render each page for every single request. Using a cache like Redis, you can cache regularly requested content and drastically decrease latency for your most requested pages, and most frameworks have hooks for caching your pages with Redis. +Simple Commands + +``` +// Set the page that will last 1 minute +SET key "..." EX 60 + +// Get the page +GET key + +``` + +### 2\. Leaderboard + +One of the places Redis shines is for leaderboards. Because Redis is in-memory, it can deal with incrementing and decrementing very fast and efficiently. Compare this to running a SQL query every request the performance gains are huge! This combined with Redis's sorted sets means you can grab only the highest rated items in the list in milliseconds, and it is stupid easy to implement. +Simple Commands + +``` +// Add an item to the sorted set +ZADD sortedSet 1 "one" + +// Get all items from the sorted set +ZRANGE sortedSet 0 -1 + +// Get all items from the sorted set with their score +ZRANGE sortedSet 0 -1 WITHSCORES + +``` + +### 3\. Session Storage + +The most common use for Redis I have seen is session storage. Unlike other session stores like Memcache, Redis can persist data so in the situation where your cache goes down when it comes back up all the data will still be there. Although this isn't mission critical to be persisted, this feature can save your users lots of headaches. No one likes their session to be randomly dropped for no reason. +Simple Commands + +``` +// Set session that will last 1 minute +SET randomHash "{userId}" EX 60 + +// Get userId +GET randomHash + +``` + +### 4\. Queue + +One of the less common, but very useful things you can do with Redis is queue things. Whether it's a queue of emails or data to be consumed by another application, you can create an efficient queue it in Redis. Using this functionality is easy and natural for any developer who is familiar with Stacks and pushing and popping items. +Simple Commands + +``` +// Add a Message +HSET messages +ZADD due + +// Recieving Message +ZRANGEBYSCORE due -inf LIMIT 0 1 +HGET messages + +// Delete Message +ZREM due +HDEL messages + +``` + +### 5\. Pub/Sub + +The final real world use for Redis I am going to bring up in this post is pub/sub. This is one of the most powerful features Redis has built in; the possibilities are limitless. You can create a real-time chat system with it, trigger notifications for friend requests on social networks, etc... This feature is one of the most underrated features Redis offers but is very powerful, yet simple to use. +Simple Commands + +``` +// Add a message to a channel +PUBLISH channel message + +// Recieve messages from a channel +SUBSCRIBE channel + +``` + +### Conclusion + +I hope you enjoyed this list of some of the many real world uses for Redis. This is just scratching the surface of what Redis can do for you, but I hope it gave you some ideas of how you can use the full potential Redis has to offer. + +-------------------------------------------------------------------------------- + +作者简介: + +Hi, my name is Ryan! I am a Software Developer with experience in many web frameworks and libraries including NodeJS, Django, Golang, and Laravel. + + +------------------- + + +via: https://ryanmccue.ca/5-real-world-uses-for-redis/ + +作者：[Ryan McCue ][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://ryanmccue.ca/author/ryan/ +[1]:https://ryanmccue.ca/author/ryan/ \ No newline at end of file diff --git a/sources/tech/20180129 A look inside Facebooks open source program.md b/sources/tech/20180129 A look inside Facebooks open source program.md new file mode 100644 index 0000000000..3610cec043 --- /dev/null +++ b/sources/tech/20180129 A look inside Facebooks open source program.md @@ -0,0 +1,68 @@ +A look inside Facebook's open source program +============================================================ + +### Facebook developer Christine Abernathy discusses how open source helps the company share insights and boost innovation. + +![A look inside Facebook's open source program](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/LAW-Internet_construction_9401467_520x292_0512_dc.png?itok=RPkPPtDe "A look inside Facebook's open source program") +Image by : opensource.com + + +Open source becomes more ubiquitous every year, appearing everywhere from [government municipalities][11] to [universities][12]. Companies of all sizes are also increasingly turning to open source software. In fact, some companies are taking open source a step further by supporting projects financially or working with developers. + +Facebook's open source program, for example, encourages others to release their code as open source, while working and engaging with the community to support open source projects. [Christine Abernathy][13], a Facebook developer, open source advocate, and member of the company's open source team, visited the Rochester Institute of Technology last November, presenting at the [November edition][14] of the FOSS Talks speaker series. In her talk, Abernathy explained how Facebook approaches open source and why it's an important part of the work the company does. + +### Facebook and open source + +Abernathy said that open source plays a fundamental role in Facebook's mission to create community and bring the world closer together. This ideological match is one motivating factor for Facebook's participation in open source. Additionally, Facebook faces unique infrastructure and development challenges, and open source provides a platform for the company to share solutions that could help others. Open source also provides a way to accelerate innovation and create better software, helping engineering teams produce better software and work more transparently. Today, Facebook's 443 projects on GitHub comprise 122,000 forks, 292,000 commits, and 732,000 followers. + + + +![open source projects by Facebook](https://opensource.com/sites/default/files/images/life-uploads/blog-article-facebook-open-source-projects.png "open source projects by Facebood") + +Some of the Facebook projects released as open source include React, GraphQL, Caffe2, and others. (Image by Christine Abernathy, used with permission) + +### Lessons learned + +Abernathy emphasized that Facebook has learned many lessons from the open source community, and it looks forward to learning many more. She identified the three most important ones: + +* Share what's useful + +* Highlight your heroes + +* Fix common pain points + + _Christine Abernathy visited RIT as part of the FOSS Talks speaker series. Every month, a guest speaker from the open source world shares wisdom, insight, and advice about the open source world with students interested in free and open source software. The [FOSS @ MAGIC][3] community is thankful to have Abernathy attend as a speaker._ + +### About the author + + [![Picture of Justin W. Flory](https://opensource.com/sites/default/files/styles/profile_pictures/public/pictures/october_2017_cropped_0.jpg?itok=gV-RgINC)][15] Justin W. Flory - Justin is a student at the [Rochester Institute of Technology][4]majoring in Networking and Systems Administration. He is currently a contributor to the [Fedora Project][5]. In Fedora, Justin is the editor-in-chief of the [Fedora Magazine][6], the lead of the [Community... ][7][more about Justin W. Flory][8][More about me][9] + +-------------------------------------------------------------------------------- + +via: https://opensource.com/article/18/1/inside-facebooks-open-source-program + +作者：[Justin W. Flory ][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://opensource.com/users/jflory +[1]:https://opensource.com/file/383786 +[2]:https://opensource.com/article/18/1/inside-facebooks-open-source-program?rate=H9_bfSwXiJfi2tvOLiDxC_tbC2xkEOYtCl-CiTq49SA +[3]:http://foss.rit.edu/ +[4]:https://www.rit.edu/ +[5]:https://fedoraproject.org/wiki/Overview +[6]:https://fedoramagazine.org/ +[7]:https://fedoraproject.org/wiki/CommOps +[8]:https://opensource.com/users/jflory +[9]:https://opensource.com/users/jflory +[10]:https://opensource.com/user/74361/feed +[11]:https://opensource.com/article/17/8/tirana-government-chooses-open-source +[12]:https://opensource.com/article/16/12/2016-election-night-hackathon +[13]:https://twitter.com/abernathyca +[14]:https://www.eventbrite.com/e/fossmagic-talks-open-source-facebook-with-christine-abernathy-tickets-38955037566# +[15]:https://opensource.com/users/jflory +[16]:https://opensource.com/users/jflory +[17]:https://opensource.com/users/jflory +[18]:https://opensource.com/article/18/1/inside-facebooks-open-source-program#comments \ No newline at end of file diff --git a/sources/tech/20180129 Advanced Python Debugging with pdb.md b/sources/tech/20180129 Advanced Python Debugging with pdb.md new file mode 100644 index 0000000000..80f17e23a3 --- /dev/null +++ b/sources/tech/20180129 Advanced Python Debugging with pdb.md @@ -0,0 +1,363 @@ +translating by lujun9972 +Advanced Python Debugging with pdb +====== + +![](https://process.filestackapi.com/cache=expiry:max/resize=width:700/compress/nygTCcWMQuyCFaOrlEnh) + +Python's built-in [`pdb`][1] module is extremely useful for interactive debugging, but has a bit of a learning curve. For a long time, I stuck to basic `print`-debugging and used `pdb` on a limited basis, which meant I missed out on a lot of features that would have made debugging faster and easier. + +In this post I will show you a few tips I've picked up over the years to level up my interactive debugging skills. + +## Print debugging vs. interactive debugging + +First, why would you want to use an interactive debugger instead of inserting `print` or `logging` statements into your code? + +With `pdb`, you have a lot more flexibility to run, resume, and alter the execution of your program without touching the underlying source. Once you get good at this, it means more time spent diving into issues and less time context switching back and forth between your editor and the command line. + +Also, by not touching the underlying source code, you will have the ability to step into third party code (e.g. modules installed from PyPI) and the standard library. + +## Post-mortem debugging + +The first workflow I used after moving away from `print` debugging was `pdb`'s "post-mortem debugging" mode. This is where you run your program as usual, but whenever an unhandled exception is thrown, you drop down into the debugger to poke around in the program state. After that, you attempt to make a fix and repeat the process until the problem is resolved. + +You can run an existing script with the post-mortem debugger by using Python's `-mpdb` option: +``` +python3 -mpdb path/to/script.py + +``` + +From here, you are dropped into a `(Pdb)` prompt. To start execution, you use the `continue` or `c` command. If the program executes successfully, you will be taken back to the `(Pdb)` prompt where you can restart the execution again. At this point, you can use `quit` / `q` or Ctrl+D to exit the debugger. + +If the program throws an unhandled exception, you'll also see a `(Pdb)` prompt, but with the program execution stopped at the line that threw the exception. From here, you can run Python code and debugger commands at the prompt to inspect the current program state. + +## Testing our basic workflow + +To see how these basic debugging steps work, I'll be using this (buggy) program: +``` +import random + +MAX = 100 + +def main(num_loops=1000): + for i in range(num_loops): + num = random.randint(0, MAX) + denom = random.randint(0, MAX) + result = num / denom + print("{} divided by {} is {:.2f}".format(num, denom, result)) + +if __name__ == "__main__": + import sys + arg = sys.argv[-1] + if arg.isdigit(): + main(arg) + else: + main() + +``` + +We're expecting the program to do some basic math operations on random numbers in a loop and print the result. Try running it normally and you will see one of the bugs: +``` +$ python3 script.py +2 divided by 30 is 0.07 +65 divided by 41 is 1.59 +0 divided by 70 is 0.00 +... +38 divided by 26 is 1.46 +Traceback (most recent call last): + File "script.py", line 16, in + main() + File "script.py", line 7, in main + result = num / denom +ZeroDivisionError: division by zero + +``` + +Let's try post-mortem debugging this error: +``` +$ python3 -mpdb script.py +> ./src/script.py(1)() +-> import random +(Pdb) c +49 divided by 46 is 1.07 +... +Traceback (most recent call last): + File "/usr/lib/python3.4/pdb.py", line 1661, in main + pdb._runscript(mainpyfile) + File "/usr/lib/python3.4/pdb.py", line 1542, in _runscript + self.run(statement) + File "/usr/lib/python3.4/bdb.py", line 431, in run + exec(cmd, globals, locals) + File "", line 1, in + File "./src/script.py", line 1, in + import random + File "./src/script.py", line 7, in main + result = num / denom +ZeroDivisionError: division by zero +Uncaught exception. Entering post mortem debugging +Running 'cont' or 'step' will restart the program +> ./src/script.py(7)main() +-> result = num / denom +(Pdb) num +76 +(Pdb) denom +0 +(Pdb) random.randint(0, MAX) +56 +(Pdb) random.randint(0, MAX) +79 +(Pdb) random.randint(0, 1) +0 +(Pdb) random.randint(1, 1) +1 + +``` + +Once the post-mortem debugger kicks in, we can inspect all of the variables in the current frame and even run new code to help us figure out what's wrong and attempt to make a fix. + +## Dropping into the debugger from Python code using `pdb.set_trace` + +Another technique that I used early on, after starting to use `pdb`, was forcing the debugger to run at a certain line of code before an error occurred. This is a common next step after learning post-mortem debugging because it feels similar to debugging with `print` statements. + +For example, in the above code, if we want to stop execution before the division operation, we could add a `pdb.set_trace` call to our program here: +``` + import pdb; pdb.set_trace() + result = num / denom + +``` + +And then run our program without `-mpdb`: +``` +$ python3 script.py +> ./src/script.py(10)main() +-> result = num / denom +(Pdb) num +94 +(Pdb) denom +19 + +``` + +The problem with this method is that you have to constantly drop these statements into your source code, remember to remove them afterwards, and switch between running your code with `python` vs. `python -mpdb`. + +Using `pdb.set_trace` gets the job done, but **breakpoints** are an even more flexible way to stop the debugger at any line (even third party or standard library code), without needing to modify any source code. Let's learn about breakpoints and a few other useful commands. + +## Debugger commands + +There are over 30 commands you can give to the interactive debugger, a list that can be seen by using the `help` command when at the `(Pdb)` prompt: +``` +(Pdb) help + +Documented commands (type help ): +======================================== +EOF c d h list q rv undisplay +a cl debug help ll quit s unt +alias clear disable ignore longlist r source until +args commands display interact n restart step up +b condition down j next return tbreak w +break cont enable jump p retval u whatis +bt continue exit l pp run unalias where + +``` + +You can use `help ` for more information on a given command. + +Instead of walking through each command, I'll list out the ones I've found most useful and what arguments they take. + +**Setting breakpoints** : + + * `l(ist)`: displays the source code of the currently running program, with line numbers, for the 10 lines around the current statement. + * `l 1,999`: displays the source code of lines 1-999. I regularly use this to see the source for the entire program. If your program only has 20 lines, it'll just show all 20 lines. + * `b(reakpoint)`: displays a list of current breakpoints. + * `b 10`: set a breakpoint at line 10. Breakpoints are referred to by a numeric ID, starting at 1. + * `b main`: set a breakpoint at the function named `main`. The function name must be in the current scope. You can also set breakpoints on functions in other modules in the current scope, e.g. `b random.randint`. + * `b script.py:10`: sets a breakpoint at line 10 in `script.py`. This gives you another way to set breakpoints in another module. + * `clear`: clears all breakpoints. + * `clear 1`: clear breakpoint 1. + + + +**Stepping through execution** : + + * `c(ontinue)`: execute until the program finishes, an exception is thrown, or a breakpoint is hit. + * `s(tep)`: execute the next line, whatever it is (your code, stdlib, third party code, etc.). Use this when you want to step down into function calls you're interested in. + * `n(ext)`: execute the next line in the current function (will not step into downstream function calls). Use this when you're only interested in the current function. + * `r(eturn)`: execute the remaining lines in the current function until it returns. Use this to skip over the rest of the function and go up a level. For example, if you've stepped down into a function by mistake. + * `unt(il) [lineno]`: execute until the current line exceeds the current line number. This is useful when you've stepped into a loop but want to let the loop continue executing without having to manually step through every iteration. Without any argument, this command behaves like `next` (with the loop skipping behavior, once you've stepped through the loop body once). + + + +**Moving up and down the stack** : + + * `w(here)`: shows an annotated view of the stack trace, with your current frame marked by `>`. + * `u(p)`: move up one frame in the current stack trace. For example, when post-mortem debugging, you'll start off on the lowest level of the stack and typically want to move `up` a few times to help figure out what went wrong. + * `d(own)`: move down one frame in the current stack trace. + + + +**Additional commands and tips** : + + * `pp `: This will "pretty print" the result of the given expression using the [`pprint`][2] module. Example: + + +``` +(Pdb) stuff = "testing the pp command in pdb with a big list of strings" +(Pdb) pp [(i, x) for (i, x) in enumerate(stuff.split())] +[(0, 'testing'), + (1, 'the'), + (2, 'pp'), + (3, 'command'), + (4, 'in'), + (5, 'pdb'), + (6, 'with'), + (7, 'a'), + (8, 'big'), + (9, 'list'), + (10, 'of'), + (11, 'strings')] + +``` + + * `!`: sometimes the Python code you run in the debugger will be confused for a command. For example `c = 1` will trigger the `continue` command. To force the debugger to execute Python code, prefix the line with `!`, e.g. `!c = 1`. + + * Pressing the Enter key at the `(Pdb)` prompt will execute the previous command again. This is most useful after the `s`/`n`/`r`/`unt` commands to quickly step through execution line-by-line. + + * You can run multiple commands on one line by separating them with `;;`, e.g. `b 8 ;; c`. + + * The `pdb` module can take multiple `-c` arguments on the command line to execute commands as soon as the debugger starts. + + + + +Example: +``` +python3 -mpdb -cc script.py # run the program without you having to enter an initial "c" at the prompt +python3 -mpdb -c "b 8" -cc script.py # sets a breakpoint on line 8 and runs the program + +``` + +## Restart behavior + +Another thing that can shave time off debugging is understanding how `pdb`'s restart behavior works. You may have noticed that after execution stops, `pdb` will give a message like, "The program finished and will be restarted," or "The script will be restarted." When I first started using `pdb`, I would always quit and re-run `python -mpdb ...` to make sure that my code changes were getting picked up, which was unnecessary in most cases. + +When `pdb` says it will restart the program, or when you use the `restart` command, code changes to the script you're debugging will be reloaded automatically. Breakpoints will still be set after reloading, but may need to be cleared and re-set due to line numbers shifting. Code changes to other imported modules will not be reloaded -- you will need to `quit` and re-run the `-mpdb` command to pick those up. + +## Watches + +One feature you may miss from other interactive debuggers is the ability to "watch" a variable change throughout the program's execution. `pdb` does not include a watch command by default, but you can get something similar by using `commands`, which lets you run arbitrary Python code whenever a breakpoint is hit. + +To watch what happens to the `denom` variable in our example program: +``` +$ python3 -mpdb script.py +> ./src/script.py(1)() +-> import random +(Pdb) b 9 +Breakpoint 1 at ./src/script.py:9 +(Pdb) commands +(com) silent +(com) print("DENOM: {}".format(denom)) +(com) c +(Pdb) c +DENOM: 77 +71 divided by 77 is 0.92 +DENOM: 27 +100 divided by 27 is 3.70 +DENOM: 10 +82 divided by 10 is 8.20 +DENOM: 20 +... + +``` + +We first set a breakpoint (which is assigned ID 1), then use `commands` to start entering a block of commands. These commands function as if you had typed them at the `(Pdb)` prompt. They can be either Python code or additional `pdb` commands. + +Once we start the `commands` block, the prompt changes to `(com)`. The `silent` command means the following commands will not be echoed back to the screen every time they're executed, which makes reading the output a little easier. + +After that, we run a `print` statement to inspect the variable, similar to what we might do when `print` debugging. Finally, we end with a `c` to continue execution, which ends the command block. Typing `c` again at the `(Pdb)` prompt starts execution and we see our new `print` statement running. + +If you'd rather stop execution instead of continuing, you can use `end` instead of `c` in the command block. + +## Running pdb from the interpreter + +Another way to run `pdb` is via the interpreter, which is useful when you're experimenting interactively and would like to drop into `pdb` without running a standalone script. + +For post-mortem debugging, all you need is a call to `pdb.pm()` after an exception has occurred: +``` +$ python3 +>>> import script +>>> script.main() +17 divided by 60 is 0.28 +... +56 divided by 94 is 0.60 +Traceback (most recent call last): + File "", line 1, in + File "./src/script.py", line 9, in main + result = num / denom +ZeroDivisionError: division by zero +>>> import pdb +>>> pdb.pm() +> ./src/script.py(9)main() +-> result = num / denom +(Pdb) num +4 +(Pdb) denom +0 + +``` + +If you want to step through normal execution instead, use the `pdb.run()` function: +``` +$ python3 +>>> import script +>>> import pdb +>>> pdb.run("script.main()") +> (1)() +(Pdb) b script:6 +Breakpoint 1 at ./src/script.py:6 +(Pdb) c +> ./src/script.py(6)main() +-> for i in range(num_loops): +(Pdb) n +> ./src/script.py(7)main() +-> num = random.randint(0, MAX) +(Pdb) n +> ./src/script.py(8)main() +-> denom = random.randint(0, MAX) +(Pdb) n +> ./src/script.py(9)main() +-> result = num / denom +(Pdb) n +> ./src/script.py(10)main() +-> print("{} divided by {} is {:.2f}".format(num, denom, result)) +(Pdb) n +66 divided by 70 is 0.94 +> ./src/script.py(6)main() +-> for i in range(num_loops): + +``` + +This one is a little trickier than `-mpdb` because you don't have the ability to step through an entire program. Instead, you'll need to manually set a breakpoint, e.g. on the first statement of the function you're trying to execute. + +## Conclusion + +Hopefully these tips have given you a few new ideas on how to use `pdb` more effectively. After getting a handle on these, you should be able to pick up the [other commands][3] and start customizing `pdb` via a `.pdbrc` file ([example][4]). + +You can also look into other front-ends for debugging, like [pdbpp][5], [pudb][6], and [ipdb][7], or GUI debuggers like the one included in PyCharm. Happy debugging! + +-------------------------------------------------------------------------------- + +via: https://www.codementor.io/stevek/advanced-python-debugging-with-pdb-g56gvmpfa + +作者：[Steven Kryskalla][a] +译者：[lujun9972](https://github.com/lujun9972) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.codementor.io/stevek +[1]:https://docs.python.org/3/library/pdb.html +[2]:https://docs.python.org/3/library/pprint.html +[3]:https://docs.python.org/3/library/pdb.html#debugger-commands +[4]:https://nedbatchelder.com/blog/200704/my_pdbrc.html +[5]:https://pypi.python.org/pypi/pdbpp/ +[6]:https://pypi.python.org/pypi/pudb/ +[7]:https://pypi.python.org/pypi/ipdb diff --git a/sources/tech/20180129 CopperheadOS Security features installing apps and more.md b/sources/tech/20180129 CopperheadOS Security features installing apps and more.md new file mode 100644 index 0000000000..fd6e110d35 --- /dev/null +++ b/sources/tech/20180129 CopperheadOS Security features installing apps and more.md @@ -0,0 +1,245 @@ +CopperheadOS: Security features, installing apps, and more +============================================================ + +### Fly your open source flag proudly with Copperhead, a mobile OS that takes its FOSS commitment seriously. + + +![Android security and privacy](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/android_security_privacy.png?itok=MPHAV5mL "Android security and privacy") +Image by : Norebbo via [Flickr][15] (Original: [public domain][16]). Modified by Opensource.com. [CC BY-SA 4.0][17]. + + _Editor's note: CopperheadOS is [licensed][11] under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 license (userspace) and GPL2 license (kernel). It is also based on Android Open Source Project (AOSP)._ + +Several years ago, I made the decision to replace proprietary technologies (mainly Apple products) with technology that ran on free and open source software (FOSS). I can't say it was easy, but I now happily use FOSS for pretty much everything. + +The hardest part involved my mobile handset. There are basically only two choices today for phones and tablets: Apple's iOS or Google's Android. Since Android is open source, it seemed the obvious choice, but I was frustrated by both the lack of open source applications on Android and the pervasiveness of Google on those devices. + +So I entered the world of custom ROMs. These are projects that take the base [Android Open Source Project][18] (AOSP) and customize it. Almost all these projects allow you to install the standard Google applications as a separate package, called GApps, and you can have as much or as little Google presence on your phone as you like. GApps packages come in a number of flavors, from the full suite of apps that Google ships with its devices to a "pico" version that includes just the minimal amount of software needed to run the Google Play Store, and from there you can add what you like. + +I started out using CyanogenMod, but when that project went in a direction I didn't like, I switched to OmniROM. I was quite happy with it, but still wondered what information I was sending to Google behind the scenes. + +Then I found out about [CopperheadOS][19]. Copperhead is a version of AOSP that focuses on delivering the most secure Android experience possible. I've been using it for a year now and have been quite happy with it. + +Unlike other custom ROMs that strive to add lots of new functionality, Copperhead runs a pretty vanilla version of AOSP. Also, while the first thing you usually do when playing with a custom ROM is to add root access to the device, not only does Copperhead prevent that, it also requires that you have a device that has verified boot, so there's no unlocking the bootloader. This is to prevent malicious code from getting access to the handset. + +Copperhead starts with a hardened version of the AOSP baseline, including full encryption, and then adds a [ton of stuff][20] I can only pretend to understand. It also applies a number of kernel and Android patches before they are applied to the mainline Android releases. + +### [copperos_extrapatches.png][1] + +![About phone with extra patches](https://opensource.com/sites/default/files/u128651/copperos_extrapatches.png "About phone with extra patches") + +It has a couple of more obvious features that I like. If you use a PIN to unlock your device, there is an option to scramble the digits. + +### [copperos_scrambleddigits.png][2] + +![Option to scramble digits](https://opensource.com/sites/default/files/u128651/copperos_scrambleddigits.png "Option to scramble digits") + +This should prevent any casual shoulder-surfer from figuring out your PIN, although it can make it a bit more difficult to unlock your device while, say, driving (but no one should be using their handset in the car, right?). + +Another issue it addresses involves tracking people by monitoring their WiFi MAC address. Most devices that use WiFi perform active scanning for wireless access points. This protocol includes the MAC address of the interface, and there are a number of ways people can use [mobile location analytics][21] to track your movement. Copperhead has an option to randomize your MAC address, which counters this process. + +### [copperos_randommac.png][3] + +![Randomize MAC address](https://opensource.com/sites/default/files/u128651/copperos_randommac.png "Randomize MAC address") + +### Installing apps + +This all sounds pretty good, right? Well, here comes the hard part. While Android is open source, much of the Google code, including the [Google Play Store][22], is not. If you install the Play Store and the code necessary for it to work, you allow Google to install software without your permission. [Google Play's terms of service][23] says: + +> "Google may update any Google app or any app you have downloaded from Google Play to a new version of such app, irrespective of any update settings that you may have selected within the Google Play app or your Device, if Google determines that the update will fix a critical security vulnerability related to the app." + +This is not acceptable from a security standpoint, so you cannot install Google applications on a Copperhead device. + +This took some getting used to, as I had come to rely on things such as Google Maps. The default application repository that ships with Copperhead is [F-Droid][24], which contains only FOSS applications. While I previously used many FOSS applications on Android, it took some effort to use  _nothing but_  free software. I did find some ways to cheat this system, and I'll cover that below. First, here are some of the applications I've grown to love from F-Droid. + +### F-Droid favorites + +**K-9 Mail** + +### [copperheados_k9mail.png][4] + +![K-9 Mail](https://opensource.com/sites/default/files/u128651/copperheados_k9mail.png "K-9 Mail") + +Even before I started using Copperhead, I loved [K-9 Mail][25]. This is simply the best mobile email client I've found, period, and it is one of the first things I install on any new device. I even use it to access my Gmail account, via IMAP and SMTP. + +**Open Camera** + +### [copperheados_cameraapi.png][5] + +![Open Camera](https://opensource.com/sites/default/files/u128651/copperheados_cameraapi.png "Open Camera") + +Copperhead runs only on rather new hardware, and I was consistently disappointed in the quality of the pictures from its default camera application. Then I discovered [Open Camera][26]. A full-featured camera app, it allows you to enable an advanced API to take advantage of the camera hardware. The only thing I miss is the ability to take a panoramic photo. + +**Amaze** + +### [copperheados_amaze.png][6] + +![Amaze](https://opensource.com/sites/default/files/u128651/copperheados_amaze.png "Amaze") + +[Amaze][27] is one of the best file managers I've ever used, free or not. When I need to navigate the filesystem, Amaze is my go-to app. + +**Vanilla Music** + +### [copperheados_vanillamusic.png][7] + +![Vanilla Music](https://opensource.com/sites/default/files/u128651/copperheados_vanillamusic.png "Vanilla Music") + +I was unhappy with the default music player, so I checked out a number of them on F-Droid and settled on [Vanilla Music][28]. It has an easy-to-use interface and interacts well with my Bluetooth devices. + +**OCReader** + +### [coperheados_ocreader.png][8] + +![OCReader](https://opensource.com/sites/default/files/u128651/coperheados_ocreader.png "OCReader") + +I am a big fan of [Nextcloud][29], particularly [Nextcloud News][30], a replacement for the now-defunct [Google Reader][31]. While I can access my news feeds through a web browser, I really missed the ability to manage them through a dedicated app. Enter [OCReader][32]. While it stands for "ownCloud Reader," it works with Nextcloud, and I've had very few issues with it. + +**Noise** + +The SMS/MMS application of choice for most privacy advocates is [Signal][33] by [Open Whisper Systems][34]. Endorsed by [Edward Snowden][35], Signal allows for end-to-end encrypted messaging. If the person you are messaging is also on Signal, your messages will be sent, encrypted, over a data connection facilitated by centralized servers maintained by Open Whisper Systems. It also, until recently, relied on [Google Cloud Messaging][36] (GCM) for notifications, which requires Google Play Services. + +The fact that Signal requires a centralized server bothered some people, so the default application on Copperhead is a fork of Signal called [Silence][37]. This application doesn't use a centralized server but does require that all parties be on Silence for encryption to work. + +Well, no one I know uses Silence. At the moment you can't even get it from the Google Play Store in the U.S. due to a trademark issue, and there is no iOS client. An encrypted SMS client isn't very useful if you can't use it for encryption. + +Enter [Noise][38]. Noise is another application maintained by Copperhead that is a fork of Signal that removes the need for GCM. While not available in the standard F-Droid repositories, Copperhead includes their own repository in the version of F-Droid they ship, which at the moment contains only the Noise application. This app will let you communicate securely with anyone else using Noise or Signal. + +### F-Droid workarounds + +**FFUpdater** + +Copperhead ships with a hardened version of the Chromium web browser, but I am a Firefox fan. Unfortunately, [Firefox is no longer included][39] in the F-Droid repository. Apps on F-Droid are all built by the F-Droid maintainers, so the process for getting into F-Droid can be complicated. The [Compass app for OpenNMS][40] isn't in F-Droid because, at the moment, it does not support builds using the [Ionic Framework][41], which Compass uses. + +Luckily, there is a simple workaround: Install the [FFUpdater][42] app on F-Droid. This allows me to install Firefox and keep it up to date through the browser itself. + +**Amazon Appstore** + +This brings me to a cool feature of Android 8, Oreo. In previous versions of Android, you had a single "known source" for software, usually the Google Play Store, and if you wanted to install software from another repository, you had to go to settings and allow "Install from Unknown Sources." I always had to remember to turn that off after an install to prevent malicious code from being able to install software on my device. + +### [copperheados_sources.png][9] + +![Allowing sources to install apps](https://opensource.com/sites/default/files/u128651/copperheados_sources.png "Allowing sources to install apps") + +With Oreo, you can permanently allow a specified application to install applications. For example, I use some applications from the [Amazon Appstore][43] (such as the Amazon Shopping and Kindle apps). When I download and install the Amazon Appstore Android package (APK), I am prompted to allow the application to install apps and then I'm not asked again. Of course, this can be turned on and off on a per-application basis. + +The Amazon Appstore has a number of useful apps, such as [IMDB][44] and [eBay][45]. Many of them don't require Google Services, but some do. For example, if I install the [Skype][46] app via Amazon, it starts up, but then complains about the operating system. The American Airlines app would start, then complain about an expired certificate. (I contacted them and was told they were no longer maintaining the version in the Amazon Appstore and it would be removed.) In any case, I can pretty simply install a couple of applications I like without using Google Play. + +**Google Play** + +Well, what about those apps you love that don't use Google Play Services but are only available through the Google Play Store? There is yet another way to safely get those apps on your Copperhead device. + +This does require some technical expertise and another device. On the second device, install the [TWRP][47] recovery application. This is usually a key first step in installing any custom ROM, and TWRP is supported on a large number of devices. You will also need the Android Debug Bridge ([ADB][48]) application from the [Android SDK][49], which can be downloaded at no cost. + +On the second device, use the Google Play Store to install the applications you want. Then, reboot into recovery. You can mount the system partition via TWRP; plug the device into a computer via a USB cable and you should be able to see it via ADB. There is a system directory called `/data/app`, and in it you will find all the APK files for your applications. Copy those you want to your computer (I use the ADB `pull`command and copy over the whole directory). + +Disconnect that phone and connect your Copperhead device. Enable the "Transfer files" option, and you should see the storage directory mounted on your computer. Copy over the APK files for the applications you want, then install them via the Amaze file manager (just navigate to the APK file and click on it). + +Note that you can do this for any application, and it might even be possible to install Google Play Services this way on Copperhead, but that kind of defeats the purpose. I use this mainly to get the [Electric Sheep][50] screensaver and a guitar tuning app I like called [Cleartune][51]. Be aware that if you install TWRP, especially on a Google Pixel, security updates may not work, as they'll expect the stock recovery. In this case you can always use [fastboot][52] to access TWRP, but leave the default recovery in place. + +### Must-have apps without a workaround + +Unfortunately, there are still a couple of Google apps I find it hard to live without. Google Maps is probably the main Google application I use, and yes, while I know I'm giving up my location to Google, it has saved hours of my life by routing me around traffic issues. [OpenStreetMap][53] has an app available via F-Droid, but it doesn't have the real-time information that makes Google Maps so useful. I also use Skype on occasion, usually when I am out of the country and have only a data connection (i.e., through a hotel WiFi network). It lets me call home and other places at a very affordable price. + +My workaround is to carry two phones. I know this isn't an option for most people, but it is the only one I've found for now. I use my Copperhead phone for anything personal (email, contacts, calendars, pictures, etc.) and my "Googlephone" for Maps, Skype, and various games. + +My dream would be for someone to perfect a hypervisor on a handset. Then I could run Copperhead and stock Google Android on the same device. I don't think anyone has a strong business reason to do it, but I do hope it happens. + +### Devices that support Copperhead + +Before you rush out to install Copperhead, there are some hurdles you'll have to jump. First, it is supported on only a [limited number of handsets][54], almost all of them late-model Google devices. The logic behind this is simple: Google tends to release Android security updates for its devices quickly, and I've found that Copperhead is able to follow suit within a day, if not within hours. Second, like any open source project, it has limited resources and it is difficult to support even a fraction of the devices now available to end users. Finally, if you want to run Copperhead on handsets like the Pixel and Pixel XL, you'll either have to build from source or [buy a device][55] from Copperhead directly. + +When I discovered Copperhead, I had a Nexus 6P, which (along with the Nexus 5X) is one of the supported devices. This allowed me to play with and get used to the operating system. I liked it so much that I donated some money to the project, but I kind of balked at the price they were asking for Pixel and Pixel XL handsets. + +Recently, though, I ended up purchasing a Pixel XL directly from Copperhead. There were a couple of reasons. One, since all of the code is available on GitHub, I set out to do [my own build][56] for a Pixel device. That process (which I never completed) made me appreciate the amount of work Copperhead puts into its project. Two, there was an article on [Slashdot][57] discussing how people were selling devices with Copperhead pre-installed and using Copperhead's update servers. I didn't appreciate that very much. Finally, I support FOSS not only by being a vocal user but also with my wallet. + +### Putting the "libre" back into free + +Another thing I love about FOSS is that I have options. There is even a new option to Copperhead being developed called [Eelo][58]. Created by [Gaël Duval][59], the developer of Mandrake Linux, this is a privacy-based Android operating system based on [LineageOS][60] (the descendant of CyanogenMod). While it should be supported on more handsets than Copperhead is, it is still in the development stage, and Copperhead is very stable and mature. I am eager to check it out, though. + +For the year I've used CopperheadOS, I've never felt safer when using a mobile device to connect to a network. I've found the open source replacements for my old apps to be more than adequate, if not better than the original apps. I've also rediscovered the browser. Where I used to have around three to four tabs open, I now have around 10, because I've found that I usually don't need to install an app to easily access a site's content. + +With companies like Google and Apple trying more and more to insinuate themselves into the lives of their users, it is nice to have an option that puts the "libre" back into free. + + +### About the author + + [![](https://opensource.com/sites/default/files/styles/profile_pictures/public/balog_tarus_-_julian_-_square.jpg?itok=ZA6yem3I)][61] + +Tarus Balog - Having been kicked out of some of the best colleges and universities in the country, I managed after seven years to get a BSEE and entered the telecommunications industry. I always ended up working on projects where we were trying to get the phone switch to talk to PCs. This got me interested in the creation and management of large communication networks. So I moved into the data communications field (they were separate back then) and started working with commercial network management tools... [more about Tarus Balog][12][More about me][13] + +-------------------------------------------------------------------------------- + +via: https://opensource.com/article/18/1/copperheados-delivers-mobile-freedom-privacy-and-security + +作者：[Tarus Balog ][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://opensource.com/users/sortova +[1]:https://opensource.com/file/384496 +[2]:https://opensource.com/file/384501 +[3]:https://opensource.com/file/384506 +[4]:https://opensource.com/file/384491 +[5]:https://opensource.com/file/384486 +[6]:https://opensource.com/file/384481 +[7]:https://opensource.com/file/384476 +[8]:https://opensource.com/file/384471 +[9]:https://opensource.com/file/384466 +[10]:https://opensource.com/article/18/1/copperheados-delivers-mobile-freedom-privacy-and-security?rate=P32BmRpJF5bYEYTHo4mW3Hp4XRk34Eq3QqMDf2oOGnw +[11]:https://copperhead.co/android/docs/building#redistribution +[12]:https://opensource.com/users/sortova +[13]:https://opensource.com/users/sortova +[14]:https://opensource.com/user/11447/feed +[15]:https://www.flickr.com/photos/mstable/17517955832 +[16]:https://creativecommons.org/publicdomain/mark/1.0/ +[17]:https://creativecommons.org/licenses/by-sa/4.0/ +[18]:https://en.wikipedia.org/wiki/Android_(operating_system)#AOSP +[19]:https://copperhead.co/ +[20]:https://copperhead.co/android/docs/technical_overview +[21]:https://en.wikipedia.org/wiki/Mobile_location_analytics +[22]:https://en.wikipedia.org/wiki/Google_Play#Compatibility +[23]:https://play.google.com/intl/en-us_us/about/play-terms.html +[24]:https://en.wikipedia.org/wiki/F-Droid +[25]:https://f-droid.org/en/packages/com.fsck.k9/ +[26]:https://f-droid.org/en/packages/net.sourceforge.opencamera/ +[27]:https://f-droid.org/en/packages/com.amaze.filemanager/ +[28]:https://f-droid.org/en/packages/ch.blinkenlights.android.vanilla/ +[29]:https://nextcloud.com/ +[30]:https://github.com/nextcloud/news +[31]:https://en.wikipedia.org/wiki/Google_Reader +[32]:https://f-droid.org/packages/email.schaal.ocreader/ +[33]:https://en.wikipedia.org/wiki/Signal_(software) +[34]:https://en.wikipedia.org/wiki/Open_Whisper_Systems +[35]:https://en.wikipedia.org/wiki/Edward_Snowden +[36]:https://en.wikipedia.org/wiki/Google_Cloud_Messaging +[37]:https://f-droid.org/en/packages/org.smssecure.smssecure/ +[38]:https://github.com/copperhead/Noise +[39]:https://f-droid.org/wiki/page/org.mozilla.firefox +[40]:https://compass.opennms.io/ +[41]:https://ionicframework.com/ +[42]:https://f-droid.org/en/packages/de.marmaro.krt.ffupdater/ +[43]:https://www.amazon.com/gp/feature.html?docId=1000626391 +[44]:https://www.imdb.com/ +[45]:https://www.ebay.com/ +[46]:https://www.skype.com/ +[47]:https://twrp.me/ +[48]:https://en.wikipedia.org/wiki/Android_software_development#ADB +[49]:https://developer.android.com/studio/index.html +[50]:https://play.google.com/store/apps/details?id=com.spotworks.electricsheep&hl=en +[51]:https://play.google.com/store/apps/details?id=com.bitcount.cleartune&hl=en +[52]:https://en.wikipedia.org/wiki/Android_software_development#Fastboot +[53]:https://f-droid.org/packages/net.osmand.plus/ +[54]:https://copperhead.co/android/downloads +[55]:https://copperhead.co/android/store +[56]:https://copperhead.co/android/docs/building +[57]:https://news.slashdot.org/story/17/11/12/024231/copperheados-fights-unlicensed-installations-on-nexus-phones +[58]:https://eelo.io/ +[59]:https://en.wikipedia.org/wiki/Ga%C3%ABl_Duval +[60]:https://en.wikipedia.org/wiki/LineageOS +[61]:https://opensource.com/users/sortova +[62]:https://opensource.com/users/sortova +[63]:https://opensource.com/users/sortova +[64]:https://opensource.com/article/18/1/copperheados-delivers-mobile-freedom-privacy-and-security#comments +[65]:https://opensource.com/tags/mobile +[66]:https://opensource.com/tags/android \ No newline at end of file diff --git a/sources/tech/20180129 How To Resume Partially Transferred Files Over SSH Using Rsync.md b/sources/tech/20180129 How To Resume Partially Transferred Files Over SSH Using Rsync.md new file mode 100644 index 0000000000..5e0583ab4f --- /dev/null +++ b/sources/tech/20180129 How To Resume Partially Transferred Files Over SSH Using Rsync.md @@ -0,0 +1,101 @@ +How To Resume Partially Transferred Files Over SSH Using Rsync +====== + +![](https://www.ostechnix.com/wp-content/uploads/2016/02/Resume-Partially-Transferred-Files-Over-SSH-Using-Rsync.png) + +There are chances that the large files which are being copied over SSH using SCP command might be interrupted or cancelled or broken due to various reasons such as power failure or network failure or user intervention. The other day I was copying the Ubuntu 16.04 ISO file to my remote system. Unfortunately, the power is gone, and the network connection is dropped immediately. The result? The copy process is terminated! This is just a simple example. The Ubuntu ISO is not so big, and I could restart the copy process as soon as the power is restored. But in production environment, you might not want to do it while you're transferring large files. + +Also, you can't always resume the aborted process using **scp** command. Because, If you do, It will simply overwrite the existing files. What would you do in such situations? No worries! This is where **Rsync** utility comes in handy! Rsync can help you to resume the interrupted copy or download process where you left it off. For those wondering, Rsync is a fast, versatile file copying utility that can be used to copy and transfer files or folders to and from remote and local systems. + +It offers a large number of options that control every aspect of its behavior and permit very flexible specification of the set of files to be copied. It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination. Rsync is widely used for backups and mirroring and as an improved copy command for everyday use. + +Just like SCP, rsync will also copy files over SSH. In case you wanted to download or transfer a big files and folders over SSH, I recommend you to use rsync utility. Be mindful that the **rsync utility should be installed on both sides** (remote and local systems) in order to resume partially transferred files. + +### Resume Partially Transferred Files Using Rsync + +Well, let me show you an example. I am going to copy Ubuntu 16.04 ISO from my local system to remote system with command: + +``` +$ scp Soft_Backup/OS\ Images/Linux/ubuntu-16.04-desktop-amd64.iso sk@192.168.43.2:/home/sk/ +``` + +Here, + + * **sk** is my remote system 's username + * **192.168.43.2** is the IP address of the remote machine. + + + +Now, I terminated it by pressing **CTRL+c**. + +**Sample output:** + +``` +sk@192.168.43.2's password: +ubuntu-16.04-desktop-amd64.iso 26% 372MB 26.2MB/s 00:39 ETA^c +``` + +[![][1]][2] + +As you see in the above output, I terminated the copy process when it reached 26%. + +If I re-run the above command, it will simply overwrite the existing file. In other words, the copy process will not resume where I left it off. + +In order to resume the copy process, we can use **rsync** command as shown below. + +``` +$ rsync -P -rsh=ssh Soft_Backup/OS\ Images/Linux/ubuntu-16.04-desktop-amd64.iso sk@192.168.43.2:/home/sk/ +``` + +**Sample output:** +``` +sk@192.168.1.103's password: +sending incremental file list +ubuntu-16.04-desktop-amd64.iso +                   380.56M 26% 41.05MB/s 0:00:25 +``` + +[![][1]][4] + +See? Now, the copying process is resumed where we left it off earlier. You also can use "-partial" instead of parameter "-P" like below. +``` +$ rsync --partial -rsh=ssh Soft_Backup/OS\ Images/Linux/ubuntu-16.04-desktop-amd64.iso sk@192.168.43.2:/home/sk/ +``` + +Here, the parameter "-partial" or "-P" tells the rsync command to keep the partial downloaded file and resumes the process. + +Alternatively, we can use the following commands as well to resume partially transferred files over SSH. + +``` +$ rsync -avP Soft_Backup/OS\ Images/Linux/ubuntu-16.04-desktop-amd64.iso sk@192.168.43.2:/home/sk/ +``` + +Or, + +``` +rsync -av --partial Soft_Backup/OS\ Images/Linux/ubuntu-16.04-desktop-amd64.iso sk@192.168.43.2:/home/sk/ +``` + +That's it. You know now how to resume the cancelled, interrupted, and partially downloaded files using rsync command. As you can see, it is not so difficult either. If rsync is installed on both systems, we can easily resume the copy process as described above. + +If you find this tutorial helpful, please share it on your social, professional networks and support OSTechNix. More good stuffs to come. Stay tuned! + +Cheers! + + + +-------------------------------------------------------------------------------- + +via: https://www.ostechnix.com/how-to-resume-partially-downloaded-or-transferred-files-using-rsync/ + +作者：[SK][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.ostechnix.com/author/sk/ +[1]:data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7 +[2]:http://www.ostechnix.com/wp-content/uploads/2016/02/scp.png () +[3]:/cdn-cgi/l/email-protection +[4]:http://www.ostechnix.com/wp-content/uploads/2016/02/rsync.png () diff --git a/sources/tech/20180129 How programmers learn to code.md b/sources/tech/20180129 How programmers learn to code.md new file mode 100644 index 0000000000..c741c01161 --- /dev/null +++ b/sources/tech/20180129 How programmers learn to code.md @@ -0,0 +1,63 @@ +How programmers learn to code +============================================================ + + [![How programmers learn to code](https://mybroadband.co.za/news/wp-content/uploads/2016/01/Programmer-working-computer-code.jpg)][8] + + +HackerRank recently published the results of its 2018 Developer Skills Report, in which it asked programmers when they started coding. + +39,441 professional and student developers completed the online survey from 16 October to 1 November 2016, with over 25% of the developers surveyed writing their first piece of code before they were 16 years old. + +### How programmers learn + +In terms of how programmers learnt to code, self-teaching is the norm for developers of all ages, stated the report. + +“Even though 67% of developers have computer science degrees, roughly 74% said they were at least partially self-taught.” + +On average, developers know four languages, but they want to learn four more. + +The thirst for learning varies by generations – developers between 18 and 24 plan to learn six languages, whereas developers older than 35 only plan to learn three. + + [![HackerRank 2018 how did you learn to code](https://mybroadband.co.za/news/wp-content/uploads/2018/01/HackerRank-2018-how-did-you-learn-to-code.jpg)][5] + +### What programmers want + +HackerRank also looked at what developers want most from an employer. + +On average, a good work-life balance, closely followed by professional growth and learning, was the most desired requirement. + +Segmenting the data by region revealed that Americans crave work-life balance more than developers Asia and Europe. + +Students tend to rank growth and learning over work-life balance, while professionals rate compensation more highly than students do. + +People who work in smaller companies tended to rank work-life balance lower, but it was still in their top three. + +Age also made a difference, with developers 25 and older rating work-life balance as most important, while those between 18 and 24 rate it as less important. + +“In some ways, we’ve discovered a slight contradiction here. Developers want work-life balance, but they also have an insatiable thirst and need for learning,” said HackerRank. + +It advised that focusing on doing what you enjoy, as opposed to trying to learning everything, can help strike a better work-life balance. + + [![HackerRank 2018 what do developers want most](https://mybroadband.co.za/news/wp-content/uploads/2018/01/HackerRank-2018-what-do-developers-want-most-640x342.jpg)][6] + + [![HackerRank 2018 how to improve work-life balance](https://mybroadband.co.za/news/wp-content/uploads/2018/01/HackerRank-2018-how-to-improve-work-life-balance-378x430.jpg)][7] + +-------------------------------------------------------------------------------- + +via: https://mybroadband.co.za/news/smartphones/246583-how-programmers-learn-to-code.html + +作者：[Staff Writer ][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://mybroadband.co.za/news/author/staff-writer +[1]:https://mybroadband.co.za/news/author/staff-writer +[2]:https://twitter.com/intent/tweet/?text=How+programmers+learn+to+code%20https://mybroadband.co.za/news/smartphones/246583-how-programmers-learn-to-code.html&via=mybroadband +[3]:mailto:?subject=How%20programmers%20learn%20to%20code&body=HackerRank%20recently%20published%20the%20results%20of%20its%202018%20Developer%20Skills%20Report.%0A%0Ahttps%3A%2F%2Fmybroadband.co.za%2Fnews%2Fsmartphones%2F246583-how-programmers-learn-to-code.html +[4]:https://mybroadband.co.za/news/smartphones/246583-how-programmers-learn-to-code.html#disqus_thread +[5]:https://mybroadband.co.za/news/wp-content/uploads/2018/01/HackerRank-2018-how-did-you-learn-to-code.jpg +[6]:https://mybroadband.co.za/news/wp-content/uploads/2018/01/HackerRank-2018-what-do-developers-want-most.jpg +[7]:https://mybroadband.co.za/news/wp-content/uploads/2018/01/HackerRank-2018-how-to-improve-work-life-balance.jpg +[8]:https://mybroadband.co.za/news/smartphones/246583-how-programmers-learn-to-code.html \ No newline at end of file diff --git a/sources/tech/20180129 How to Use DockerHub.md b/sources/tech/20180129 How to Use DockerHub.md new file mode 100644 index 0000000000..3793a6b718 --- /dev/null +++ b/sources/tech/20180129 How to Use DockerHub.md @@ -0,0 +1,135 @@ +How to Use DockerHub +====== + +![](https://www.linux.com/sites/lcom/files/styles/rendered_file/public/dockerhub-container.jpg?itok=tvraxGzh) + +In the previous articles, we learned the basics of [Docker terminology][1], [how to install Docker][2] on desktop Linux, macOS, and Windows, and [how to create container images][3] and run them on your system. In this last article in the series, we will talk about using images from DockerHub and publishing your own images to DockerHub. + +First things first: what is DockerHub and why is it important? DockerHub is a cloud-based repository run and managed by Docker Inc. It's an online repository where Docker images can be published and used by other users. There are both public and private repositories. If you are a company, you can have a private repository for use within your own organization, whereas public images can be used by anyone. + +You can also use official Docker images that are published publicly. I use many such images, including for my test WordPress installations, KDE plasma apps, and more. Although we learned last time how to create your own Docker images, you don't have to. There are thousands of images published on DockerHub for you to use. DockerHub is hardcoded into Docker as the default registry, so when you run the docker pull command against any image, it will be downloaded from DockerHub. + +### Download images from Docker Hub and run locally + +Please check out the previous articles in the series to get started. Then, once you have Docker running on your system, you can open the terminal and run: +``` +$ docker images +``` + +This command will show all the docker images currently on your system. Let's say you want to deploy Ubuntu on your local machine; you would do: +``` +$ docker pull ubuntu +``` + +If you already have Ubuntu image on your system, the command will automatically update that image to the latest version. So, if you want to update the existing images, just run the docker pull command, easy peasy. It's like apt-get upgrade without any muss and fuss. + +You already know how to run an image: +``` +$ docker run -it + +$ docker run -it ubuntu +``` + +The command prompt should change to something like this: +``` +root@1b3ec4621737:/# +``` + +Now you can run any command and utility that you use on Ubuntu. It's all safe and contained. You can run all the experiments and tests you want on that Ubuntu. Once you are done testing, you can nuke the image and download a new one. There is no system overhead that you would get with a virtual machine. + +You can exit that container by running the exit command: +``` +$ exit +``` + +Now let's say you want to install Nginx on your system. Run search to find the desired image: +``` +$ docker search nginx + +aizMFFysICAEsgDDYrsrlqwoCgGbWVHtcOzgV9mA +``` + +As you can see, there are many images of Nginx on DockerHub. Why? Because anyone can publish an image. Various images are optimized for different projects, so you can choose the appropriate image. You just need to install the appropriate image for your use-case. + +Let's say you want to pull Bitnami's Nginx container: +``` +$ docker pull bitnami/nginx +``` + +Now run it with: +``` +$ docker run -it bitnami/nginx +``` + +### How to publish images to Docker Hub? + +Previously, [we learned how to create a Docker image][3], and we can easily publish that image to DockerHub. First, you need to log into DockerHub. If you don't already have an account, please [create one][5]. Then, you can open terminal app and log in: +``` +$ docker login --username= +``` + +Replace with the name of your username for Docker Hub. In my case it's arnieswap: +``` +$ docker login --username=arnieswap> +``` + +Enter the password, and you are logged in. Now run the docker images command to get the ID of the image that you created last time. +``` +$ docker images + +tW1jDOugkX7J2FfyFyToM6B8m5OYFwMba-Ag5aez +``` + +Now, suppose you want to push the ng image to DockerHub. First, we need to tag that image ([learn more about tags][1]): +``` +$ docker tag e7083fd898c7 arnieswap/my_repo:testing +``` + +Now push that image: +``` +$ docker push arnieswap/my_repo +``` + +The push refers to repository [docker.io/arnieswap/my_repo] +``` +12628b20827e: Pushed + +8600ee70176b: Mounted from library/ubuntu + +2bbb3cec611d: Mounted from library/ubuntu + +d2bb1fc88136: Mounted from library/ubuntu + +a6a01ad8b53f: Mounted from library/ubuntu + +833649a3e04c: Mounted from library/ubuntu + +testing: digest: sha256:286cb866f34a2aa85c9fd810ac2cedd87699c02731db1b8ca1cfad16ef17c146 size: 1569 + +``` + +Eureka! Your image is being uploaded. Once finished, open DockerHub, log into your account, and you can see your very first Docker image. Now anyone can deploy your image. It's the easiest and fastest way to develop and distribute software. Whenever you update the image, users can simply run: +``` +$ docker run arnieswap/my_repo +``` + +Now you know why people love Docker containers. They solve many problems that traditional workloads face and allow you develop, test, and deploy applications in no time. And, by following the steps in this series, you can try them out for yourself. + + +-------------------------------------------------------------------------------- + +via: https://www.linux.com/blog/learn/intro-to-linux/2018/1/how-use-dockerhub + +作者：[Swapnil Bhartiya][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.linux.com/users/arnieswap +[1]:https://www.linux.com/blog/intro-to-linux/2017/12/container-basics-terms-you-need-know +[2]:https://www.linux.com/blog/learn/intro-to-linux/how-install-docker-ce-your-desktop +[3]:https://www.linux.com/blog/learn/intro-to-linux/2018/1/how-create-docker-image +[4]:https://lh3.googleusercontent.com/aizMFFysICAEsgDDYrsrlqwoCgGbWVHtcOzgV9mAtV8IdBZgHPJTdHIZhWBNCRvOyJb108ZBajJ_Nz10yCxGSvk-AF-yvFxpojLdVu3Jjihcwaup6CQLc67A5nglBuGDaOZWcrbV +[5]:https://hub.docker.com/ +[6]:https://lh6.googleusercontent.com/tW1jDOugkX7J2FfyFyToM6B8m5OYFwMba-Ag5aezVGf2A5gsKJ47QrCh_TOKWgIKfE824Uc2Cwwwj9jWps1yJlUZqDyIceVQs-nEbKavFDxuUxLyd4thBA4_rsXrQH4r7hrG8FnD diff --git a/sources/tech/20180129 How to make your LXD containers get IP addresses from your LAN using a bridge.md b/sources/tech/20180129 How to make your LXD containers get IP addresses from your LAN using a bridge.md new file mode 100644 index 0000000000..6f26f182b8 --- /dev/null +++ b/sources/tech/20180129 How to make your LXD containers get IP addresses from your LAN using a bridge.md @@ -0,0 +1,173 @@ +How to make your LXD containers get IP addresses from your LAN using a bridge +====== +**Background** : LXD is a hypervisor that manages machine containers on Linux distributions. You install LXD on your Linux distribution and then you can launch machine containers into your distribution running all sort of (other) Linux distributions. + +In the previous post, we saw how to get our LXD container to receive an IP address from the local network (instead of getting the default private IP address), using **macvlan**. + +In this post, we are going to see how to use a **bridge** to make our containers get an IP address from the local network. Specifically, we are going to see how to do this using NetworkManager. If you have several public IP addresses, you can use this method (or the other with the **macvlan** ) in order to expose your LXD containers directly to the Internet. + +### Creating the bridge with NetworkManager + +See this post [How to configure a Linux bridge with Network Manager on Ubuntu][1] on how to create the bridge with NetworkManager. It explains that you + + 1. Use **NetworkManager** to **Add a New Connection** , a **Bridge**. + 2. When configuring the **Bridge** , you specify the real network connection (the device, like **eth0** or **enp3s12** ) that will be **the slave of the bridge**. You can verify the device of the network connection if you run **ip route list 0.0.0.0/0**. + 3. Then, you can remove the old network connection and just keep the slave. The slave device ( **bridge0** ) will now be the device that gets you your LAN IP address. + + + +At this point you would have again network connectivity. Here is the new device, **bridge0**. +``` +$ ifconfig bridge0 +bridge0 Link encap:Ethernet HWaddr 00:e0:4b:e0:a8:c2 + inet addr:192.168.1.64 Bcast:192.168.1.255 Mask:255.255.255.0 + inet6 addr: fe80::d3ca:7a11:f34:fc76/64 Scope:Link + UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 + RX packets:9143 errors:0 dropped:0 overruns:0 frame:0 + TX packets:7711 errors:0 dropped:0 overruns:0 carrier:0 + collisions:0 txqueuelen:1000 + RX bytes:7982653 (7.9 MB) TX bytes:1056263 (1.0 MB) +``` + +### Creating a new profile in LXD for bridge networking + +In LXD, there is a default profile and then you can create additional profile that either are independent from the default (like in the **macvlan** post), or can be chained with the default profile. Now we see the latter. + +First, create a new and empty LXD profile, called **bridgeprofile**. +``` +$ lxc create profile bridgeprofile +``` + +Here is the fragment to add to the new profile. The **eth0** is the interface name in the container, so for the Ubuntu containers it does not change. Then, **bridge0** is the interface that was created by NetworkManager. If you created that bridge by some other way, add here the appropriate interface name. The **EOF** at the end is just a marker when we copy and past to the profile. +``` +description: Bridged networking LXD profile +devices: + eth0: + name: eth0 + nictype: bridged + parent: bridge0 + type: nic +**EOF** +``` + +Paste the fragment to the new profile. +``` +$ cat <:] [:][] [--ephemeral|-e] [--profile|-p ...] [--config|-c ...] [--type|-t ] + +Create and start containers from images. + +Not specifying -p will result in the default profile. +Specifying "-p" with no argument will result in no profile. + +Examples: + lxc launch ubuntu:16.04 u1 + +Options: + -c, --config (= map[]) Config key/value to apply to the new container + --debug (= false) Enable debug mode + -e, --ephemeral (= false) Ephemeral container + --force-local (= false) Force using the local unix socket + --no-alias (= false) Ignore aliases when determining what command to run + -p, --profile (= []) Profile to apply to the new container +**-t (= "") Instance type** + --verbose (= false) Enable verbose mode +``` + +What do we put for Instance type? Here is the documentation, + + + +Simply put, an instance type is just a mnemonic shortcut for specific pair of CPU cores and RAM memory settings. For CPU you specify the number of cores and for RAM memory the amount in GB (assuming your own computer has enough cores and RAM so that LXD can allocate them to the newly created container). + +You would need an instance type if you want to create a machine container that resembles in the specs as close as possible what you will be installing later on, on AWS (Amazon), Azure (Microsoft) or GCE (Google). + +The instance type can have any of the following forms, + + * `` for example: **t2.micro** (LXD figures out that this refers to AWS t2.micro, therefore 1 core, 1GB RAM). + * `:` for example, **aws:t2.micro** (LXD quickly looks into the AWS types, therefore 1core, 1GB RAM). + * `c-m` for example, **c1-m1** (LXD explicitly allocates one core, and 1GB RAM). + + + +Where do these mnemonics like **t2.micro** come from? The documentation says from + +[![][1]][2] + +There are three sets of instance types, **aws** , **azure** and **gce**. Their names are listed in [the LXD instance type index file ][3]**.yaml,** +``` +aws: "aws.yaml" +gce: "gce.yaml" +azure: "azure.yaml" + +``` + +Over there, there are YAML configuration files for each of AWS, Azure and GCE, and in them there are settings for CPU cores and RAM memory. + +The actual URLs that the LXD client will be using, are + + + +Sample for AWS: +``` +t2.large: + cpu: 2.0 + mem: 8.0 +t2.medium: + cpu: 2.0 + mem: 4.0 +t2.micro: + cpu: 1.0 + mem: 1.0 +t2.nano: + cpu: 1.0 + mem: 0.5 +t2.small: + cpu: 1.0 + mem: 2.0 +``` + + + +Sample for Azure: +``` +ExtraSmall: + cpu: 1.0 + mem: 0.768 +Large: + cpu: 4.0 + mem: 7.0 +Medium: + cpu: 2.0 + mem: 3.5 +Small: + cpu: 1.0 + mem: 1.75 +Standard_A1_v2: + cpu: 1.0 + mem: 2.0 +``` + + + +Sample for GCE: +``` +f1-micro: + cpu: 0.2 + mem: 0.6 +g1-small: + cpu: 0.5 + mem: 1.7 +n1-highcpu-16: + cpu: 16.0 + mem: 14.4 +n1-highcpu-2: + cpu: 2.0 + mem: 1.8 +n1-highcpu-32: + cpu: 32.0 + mem: 28.8 +``` + +Let's see an example. Here, all of the following are all equivalent! Just run one of them to get a 1 CPU core/1GB RAM container. +``` +$ lxc launch ubuntu:x -t t2.micro aws-t2-micro + +$ lxc launch ubuntu:x -t aws:t2.micro aws-t2-micro + +$ lxc launch ubuntu:x -t c1-m1 aws-t2-micro +``` + +Let's verify that the constraints have been actually set for the container. +``` +$ lxc config get aws-t2-micro limits.cpu +1 + +$ lxc config get aws-t2-micro limits.cpu.allowance + + +$ lxc config get aws-t2-micro limits.memory +1024MB + +$ lxc config get aws-t2-micro limits.memory.enforce + + +``` + +There are generic limits for 1 CPU core and 1024MB/1GB RAM. For more, see [LXD resource control][4]. + +If you already have a running container and you wanted to set limits live (no need to restart it), here is how you would do that. +``` +$ lxc launch ubuntu:x mycontainer +Creating mycontainer +Starting mycontainer + +$ lxc config set mycontainer limits.cpu 1 +$ lxc config set mycontainer limits.memory 1GB +``` + +Let's see the config with the limits, +``` +$ lxc config show mycontainer +architecture: x86_64 +config: + image.architecture: amd64 + image.description: ubuntu 16.04 LTS amd64 (release) (20180126) + image.label: release + image.os: ubuntu + image.release: xenial + image.serial: "20180126" + image.version: "16.04" + limits.cpu: "1" + limits.memory: 1GB +... +``` + +### Troubleshooting + +#### I tried to the the memory limit but I get an error! + +I got this error, +``` +$ lxc config set mycontainer limits.memory 1 +error: Failed to set cgroup memory.limit_in_bytes="1": setting cgroup item for the container failed +Exit 1 +``` + +When you set the memory limit ( **limits.memory** ), you need to append a specifier like **GB** (as in 1GB). Because the number there is in bytes if no specifier is present, and one byte of memory is not going to work. + +#### I cannot set the limits in lxc launch -config! + +How do I use **lxc launch -config ConfigurationGoesHere**? + +Here is the documentation: +``` +$ lxc launch --help +Usage: lxc launch [ :] ... [--config|-c ...] +``` + +Here it is, +``` +$ lxc launch ubuntu:x --config limits.cpu=1 --config limits.memory=1GB mycontainer +Creating mycontainer +Starting mycontainer +``` + +That is, use multiple **- config** parameters. + + +-------------------------------------------------------------------------------- + +via: https://blog.simos.info/how-to-use-lxd-instance-types/ + +作者：[Simos Xenitellis][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://blog.simos.info/author/simos/ +[1]:https://i1.wp.com/blog.simos.info/wp-content/uploads/2018/01/lxd-instance-types.png?resize=750%2C277&ssl=1 +[2]:https://i1.wp.com/blog.simos.info/wp-content/uploads/2018/01/lxd-instance-types.png?ssl=1 +[3]:https://uk.images.linuxcontainers.org/meta/instance-types/.yaml +[4]:https://stgraber.org/2016/03/26/lxd-2-0-resource-control-412/ diff --git a/sources/tech/20180129 Install Zabbix Monitoring Server and Agent on Debian 9.md b/sources/tech/20180129 Install Zabbix Monitoring Server and Agent on Debian 9.md new file mode 100644 index 0000000000..308b6f1341 --- /dev/null +++ b/sources/tech/20180129 Install Zabbix Monitoring Server and Agent on Debian 9.md @@ -0,0 +1,401 @@ +Install Zabbix Monitoring Server and Agent on Debian 9 +====== + +Monitoring tools are used to continuously keep track of the status of the system and send out alerts and notifications if anything goes wrong. Also, monitoring tools help you to ensure that your critical systems, applications and services are always up and running. Monitoring tools are a supplement for your network security, allowing you to detect malicious traffic, where it's coming from and how to cancel it. + +Zabbix is a free, open source and the ultimate enterprise-level monitoring tool designed for real-time monitoring of millions of metrics collected from tens of thousands of servers, virtual machines and network devices. Zabbix has been designed to skill from small environment to large environment. Its web front-end is written in PHP, backend is written in C and uses MySQL, PostgreSQL, SQLite, Oracle or IBM DB2 to store data. Zabbix provides graphing functionality that allows you to get an overview of the current state of specific nodes and the network + +Some of the major features of the Zabbix are listed below: + + * Monitoring Servers, Databases, Applications, Network Devices, Vmware hypervisor, Virtual Machines and much more. + * Special designed to support small to large environments to improve the quality of your services and reduce operating costs by avoiding downtime. + * Fully open source, so you don't need to pay anything. + * Provide user friendly web interface to do everything from a central location. + * Comes with SNMP to monitor Network device and IPMI to monitor Hardware device. + * Web-based front end that allows full system control from a browser. + +This tutorial will walk you through the step by step instruction of how to install Zabbix Server and Zabbix agent on Debian 9 server. We will also explain how to add the Zabbix agent to the Zabbix server for monitoring. + +#### Requirements + + * Two system with Debian 9 installed. + * Minimum 1 GB of RAM and 10 DB of disk space required. Amount of RAM and Disk space depends on the number of hosts and the parameters that are being monitored. + * A non-root user with sudo privileges setup on your server. + + + +#### Getting Started + +Before starting, it is necessary to update your server's package repository to the latest stable version. You can update it by just running the following command on both instances: + +``` +sudo apt-get update -y +sudo apt-get upgrade -y +``` + +Next, restart your system to apply these changes. + +#### Install Apache, PHP and MariaDB + +Zabbix runs on Apache web server, written in PHP and uses MariaDB/MySQL to store their data. So in order to install Zabbix, you will require Apache, MariaDB and PHP to work. First, install Apache, PHP and Other PHP modules by running the following command: + +``` +sudo apt-get install apache2 libapache2-mod-php7.0 php7.0 php7.0-xml php7.0-bcmath php7.0-mbstring -y +``` + +Next, you will need to add MariaDB repository to your system. Because, latest version of the MariaDB is not available in Debian 9 default repository. + +You can add the repository by running the following command: + +``` +sudo apt-get install software-properties-common -y +sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 0xF1656F24C74CD1D8 +sudo add-apt-repository 'deb [arch=amd64] http://www.ftp.saix.net/DB/mariadb/repo/10.1/debian stretch main' +``` + +Next, update the repository by running the following command: + +``` +sudo apt-get update -y +``` + +Finally, install the MariaDB server with the following command: + +``` +sudo apt-get install mariadb-server -y +``` + +By default, MariaDB installation is not secured. So you will need to secure it first. You can do this by running the mysql_secure_installation script. + +``` +sudo mysql_secure_installation +``` + +Answer all the questions as shown below: +``` + +Enter current password for root (enter for none): Enter +Set root password? [Y/n]: Y +New password: +Re-enter new password: +Remove anonymous users? [Y/n]: Y +Disallow root login remotely? [Y/n]: Y +Remove test database and access to it? [Y/n]: Y +Reload privilege tables now? [Y/n]: Y + +``` + +The above script will set the root password, remove test database, remove anonymous user and Disallow root login from a remote location. + +Once the MariaDB installation is secured, start the Apache and MariaDB service and enable them to start on boot time by running the following command: + +``` +sudo systemctl start apache2 +sudo systemctl enable apache2 +sudo systemctl start mysql +sudo systemctl enable mysql +``` + +#### Installing Zabbix Server + +By default, Zabbix is available in the Debian 9 repository, but it might be outdated. So it is recommended to install most recent version from the official Zabbix repositories. You can download and add the latest version of the Zabbix repository with the following command: + +``` +wget http://repo.zabbix.com/zabbix/3.0/debian/pool/main/z/zabbix-release/zabbix-release_3.0-2+stretch_all.deb +``` + +Next, install the downloaded repository with the following command: + +``` +sudo dpkg -i zabbix-release_3.0-2+stretch_all.deb +``` + +Next, update the package cache and install Zabbix server with web front-end and Mysql support by running the following command: + +``` +sudo apt-get update -y +sudo apt-get install zabbix-server-mysql zabbix-frontend-php -y +``` + +You will also need to install the Zabbix agent to collect data about the Zabbix server status itself: + +``` +sudo apt-get install zabbix-agent -y +``` + +After installing Zabbix agent, start the Zabbix agent service and enable it to start on boot time by running the following command: + +``` +sudo systemctl start zabbix-agent +sudo systemctl enable zabbix-agent +``` + +#### Configuring Zabbix Database + +Zabbix uses MariaDB/MySQL as a database backend. So, you will need to create a MySQL database and User for zabbix installation: + +First, log into MySQL shell with the following command: + +``` +mysql -u root -p +``` + +Enter your root password, then create a database for Zabbix with the following command: + +``` +MariaDB [(none)]> CREATE DATABASE zabbixdb character set utf8 collate utf8_bin; +``` + +Next, create a user for Zabbix, assign a password and grant all privileges on Zabbix database with the following command: + +``` +MariaDB [(none)]> CREATE user zabbix identified by 'password'; +MariaDB [(none)]> GRANT ALL PRIVILEGES on zabbixdb.* to zabbixuser@localhost identified by 'password'; +``` + +Next, flush the privileges with the following command: + +``` +MariaDB [(none)]> FLUSH PRIVILEGES; +``` + +Finally, exit from the MySQL shell with the following command: + +``` +MariaDB [(none)]> exit; +``` + +Next, import initial schema and data to the newly created database with the following command: + +``` +cd /usr/share/doc/zabbix-server-mysql*/ +zcat create.sql.gz | mysql -u zabbix -p zabbixdb +``` + +#### Configuring Zabbix + +Zabbix creates its own configuration file at `/etc/zabbix/apache.conf`. Edit this file and update the Timezone and PHP setting as per your need: + +``` +sudo nano /etc/zabbix/apache.conf +``` + +Change the file as shown below: +``` + php_value max_execution_time 300 + php_value memory_limit 128M + php_value post_max_size 32M + php_value upload_max_filesize 8M + php_value max_input_time 300 + php_value always_populate_raw_post_data -1 + php_value date.timezone Asia/Kolkata + +``` + +Save the file when you are finished. + +Next, you will need to update the database details for Zabbix. You can do this by editing `/etc/zabbix/zabbix_server.conf` file: + +``` +sudo nano /etc/zabbix/zabbix_server.conf +``` + +Change the following lines: +``` +DBHost=localhost +DBName=zabbixdb +DBUser=zabbixuser +DBPassword=password + +``` + +Save and close the file when you are finished. Then restart all the services with the following command: + +``` +sudo systemctl restart apache2 +sudo systemctl restart mysql +sudo systemctl restart zabbix-server +``` + +#### Configuring Firewall + +Before proceeding, you will need to configure the UFW firewall to secure Zabbix server. + +First, make sure UFW is installed on your system. Otherewise, you can install it by running the following command: + +``` +sudo apt-get install ufw -y +``` + +Next, enable the UFW firewall: + +``` +sudo ufw enable +``` + +Next, allow port 10050, 10051 and 80 through UFW with the following command: + +``` +sudo ufw allow 10050/tcp +sudo ufw allow 10051/tcp +sudo ufw allow 80/tcp +``` + +Finally, reload the firewall to apply these changes with the following command: + +``` +sudo ufw reload +``` + +Once the UFW firewall is configured you can proceed to install the Zabbix server via web interface. + +#### Accessing Zabbix Web Installation Wizard + +Once everything is fine, it's time to access Zabbix web installation wizard. + +Open your web browser and navigate the URL , you will be redirected to the following page: + +[![Zabbix 3.0][2]][3] + +Click on the **Next step** button, you should see the following page: + +[![Zabbix Prerequisites][4]][5] + +Here, all the Zabbix pre-requisites are checked and verified, then click on the **Next step** button you should see the following page: + +[![Database Configuration][6]][7] + +Here, provide the Zabbix database name, database user and password then click on the **Next step** button, you should see the following page: + +[![Zabbix Server Details][8]][9] + +Here, specify the Zabbix server details and Port number then click on the **Next step** button, you should see the pre-installation summary of Zabbix Server in following page: + +[![Installation summary][10]][11] + +Next, click on the **Next step** button to start the Zabbix installation. Once the Zabbix installation is completed successfully, you should see the following page: + +[![Zabbix installed successfully][12]][13] + +Here, click on the **Finish** button, it will redirect to the Zabbix login page as shown below: + +[![Login to Zabbix][14]][15] + +Here, provide username as Admin and password as zabbix then click on the **Sign in** button. You should see the Zabbix server dashboard in the following image: + +[![Zabbix Dashboard][16]][17] + +Your Zabbix web installation is now finished. + +#### Install Zabbix Agent + +Now your Zabbix server is up and functioning. It's time to add Zabbix agent node to the Zabbix Server for Monitoring. + +First, log into Zabbix agent instance and add the Zabbix repository with the following command: + +``` +wget http://repo.zabbix.com/zabbix/3.0/debian/pool/main/z/zabbix-release/zabbix-release_3.0-2+stretch_all.deb +sudo dpkg -i zabbix-release_3.0-2+stretch_all.deb +sudo apt-get update -y +``` + +Once you have configured Zabbix repository on your system, install the Zabbix agent by just running the following command: + +``` +sudo apt-get install zabbix-agent -y +``` + +Once the Zabbix agent is installed, you will need to configure Zabbix agent to communicate with Zabbix server. You can do this by editing the Zabbix agent configuration file: + +``` +sudo nano /etc/zabbix/zabbix_agentd.conf +``` + +Change the file as shown below: +``` + #Zabbix Server IP Address / Hostname + + Server=192.168.0.103 + + #Zabbix Agent Hostname + + Hostname=zabbix-agent + + +``` + +Save and close the file when you are finished, then restart the Zabbix agent service and enable it to start on boot time with the following command: + +``` +sudo systemctl restart zabbix-agent +sudo systemctl enable zabbix-agent +``` + +#### Add Zabbix Agent Node to Zabbix Server + +Next, you will need to add the Zabbix agent node to the Zabbix server for monitoring. First, log in to the Zabbix server web interface. + +[![Zabbix UI][18]][19] + +Next, Click on **Configuration --> Hosts -> Create Host**, you should see the following page: + +[![Create Host in Zabbix][20]][21] + +Here, specify the Hostname, IP address and Group names of Zabbix agent. Then navigate to Templates tab, you should see the following page: + +[![specify the Hostname, IP address and Group name][22]][23] + +Here, search appropriate templates and click on **Add** button, you should see the following page: + +[![OS Template][24]][25] + +Finally, click on **Add** button again. You will see your new host with green labels indicating that everything is working fine. + +[![Hast successfully added to Zabbix][26]][27] + +If you have extra servers and network devices that you want to monitor, log into each host, install the Zabbix agent and add each host from the Zabbix web interface. + +#### Conclusion + +Congratulations! you have successfully installed the Zabbix server and Zabbix agent in Debian 9 server. You have also added Zabbix agent node to the Zabbix server for monitoring. You can now easily list the current issue and past history, get the latest data of hosts, list the current problems and also visualized the collected resource statistics such as CPU load, CPU utilization, Memory usage, etc via graphs. I hope you can now easily install and configure Zabbix on Debian 9 server and deploy it on production environment. Compared to other monitoring software, Zabbix allows you to build your own maps of different network segments while monitoring many hosts. You can also monitor Windows host using Zabbix windows agent. For more information, you can refer the [Zabbix Documentation Page][28]. Feel free to ask me if you have any questions. + + +-------------------------------------------------------------------------------- + +via: https://www.howtoforge.com/tutorial/install-zabbix-monitoring-server-and-agent-on-debian-9/ + +作者：[Hitesh Jethva][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.howtoforge.com +[1]:/cdn-cgi/l/email-protection +[2]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/Screenshot-of-zabbix-welcome-page.png +[3]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/big/Screenshot-of-zabbix-welcome-page.png +[4]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/Screenshot-of-zabbix-pre-requisite-check-page.png +[5]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/big/Screenshot-of-zabbix-pre-requisite-check-page.png +[6]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/Screenshot-of-zabbix-db-config-page.png +[7]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/big/Screenshot-of-zabbix-db-config-page.png +[8]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/Screenshot-of-zabbix-server-details.png +[9]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/big/Screenshot-of-zabbix-server-details.png +[10]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/Screenshot-of-pre-installation-summary.png +[11]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/big/Screenshot-of-pre-installation-summary.png +[12]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/Screenshot-of-zabbix-install-success.png +[13]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/big/Screenshot-of-zabbix-install-success.png +[14]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/Screenshot-of-zabbix-login-page.png +[15]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/big/Screenshot-of-zabbix-login-page.png +[16]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/Screenshot-of-zabbix-welcome-dashboard.png +[17]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/big/Screenshot-of-zabbix-welcome-dashboard.png +[18]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/Screenshot-of-zabbix-welcome-dashboard1.png +[19]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/big/Screenshot-of-zabbix-welcome-dashboard1.png +[20]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/Screenshot-of-zabbix-agent-host1.png +[21]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/big/Screenshot-of-zabbix-agent-host1.png +[22]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/Screenshot-of-zabbix-agent-add-templates.png +[23]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/big/Screenshot-of-zabbix-agent-add-templates.png +[24]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/Screenshot-of-zabbix-agent-select-templates.png +[25]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/big/Screenshot-of-zabbix-agent-select-templates.png +[26]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/Screenshot-of-zabbix-agent-dashboard.png +[27]:https://www.howtoforge.com/images/install_zabbix_monitoring_server_and_agent_on_debian_9/big/Screenshot-of-zabbix-agent-dashboard.png +[28]:https://www.zabbix.com/documentation/3.2/ diff --git a/sources/tech/20180129 Parsing HTML with Python.md b/sources/tech/20180129 Parsing HTML with Python.md new file mode 100644 index 0000000000..d0dbee596f --- /dev/null +++ b/sources/tech/20180129 Parsing HTML with Python.md @@ -0,0 +1,212 @@ +Parsing HTML with Python +====== + +![](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/bus_html_code.png?itok=VjUmGsnl) + +Image by : Jason Baker for Opensource.com. + +As a long-time member of the documentation team at Scribus, I keep up-to-date with the latest updates of the source so I can help make updates and additions to the documentation. When I recently did a "checkout" using Subversion on a computer I had just upgraded to Fedora 27, I was amazed at how long it took to download the documentation, which consists of HTML pages and associated images. I became concerned that the project's documentation seemed much larger than it should be and suspected that some of the content was "zombie" documentation--HTML files that aren't used anymore and images that have lost all references in the currently used HTML. + +I decided to create a project for myself to figure this out. One way to do this is to search for existing image files that aren't used. If I could scan through all the HTML files for image references, then compare that list to the actual image files, chances are I would see a mismatch. + +Here is a typical image tag: +``` +Edit examples +``` + +I'm interested in the part between the first set of quotation marks, after `src=`. After some searching for a solution, I found a Python module called [BeautifulSoup][1]. The tasty part of the script I wrote looks like this: +``` +soup = BeautifulSoup(all_text, 'html.parser') +match = soup.findAll("img") +if len(match) > 0: + for m in match: + imagelist.append(str(m)) +``` + +We can use this `findAll` method to pluck out the image tags. Here is a tiny piece of the output: +``` + + +GSview - Advanced Options PanelScribus External Tools Preferences +``` + +So far, so good. I thought that the next step might be to just carve this down, but when I tried some string methods in the script, it returned errors about this being tags and not strings. I saved the output to a file and went through the process of editing in [KWrite][2]. One nice thing about KWrite is that you can do a "find & replace" using regular expressions (regex), so I could replace `', all_text) +if len(match)>0: + for m in match: + imagelist.append(m) +``` + +And a tiny piece of its output looks like this: +``` +images/cmcanvas.png" title="Context Menu for the document canvas" alt="Context Menu for the document canvas" />
+``` + +I decided to home in on the `src=` piece. One way would be to wait for an occurrence of `s`, then see if the next character is `r`, the next `c`, and the next `=`. If so, bingo! Then what follows between two sets of double quotation marks is what I need. The problem with this is the structure it takes to hang onto these. One way of looking at a string of characters representing a line of HTML text would be: +``` +for c in all_text: +``` + +But the logic was just too messy to hang onto the previous `c`, and the one before that, the one before that, and the one before that. + +In the end, I decided to focus on the `=` and to use an indexing method whereby I could easily reference any prior or future character in the string. Here is the searching part: +``` + index = 3 + while index < linelength: + if (all_text[index] == '='): + if (all_text[index-3] == 's') and (all_text[index-2] == 'r') and (all_text[index-1] == 'c'): + imagefound(all_text, imagelist, index) + index += 1 + else: + index += 1 + else: + index += 1 +``` + +I start the search with the fourth character (indexing starts at 0), so I don't get an indexing error down below, and realistically, there will not be an equal sign before the fourth character of a line. The first test is to see if we find `=` as we're marching through the string, and if not, we march on. If we do see one, then we ask if the three previous characters were `s`, `r`, and `c`, in that order. If that happens, we call the function `imagefound`: +``` +def imagefound(all_text, imagelist, index): + end = 0 + index += 2 + newimage = '' + while end == 0: + if (all_text[index] != '"'): + newimage = newimage + all_text[index] + index += 1 + else: + newimage = newimage + '\n' + imagelist.append(newimage) + end = 1 + return +``` + +We're sending the function the current index, which represents the `=`. We know the next character will be `"`, so we jump two characters and begin adding characters to a holding string named `newimage`, until we reach the following `"`, at which point we're done. We add the string plus a `newline` character to our list `imagelist` and `return`, keeping in mind there may be more image tags in this remaining string of HTML, so we're right back in the middle of our searching loop. + +Here's what our output looks like now: +``` +images/text-frame-link.png +images/text-frame-unlink.png +images/gimpoptions1.png +images/gimpoptions3.png +images/gimpoptions2.png +images/fontpref3.png +images/font-subst.png +images/fontpref2.png +images/fontpref1.png +images/dtp-studio.png +``` + +Ahhh, much cleaner, and this only took a few seconds to run. I could have jumped seven more index spots to cut out the `images/` part, but I like having it there to make sure I haven't chopped off the first letter of the image filename, and this is so easy to edit out with KWrite--you don't even need regex. After doing that and saving the file, the next step was to run another script I wrote called `sortlist.py`: +``` +#!/usr/bin/env python +# -*- coding: utf-8 -*- +# sortlist.py + +import os + +imagelist = [] +for line in open('/tmp/imagelist_parse4.txt').xreadlines(): + imagelist.append(line) + +imagelist.sort() + +outfile = open('/tmp/imagelist_parse4_sorted.txt', 'w') +outfile.writelines(imagelist) +outfile.close() +``` + +This pulls in the file contents as a list, sorts it, then saves it as another file. After that I could just do the following: +``` +ls /home/gregp/development/Scribus15x/doc/en/images/*.png > '/tmp/actual_images.txt' +``` + +Then I need to run `sortlist.py` on that file too, since the method `ls` uses to sort is different from Python. I could have run a comparison script on these files, but I preferred to do this visually. In the end, I ended up with 42 images that had no HTML reference from the documentation. + +Here is my parsing script in its entirety: +``` +#!/usr/bin/env python +# -*- coding: utf-8 -*- +# parseimg4.py + +import os + +def imagefound(all_text, imagelist, index): + end = 0 + index += 2 + newimage = '' + while end == 0: + if (all_text[index] != '"'): + newimage = newimage + all_text[index] + index += 1 + else: + newimage = newimage + '\n' + imagelist.append(newimage) + end = 1 + return + +htmlnames = [] +imagelist = [] +tempstring = '' +filenames = os.listdir('/home/gregp/development/Scribus15x/doc/en/') +for name in filenames: + if name.endswith('.html'): + htmlnames.append(name) +#print htmlnames +for htmlfile in htmlnames: + all_text = open('/home/gregp/development/Scribus15x/doc/en/' + htmlfile).read() + linelength = len(all_text) + index = 3 + while index < linelength: + if (all_text[index] == '='): + if (all_text[index-3] == 's') and (all_text[index-2] == 'r') and +(all_text[index-1] == 'c'): + imagefound(all_text, imagelist, index) + index += 1 + else: + index += 1 + else: + index += 1 + +outfile = open('/tmp/imagelist_parse4.txt', 'w') +outfile.writelines(imagelist) +outfile.close() +imageno = len(imagelist) +print str(imageno) + " images were found and saved" +``` + +Its name, `parseimg4.py`, doesn't really reflect the number of scripts I wrote along the way, with both minor and major rewrites, plus discards and starting over. Notice that I've hardcoded these directory and filenames, but it would be easy enough to generalize, asking for user input for these pieces of information. Also as they were working scripts, I sent the output to `/tmp`, so they disappear once I reboot my system. + +This wasn't the end of the story, since the next question was: What about zombie HTML files? Any of these files that are not used might reference images not picked up by the previous method. We have a `menu.xml` file that serves as the table of contents for the online manual, but I also needed to consider that some files listed in the TOC might reference files not in the TOC, and yes, I did find some. + +I'll conclude by saying that this was a simpler task than this image search, and it was greatly helped by the processes I had already developed. + + +### About the author + + [![](https://opensource.com/sites/default/files/styles/profile_pictures/public/20150529_gregp.jpg?itok=nv02g6PV)][7] Greg Pittman - Greg is a retired neurologist in Louisville, Kentucky, with a long-standing interest in computers and programming, beginning with Fortran IV in the 1960s. When Linux and open source software came along, it kindled a commitment to learning more, and eventually contributing. He is a member of the Scribus Team.[More about me][8] + +-------------------------------------------------------------------------------- + +via: https://opensource.com/article/18/1/parsing-html-python + +作者：[Greg Pittman][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://opensource.com/users/greg-p +[1]:https://www.crummy.com/software/BeautifulSoup/ +[2]:https://www.kde.org/applications/utilities/kwrite/ +[7]:https://opensource.com/users/greg-p +[8]:https://opensource.com/users/greg-p diff --git a/sources/tech/20180129 Rapid, Secure Patching- Tools and Methods.md b/sources/tech/20180129 Rapid, Secure Patching- Tools and Methods.md new file mode 100644 index 0000000000..9ac7340c14 --- /dev/null +++ b/sources/tech/20180129 Rapid, Secure Patching- Tools and Methods.md @@ -0,0 +1,583 @@ +Rapid, Secure Patching: Tools and Methods +====== + +It was with some measure of disbelief that the computer science community greeted the recent [EternalBlue][1]-related exploits that have torn through massive numbers of vulnerable systems. The SMB exploits have kept coming (the most recent being [SMBLoris][2] presented at the last DEF CON, which impacts multiple SMB protocol versions, and for which Microsoft will issue no corrective patch. Attacks with these tools [incapacitated critical infrastructure][3] to the point that patients were even turned away from the British National Health Service. + +It is with considerable sadness that, during this SMB catastrophe, we also have come to understand that the famous Samba server presented an exploitable attack surface on the public internet in sufficient numbers for a worm to propagate successfully. I previously [have discussed SMB security][4] in Linux Journal, and I am no longer of the opinion that SMB server processes should run on Linux. + +In any case, systems administrators of all architectures must be able to down vulnerable network servers and patch them quickly. There is often a need for speed and competence when working with a large collection of Linux servers. Whether this is due to security situations or other concerns is immaterial—the hour of greatest need is not the time to begin to build administration tools. Note that in the event of an active intrusion by hostile parties, [forensic analysis][5] may be a legal requirement, and no steps should be taken on the compromised server without a careful plan and documentation. Especially in this new era of the black hats, computer professionals must step up their game and be able to secure vulnerable systems quickly. + +### Secure SSH Keypairs + +Tight control of a heterogeneous UNIX environment must begin with best-practice use of SSH authentication keys. I'm going to open this section with a simple requirement. SSH private keys must be one of three types: Ed25519, ECDSA using the E-521 curve or RSA keys of 3072 bits. Any key that does not meet those requirements should be retired (in particular, DSA keys must be removed from service immediately). + +The [Ed25519][6] key format is associated with Daniel J. Bernstein, who has such a preeminent reputation in modern cryptography that the field is becoming a DJB [monoculture][7]. The Ed25519 format is deigned for speed, security and size economy. If all of your SSH servers are recent enough to support Ed25519, then use it, and consider nothing else. + +[Guidance on creating Ed25519 keys][8] suggests 100 rounds for a work factor in the "-o" secure format. Raising the number of rounds raises the strength of the encrypted key against brute-force attacks (should a file copy of the private key fall into hostile hands), at the cost of more work and time in decrypting the key when ssh-add is executed. Although there always is [controversy and discussion][9] with security advances, I will repeat the guidance here and suggest that the best format for a newly created SSH key is this: + +``` + +ssh-keygen -a 100 -t ed25519 + +``` + +Your systems might be too old to support Ed25519—Oracle/CentOS/Red Hat 7 have this problem (the 7.1 release introduced support). If you cannot upgrade your old SSH clients and servers, your next best option is likely E-521, available in the ECDSA key format. + +The ECDSA curves came from the US government's National Institute of Standards (NIST). The best known and most implemented of all of the NIST curves are P-256, P-384 and E-521\. All three curves are approved for secret communications by a variety of government entities, but a number of cryptographers have [expressed growing suspicion][10] that the P-256 and P-384 curves are tainted. Well known cryptographer Bruce Schneier [has remarked][11]: "I no longer trust the constants. I believe the NSA has manipulated them through their relationships with industry." However, DJB [has expressed][12] limited praise of the E-521 curve: "To be fair I should mention that there's one standard NIST curve using a nice prime, namely 2521 – 1; but the sheer size of this prime makes it much slower than NIST P-256." All of the NIST curves have greater issues with "side channel" attacks than Ed25519—P-521 is certainly a step down, and many assert that none of the NIST curves are safe. In summary, there is a slight risk that a powerful adversary exists with an advantage over the P-256 and P-384 curves, so one is slightly inclined to avoid them. Note that even if your OpenSSH (source) release is capable of E-521, it may be [disabled by your vendor][13] due to patent concerns, so E-521 is not an option in this case. If you cannot use DJB's 2255 – 19 curve, this command will generate an E-521 key on a capable system: + +``` + +ssh-keygen -o -a 100 -b 521 -t ecdsa + +``` + +And, then there is the unfortunate circumstance with SSH servers that support neither ECDSA nor Ed25519\. In this case, you must fall back to RSA with much larger key sizes. An absolute minimum is the modern default of 2048 bits, but 3072 is a wiser choice: + +``` + +ssh-keygen -o -a 100 -b 3072 -t rsa + +``` + +Then in the most lamentable case of all, when you must use old SSH clients that are not able to work with private keys created with the -o option, you can remove the password on id_rsa and create a naked key, then use OpenSSL to encrypt it with AES256 in the PKCS#8 format, as [first documented by Martin Kleppmann][14]. Provide a blank new password for the keygen utility below, then supply a new password when OpenSSL reprocesses the key: + +``` + +$ cd ~/.ssh + +$ cp id_rsa id_rsa-orig + +$ ssh-keygen -p -t rsa +Enter file in which the key is (/home/cfisher/.ssh/id_rsa): +Enter old passphrase: +Key has comment 'cfisher@localhost.localdomain' +Enter new passphrase (empty for no passphrase): +Enter same passphrase again: +Your identification has been saved with the new passphrase. + +$ openssl pkcs8 -topk8 -v2 aes256 -in id_rsa -out id_rsa-strong +Enter Encryption Password: +Verifying - Enter Encryption Password: + +mv id_rsa-strong id_rsa +chmod 600 id_rsa + +``` + +After creating all of these keys on a newer system, you can compare the file sizes: + +``` + +$ ll .ssh +total 32 +-rw-------. 1 cfisher cfisher 801 Aug 10 21:30 id_ecdsa +-rw-r--r--. 1 cfisher cfisher 283 Aug 10 21:30 id_ecdsa.pub +-rw-------. 1 cfisher cfisher 464 Aug 10 20:49 id_ed25519 +-rw-r--r--. 1 cfisher cfisher 111 Aug 10 20:49 id_ed25519.pub +-rw-------. 1 cfisher cfisher 2638 Aug 10 21:45 id_rsa +-rw-------. 1 cfisher cfisher 2675 Aug 10 21:42 id_rsa-orig +-rw-r--r--. 1 cfisher cfisher 583 Aug 10 21:42 id_rsa.pub + +``` + +Although they are relatively enormous, all versions of OpenSSH that I have used have been compatible with the RSA private key in PKCS#8 format. The Ed25519 public key is now small enough to fit in 80 columns without word wrap, and it is as convenient as it is efficient and secure. + +Note that PuTTY may have problems using various versions of these keys, and you may need to remove passwords for a successful import into the PuTTY agent. + +These keys represent the most secure formats available for various OpenSSH revisions. They really aren't intended for PuTTY or other general interactive activity. Although one hopes that all users create strong keys for all situations, these are enterprise-class keys for major systems activities. It might be wise, however, to regenerate your system host keys to conform to these guidelines. + +These key formats may soon change. Quantum computers are causing increasing concern for their ability to run [Shor's Algorithm][15], which can be used to find prime factors to break these keys in reasonable time. The largest commercially available quantum computer, the [D-Wave 2000Q][16], effectively [presents under 200 qubits][17] for this activity, which is not (yet) powerful enough for a successful attack. NIST [announced a competition][18] for a new quantum-resistant public key system with a deadline of November 2017 In response, a team including DJB has released source code for [NTRU Prime][19]. It does appear that we will likely see a post-quantum public key format for OpenSSH (and potentially TLS 1.3) released within the next two years, so take steps to ease migration now. + +Also, it's important for SSH servers to restrict their allowed ciphers, MACs and key exchange lest strong keys be wasted on broken crypto (3DES, MD5 and arcfour should be long-disabled). My [previous guidance][20] on the subject involved the following (three) lines in the SSH client and server configuration (note that formatting in the sshd_config file requires all parameters on the same line with no spaces in the options; line breaks have been added here for clarity): + +``` + +Ciphers chacha20-poly1305@openssh.com, + aes256-gcm@openssh.com, + aes128-gcm@openssh.com, + aes256-ctr, + aes192-ctr, + aes128-ctr + +MACs hmac-sha2-512-etm@openssh.com, + hmac-sha2-256-etm@openssh.com, + hmac-ripemd160-etm@openssh.com, + umac-128-etm@openssh.com, + hmac-sha2-512, + hmac-sha2-256, + hmac-ripemd160, + umac-128@openssh.com + +KexAlgorithms curve25519-sha256@libssh.org, + diffie-hellman-group-exchange-sha256 + +``` + +Since the previous publication, RIPEMD160 is likely no longer safe and should be removed. Older systems, however, may support only SHA1, MD5 and RIPEMD160\. Certainly remove MD5, but users of PuTTY likely will want to retain SHA1 when newer MACs are not an option. Older servers can present a challenge in finding a reasonable Cipher/MAC/KEX when working with modern systems. + +At this point, you should have strong keys for secure clients and servers. Now let's put them to use. + +### Scripting the SSH Agent + +Modern OpenSSH distributions contain the ssh-copy-id shell script for easy key distribution. Below is an example of installing a specific, named key in a remote account: + +``` + +$ ssh-copy-id -i ~/.ssh/some_key.pub person@yourserver.com +ssh-copy-id: INFO: Source of key(s) to be installed: + "/home/cfisher/.ssh/some_key.pub" +ssh-copy-id: INFO: attempting to log in with the new key(s), + to filter out any that are already installed +ssh-copy-id: INFO: 1 key(s) remain to be installed -- + if you are prompted now it is to install the new keys +person@yourserver.com's password: + +Number of key(s) added: 1 + +Now try logging into the machine, with: + "ssh 'person@yourserver.com'" +and check to make sure that only the key(s) you wanted were added. + +``` + +If you don't have the ssh-copy-id script, you can install a key manually with the following command: + +``` + +$ ssh person@yourserver.com 'cat >> ~/.ssh/authorized_keys' < \ + ~/.ssh/some_key.pub + +``` + +If you have SELinux enabled, you might have to mark a newly created authorized_keys file with a security type; otherwise, the sshd server dæmon will be prevented from reading the key (the syslog may report this issue): + +``` + +$ ssh person@yourserver.com 'chcon -t ssh_home_t + ↪~/.ssh/authorized_keys' + +``` + +Once your key is installed, test it in a one-time use with the -i option (note that you are entering a local key password, not a remote authentication password): + +``` + +$ ssh -i ~/.ssh/some_key person@yourserver.com +Enter passphrase for key '/home/v-fishecj/.ssh/some_key': +Last login: Wed Aug 16 12:20:26 2017 from 10.58.17.14 +yourserver $ + +``` + +General, interactive users likely will cache their keys with an agent. In the example below, the same password is used on all three types of keys that were created in the previous section: + +``` + +$ eval $(ssh-agent) +Agent pid 4394 + +$ ssh-add +Enter passphrase for /home/cfisher/.ssh/id_rsa: +Identity added: ~cfisher/.ssh/id_rsa (~cfisher/.ssh/id_rsa) +Identity added: ~cfisher/.ssh/id_ecdsa (cfisher@init.com) +Identity added: ~cfisher/.ssh/id_ed25519 (cfisher@init.com) + +``` + +The first command above launches a user agent process, which injects environment variables (named SSH_AGENT_SOCK and SSH_AGENT_PID) into the parent shell (via eval). The shell becomes aware of the agent and passes these variables to the programs that it runs from that point forward. + +When launched, the ssh-agent has no credentials and is unable to facilitate SSH activity. It must be primed by adding keys, which is done with ssh-add. When called with no arguments, all of the default keys will be read. It also can be called to add a custom key: + +``` + +$ ssh-add ~/.ssh/some_key +Enter passphrase for /home/cfisher/.ssh/some_key: +Identity added: /home/cfisher/.ssh/some_key + ↪(cfisher@localhost.localdomain) + +``` + +Note that the agent will not retain the password on the key. ssh-add uses any and all passwords that you enter while it runs to decrypt keys that it finds, but the passwords are cleared from memory when ssh-add terminates (they are not sent to ssh-agent). This allows you to upgrade to new key formats with minimal inconvenience, while keeping the keys reasonably safe. + +The current cached keys can be listed with ssh-add -l (from, which you can deduce that "some_key" is an Ed25519): + +``` + +$ ssh-add -l +3072 SHA256:cpVFMZ17oO5n/Jfpv2qDNSNcV6ffOVYPV8vVaSm3DDo + /home/cfisher/.ssh/id_rsa (RSA) +521 SHA256:1L9/CglR7cstr54a600zDrBbcxMj/a3RtcsdjuU61VU + cfisher@localhost.localdomain (ECDSA) +256 SHA256:Vd21LEM4lixY4rIg3/Ht/w8aoMT+tRzFUR0R32SZIJc + cfisher@localhost.localdomain (ED25519) +256 SHA256:YsKtUA9Mglas7kqC4RmzO6jd2jxVNCc1OE+usR4bkcc + cfisher@localhost.localdomain (ED25519) + +``` + +While a "primed" agent is running, the SSH clients may use (trusting) remote servers fluidly, with no further prompts for credentials: + +``` + +$ sftp person@yourserver.com +Connected to yourserver.com. +sftp> quit + +$ scp /etc/passwd person@yourserver.com:/tmp +passwd 100% 2269 65.8KB/s 00:00 + +$ ssh person@yourserver.com + (motd for yourserver.com) +$ ls -l /tmp/passwd +-rw-r--r-- 1 root wheel 2269 Aug 16 09:07 /tmp/passwd +$ rm /tmp/passwd +$ exit +Connection to yourserver.com closed. + +``` + +The OpenSSH agent can be locked, preventing any further use of the credentials that it holds (this might be appropriate when suspending a laptop): + +``` + +$ ssh-add -x +Enter lock password: +Again: +Agent locked. + +$ ssh yourserver.com +Enter passphrase for key '/home/cfisher/.ssh/id_rsa': ^C + +``` + +It will provide credentials again when it is unlocked: + +``` + +$ ssh-add -X +Enter lock password: +Agent unlocked. + +``` + +You also can set ssh-agent to expire keys after a time limit with the -t option, which may be useful for long-lived agents that must clear keys after a set daily shift. + +General shell users may cache many types of keys with a number of differing agent implementations. In addition to the standard OpenSSH agent, users may rely upon PuTTY's pageant.exe, GNOME keyring or KDE Kwallet, among others (the use of the PUTTY agent could likely fill an article on its own). + +However, the goal here is to create "enterprise" keys for critical server controls. You likely do not want long-lived agents in order to limit the risk of exposure. When scripting with "enterprise" keys, you will run an agent only for the duration of the activity, then kill it at completion. + +There are special options for accessing the root account with OpenSSH—the PermitRootLogin parameter can be added to the sshd_config file (usually found in /etc/ssh). It can be set to a simple yes or no, forced-commands-only, which will allow only explicitly-authorized programs to be executed, or the equivalent options prohibit-password or without-password, both of which will allow access to the keys generated here. + +Many hold that root should not be allowed any access. [Michael W. Lucas][21] addresses the question in SSH Mastery: + +> Sometimes, it seems that you need to allow users to SSH in to the system as root. This is a colossally bad idea in almost all environments. When users must log in as a regular user and then change to root, the system logs record the user account, providing accountability. Logging in as root destroys that audit trail....It is possible to override the security precautions and make sshd permit a login directly as root. It's such a bad idea that I'd consider myself guilty of malpractice if I told you how to do it. Logging in as root via SSH almost always means you're solving the wrong problem. Step back and look for other ways to accomplish your goal. + +When root action is required quickly on more than a few servers, the above advice can impose painful delays. Lucas' direct criticism can be addressed by allowing only a limited set of "bastion" servers to issue root commands over SSH. Administrators should be forced to log in to the bastions with unprivileged accounts to establish accountability. + +However, one problem with remotely "changing to root" is the [statistical use of the Viterbi algorithm][22] Short passwords, the su - command and remote SSH calls that use passwords to establish a trinary network configuration are all uniquely vulnerable to timing attacks on a user's keyboard movement. Those with the highest security concerns will need to compensate. + +For the rest of us, I recommend that PermitRootLogin without-password be set for all target machines. + +Finally, you can easily terminate ssh-agent interactively with the -k option: + +``` + +$ eval $(ssh-agent -k) +Agent pid 4394 killed + +``` + +With these tools and the intended use of them in mind, here is a complete script that runs an agent for the duration of a set of commands over a list of servers for a common named user (which is not necessarily root): + +``` + +# cat artano + +#!/bin/sh + +if [[ $# -lt 1 ]]; then echo "$0 - requires commands"; exit; fi + +R="-R5865:127.0.0.1:5865" # set to "-2" if you don't want + ↪port forwarding + +eval $(ssh-agent -s) + +function cleanup { eval $(ssh-agent -s -k); } + +trap cleanup EXIT + +function remsh { typeset F="/tmp/${1}" h="$1" p="$2"; + ↪shift 2; echo "#$h" + if [[ "$ARTANO" == "PARALLEL" ]] + then ssh "$R" -p "$p" "$h" "$@" < /dev/null >>"${F}.out" + ↪2>>"${F}.err" & + else ssh "$R" -p "$p" "$h" "$@" + fi } # HOST PORT CMD + +if ssh-add ~/.ssh/master_key +then remsh yourserver.com 22 "$@" + remsh container.yourserver.com 2200 "$@" + remsh anotherserver.com 22 "$@" + # Add more hosts here. +else echo Bad password - killing agent. Try again. +fi + +wait + +####################################################################### +# Examples: # Artano is an epithet of a famous mythical being +# artano 'mount /patchdir' # you will need an fstab entry for this +# artano 'umount /patchdir' +# artano 'yum update -y 2>&1' +# artano 'rpm -Fvh /patchdir/\*.rpm' +####################################################################### + +``` + +This script runs all commands in sequence on a collection of hosts by default. If the ARTANO environment variable is set to PARALLEL, it instead will launch them all as background processes simultaneously and append their STDOUT and STDERR to files in /tmp (this should be no problem when dealing with fewer than a hundred hosts on a reasonable server). The PARALLEL setting is useful not only for pushing changes faster, but also for collecting audit results. + +Below is an example using the yum update agent. The source of this particular invocation had to traverse a firewall and relied on a proxy setting in the /etc/yum.conf file, which used the port-forwarding option (-R) above: + +``` + +# ./artano 'yum update -y 2>&1' +Agent pid 3458 +Enter passphrase for /root/.ssh/master_key: +Identity added: /root/.ssh/master_key (/root/.ssh/master_key) +#yourserver.com +Loaded plugins: langpacks, ulninfo +No packages marked for update +#container.yourserver.com +Loaded plugins: langpacks, ulninfo +No packages marked for update +#anotherserver.com +Loaded plugins: langpacks, ulninfo +No packages marked for update +Agent pid 3458 killed + +``` + +The script can be used for more general maintenance functions. Linux installations running the XFS filesystem should "defrag" periodically. Although this normally would be done with cron, it can be a centralized activity, stored in a separate script that includes only on the appropriate hosts: + +``` + +&1' +Agent pid 7897 +Enter passphrase for /root/.ssh/master_key: +Identity added: /root/.ssh/master_key (/root/.ssh/master_key) +#yourserver.com +#container.yourserver.com +#anotherserver.com +Agent pid 7897 killed + +``` + +An easy method to collect the contents of all authorized_keys files for all users is the following artano script (this is useful for system auditing and is coded to remove file duplicates): + +``` + +artano 'awk -F: {print\$6\"/.ssh/authorized_keys\"} \ + /etc/passwd | sort -u | xargs grep . 2> /dev/null' + +``` + +It is convenient to configure NFS mounts for file distribution to remote nodes. Bear in mind that NFS is clear text, and sensitive content should not traverse untrusted networks while unencrypted. After configuring an NFS server on host 1.2.3.4, I add the following line to the /etc/fstab file on all the clients and create the /patchdir directory. After the change, the artano script can be used to mass-mount the directory if the network configuration is correct: + +``` + +# tail -1 /etc/fstab +1.2.3.4:/var/cache/yum/x86_64/7Server/ol7_latest/packages + ↪/patchdir nfs4 noauto,proto=tcp,port=2049 0 0 + +``` + +Assuming that the NFS server is mounted, RPMs can be upgraded from images stored upon it (note that Oracle Spacewalk or Red Hat Satellite might be a more capable patch method): + +``` + +# ./artano 'rpm -Fvh /patchdir/\*.rpm' +Agent pid 3203 +Enter passphrase for /root/.ssh/master_key: +Identity added: /root/.ssh/master_key (/root/.ssh/master_key) +#yourserver.com +Preparing... ######################## +Updating / installing... +xmlsec1-1.2.20-7.el7_4 ######################## +xmlsec1-openssl-1.2.20-7.el7_4 ######################## +Cleaning up / removing... +xmlsec1-openssl-1.2.20-5.el7 ######################## +xmlsec1-1.2.20-5.el7 ######################## +#container.yourserver.com +Preparing... ######################## +Updating / installing... +xmlsec1-1.2.20-7.el7_4 ######################## +xmlsec1-openssl-1.2.20-7.el7_4 ######################## +Cleaning up / removing... +xmlsec1-openssl-1.2.20-5.el7 ######################## +xmlsec1-1.2.20-5.el7 ######################## +#anotherserver.com +Preparing... ######################## +Updating / installing... +xmlsec1-1.2.20-7.el7_4 ######################## +xmlsec1-openssl-1.2.20-7.el7_4 ######################## +Cleaning up / removing... +xmlsec1-openssl-1.2.20-5.el7 ######################## +xmlsec1-1.2.20-5.el7 ######################## +Agent pid 3203 killed + +``` + +I am assuming that my audience is already experienced with package tools for their preferred platforms. However, to avoid criticism that I've included little actual discussion of patch tools, the following is a quick reference of RPM manipulation commands, which is the most common package format on enterprise systems: + +* rpm -Uvh package.i686.rpm — install or upgrade a package file. + +* rpm -Fvh package.i686.rpm — upgrade a package file, if an older version is installed. + +* rpm -e package — remove an installed package. + +* rpm -q package — list installed package name and version. + +* rpm -q --changelog package — print full changelog for installed package (including CVEs). + +* rpm -qa — list all installed packages on the system. + +* rpm -ql package — list all files in an installed package. + +* rpm -qpl package.i686.rpm — list files included in a package file. + +* rpm -qi package — print detailed description of installed package. + +* rpm -qpi package — print detailed description of package file. + +* rpm -qf /path/to/file — list package that installed a particular file. + +* rpm --rebuild package.src.rpm — unpack and build a binary RPM under /usr/src/redhat. + +* rpm2cpio package.src.rpm | cpio -icduv — unpack all package files in the current directory. + +Another important consideration for scripting the SSH agent is limiting the capability of an authorized key. There is a [specific syntax][23] for such limitations Of particular interest is the from="" clause, which will restrict logins on a key to a limited set of hosts. It is likely wise to declare a set of "bastion" servers that will record non-root logins that escalate into controlled users who make use of the enterprise keys. + +An example entry might be the following (note that I've broken this line, which is not allowed syntax but done here for clarity): + +``` + +from="*.c2.security.yourcompany.com,4.3.2.1" ssh-ed25519 + ↪AAAAC3NzaC1lZDI1NTE5AAAAIJSSazJz6A5x6fTcDFIji1X+ +↪svesidBonQvuDKsxo1Mx + +``` + +A number of other useful restraints can be placed upon authorized_keys entries. The command="" will restrict a key to a single program or script and will set the SSH_ORIGINAL_COMMAND environment variable to the client's attempted call—scripts can set alarms if the variable does not contain approved contents. The restrict option also is worth consideration, as it disables a large set of SSH features that can be both superfluous and dangerous. + +Although it is possible to set server identification keys in the known_hosts file to a @revoked status, this cannot be done with the contents of authorized_keys. However, a system-wide file for forbidden keys can be set in the sshd_config with RevokedKeys. This file overrides any user's authorized_keys. If set, this file must exist and be readable by the sshd server process; otherwise, no keys will be accepted at all (so use care if you configure it on a machine where there are obstacles to physical access). When this option is set, use the artano script to append forbidden keys to the file quickly when they should be disallowed from the network. A clear and convenient file location would be /etc/ssh/revoked_keys. + +It is also possible to establish a local Certificate Authority (CA) for OpenSSH that will [allow keys to be registered with an authority][24] with expiration dates. These CAs can [become quite elaborate][25] in their control over an enterprise. Although the maintenance of an SSH CA is beyond the scope of this article, keys issued by such CAs should be strong by adhering to the requirements for Ed25519/E-521/RSA-3072. + +### pdsh + +Many higher-level tools for the control of collections of servers exist that are much more sophisticated than the script I've presented here. The most famous is likely [Puppet][26], which is a Ruby-based configuration management system for enterprise control. Puppet has a somewhat short list of supported operating systems. If you are looking for low-level control of Android, Tomato, Linux smart terminals or other "exotic" POSIX, Puppet is likely not the appropriate tool. Another popular Ruby-based tool is [Chef][27], which is known for its complexity. Both Puppet and Chef require Ruby installations on both clients and servers, and they both will catalog any SSH keys that they find, so this key strength discussion is completely applicable to them. + +There are several similar Python-based tools, including [Ansible][28], [Bcfg2][29], [Fabric][30] and [SaltStack][31]. Of these, only Ansible can run "agentless" over a bare SSH connection; the rest will require agents that run on target nodes (and this likely includes a Python runtime). + +Another popular configuration management tool is [CFEngine][32], which is coded in C and claims very high performance. [Rudder][33] has evolved from portions of CFEngine and has a small but growing user community. + +Most of the previously mentioned packages are licensed commercially and some are closed source. + +The closest low-level tool to the activities presented here is the Parallel Distributed Shell (pdsh), which can be found in the [EPEL repository][34]. The pdsh utilities grew out of an IBM-developed package named dsh designed for the control of compute clusters. Install the following packages from the repository to use pdsh: + +``` + +# rpm -qa | grep pdsh +pdsh-2.31-1.el7.x86_64 +pdsh-rcmd-ssh-2.31-1.el7.x86_64 + +``` + +An SSH agent must be running while using pdsh with encrypted keys, and there is no obvious way to control the destination port on a per-host basis as was done with the artano script. Below is an example using pdsh to run a command on three remote servers: + +``` + +# eval $(ssh-agent) +Agent pid 17106 + +# ssh-add ~/.ssh/master_key +Enter passphrase for /root/.ssh/master_key: +Identity added: /root/.ssh/master_key (/root/.ssh/master_key) + +# pdsh -w hosta.com,hostb.com,hostc.com uptime +hosta: 13:24:49 up 13 days, 2:13, 6 users, load avg: 0.00, 0.01, 0.05 +hostb: 13:24:49 up 7 days, 21:15, 5 users, load avg: 0.05, 0.04, 0.05 +hostc: 13:24:49 up 9 days, 3:26, 3 users, load avg: 0.00, 0.01, 0.05 + +# eval $(ssh-agent -k) +Agent pid 17106 killed + +``` + +The -w option above defines a host list. It allows for limited arithmetic expansion and can take the list of hosts from standard input if the argument is a dash (-). The PDSH_SSH_ARGS and PDSH_SSH_ARGS_APPEND environment variables can be used to pass custom options to the SSH call. By default, 32 sessions will be launched in parallel, and this "fanout/sliding window" will be maintained by launching new host invocations as existing connections complete and close. You can adjust the size of the "fanout" either with the -f option or the FANOUT environment variable. It's interesting to note that there are two file copy commands: pdcp and rpdcp, which are analogous to scp. + +Even a low-level utility like pdsh lacks some flexibility that is available by scripting OpenSSH, so prepare to feel even greater constraints as more complicated tools are introduced. + +### Conclusion + +Modern Linux touches us in many ways on diverse platforms. When the security of these systems is not maintained, others also may touch our platforms and turn them against us. It is important to realize the maintenance obligations when you add any Linux platform to your environment. This obligation always exists, and there are consequences when it is not met. + +In a security emergency, simple, open and well understood tools are best. As tool complexity increases, platform portability certainly declines, the number of competent administrators also falls, and this likely impacts speed of execution. This may be a reasonable trade in many other aspects, but in a security context, it demands a much more careful analysis. Emergency measures must be documented and understood by a wider audience than is required for normal operations, and using more general tools facilitates that discussion. + +I hope the techniques presented here will prompt that discussion for those who have not yet faced it. + +### Disclaimer + +The views and opinions expressed in this article are those of the author and do not necessarily reflect those of Linux Journal. + +### Note: + +An exploit [compromising Ed25519][35] was recently demonstrated that relies upon custom hardware changes to derive a usable portion of a secret key. Physical hardware security is a basic requirement for encryption integrity, and many common algorithms are further vulnerable to cache timing or other side channel attacks that can be performed by the unprivileged processes of other users. Use caution when granting access to systems that process sensitive data. + + +-------------------------------------------------------------------------------- + +via: http://www.linuxjournal.com/content/rapid-secure-patching-tools-and-methods + +作者：[Charles Fisher][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://www.linuxjournal.com/users/charles-fisher +[1]:https://en.wikipedia.org/wiki/EternalBlue +[2]:http://securityaffairs.co/wordpress/61530/hacking/smbloris-smbv1-flaw.html +[3]:http://www.telegraph.co.uk/news/2017/05/13/nhs-cyber-attack-everything-need-know-biggest-ransomware-offensive +[4]:http://www.linuxjournal.com/content/smbclient-security-windows-printing-and-file-transfer +[5]:https://staff.washington.edu/dittrich/misc/forensics +[6]:https://ed25519.cr.yp.to +[7]:http://www.metzdowd.com/pipermail/cryptography/2016-March/028824.html +[8]:https://blog.g3rt.nl/upgrade-your-ssh-keys.html +[9]:https://news.ycombinator.com/item?id=12563899 +[10]:http://safecurves.cr.yp.to/rigid.html +[11]:https://en.wikipedia.org/wiki/Curve25519 +[12]:http://blog.cr.yp.to/20140323-ecdsa.html +[13]:https://lwn.net/Articles/573166 +[14]:http://martin.kleppmann.com/2013/05/24/improving-security-of-ssh-private-keys.html +[15]:https://en.wikipedia.org/wiki/Shor's_algorithm +[16]:https://www.dwavesys.com/d-wave-two-system +[17]:https://crypto.stackexchange.com/questions/40893/can-or-can-not-d-waves-quantum-computers-use-shors-and-grovers-algorithm-to-f +[18]:https://yro.slashdot.org/story/16/12/21/2334220/nist-asks-public-for-help-with-quantum-proof-cryptography +[19]:https://ntruprime.cr.yp.to/index.html +[20]:http://www.linuxjournal.com/content/cipher-security-how-harden-tls-and-ssh +[21]:https://www.michaelwlucas.com/tools/ssh +[22]:https://people.eecs.berkeley.edu/~dawnsong/papers/ssh-timing.pdf +[23]:https://man.openbsd.org/sshd#AUTHORIZED_KEYS_FILE_FORMAT +[24]:https://ef.gy/hardening-ssh +[25]:https://code.facebook.com/posts/365787980419535/scalable-and-secure-access-with-ssh +[26]:https://puppet.com +[27]:https://www.chef.io +[28]:https://www.ansible.com +[29]:http://bcfg2.org +[30]:http://www.fabfile.org +[31]:https://saltstack.com +[32]:https://cfengine.com +[33]:http://www.rudder-project.org/site +[34]:https://fedoraproject.org/wiki/EPEL +[35]:https://research.kudelskisecurity.com/2017/10/04/defeating-eddsa-with-faults diff --git a/sources/tech/20180129 WebSphere MQ programming in Python with Zato.md b/sources/tech/20180129 WebSphere MQ programming in Python with Zato.md new file mode 100644 index 0000000000..3e53d67201 --- /dev/null +++ b/sources/tech/20180129 WebSphere MQ programming in Python with Zato.md @@ -0,0 +1,262 @@ +WebSphere MQ programming in Python with Zato +====== +[WebSphere MQ][1] is a messaging middleware product by IBM - a message queue server - and this post shows how to integrate with MQ from Python and [Zato][2]. + +The article will go through a short process that will let you: + + * Send messages to queues in 1 line of Python code + * Receive messages from queues without coding + * Seamlessly integrate with Java JMS applications - frequently found in WebSphere MQ environments + * Push MQ messages from [Django][3] or [Flask][4] + + + +### Prerequisites + + * [Zato][2] 3.0+ (e.g. from [source code][5]) + * WebSphere MQ 6.0+ + + + +### Preliminary steps + + * Obtain connection details and credentials to the queue manager that you will be connecting to: + + * host, e.g. 10.151.13.11 + * port, e.g. 1414 + * channel name, e.g. DEV.SVRCONN.1 + * queue manager name (optional) + * username (optional) + * password (optional) + * Install [Zato][6] + + * On the same system that Zato is on, install a [WebSphere MQ Client][7] \- this is an umbrella term for a set of development headers and libraries that let applications connect to remote queue managers + + * Install [PyMQI][8] \- an additional dependency implementing the low-level proprietary MQ protocol. Note that you need to use the pip command that Zato ships with: + + + +``` +# Assuming Zato is in /opt/zato/current +zato$ cd /opt/zato/current/bin +zato$ ./pip install pymqi + +``` + + * That is it - everything is installed and the rest is a matter of configuration + + + +### Understanding definitions, outgoing connections and channels + +Everything in Zato revolves around re-usability and hot-reconfiguration - each individual piece of configuration can be changed on the fly, while servers are running, without restarts. + +Note that the concepts below are presented in the context of WebSphere MQ but they apply to other connection types in Zato too. + + * **Definitions** \- encapsulate common details that apply to other parts of configuration, e.g. a connection definition may contain remote host and port + * **Outgoing connections** \- objects through which data is sent to remote resources, such as MQ queues + * **Channels** \- objects through which data can be received, for instance, from MQ queues + + + +It is usually most convenient to configure environments during development using [web-admin GUI][9] but afterwards this can be automated with [enmasse][10], [API][11] or [command-line interface][12]. + +Once configuration is defined, it can be used from Zato services which in turn represent APIs that Zato clients invoke. Then, external applications, such as a Django or Flask, will connect using HTTP to a Zato service which will on their behalf send messages to MQ queues. + +Let's use web-admin to define all the Zato objects required for MQ integrations. (Hint: web-admin by default runs on ) + +### Definition + + * Go to Connections -> Definitions -> WebSphere MQ + * Fill out the form and click OK + * Observe the 'Use JMS' checkbox - more about it later on + + + +![Screenshots][13] + + * Note that a password is by default set to an unusable one (a random UUID4) so once a definition is created, click on Change password to set it to a required one + + + +![Screenshots][14] + + * Click Ping to confirm that connections to the remote queue manager can be established + + + +![Screenshots][15] + +### Outgoing connection + + * Go to Connections -> Outgoing -> WebSphere MQ + * Fill out the form - the connection's name is just a descriptive label + * Note that you do not specify a queue name here - this is because a single connection can be used with as many queues as needed + + + +![Screenshots][16] + + * You can now send a test MQ message directly from web-admin after click Send a message + + + +![Screenshots][17] + +![Screenshots][18] + +### API services + + * Having carried out the steps above, you can now send messages to queue managers from web-admin, which is a great way to confirm MQ-level connectivity but the crucial point of using Zato is to offer API services to client applications so let's create two services now, one for sending messages to MQ and one that will receive them. + + + +``` +# -*- coding: utf-8 -*- + +from __future__ import absolute_import, division, print_function, unicode_literals + +# Zato +from zato.server.service import Service + +class MQSender(Service): + """ Sends all incoming messages as they are straight to a remote MQ queue. + """ + def handle(self): + + # This single line suffices + self.out.wmq.send(self.request.raw_request, 'customer.updates', 'CUSTOMER.1') +``` + + * In practice, a service such as the one above could perform transformation on incoming messages or read its destination queue names from configuration files but it serves to illustrate the point that literally 1 line of code is needed to send MQ messages + + * Let's create a channel service now - one that will act as a callback invoked for each message consumed off a queue: + + + +``` +# -*- coding: utf-8 -*- + +from __future__ import absolute_import, division, print_function, unicode_literals + +# Zato +from zato.server.service import Service + +class MQReceiver(Service): + """ Invoked for each message taken from a remote MQ queue + """ + def handle(self): + self.logger.info(self.request.raw_request) +``` + +But wait - if this is the service that is a callback one then how does it know which queue to get messages from? + +That is the key point of Zato architecture - services do not need to know it and unless you really need it, they won't ever access this information. + +Such configuration details are configured externally (for instance, in web-admin) and a service is just a black box that receives some input, operates on it and produces output. + +In fact, the very same service could be mounted not only on WebSphere MQ ones but also on REST or AMQP channels. + +Without further ado, let's create a channel in that case, but since this is an article about MQ, only this connection type will be shown even if the same principle applies to other channel types. + +### Channel + + * Go to Connections -> Channels -> WebSphere MQ + * Fill out the form and click OK + * Data format may be JSON, XML or blank if no automatic de-serialization is required + + + +![Screenshots][19] + +After clicking OK a lightweight background task will start to listen for messages pertaining to a given queue and upon receiving any, the service configured for channel will be invoked. + +You can start as many channels as there are queues to consume messages from, that is, each channel = one input queue and each channel may declare a different service. + +### JMS Java integration + +In many MQ environments the majority of applications will be based on Java JMS and Zato implements the underlying wire-level MQ JMS protocol to let services integrate with such systems without any effort from a Python programmer's perspective. + +When creating connection definitions, merely check Use JMS and everything will be taken care of under the hood - all the necessary wire headers will be added or removed when it needs to be done. + +![Screenshots][20] + +### No restarts required + +It's worth to emphasize again that at no point are server restarts required to reconfigure connection details. + +No matter how many definitions, outgoing connections, channels there are, and no matter of what kind they are (MQ or not), changing any of them will only update that very one across the whole cluster of Zato servers without interrupting other API services running concurrently. + +### Configuration wrap-up + + * MQ connection definitions are re-used across outgoing connections and channels + * Outgoing connections are used by services to send messages to queues + * Data from queues is read through channels that invoke user-defined services + * Everything is reconfigurable on the fly + + + +Let's now check how to add a REST channel for the MQSender service thus letting Django and Flask push MQ messages. + +### Django and Flask integration + + * Any Zato-based API service can be mounted on a channel + * For Django and Flask, it is most convenient to mount one's services on REST channels and invoke them using the [zato-client][21] from PyPI + * zato-client is a set of convenience clients that lets any Python application, including ones based on Django or Flask, to invoke Zato services in just a few steps + * There is [a dedicated chapter][22] in documentation about Django and Flask, including a sample integration scenario + * It's recommended to go through the chapter step-by-step - since all Zato configuration objects share the same principles, the whole of its information applies to any sort of technology that Django or Flask may need to integrate with, including WebSphere MQ + * After completing that chapter, to push messages to MQ, you will only need to: + * Create a security definition for a new REST channel for Django or Flask + * Create the REST channel itself + * Assign a service to it (e.g. MQSender) + * Use a Python client from zato-client to invoke that channel from Django or Flask + * And that is it - no MQ programming is needed to send messages to MQ queues from any Python application :-) + + + +### Summary + + * Zato lets Python programmers integrate with WebSphere MQ with little to no effort + * Built-in support for JMS lets one integrate with existing Java applications in a transparent manner + * Built-in Python clients offer trivial access to Zato-based API services from other Python applications, including Django or Flask + + + +Where to next? Start off with the [tutorial][23], then consult the [documentation][24], there is a lot of information for all types of API and integration projects, and have a look at [support options][25] in case you need absolutely any sort of assistance! + +-------------------------------------------------------------------------------- + +via: https://zato.io/blog/posts/websphere-mq-python-zato.html + +作者：[zato][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://zato.io +[1]:https://en.wikipedia.org/wiki/IBM_WebSphere_MQ +[2]:https://zato.io/docs +[3]:https://www.djangoproject.com/ +[4]:http://flask.pocoo.org/ +[5]:https://zato.io/docs/admin/guide/install/source.html +[6]:https://zato.io/docs/admin/guide/install/index.html +[7]:https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_7.0.1/com.ibm.mq.csqzaf.doc/cs10230_.htm +[8]:https://github.com/dsuch/pymqi/ +[9]:https://zato.io/docs/web-admin/intro.html +[10]:https://zato.io/docs/admin/guide/enmasse.html +[11]:https://zato.io/docs/public-api/intro.html +[12]:https://zato.io/docs/admin/cli/index.html +[13]:https://zato.io/blog/images/wmq-python-zato/def-create.png +[14]:https://zato.io/blog/images/wmq-python-zato/def-options.png +[15]:https://zato.io/blog/images/wmq-python-zato/def-ping.png +[16]:https://zato.io/blog/images/wmq-python-zato/outconn-create.png +[17]:https://zato.io/blog/images/wmq-python-zato/outconn-options.png +[18]:https://zato.io/blog/images/wmq-python-zato/outconn-send.png +[19]:https://zato.io/blog/images/wmq-python-zato/channel-create.png +[20]:https://zato.io/blog/images/wmq-python-zato/def-create-jms.png +[21]:https://pypi.python.org/pypi/zato-client +[22]:https://zato.io/docs/progguide/clients/django-flask.html +[23]:https://zato.io/docs/tutorial/01.html +[24]:https://zato.io/docs/ +[25]:https://zato.io/support.html diff --git a/sources/tech/20180129 What Happens When You Want to Create a Special Fille with All Special Characters in Linux.md b/sources/tech/20180129 What Happens When You Want to Create a Special Fille with All Special Characters in Linux.md new file mode 100644 index 0000000000..60e923fd46 --- /dev/null +++ b/sources/tech/20180129 What Happens When You Want to Create a Special Fille with All Special Characters in Linux.md @@ -0,0 +1,189 @@ +What Happens When You Want to Create a Special File with All Special Characters in Linux? +============================================================ + + +![special chars](https://www.linux.com/sites/lcom/files/styles/rendered_file/public/special-chars.png?itok=EEvlt5Nw "special chars") +Learn how to handle creation of a special file filled with special characters.[Used with permission][1] + +I recently joined Holberton School as a student, hoping to learn full-stack software development. What I did not expect was that in two weeks I would be pretty much proficient with creating shell scripts that would make my coding life easy and fast! + +So what is the post about? It is about a novel problem that my peers and I faced when we were asked to create a file with no regular alphabets/ numbers but instead special characters!! Just to give you a look at what kind of file name we were dealing with — + +### \*\\’”Holberton School”\’\\*$\?\*\*\*\*\*:) + +What a novel file name! Of course, this question was met with the collective groaning and long drawn sighs of all 55 (batch #5) students! + +![1*Lf_XPhmgm-RB5ipX_lBjsQ.gif](https://cdn-images-1.medium.com/max/1600/1*Lf_XPhmgm-RB5ipX_lBjsQ.gif) + +Some proceeded to make their lives easier by breaking the file name into pieces on a doc file and adding in the **“\\” or “\”** in front of certain special character which kind of resulted in this format - + +#### \\*\\\\’\”Holberton School\”\\’\\\\*$\\?\\*\\*\\*\\*\\*:) + +![1*p6s8WlysClalj0x2fQhGOg.gif](https://cdn-images-1.medium.com/max/1600/1*p6s8WlysClalj0x2fQhGOg.gif) + +Everyone trying to get the \\ right + +bamboozled? me, too! I did not want to believe that this was the only way to solve this, as I was getting frustrated with every “\\” or “\” that was required to escape and print those special characters as normal characters! + +If you’re new to shell scripting, here is a quick walk through on why so many “\\” , “\” were required and where. + +In shell scripting “ ” and ‘ ’ have special usage and once you understand and remember when and where to use them it can make your life easier! + +#### Double Quoting + +The first type of quoting we will look at is double quotes. **If you place text inside double quotes, all the special characters used by the shell lose their special meaning and are treated as ordinary characters. The exceptions are “$”, “\” (backslash), and “`” (back- quote).**This means that word-splitting, pathname expansion, tilde expansion, and brace expansion are suppressed, but parameter expansion, arithmetic expansion, and command substitution are still carried out. Using double quotes, we can cope with filenames containing embedded spaces. + +So this means that you can create file with names that have spaces between words — if that is your thing, but I would suggest you to not do that as it is inconvenient and rather an unpleasant experience for you to try to find that file when you need ! + +**Quoting “THE” guide for linux I follow and read like it is the Harry Potter of the linux coding world —** + +Say you were the unfortunate victim of a file called two words.txt. If you tried to use this on the command line, word-splitting would cause this to be treated as two separate arguments rather than the desired single argument: + +**[[me@linuxbox][3] me]$ ls -l two words.txt** + +``` +ls: cannot access two: No such file or directory +ls: cannot access words.txt: No such file or directory +``` + +By using double quotes, you can stop the word-splitting and get the desired result; further, you can even repair the damage: + +``` +[me@linuxbox me]$ ls -l “two words.txt” +-rw-rw-r — 1 me me 18 2008–02–20 13:03 two words.txt +[me@linuxbox me]$ mv “two words.txt” two_words.t +``` + +There! Now we don’t have to keep typing those pesky double quotes. + +Now, let us talk about single quotes and what is their significance in shell — + +#### Single Quotes + +Enclosing characters in single quotes (‘’’) preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash. + +Yes! that got me and I was wondering how will I use it, apparently when I was googling to find and easier way to do it I stumbled across this piece of information on the internet — + +### Strong quoting + +Strong quoting is very easy to explain: + +Inside a single-quoted string **nothing** is interpreted, except the single-quote that closes the string. + +``` +echo 'Your PATH is: $PATH' +``` + +`$PATH` won't be expanded, it's interpreted as ordinary text because it's surrounded by strong quotes. + +In practice that means to produce a text like `Here's my test…` as a single-quoted string, **you have to leave and re-enter the single quoting to get the character "`'`" as literal text:** + +``` +# WRONG +echo 'Here's my test...' +``` + +``` +# RIGHT +echo 'Here'\''s my test...' +``` + +``` +# ALTERNATIVE: It's also possible to mix-and-match quotes for readability: +echo "Here's my test" +``` + +Well now you’re wondering — “well that explains the quotes but what about the “\”??” + +So for certain characters we need a special way to escape those pesky “\” we saw in that file name. + +#### Escaping Characters + +Sometimes you only want to quote a single character. To do this, you can precede a character with a backslash, which in this context is called the  _escape character_ . Often this is done inside double quotes to selectively prevent an expansion: + +``` +[me@linuxbox me]$ echo “The balance for user $USER is: \$5.00” +The balance for user me is: $5.00 +``` + +It is also common to use escaping to eliminate the special meaning of a character in a filename. For example, it is possible to use characters in filenames that normally have special meaning to the shell. These would include “$”, “!”, “&”, “ “, and others. To include a special character in a filename you can to this: + +``` +[me@linuxbox me]$ mv bad\&filename good_filename +``` + +> _**To allow a backslash character to appear, escape it by typing “\\”. Note that within single quotes, the backslash loses its special meaning and is treated as an ordinary character.**_ + +Looking at the filename now we can understand better as to why the “\\” were used in front of all those “\”s. + +So, to print the file name without losing “\” and other special characters what others did was to suppress the “\” with “\\” and to print the single quotes there are a few ways you can do that. + +``` +1. echo $'It\'s Shell Programming' # ksh, bash, and zsh only, does not expand variables +2. echo "It's Shell Programming" # all shells, expands variables +3. echo 'It'\''s Shell Programming' # all shells, single quote is outside the quotes +4\. echo 'It'"'"'s Shell Programming' # all shells, single quote is inside double quotes +``` + +``` +for further reading please follow this link +``` + +Looking at option 3, I realized this would mean that I would only need to use “\” and single quotes at certain places to be able to write the whole file without getting frustrated with “\\” placements. + +So with the hope in mind and lesser trial and errors I was actually able to print out the file name like this: + +#### ‘\*\\’\’’”Holberton School”\’\’’\\*$\?\*\*\*\*\*:)’ + +to understand better I have added an **“a”** instead of my single quotes so that the file name and process becomes more clearer. For a better understanding, I’ll break them down into modules: + +![1*hP1gmzbn7G7gUEhoynj1ew.gif](https://cdn-images-1.medium.com/max/1600/1*hP1gmzbn7G7gUEhoynj1ew.gif) + +#### a\*\\a \’ a”Holberton School”\a \’ a\\*$\?\*\*\*\*\*:)a + +#### Module 1 — a\*\\a + +Here the use of single quote (a) creates a safe suppression for \*\\ and as mentioned before in strong quoting, the only way we can print the ‘ is to leave and re-enter the single quoting to get the character. + +#### Module 2 , 4— \’ + +The \ suppresses the single quote as a standalone module. + +#### Module 3 — a”Holberton School”\a + +Here the use of single quote (a) creates a safe suppression for double quotes and \ along with regular text. + +#### Module 5 — a\\*$\?\*\*\*\*\*:)a + +Here the use of single quote (a) creates a safe suppression for all special characters being used such as *, \, $, ?, : and ). + +so in the end I was able to be lazy and maintain my sanity, and got away with only using single quotes to create small modules and “\” in certain places. + +![1*rO34jp-bYSkCnHSdwoO3qQ.gif](https://cdn-images-1.medium.com/max/1600/1*rO34jp-bYSkCnHSdwoO3qQ.gif) + +And, that is how I was able to get the file to work right! After a few misses, it felt amazing and it was great to learn a new way to do things! + +![1*PE9_VtcfGGQjnYMwJ8YB1A.gif](https://cdn-images-1.medium.com/max/1600/1*PE9_VtcfGGQjnYMwJ8YB1A.gif) + +Handled that curve-ball pretty well! Hope this helps you in the future when, someday you might need to create a special file for a special reason in shell! + + _**Mitali Sengupta **is a former digital marketing professional, currently enrolled as a full-stack engineering student at Holberton School. She is passionate about innovation in AI and Blockchain technologies.. You can contact Mitali on [Twitter][4], [LinkedIn][5] or [GitHub][6]._ + +-------------------------------------------------------------------------------- + +via: https://www.linux.com/blog/what-happens-when-you-want-create-special-file-all-special-characters-linux + +作者：[MITALI SENGUPTA ][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.linux.com/users/mitalisengupta +[1]:https://www.linux.com/licenses/category/used-permission +[2]:https://www.linux.com/files/images/special-charspng +[3]:mailto:me@linuxbox +[4]:https://twitter.com/aadhiBangalan +[5]:https://www.linkedin.com/in/mitali-sengupta-auger +[6]:https://github.com/MitaliSengupta +[7]:http://mywiki.wooledge.org/Quotes#Examples \ No newline at end of file diff --git a/sources/tech/20180130 An introduction to the DomTerm terminal emulator for Linux.md b/sources/tech/20180130 An introduction to the DomTerm terminal emulator for Linux.md new file mode 100644 index 0000000000..4553570166 --- /dev/null +++ b/sources/tech/20180130 An introduction to the DomTerm terminal emulator for Linux.md @@ -0,0 +1,126 @@ +An introduction to the DomTerm terminal emulator for Linux +====== +![](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/osdc_terminals.png?itok=CfBqYBah) + +[DomTerm][1] is a modern terminal emulator that uses a browser engine as a "GUI toolkit." This enables some neat features, such as embeddable graphics and links, HTML rich text, and foldable (show/hide) commands. Otherwise it looks and feels like a feature-full, standalone terminal emulator, with excellent xterm compatibility (including mouse handling and 24-bit color), and appropriate "chrome" (menus). In addition, there is built-in support for session management and sub-windows (as in `tmux` and `GNU screen`), basic input editing (as in `readline`), and paging (as in `less`). + +![](https://opensource.com/sites/default/files/u128651/domterm1.png) +Image 1: The DomTerminal terminal emulator. View larger image. + +Below we'll look more at these features. We'll assume you have `domterm` installed (skip to the end of this article if you need to get and build DomTerm). First, though, here's a quick overview of the technology. + +### Frontend vs. backend + +Most of DomTerm is written in JavaScript and runs in a browser engine. This can be a desktop web browser, such as Chrome or Firefox (see image 3), or it can be an embedded browser. Using a general web browser works fine, but the user experience isn't as nice (as the menus are designed for general browsing, not for a terminal emulator), and the security model gets in the way, so using an embedded browser is nicer. + +The following are currently supported: + + * `qtdomterm`, which uses the Qt toolkit and `QtWebEngine` + * An `[Electron][2]` embedding (see image 1) + * `atom-domterm` runs DomTerm as a package in the [Atom text editor][3] (which is also based on Electron) and integrates with the Atom pane system (see image 2) + * A wrapper for JavaFX's `WebEngine`, which is useful for code written in Java (see image 4) + * Previously, the preferred frontend used [Firefox-XUL][4], but Mozilla has since dropped XUL + + + +![DomTerm terminal panes in Atom editor][6] + +Image 2: DomTerm terminal panes in Atom editor. [View larger image.][7] + +Currently, the Electron frontend is probably the nicest option, closely followed by the Qt frontend. If you use Atom, `atom-domterm` works pretty well. + +The backend server is written in C. It manages pseudo terminals (PTYs) and sessions. It is also an HTTP server that provides the JavaScript and other files to the frontend. The `domterm` command starts terminal jobs and performs other requests. If there is no server running, `domterm` daemonizes itself. Communication between the backend and the server is normally done using WebSockets (with [libwebsockets][8] on the server). However, the JavaFX embedding uses neither WebSockets nor the DomTerm server; instead Java applications communicate directly using the Java-JavaScript bridge. + +### A solid xterm-compatible terminal emulator + +DomTerm looks and feels like a modern terminal emulator. It handles mouse events, 24-bit color, Unicode, double-width (CJK) characters, and input methods. DomTerm does a very good job on the [vttest testsuite][9]. + +Unusual features include: + +**Show/hide buttons ("folding"):** The little triangles (seen in image 2 above) are buttons that hide/show the corresponding output. To create the buttons, just add certain [escape sequences][10] in the [prompt text][11]. + +**Mouse-click support for`readline` and similar input editors:** If you click in the (yellow) input area, DomTerm will send the right sequence of arrow-key keystrokes to the application. (This is enabled by escape sequences in the prompt; you can also force it using Alt+Click.) + +**Style the terminal using CSS:** This is usually done in `~/.domterm/settings.ini`, which is automatically reloaded when saved. For example, in image 2, terminal-specific background colors were set. + +### A better REPL console + +A classic terminal emulator works on rectangular grids of character cells. This works for a REPL (command shell), but it is not ideal. Here are some DomTerm features useful for REPLs that are not typically found in terminal emulators: + +**A command can "print" an image, a graph, a mathematical formula, or a set of clickable links:** An application can send an escape sequence containing almost any HTML. (The HTML is scrubbed to remove JavaScript and other dangerous features.) + +The image 3 shows a fragment from a [`gnuplot`][12] session. Gnuplot (2.1 or later) supports `domterm` as a terminal type. Graphical output is converted to an [SVG image][13], which is then printed to the terminal. My blog post [Gnuplot display on DomTerm][14] provides more information on this. + +![](https://opensource.com/sites/default/files/dt-gnuplot.png) +Image 3: Gnuplot screenshot. View larger image. + +The [Kawa][15] language has a library for creating and transforming [geometric picture values][16]. If you print such a picture value to a DomTerm terminal, the picture is converted to SVG and embedded in the output. + +![](https://opensource.com/sites/default/files/dt-kawa1.png) +Image 4: Computable geometry in Kawa. View larger image. + +**Rich text in output:** Help messages are more readable and look nicer with HTML styling. The lower pane of image 1 shows the ouput from `domterm help`. (The output is plaintext if not running under DomTerm.) Note the `PAUSED` message from the built-in pager. + +**Error messages can include clickable links:** DomTerm recognizes the syntax `filename:line:column:` and turns it into a link that opens the file and line in a configurable text editor. (This works for relative filenames if you use `PROMPT_COMMAND` or similar to track directories.) + +A compiler can detect that it is running under DomTerm and directly emit file links in an escape sequence. This is more robust than depending on DomTerm's pattern matching, as it handles spaces and other special characters, and it does not depend on directory tracking. In image 4, you can see error messages from the [Kawa compiler][15]. Hovering over the file position causes it to be underlined, and the `file:` URL shows in the `atom-domterm` message area (bottom of the window). (When not using `atom-domterm`, such messages are shown in an overlay box, as seen for the `PAUSED` message in image 1.) + +The action when clicking on a link is configurable. The default action for a `file:` link with a `#position` suffix is to open the file in a text editor. + +**Structured internal representation:** The following are all represented in the internal node structure: Commands, prompts, input lines, normal and error output, tabs, and preserving the structure if you "Save as HTML." The HTML file is compatible with XML, so you can use XML tools to search or transform the output. The command `domterm view-saved` opens a saved HTML file in a way that enables command folding (show/hide buttons are active) and reflow on window resize. + +**Built-in Lisp-style pretty-printing:** You can include pretty-printing directives (e.g., grouping) in the output such that line breaks are recalculated on window resize. See my article [Dynamic pretty-printing in DomTerm][17] for a deeper discussion. + +**Basic built-in line editing** with history (like `GNU readline`): This uses the browser's built-in editor, so it has great mouse and selection handling. You can switch between normal character-mode (most characters typed are sent directly to the process); or line-mode (regular characters are inserted while control characters cause editing actions, with Enter sending the edited line to the process). The default is automatic mode, where DomTerm switches between character-mode and line-mode depending on whether the PTY is in raw or canonical mode. + +**A built-in pager** (like a simplified `less`): Keyboard shortcuts will control scrolling. In "paging mode," the output pauses after each new screen (or single line, if you move forward line-by-line). The paging mode is unobtrusive and smart about user input, so you can (if you wish) run it without it interfering with interactive programs. + +### Multiplexing and sessions + +**Tabs and tiling:** Not only can you create multiple terminal tabs, you can also tile them. You can use either the mouse or a keyboard shortcut to move between panes and tabs as well as create new ones. They can be rearranged and resized with the mouse. This is implemented using the [GoldenLayout][18] JavaScript library. [Image 1][19] shows a window with two panes. The top one has two tabs, with one running [Midnight Commander][20]; the bottom pane shows `domterm help` output as HTML. However, on Atom we instead use its built-in draggable tiles and tabs; you can see this in image 2. + +**Detaching and reattaching to sessions:** DomTerm supports sessions arrangement, similar to `tmux` and GNU `screen`. You can even attach multiple windows or panes to the same session. This supports multi-user session sharing and remote connections. (For security, all sessions of the same server need to be able to read a Unix domain socket and a local file containing a random key. This restriction will be lifted when we have a good, safe remote-access story.) + +**The** **`domterm`** **command** is also like `tmux` or GNU `screen` in that has multiple options for controlling or starting a server that manages one or more sessions. The major difference is that, if it's not already running under DomTerm, the `domterm` command creates a new top-level window, rather than running in the existing terminal. + +The `domterm` command has a number of sub-commands, similar to `tmux` or `git`. Some sub-commands create windows or sessions. Others (such as "printing" an image) only work within an existing DomTerm session. + +The command `domterm browse` opens a window or pane for browsing a specified URL, such as when browsing documentation. + +### Getting and installing DomTerm + +DomTerm is available from its [GitHub repository][21]. Currently, there are no prebuilt packages, but there are [detailed instructions][22]. All prerequisites are available on Fedora 27, which makes it especially easy to build. + +-------------------------------------------------------------------------------- + +via: https://opensource.com/article/18/1/introduction-domterm-terminal-emulator + +作者：[Per Bothner][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://opensource.com/users/perbothner +[1]:http://domterm.org/ +[2]:https://electronjs.org/ +[3]:https://atom.io/ +[4]:https://en.wikipedia.org/wiki/XUL +[5]:/file/385346 +[6]:https://opensource.com/sites/default/files/images/dt-atom1.png (DomTerm terminal panes in Atom editor) +[7]:https://opensource.com/sites/default/files/images/dt-atom1.png +[8]:https://libwebsockets.org/ +[9]:http://invisible-island.net/vttest/ +[10]:http://domterm.org/Wire-byte-protocol.html +[11]:http://domterm.org/Shell-prompts.html +[12]:http://www.gnuplot.info/ +[13]:https://developer.mozilla.org/en-US/docs/Web/SVG +[14]:http://per.bothner.com/blog/2016/gnuplot-in-domterm/ +[15]:https://www.gnu.org/software/kawa/ +[16]:https://www.gnu.org/software/kawa/Composable-pictures.html +[17]:http://per.bothner.com/blog/2017/dynamic-prettyprinting/ +[18]:https://golden-layout.com/ +[19]:https://opensource.com/sites/default/files/u128651/domterm1.png +[20]:https://midnight-commander.org/ +[21]:https://github.com/PerBothner/DomTerm +[22]:http://domterm.org/Downloading-and-building.html diff --git a/sources/tech/20180130 Ansible- Making Things Happen.md b/sources/tech/20180130 Ansible- Making Things Happen.md new file mode 100644 index 0000000000..88210cd20c --- /dev/null +++ b/sources/tech/20180130 Ansible- Making Things Happen.md @@ -0,0 +1,174 @@ +Ansible: Making Things Happen +====== +In my [last article][1], I described how to configure your server and clients so you could connect to each client from the server. Ansible is a push-based automation tool, so the connection is initiated from your "server", which is usually just a workstation or a server you ssh in to from your workstation. In this article, I explain how modules work and how you can use Ansible in ad-hoc mode from the command line. + +Ansible is supposed to make your job easier, so the first thing you need to learn is how to do familiar tasks. For most sysadmins, that means some simple command-line work. Ansible has a few quirks when it comes to command-line utilities, but it's worth learning the nuances, because it makes for a powerful system. + +### Command Module + +This is the safest module to execute remote commands on the client machine. As with most Ansible modules, it requires Python to be installed on the client, but that's it. When Ansible executes commands using the Command Module, it does not process those commands through the user's shell. This means some variables like $HOME are not available. It also means stream functions (redirects, pipes) don't work. If you don't need to redirect output or to reference the user's home directory as a shell variable, the Command Module is what you want to use. To invoke the Command Module in ad-hoc mode, do something like this: + +``` + +ansible host_or_groupname -m command -a "whoami" + +``` + +Your output should show SUCCESS for each host referenced and then return the user name that the user used to log in. You'll notice that the user is not root, unless that's the user you used to connect to the client computer. + +If you want to see the elevated user, you'll add another argument to the ansible command. You can add -b in order to "become" the elevated user (or the sudo user). So, if you were to run the same command as above with a "-b" flag: + +``` + +ansible host_or_groupname -b -m command -a "whoami" + +``` + +you should see a similar result, but the whoami results should say root instead of the user you used to connect. That flag is important to use, especially if you try to run remote commands that require root access! + +### Shell Module + +There's nothing wrong with using the Shell Module to execute remote commands. It's just important to know that since it uses the remote user's environment, if there's something goofy with the user's account, it might cause problems that the Command Module avoids. If you use the Shell Module, however, you're able to use redirects and pipes. You can use the whoami example to see the difference. This command: + +``` + +ansible host_or_groupname -m command -a "whoami > myname.txt" + +``` + +should result in an error about > not being a valid argument. Since the Command Module doesn't run inside any shell, it interprets the greater-than character as something you're trying to pass to the whoami command. If you use the Shell Module, however, you have no problems: + +``` + +ansible host_or_groupname -m shell -a "whom > myname.txt" + +``` + +This should execute and give you a SUCCESS message for each host, but there should be nothing returned as output. On the remote machine, however, there should be a file called myname.txt in the user's home directory that contains the name of the user. My personal policy is to use the Command Module whenever possible and to use the Shell Module if needed. + +### The Raw Module + +Functionally, the Raw Module works like the Shell Module. The key difference is that Ansible doesn't do any error checking, and STDERR, STDOUT and Return Code is returned. Other than that, Ansible has no idea what happens, because it just executes the command over SSH directly. So while the Shell Module will use /bin/sh by default, the Raw Module just uses whatever the user's personal default shell might be. + +Why would a person decide to use the Raw Module? It doesn't require Python on the remote computer—at all. Although it's true that most servers have Python installed by default, or easily could have it installed, many embedded devices don't and can't have Python installed. For most configuration management tools, not having an agent program installed means the remote device can't be managed. With Ansible, if all you have is SSH, you still can execute remote commands using the Raw Module. I've used the Raw Module to manage Bitcoin miners that have a very minimal embedded environment. It's a powerful tool, and when you need it, it's invaluable! + +### Copy Module + +Although it's certainly possible to do file and folder manipulation with the Command and Shell Modules, Ansible includes a module specifically for copying files to the server. Even though it requires learning a new syntax for copying files, I like to use it because Ansible will check to see whether a file exists, and whether it's the same file. That means it copies the file only if it needs to, saving time and bandwidth. It even will make backups of existing files! I can't tell you how many times I've used scp and sshpass in a Bash FOR loop and dumped files on servers, even if they didn't need them. Ansible makes it easy and doesn't require FOR loops and IP iterations. + +The syntax is a little more complicated than with Command, Shell or Raw. Thankfully, as with most things in the Ansible world, it's easy to understand—for example: + +``` + +ansible host_or_groupname -b -m copy \ + -a "src=./updated.conf dest=/etc/ntp.conf \ + owner=root group=root mode=0644 backup=yes" + +``` + +This will look in the current directory (on the Ansible server/workstation) for a file called updated.conf and then copy it to each host. On the remote system, the file will be put in /etc/ntp.conf, and if a file already exists, and it's different, the original will be backed up with a date extension. If the files are the same, Ansible won't make any changes. + +I tend to use the Copy Module when updating configuration files. It would be perfect for updating configuration files on Bitcoin miners, but unfortunately, the Copy Module does require that the remote machine has Python installed. Nevertheless, it's a great way to update common files on many remote machines with one simple command. It's also important to note that the Copy Module supports copying remote files to other locations on the remote filesystem using the remote_src=true directive. + +### File Module + +The File Module has a lot in common with the Copy Module, but if you try to use the File Module to copy a file, it doesn't work as expected. The File Module does all its actions on the remote machine, so src and dest are all references to the remote filesystem. The File Module often is used for creating directories, creating links or deleting remote files and folders. The following will simply create a folder named /etc/newfolder on the remote servers and set the mode: + +``` + +ansible host_or_groupname -b -m file \ + -a "path=/etc/newfolder state=directory mode=0755" + +``` + +You can, of course, set the owner and group, along with a bunch of other options, which you can learn about on the Ansible doc site. I find I most often will either create a folder or symbolically link a file using the File Module. To create a symlink: + +``` + +sensible host_or_groupname -b -m file \ + -a "src=/etc/ntp.conf dest=/home/user/ntp.conf \ + owner=user group=user state=link" + +``` + +Notice that the state directive is how you inform Ansible what you actually want to do. There are several state options: + +* link — create symlink. + +* directory — create directory. + +* hard — create hardlink. + +* touch — create empty file. + +* absent — delete file or directory recursively. + +This might seem a bit complicated, especially when you easily could do the same with a Command or Shell Module command, but the clarity of using the appropriate module makes it more difficult to make mistakes. Plus, learning these commands in ad-hoc mode will make playbooks, which consist of many commands, easier to understand (I plan to cover this in my next article). + +### File Management + +Anyone who manages multiple distributions knows it can be tricky to handle the various package managers. Ansible handles this in a couple ways. There are specific modules for apt and yum, but there's also a generic module called "package" that will install on the remote computer regardless of whether it's Red Hat- or Debian/Ubuntu-based. + +Unfortunately, while Ansible usually can detect the type of package manager it needs to use, it doesn't have a way to fix packages with different names. One prime example is Apache. On Red Hat-based systems, the package is "httpd", but on Debian/Ubuntu systems, it's "apache2". That means some more complex things need to happen in order to install the correct package automatically. The individual modules, however, are very easy to use. I find myself just using apt or yum as appropriate, just like when I manually manage servers. Here's an apt example: + +``` + +ansible host_or_groupname -b -m apt \ + -a "update_cache=yes name=apache2 state=latest" + +``` + +With this one simple line, all the host machines will run apt-get update (that's the update_cache directive at work), then install apache2's latest version including any dependencies required. Much like the File Module, the state directive has a few options: + +* latest — get the latest version, upgrading existing if needed. + +* absent — remove package if installed. + +* present — make sure package is installed, but don't upgrade existing. + +The Yum Module works similarly to the Apt Module, but I generally don't bother with the update_cache directive, because yum updates automatically. Although very similar, installing Apache on a Red Hat-based system looks like this: + +``` + +ansible host_or_groupname -b -m yum \ + -a "name=httpd state=present" + +``` + +The difference with this example is that if Apache is already installed, it won't update, even if an update is available. Sometimes updating to the latest version isn't want you want, so this stops that from accidentally happening. + +### Just the Facts, Ma'am + +One frustrating thing about using Ansible in ad-hoc mode is that you don't have access to the "facts" about the remote systems. In my next article, where I plan to explore creating playbooks full of various tasks, you'll see how you can reference the facts Ansible learns about the systems. It makes Ansible far more powerful, but again, it can be utilized only in playbook mode. Nevertheless, it's possible to use ad-hoc mode to peek at the sorts information Ansible gathers. If you run the setup module, it will show you all the details from a remote system: + +``` + +ansible host_or_groupname -b -m setup + +``` + +That command will spew a ton of variables on your screen. You can scroll through them all to see the vast amount of information Ansible pulls from the host machines. In fact, it shows so much information, it can be overwhelming. You can filter the results: + +``` + +ansible host_or_groupname -b -m setup -a "filter=*family*" + +``` + +That should just return a single variable, ansible_os_family, which likely will be Debian or Red Hat. When you start building more complex Ansible setups with playbooks, it's possible to insert some logic and conditionals in order to use yum where appropriate and apt where the system is Debian-based. Really, the facts variables are incredibly useful and make building playbooks that much more exciting. + +But, that's for another article, because you've come to the end of the second installment. Your assignment for now is to get comfortable using Ansible in ad-hoc mode, doing one thing at a time. Most people think ad-hoc mode is just a stepping stone to more complex Ansible setups, but I disagree. The ability to configure hundreds of servers consistently and reliably with a single command is nothing to scoff at. I love making elaborate playbooks, but just as often, I'll use an ad-hoc command in a situation that used to require me to ssh in to a bunch of servers to do simple tasks. Have fun with Ansible; it just gets more interesting from here! + + +-------------------------------------------------------------------------------- + +via: http://www.linuxjournal.com/content/ansible-making-things-happen + +作者：[Shawn Powers][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://www.linuxjournal.com/users/shawn-powers +[1]:http://www.linuxjournal.com/content/ansible-automation-framework-thinks-sysadmin diff --git a/sources/tech/20180130 Create and manage MacOS LaunchAgents using Go.md b/sources/tech/20180130 Create and manage MacOS LaunchAgents using Go.md new file mode 100644 index 0000000000..8bd6b8bf64 --- /dev/null +++ b/sources/tech/20180130 Create and manage MacOS LaunchAgents using Go.md @@ -0,0 +1,305 @@ +Create and manage MacOS LaunchAgents using Go +============================================================ + +If you have ever tried writing a daemon for MacOS you have met with `launchd`. For those that don’t have the experience, think of it as a framework for starting, stopping and managing daemons, applications, processes, and scripts. If you have any *nix experience the word daemon should not be too alien to you. + +For those unfamiliar, a daemon is a program running in the background without requiring user input. A typical daemon might, for instance, perform daily maintenance tasks or scan a device for malware when connected. + +This post is aimed at folks that know a little bit about what daemons are, what is the common way of using them and know a bit about Go. Also, if you have ever written a daemon for any other *nix system, you will have a good idea of what we are going to talk here. If you are an absolute beginner in Go or systems this might prove to be an overwhelming article. Still, feel free to give it a shot and let me know how it goes. + +If you ever find yourself wanting to write a MacOS daemon with Go you would like to know most of the stuff we are going to talk about in this article. Without further ado, let’s dive in. + +### What is `launchd` and how it works? + +`launchd` is a unified service-management framework, that starts, stops and manages daemons, applications, processes, and scripts in MacOS. + +One of its key features is that it differentiates between agents and daemons. In `launchd` land, an agent runs on behalf of the logged in user while a daemon runs on behalf of the root user or any specified user. + +### Defining agents and daemons + +An agent/daemon is defined in an XML file, which states the properties of the program that will execute, among a list of other properties. Another aspect to keep in mind is that `launchd` decides if a program will be treated as a daemon or an agent by where the program XML is located. + +Over at [launchd.info][3], there’s a simple table that shows where you would (or not) place your program’s XML: + +``` ++----------------+-------------------------------+----------------------------------------------------+| Type | Location | Run on behalf of |+----------------+-------------------------------+----------------------------------------------------+| User Agents | ~/Library/LaunchAgents | Currently logged in user || Global Agents | /Library/LaunchAgents | Currently logged in user || Global Daemons | /Library/LaunchDaemons | root or the user specified with the key 'UserName' || System Agents | /System/Library/LaunchAgents | Currently logged in user || System Daemons | /System/Library/LaunchDaemons | root or the user specified with the key 'UserName' |+----------------+-------------------------------+----------------------------------------------------+ +``` + +This means that when we set our XML file in, for example, the `/Library/LaunchAgents` path our process will be treated as a global agent. The main difference between the daemons and agents is that LaunchDaemons will run as root, and are generally background processes. On the other hand, LaunchAgents are jobs that will run as a user or in the context of userland. These may be scripts or other foreground items and they also have access to the MacOS UI (e.g. you can send notifications, control the windows, etc.) + +So, how do we define an agent? Let’s take a look at a simple XML file that `launchd`understands: + +``` + Label com.example.app Program /Users/Me/Scripts/cleanup.sh RunAtLoad +``` + +The XML is quite self-explanatory, unless it’s the first time you are seeing an XML file. The file has three main properties, with values. In fact, if you take a better look you will see the `dict` keyword which means `dictionary`. This actually means that the XML represents a key-value structure, so in Go it would look like: + +``` +map[string]string{ "Label": "com.example.app", "Program": "/Users/Me/Scripts/cleanup.sh", "RunAtLoad": "true",} +``` + +Let’s look at each of the keys: + +1. `Label` - The job definition or the name of the job. This is the unique identifier for the job within the `launchd` instance. Usually, the label (and hence the name) is written in [Reverse domain name notation][1]. + +2. `Program` - This key defines what the job should start, in our case a script with the path `/Users/Me/Scripts/cleanup.sh`. + +3. `RunAtLoad` - This key specifies when the job should be run, in this case right after it’s loaded. + +As you can see, the keys used in this XML file are quite self-explanatory. This is the case for the remaining 30-40 keys that `launchd` supports. Last but not least these files although have an XML syntax, in fact, they have a `.plist` extension (which means `Property List`). Makes a lot of sense, right? + +### `launchd` v.s. `launchctl` + +Before we continue with our little exercise of creating daemons/agents with Go, let’s first see how `launchd` allows us to control these jobs. While `launchd`’s job is to boot the system and to load and maintain services, there is a different command used for jobs management - `launchctl`. With `launchd` facilitating jobs, the control of services is centralized in the `launchctl` command. + +`launchctl` has a long list of subcommands that we can use. For example, loading or unloading a job is done via: + +``` +launchctl unload/load ~/Library/LaunchAgents/com.example.app.plist +``` + +Or, starting/stopping a job is done via: + +``` +launchctl start/stop ~/Library/LaunchAgents/com.example.app.plist +``` + +To get any confusion out of the way, `load` and `start` are different. While `start`only starts the agent/daemon, `load` loads the job and it might also start it if the job is configured to run on load. This is achieved by setting the `RunAtLoad` property in the property list XML of the job: + +``` + Label com.example.app Program /Users/Me/Scripts/cleanup.sh RunAtLoad +``` + +If you would like to see what other commands `launchctl` supports, you can run`man launchctl` in your terminal and see the options in detail. + +### Automating with Go + +After getting the basics of `launchd` and `launctl` out of the way, why don’t we see how we can add an agent to any Go package? For our example, we are going to write a simple way of plugging in a `launchd` agent for any of your Go packages. + +As we already established before, `launchd` speaks in XML. Or, rather, it understands XML files, called  _property lists_  (or `.plist`). This means, for our Go package to have an agent running on MacOS, it will need to tell `launchd` “hey, `launchd`, run this thing!”. And since `launch` speaks only in `.plist`, that means our package needs to be capable of generating XML files. + +### Templates in Go + +While one could have a hardcoded `.plist` file in their project and copy it across to the `~/Library/LaunchAgents` path, a more programmatical way to do this would be to use a template to generate these XML files. The good thing is Go’s standard library has us covered - the `text/template` package ([docs][4]) does exactly what we need. + +In a nutshell, `text/template` implements data-driven templates for generating textual output. Or in other words, you give it a template and a data structure, it will mash them up together and produce a nice and clean text file. Perfect. + +Let’s say the `.plist` we need to generate in our case is the following: + +``` + LabelTicker Program/usr/local/bin/ticker StandardOutPath/tmp/ticker.out.log StandardErrorPath/tmp/ticker.err.log KeepAlive RunAtLoad +``` + +We want to keep it quite simple in our little exercise. It will contain only six properties: `Label`, `Program`, `StandardOutPath`, `StandardErrorPath`, `KeepAlive` and `RunAtLoad`. To generate such a XML, its template would look something like this: + +``` + + + + + Label{{.Label}} + Program{{.Program}} + StandardOutPath/tmp/{{.Label}}.out.log + StandardErrorPath/tmp/{{.Label}}.err.log + KeepAlive<{{.KeepAlive}}/> + RunAtLoad<{{.RunAtLoad}}/> + + + +``` + +As you can see, the difference between the two XMLs is that the second one has the double curly braces with expressions in them in places where the first XML has some sort of a value. These are called “actions”, which can be data evaluations or control structures and are delimited by “ and “. Any of the text outside actions is copied to the output untouched. + +### Injecting your data + +Now that we have our template with its glorious XML and curly braces (or actions), let’s see how we can inject our data into it. Since things are generally simple in Go, especially when it comes to its standard library, you should not worry - this will be easy! + +To keep thing simple, we will store the whole XML template in a plain old string. Yes, weird, I know. The best way would be to store it in a file and read it from there, or embed it in the binary itself, but in our little example let’s keep it simple: + +``` +// template.go +package main + +func Template() string { + return ` + + + + + Label{{.Label}} + Program{{.Program}} + StandardOutPath/tmp/{{.Label}}.out.log + StandardErrorPath/tmp/{{.Label}}.err.log + KeepAlive<{{.KeepAlive}}/> + RunAtLoad<{{.RunAtLoad}}/> + + +` +} + +``` + +And the program that will use our little template function: + +``` +// main.gopackage mainimport ( "log" "os" "text/template")func main() { data := struct { Label string Program string KeepAlive bool RunAtLoad bool }{ Label: "ticker", Program: "/usr/local/bin/ticker", KeepAlive: true, RunAtLoad: true, } t := template.Must(template.New("launchdConfig").Parse(Template())) err := t.Execute(os.Stdout, data) if err != nil { log.Fatalf("Template generation failed: %s", err) }} +``` + +So, what happens there, in the `main` function? It’s actually quite simple: + +1. We declare a small `struct`, which has only the properties that will be needed in the template, and we immediately initialize it with the values for our program. + +2. We build a new template, using the `template.New` function, with the name`launchdConfig`. Then, we invoke the `Parse` function on it, which takes the XML template as an argument. + +3. We invoke the `template.Must` function, which takes our built template as argument. From the documentation, `template.Must` is a helper that wraps a call to a function returning `(*Template, error)` and panics if the error is non-`nil`. Actually, `template.Must` is built to, in a way, validate if the template can be understood by the `text/template` package. + +4. Finally, we invoke `Execute` on our built template, which takes a data structure and applies its attributes to the actions in the template. Then it sends the output to `os.Stdout`, which does the trick for our example. Of course, the output can be sent to any struct that implements the `io.Writer` interface, like a file (`os.File`). + +### Make and load my `.plist` + +Instead of sending all this nice XML to standard out, let’s throw in an open file descriptor to the `Execute` function and finally save our `.plist` file in`~/Library/LaunchAgents`. There are a couple of main points we need to change. + +First, getting the location of the binary. Since it’s a Go binary, and we will install it via `go install`, we can assume that the path will be at `$GOPATH/bin`. Second, since we don’t know the actual `$HOME` of the current user, we will have to get it through the environment. Both of these can be done via `os.Getenv` ([docs][5]) which takes a variable name and returns its value. + +``` +// main.gopackage mainimport ( "log" "os" "text/template")func main() { data := struct { Label string Program string KeepAlive bool RunAtLoad bool }{ Label: "com.ieftimov.ticker", // Reverse-DNS naming convention Program: fmt.Sprintf("%s/bin/ticker", os.Getenv("GOPATH")), KeepAlive: true, RunAtLoad: true, } plistPath := fmt.Sprintf("%s/Library/LaunchAgents/%s.plist", os.Getenv("HOME"), data.Label) f, err := os.Open(plistPath) t := template.Must(template.New("launchdConfig").Parse(Template())) err := t.Execute(f, data) if err != nil { log.Fatalf("Template generation failed: %s", err) }} +``` + +That’s about it. The first part, about setting the correct `Program` property, is done by concatenating the name of the program and `$GOPATH`: + +``` +fmt.Sprintf("%s/bin/ticker", os.Getenv("GOPATH"))// Output: /Users//go/bin/ticker +``` + +The second part is slightly more complex, and it’s done by concatenating three strings, the `$HOME` environment variable, the `Label` property of the program and the `/Library/LaunchAgents` string: + +``` +fmt.Sprintf("%s/Library/LaunchAgents/%s.plist", os.Getenv("HOME"), data.Label)// Output: /Users//Library/LaunchAgents/com.ieftimov.ticker.plist +``` + +By having these two paths, opening the file and writing to it is very trivial - we open the file via `os.Open` and we pass in the `os.File` structure to `t.Execute` which writes to the file descriptor. + +### What about the Launch Agent? + +We will keep this one simple as well. Let’s throw in a command to our package, make it installable via `go install` (not that there’s much to it) and make it runnable by our `.plist` file: + +``` +// cmd/ticker/main.gopackage tickerimport ( "time" "fmt")func main() { for range time.Tick(30 * time.Second) { fmt.Println("tick!") }} +``` + +This the `ticker` program will use `time.Tick`, to execute an action every 30 seconds. Since this will be an infinite loop, `launchd` will kick off the program on boot (because `RunAtLoad` is set to `true` in the `.plist` file) and will keep it running. But, to make the program controllable from the operating system, we need to make the program react to some OS signals, like `SIGINT` or `SIGTERM`. + +### Understanding and handling OS signals + +While there’s quite a bit to be learned about OS signals, in our example we will scratch a bit off the surface. (If you know a lot about inter-process communication this might be too much of an oversimplification to you - and I apologize up front. Feel free to drop some links on the topic in the comments so others can learn more!) + +The best way to think about a signal is that it’s a message from the operating system or another process, to a process. It is an asynchronous notification sent to a process or to a specific thread within the same process to notify it of an event that occurred. + +There are quite a bit of various signals that can be sent to a process (or a thread), like `SIGKILL` (which kills a process), `SIGSTOP` (stop), `SIGTERM` (termination), `SIGILL`and so on and so forth. There’s an exhaustive list of signal types on [Wikipedia’s page][6]on signals. + +To get back to `launchd`, if we look at its documentation about stopping a job we will notice the following: + +> Stopping a job will send the signal `SIGTERM` to the process. Should this not stop the process launchd will wait `ExitTimeOut` seconds (20 seconds by default) before sending `SIGKILL`. + +Pretty self-explanatory, right? We need to handle one signal - `SIGTERM`. Why not `SIGKILL`? Because `SIGKILL` is a special signal that cannot be caught - it kills the process without any chance for a graceful shutdown, no questions asked. That’s why there’s a termination signal and a “kill” signal. + +Let’s throw in a bit of signal handling in our code, so our program knows that it needs to exit when it gets told to do so: + +``` +package mainimport ( "fmt" "os" "os/signal" "syscall" "time")func main() { sigs := make(chan os.Signal, 1) signal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM) go func() { <-sigs os.Exit(0) }() for range time.Tick(30 * time.Second) { fmt.Println("tick!") }} +``` + +In the new version, the agent program has two new packages imported: `os/signal`and `syscall`. `os/signal` implements access to incoming signals, that are primarily used on Unix-like systems. Since in this article we are specifically interested in MacOS, this is exactly what we need. + +Package `syscall` contains an interface to the low-level operating system primitives. An important note about `syscall` is that it is locked down since Go v1.4\. This means that any code outside of the standard library that uses the `syscall` package should be migrated to use the new `golang.org/x/sys` [package][7]. Since we are using **only**the signals constants of `syscall` we can get away with this. + +(If you want to read more about the package lockdown, you can see [the rationale on locking it down][8] by the Go team and the new [golang.org/s/sys][9] package.) + +Having the basics of the packages out of the way, let’s go step by step through the new lines of code added: + +1. We make a buffered channel of type `os.Signal`, with a size of `1`. `os.Signal`is a type that represents an operating system signal. + +2. We call `signal.Notify` with the new channel as an argument, plus`syscall.SIGINT` and `syscall.SIGTERM`. This function states “when the OS sends a `SIGINT` or a `SIGTERM` signal to this program, send the signal to the channel”. This allows us to somehow handle the sent OS signal. + +3. The new goroutine that we spawn waits for any of the signals to arrive through the channel. Since we know that any of the signals that will arrive are about shutting down the program, after receiving any signal we use `os.Exit(0)`([docs][2]) to gracefully stop the program. One caveat here is that if we had any `defer`red calls they would not be run. + +Now `launchd` can run the agent program and we can `load` and `unload`, `start`and `stop` it using `launchctl`. + +### Putting it all together + +Now that we have all the pieces ready, we need to put them together to a good use. Our application will consist of two binaries - a CLI tool and an agent (daemon). Both of the programs will be stored in separate subdirectories of the `cmd` directory. + +The CLI tool: + +``` +// cmd/cli/main.gopackage mainimport ( "log" "os" "text/template")func main() { data := struct { Label string Program string KeepAlive bool RunAtLoad bool }{ Label: "com.ieftimov.ticker", // Reverse-DNS naming convention Program: fmt.Sprintf("%s/bin/ticker", os.Getenv("GOPATH")), KeepAlive: true, RunAtLoad: true, } plistPath := fmt.Sprintf("%s/Library/LaunchAgents/%s.plist", os.Getenv("HOME"), data.Label) f, err := os.Open(plistPath) t := template.Must(template.New("launchdConfig").Parse(Template())) err := t.Execute(f, data) if err != nil { log.Fatalf("Template generation failed: %s", err) }} +``` + +And the ticker program: + +``` +// cmd/ticker/main.gopackage mainimport ( "fmt" "os" "os/signal" "syscall" "time")func main() { sigs := make(chan os.Signal, 1) signal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM) go func() { <-sigs os.Exit(0) }() for range time.Tick(30 * time.Second) { fmt.Println("tick!") }} +``` + +To install them both, we need to run `go install ./...` in the project root. The command will install all the sub-packages that are located within the project. This will leave us with two available binaries, installed in the `$GOPATH/bin` path. + +To install our launch agent, we need to run only the CLI tool, via the `cli` command. This will generate the `.plist` file and place it in the `~/Library/LaunchAgents`path. We don’t need to touch the `ticker` binary - that one will be managed by `launchd`. + +To load the newly created `.plist` file, we need to run: + +``` +launchctl load ~/Library/LaunchAgents/com.ieftimov.ticker.plist +``` + +When we run it, we will not see anything immediately, but after 30 seconds the ticker will add a `tick!` line in `/tmp/ticker.out.log`. We can `tail` the file to see the new lines being added. If we want to unload the agent, we can use: + +``` +launchctl unload ~/Library/LaunchAgents/com.ieftimov.ticker.plist +``` + +This will unload the launch agent and will stop the ticker from running. Remember the signal handling we added? This is the case where it’s being used! Also, we could have automated the (un)loading of the file via the CLI tool but for simplicity, we left it out. You can try to improve the CLI tool by making it a bit smarter with subcommands and flags, as a follow-up exercise from this tutorial. + +Finally, if you decide to completely delete the launch agent, you can remove the`.plist` file: + +``` +rm ~/Library/LaunchAgents/com.ieftimov.ticker.plist +``` + +### In closing + +As part of this (quite long!) article, we saw how we can work with `launchd` and Golang. We took a detour, like learning about `launchd` and `launchctl`, generating XML files using the `text/template` package, we took a look at OS signals and how we can gracefully shutdown a Go program by handling the `SIGINT` and `SIGTERM`signals. There was quite a bit to learn and see, but we got to the end. + +Of course, we only scratched the surface with this article. For example, `launchd` is quite an interesting tool. You can use it also like `crontab` because it allows running programs at explicit time/date combinations or on specific days. Or, for example, the XML template can be embedded in the program binary using tools like [`go-bindata`][10], instead of hardcoding it in a function. Also, you explore more about signals, how they work and how Go implements these low-level primitives so you can use them with ease in your programs. The options are plenty, feel free to explore! + +If you have found any mistakes in the article, feel free to drop a comment below - I will appreciate it a ton. I find learning through teaching (blogging) a very pleasant experience and would like to have all the details fully correct in my posts. + +-------------------------------------------------------------------------------- + +作者简介： + +Backend engineer, interested in Ruby, Go, microservices, building resilient architectures and solving challenges at scale. I coach at Rails Girls in Amsterdam, maintain a list of small gems and often contribute to Open Source. +This is where I write about software development, programming languages and everything else that interests me. + +--------------------- + + +via: https://ieftimov.com/create-manage-macos-launchd-agents-golang + +作者：[Ilija Eftimov ][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://ieftimov.com/about +[1]:https://ieftimov.com/en.wikipedia.org/wiki/Reverse_domain_name_notation +[2]:https://godoc.org/os#Exit +[3]:https://launchd.info/ +[4]:https://godoc.org/text/template +[5]:https://godoc.org/os#Getenv +[6]:https://en.wikipedia.org/wiki/Signal_(IPC) +[7]:https://golang.org/x/sys +[8]:https://docs.google.com/document/d/1QXzI9I1pOfZPujQzxhyRy6EeHYTQitKKjHfpq0zpxZs/edit +[9]:https://golang.org/x/sys +[10]:https://github.com/jteeuwen/go-bindata \ No newline at end of file diff --git a/sources/tech/20180130 Graphics and music tools for game development.md b/sources/tech/20180130 Graphics and music tools for game development.md new file mode 100644 index 0000000000..7414e89704 --- /dev/null +++ b/sources/tech/20180130 Graphics and music tools for game development.md @@ -0,0 +1,179 @@ +Graphics and music tools for game development +====== + +![](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/OSDC_Life_opengame.png?itok=JPxruL3k) + +In early October, our club, [Geeks and Gadgets][1] from Marshall University, participated in the inaugural [Open Jam][2], a game jam that celebrated the best of open source tools. Game jams are events where participants work as teams to develop computer games for fun. Jams tend to be very short--only three days long--and very exhausting. Opensource.com [announced][3] Open Jam in late August, and more than [three dozen games][4] were entered into the competition. + +Our club likes to create and use open source software in our projects, so Open Jam was naturally the jam we wanted to participate in. Our submission was an experimental game called [Mark My Words][5]. We used a variety of free and open source (FOSS) tools to develop it; in this article we'll discuss some of the tools we used and potential stumbling blocks to be aware of. + +### Audio tools + +#### MilkyTracker + +[MilkyTracker][6] is one of the best software packages available for composing old-style video game music. It is an example of a [music tracker][7], a powerful MOD and XM file creator with a characteristic grid-based pattern editor. We used it to compose most of the musical pieces in our game. One of the great things about this program is that it consumed much less disk space and RAM than most of our other tools. Even so, MilkyTracker is still extremely powerful. + +![](https://opensource.com/sites/default/files/u128651/mtracker.png) + +The user interface took a while to get used to, so here are some pointers for any musician who wants to try out MilkyTracker: + + * Go to Config > Misc. and set the edit mode control style to "MilkyTracker." This will give you modern keyboard shortcuts for almost everything + * Undo with Ctrl+Z + * Redo with Ctrl+Y + * Toggle pattern-edit mode with the Spacebar + * Delete the previous note with the Backspace key + * Insert a row with the Insert key + * By default, a note will continue playing until it is replaced on that channel. You can end a note explicitly by inserting a KeyOff note with the backquote (`) key + * You will have to create or find samples before you can start composing. We recommend finding [Creative Commons][8] licensed samples at websites such as [Freesound][9] or [ccMixter][10] + + + +In addition, keep the [MilkyTracker documentation page][11] handy. It contains links to numerous tutorials and manuals. A good starting point is the [MilkyTracker Guide][12] on the project's wiki. + +#### LMMS + +Two of our musicians used the versatile and modern music creation tool [LMMS][13]. It comes with a library of cool samples and effects, plus a variety of flexible plugins for generating unique sounds. The learning curve for LMMS was surprisingly low, in part due to the nice beat/bassline editor. + +![](https://opensource.com/sites/default/files/u128651/lmms_plugins.png) + +We have one suggestion for musicians trying out LMMS: Use the plugins. For [chiptune][14]-style music, we recommend [sfxr][15], [BitInvader][16], and [FreeBoy][17]. For other styles, [ZynAddSubFX][18] is a good choice. It comes with a wide range of synthesized instruments that can be altered however you see fit. + +### Graphics tools + +#### Tiled + +[Tiled][19] is a popular tilemap editor in open source game development. We used it to assemble consistent, retro-looking backgrounds for our in-game scenes. + +![](https://opensource.com/sites/default/files/u128651/tiled.png) + +Tiled can export maps as XML, JSON, or as flattened images. It is stable and cross-platform. + +One of Tiled's features, which we did not use during the jam, allows you to define and place arbitrary game objects, such as coins and powerups, onto the map. All you have to do is load the object's graphics as a tileset, then place them using Insert Tile. + +Overall, Tiled is a stellar piece of software that we recommend for any project that needs a map editor. + +#### Piskel + +[Piskel][20] is a pixel art editor whose source code is licensed under the [Apache License, Version 2.0][21]. We used Piskel for almost all our graphical assets during the jam, and we will certainly be using it in future projects as well. + +Two features of Piskel that helped us immensely during the jam are onion skin and spritesheet exporting. + +##### Onion skin + +The onion skin feature will make Piskel show a ghostly overlay of the previous and next frames of your animation as you edit, like this: + +![](https://opensource.com/sites/default/files/u128651/onionshow.gif) + +Onion skin is handy because it serves as a drawing guide and helps you maintain consistent shapes and volumes on your characters throughout the animation process. To enable it, just click the onion-shaped icon underneath the preview window on the top-right of the screen. + +![](https://opensource.com/sites/default/files/u128651/onionenable.png) + +##### Spritesheet exporting + +Piskel's ability to export animations as a spritesheet was also very helpful. A spritesheet is a single raster image that contains all the frames of an animation. For example, here is a spritesheet we exported from Piskel: + +![](https://opensource.com/sites/default/files/u128651/sprite-artist.png) + +The spritesheet consists of two frames. One frame is in the top half of the image and the other frame is in the bottom half of the image. Spritesheets greatly simplify a game's code by enabling an entire animation to be loaded from a single file. Here is an animated version of the above spritesheet: + +![](https://opensource.com/sites/default/files/u128651/sprite-artist-anim.gif) + +##### Unpiskel.py + +There were several times during the jam when we wanted to batch convert Piskel files into PNGs. Since the Piskel file format is based on JSON, we wrote a small GPLv3-licensed Python script called [unpiskel.py][22] to do the conversion. + +It is invoked like this: +``` + + +python unpiskel.py input.piskel +``` + +The script will extract the PNG data frames and layers from a Piskel file (here `input.piskel`) and store them in their own files. The files follow the pattern `NAME_XX_YY.png` where `NAME` is the truncated name of the Piskel file, `XX` is the frame number, and `YY` is the layer number. + +Because the script can be invoked from a shell, it can be used on a whole list of files. +``` +for f in *.piskel; do python unpiskel.py "$f"; done +``` + +### Python, Pygame, and cx_Freeze + +#### Python and Pygame + +We used the [Python][23] language to make our game. It is a scripting language that is commonly used for text processing and desktop app development. It can also be used for game development, as projects like [Angry Drunken Dwarves][24] and [Ren'Py][25] have shown. Both of these projects use a Python library called [Pygame][26] to display graphics and produce sound, so we decided to use this library in Open Jam, too. + +Pygame turned out to be both stable and featureful, and it was great for the arcade-style game we were creating. The library's speed was fast enough at low resolutions, but its CPU-only rendering starts to slow down at higher resolutions. This is because Pygame does not use hardware-accelerated rendering. However, the infrastructure is there for developers to take full advantage of OpenGL. + +If you're looking for a good 2D game programming library, Pygame is one to keep your eye on. Its website has [a good tutorial][27] to get started. Be sure to check it out! + +#### cx_Freeze + +Prepping our game for distribution was interesting. We knew that Windows users were unlikely to have a Python installation, and asking them to install it would have been too much. On top of that, they would have had to also install Pygame, which is not an intuitive task on Windows. + +One thing was clear: We had to put our game into a more convenient form. Many of the other Open Jam participants used the proprietary game engine Unity, which enabled their games to be played in the web browser. This made them extremely convenient to play. Convenience was one thing our game didn't have even a sliver of. But, thanks to a vibrant Python ecosystem, we had options. Tools exist to help Python programmers prepare their programs for distribution on Windows. The two that we considered were [cx_Freeze][28] and [Pygame2exe][29] (which uses [py2exe][30]). We decided on cx_Freeze because it was cross-platform. + +In cx_Freeze, you can pack a single-script game for distribution just by running a command like this in the shell: +``` +cxfreeze main.py --target-dir dist +``` + +This invocation of `cxfreeze` will take your script (here `main.py`) and the Python interpreter on your system and bundle them up into the `dist` directory. Once this is done, all you have to do is manually copy your game's data files into the `dist` directory. You will find that the `dist` directory contains an executable file that can be run to start your game. + +There is a more involved way to use cx_Freeze that allows you to automate the copying of data files, but we found the straightforward invocation of `cxfreeze` to be good enough for our needs. Thanks to this tool, we made our game a little more convenient to play. + +### Celebrating open source + +Open Jam is important because it celebrates the open source model of software development. This is an opportunity to analyze the current state of open source tools and what we need to work on in the future. Game jams are perhaps the best time for game devs to try to push their tools to the limit, to learn what must be improved for the good of future game devs. + +Open source tools enable people to explore their creativity without compromising their freedom and without investing money upfront. Although we might not become professional game developers, we were still able to get a small taste of it with our short, experimental game called [Mark My Words][5]. It is a linguistically themed game that depicts the evolution of a fictional writing system throughout its history. There were many other delightful submissions to Open Jam, and they are all worth checking out. Really, [go look][31]! + +Before closing, we would like to thank all the [club members who participated][32] and made this experience truly worthwhile. We would also like to thank [Michael Clayton][33], [Jared Sprague][34], and [Opensource.com][35] for hosting Open Jam. It was a blast. + +Now, we have some questions for readers. Are you a FOSS game developer? What are your tools of choice? Be sure to leave a comment below! + +-------------------------------------------------------------------------------- + +via: https://opensource.com/article/18/1/graphics-music-tools-game-dev + +作者：[Charlie Murphy][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://opensource.com/users/rsg167 +[1]:http://mugeeks.org/ +[2]:https://itch.io/jam/open-jam-1 +[3]:https://opensource.com/article/17/8/open-jam-announcement +[4]:https://opensource.com/article/17/11/open-jam +[5]:https://mugeeksalpha.itch.io/mark-omy-words +[6]:http://milkytracker.titandemo.org/ +[7]:https://en.wikipedia.org/wiki/Music_tracker +[8]:https://creativecommons.org/ +[9]:https://freesound.org/ +[10]:http://ccmixter.org/view/media/home +[11]:http://milkytracker.titandemo.org/documentation/ +[12]:https://github.com/milkytracker/MilkyTracker/wiki/MilkyTracker-Guide +[13]:https://lmms.io/ +[14]:https://en.wikipedia.org/wiki/Chiptune +[15]:https://github.com/grimfang4/sfxr +[16]:https://lmms.io/wiki/index.php?title=BitInvader +[17]:https://lmms.io/wiki/index.php?title=FreeBoy +[18]:http://zynaddsubfx.sourceforge.net/ +[19]:http://www.mapeditor.org/ +[20]:https://www.piskelapp.com/ +[21]:https://github.com/piskelapp/piskel/blob/master/LICENSE +[22]:https://raw.githubusercontent.com/MUGeeksandGadgets/MarkMyWords/master/tools/unpiskel.py +[23]:https://www.python.org/ +[24]:https://www.sacredchao.net/~piman/angrydd/ +[25]:https://renpy.org/ +[26]:https://www.Pygame.org/ +[27]:http://Pygame.org/docs/tut/PygameIntro.html +[28]:https://anthony-tuininga.github.io/cx_Freeze/ +[29]:https://Pygame.org/wiki/Pygame2exe +[30]:http://www.py2exe.org/ +[31]:https://itch.io/jam/open-jam-1/entries +[32]:https://github.com/MUGeeksandGadgets/MarkMyWords/blob/3e1e8aed12ebe13acccf0d87b06d4f3bd124b9db/README.md#credits +[33]:https://twitter.com/mwcz +[34]:https://twitter.com/caramelcode +[35]:https://opensource.com/ diff --git a/sources/tech/20180130 Install AWFFull web server log analysis application on ubuntu 17.10.md b/sources/tech/20180130 Install AWFFull web server log analysis application on ubuntu 17.10.md new file mode 100644 index 0000000000..03e15878b9 --- /dev/null +++ b/sources/tech/20180130 Install AWFFull web server log analysis application on ubuntu 17.10.md @@ -0,0 +1,95 @@ +Install AWFFull web server log analysis application on ubuntu 17.10 +====== + + +AWFFull is a web server log analysis program based on "The Webalizer".AWFFull produces usage statistics in HTML format for viewing with a browser. The results are presented in both columnar and graphical format, which facilitates interpretation. Yearly, monthly, daily and hourly usage statistics are presented, along with the ability to display usage by site, URL, referrer, user agent (browser), user name,search strings, entry/exit pages, and country (some information may not be available if not present in the log file being processed). + + + +AWFFull supports CLF (common log format) log files, as well as Combined log formats as defined by NCSA and others, and variations of these which it attempts to handle intelligently. In addition, AWFFull also supports wu-ftpd xferlog formatted log files, allowing analysis of ftp servers, and squid proxy logs. Logs may also be compressed, via gzip. + +AWFFull is a web server log analysis program based on "The Webalizer".AWFFull produces usage statistics in HTML format for viewing with a browser. The results are presented in both columnar and graphical format, which facilitates interpretation. Yearly, monthly, daily and hourly usage statistics are presented, along with the ability to display usage by site, URL, referrer, user agent (browser), user name,search strings, entry/exit pages, and country (some information may not be available if not present in the log file being processed).AWFFull supports CLF (common log format) log files, as well as Combined log formats as defined by NCSA and others, and variations of these which it attempts to handle intelligently. In addition, AWFFull also supports wu-ftpd xferlog formatted log files, allowing analysis of ftp servers, and squid proxy logs. Logs may also be compressed, via gzip. + +If a compressed log file is detected, it will be automatically uncompressed while it is read. Compressed logs must have the standard gzip extension of .gz. + +### Changes from Webalizer + +AWFFull is based on the Webalizer code and has a number of large and small changes. These include: + +o Beyond the raw statistics: Making use of published formulae to provide additional insights into site usage. + +o GeoIP IP Address look-ups for more accurate country detection. + +o Resizable graphs. + +o Integration with GNU gettext allowing for ease of translations.Currently 32 languages are supported. + +o Display more than 12 months of the site history on the front page. + +o Additional page count tracking and sort by same. + +o Some minor visual tweaks, including Geolizer's use of Kb, Mb etc for Volumes. + +o Additional Pie Charts for URL counts, Entry and Exit Pages, and Sites. + +o Horizontal lines on graphs that are more sensible and easier to read. + +o User Agent and Referral tracking is now calculated via PAGES not HITS. + +o GNU style long command line options are now supported (eg --help). + +o Can choose what is a page by excluding "what isn't" vs the original "what is" method. + +o Requests to the site being analysed are displayed with the matching referring URL. + +o A Table of 404 Errors, and the referring URL can be generated. + +o An external CSS file can be used with the generated html. + +o Manual performance optimisation of the config file is now easier with a post analysis summary output. + +o Specified IP's & Addresses can be assigned to a given country. + +o Additional Dump options for detailed analysis with other tools. + +o Lotus Domino v6 logs are now detected and processed. + +**Install awffull on ubuntu 17.10** + +> sudo apt-get install awffull + +### Configuring AWFFULL + +You have to edit awffull config file at /etc/awffull/awffull.conf. If you have multiple virtual websites running in the same machine, you can make several copies of the default config file. + +> sudo vi /etc/awffull/awffull.conf + +Make sure the following lines are there + +> LogFile /var/log/apache2/access.log.1 +> OutputDir /var/www/html/awffull + +Save and exit the file + +You can run the awffull config using the following command + +> awffull -c [your config file name] + +This will create all the required files under /var/www/html/awffull directory so you can access your webserver stats using http://serverip/awffull/ + +You should see similar to the following screen + +If you have more site and you can automate the process using shell script and cron job. + + +-------------------------------------------------------------------------------- + +via: http://www.ubuntugeek.com/install-awffull-web-server-log-analysis-application-on-ubuntu-17-10.html + +作者：[ruchi][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://www.ubuntugeek.com/author/ubuntufix diff --git a/sources/tech/20180130 Introduction to AWS for Data Scientists.md b/sources/tech/20180130 Introduction to AWS for Data Scientists.md new file mode 100644 index 0000000000..ada3585745 --- /dev/null +++ b/sources/tech/20180130 Introduction to AWS for Data Scientists.md @@ -0,0 +1,212 @@ +Introduction to AWS for Data Scientists +====== +![sky-690293_1920][1] + +These days, many businesses use cloud based services; as a result various companies have started building and providing such services. Amazon [began the trend][2], with Amazon Web Services (AWS). While AWS began in 2006 as a side business, it now makes [$14.5 billion in revenue each year][3]. + +Other leaders in this area include: + + * Google--Google Cloud Platform (GCP) + * Microsoft--Azure Cloud Services + * IBM--IBM Cloud + + + +Cloud services are useful to businesses of all sizes--small companies benefit from the low cost, as compared to buying servers. Larger companies gain reliability and productivity, with less cost, since the services run on optimum energy and maintenance. + +These services are also powerful tools that you can use to ease your work. Setting up a Hadoop cluster to work with Spark manually could take days if it's your first time, but AWS sets that up for you in minutes. + +We are going to focus on AWS here because it comes with more products relevant to data scientists. In general, we can say familiarity with AWS helps data scientists to: + + 1. Prepare the infrastructure they need for their work (e.g. Hadoop clusters) with ease + 2. Easily set up necessary tools (e.g. Spark) + 3. Decrease expenses significantly--such as by paying for huge Hadoop clusters only when needed + 4. Spend less time on maintenance, as there's no need for tasks like manually backing up data + 5. Develop products and features that are ready to launch without needing help from engineers (or, at least, needing very little help) + + + +In this post, I'll give an overview of useful AWS services for data scientists -- what they are, why they're useful, and how much they cost. + +### Elastic Compute Cloud (EC2) + +Many other AWS services are built around EC2, making it a core piece of AWS. EC2s are in fact (virtual) servers that you can rent from Amazon and set up or run any program/application on it. These servers come in different operating systems and Amazon charges you based on the computing power and capacity of the server (i.e. Hard Drive capacity, CPU, Memory, etc.) and the duration the server been up. + +#### EC2 benefits + +For example, you can rent a Linux or Windows server with computation power and storage capacity that fits your specific needs and Amazon charges you based on these specifications and the duration you use the server. Note that previously AWS charged at least for one hour for each instance you run, but they recently changed their policy to [per-second billing][4]. + +One of the good things about EC2 is its scalability--by changing memory, number of vCPUs, bandwidth, and so on, you can easily scale your system up or down. Therefore, if you think a system doesn't have enough power for running a specific task or a calculation in your project is taking too long, you can scale up to finish your work and later scale down again to reduce the cost. EC2 is also very reliable, since Amazon takes care of the maintenance. + +#### EC2 cost + +EC2 instances are relatively low-cost, and there are different types of instances for different use cases. For example, there are instances that are optimized for computation and those have relatively lower cost on CPU usage. Or those optimized for memory have lower cost on memory usage. + +To give you an idea on EC2 cost, a general purpose medium instance with 2 vCPUs and 4 GIG of memory (at the time of writing this article) costs $0.0464 per hour for a linux server, see [Amazon EC2 Pricing][5] for prices and more information. AWS also now has [spot instance pricing][6], which calculates the price based on supply/demand at the time and provides up to a 90% discount for short term usages depending on the time you want to use the instance. For example, the same instance above costs $0.0173 per hour on spot pricing plan. + +Note that you have to add storage costs to the above as well. Most EC2 instances use Elastic Block Store (EBS) systems, which cost around $0.1/GIG/month; see the prices [here][7]. [Storage optimized instances][8] use Solid State Drive (SSD) systems, which are more expensive. + +![Ec2cost][9] + +EBS acts like an external hard drive. You can attach it to an instance, de-attach it, and re-attach it to another instance. You can also stop or terminate an instance after your work is done and not pay for the instance when it is idle. + +If you stop an instance, AWS will still keep the EBS live and as a result the data you have on the hard drive will remain intact (it's like powering off your computer). Later you can restart stopped instances and get access to the data you generated, or even tools you installed there in the previous sessions. However, when you stop an instance instead of terminating it, Amazon will still charge you for the attached EBS (~$0.1/GIG/month). If you terminate the instance, the EBS will get cleaned so you will lose all the data on that instance, but you no longer need to pay for the EBS. + +If you need to keep the data on EBS for your future use (let's say you have custom tools installed on that instance and you don't want to redo your work again later) you can make a snapshot of the EBS and can later restore it in a new EBS and attach it to a new instance. + +Snapshots get stored on S3 (Amazon's cheap storage system; we will get to it later) so it will cost you less ($0.05 per GB-month) to keep the data in EBS like that. However, it takes time (depending on the size of the EBS) to get snapshot and restoring it. Besides, reattaching a restored EBS to EC2 instance is not that straight forward, so it only make sense to use a snapshot like that if you know you are not going to use that EBS for a while. + +Note that to scale an instance up or down, you have to first stop the instance and then change the instance specifications. You can't decrease the EBS size, only increase it, and it's more difficult. You have to: + + 1. Stop the instance + 2. Make a snapshot out of the EBS + 3. Restore the snapshot in an EBS with the new size + 4. De-attach previous EBS + 5. Attach the new one. + + + +### Simple Storage Service (S3) + +S3 is AWS object (file) storage service. S3 is like Dropbox or Google drive, but way more scalable and is made particularly to work with codes and applications. + +S3 doesn't provide a user friendly interface since it is designed to work with online applications, not the end user. Therefore, working with S3 through APIs is easier than through its web console and there are many libraries and APIs developed (in various languages) to work with this service. For example, [Boto3][10] is a S3 library written in Python (in fact Boto3 is suitable for working with many other AWS services as well) . + +S3 stores files based on `bucket`s and `key`s. Buckets are similar to root folders, and keys are similar to subfolders and files. So if you store a file named `my_file.txt` on s3 like `myproject/mytextfiles/my_file.txt`, then "myproject" is the bucket you are using and then `mytextfiles/my_file.txt` is the key to that file. This is important to know since APIs will ask for the bucket and key separately when you want to retrieve your file from s3. + +#### S3 benefits + +There is no limit on the size of data you can store on S3--you just have to pay for the storage based on the size you need per month. + +S3 is also very reliable and "[it is designed to deliver 99.999999999% durability][11]". However, the service may not be always up. On February 28th, 2017 some of s3 servers went down for couple of hours and that disrupted many applications such as Slack, Trello, etc. see [these][12] [articles][13] for more information on this incident. + +#### S3 cost + +The cost is low, starting at $0.023 per GB per month for standard access, if you want to get access to these files regularly. It could go down even lower if you don't need to load data too frequently. See [Amazon S3 Pricing][14] for more information. + +AWS may charge you for other S3 related actions such as requests through APIs, but the cost for those are insignificant (less than $0.05 per 1,000 requests in most cases). + +### Relational Database Service (RDS) + +AWS RDS is a relational database service in the cloud. RDS currently supports SQL Server, MySQL, PostgreSQL, ORACLE, and a couple of other SQL-based frameworks. AWS sets up the system you need and configures the parameters so you can have a relational database up and running in minutes. RDS also handles backup, recovery, software patching, failure detection, and repairs by itself so you don't need to maintain the system. + +#### RDS benefits + +RDS is scalable, both computing power and the storage capacity can be scaled up or down easily. RDS system runs on EC2 servers (as I mentioned EC2 servers are the core of most of AWS services, including RDS service) so by computing power here we mean the computing power of the EC2 server our RDS service is running on, and you can scale up the computing power of this system up to 32 vCPUs and 244 GiB of RAM and changing the scale would not take more than few minutes. + +Scaling the storage requirements up or down is also possible. [Amazon Aurora][15] is a version of MySQL and PostgreSQL with some additional features, and can automatically scale up when more storage space is needed (you can define the maximum). The MySQL, MariaDB, Oracle, and PostgreSQL engines allow you to scale up on the fly without downtime. + +#### RDS cost + +The [cost of RDS servers][16] is based on three factors: computational power, storage, and data transfer. + +![RDSpricing][17] + +For example, a PostgreSQL system with medium computational power (2 vCPUs and 8 gig of memory) costs $0.182 per hour; you can pay less if you go under a one- or three-year contract. + +For storage, there are a [variety of options and prices][18]. If you choose single availability zone General Purpose SSD Storage (gp2), a good option for data scientists, the cost for a server in north Virginia at the time of writing this article is $0.115 per GB-month, and you can select from 5 GB to 16 TB of SSD. + +For data transfer, the cost varies a little based on the source and destination of data (one of which is RDS). For example, all data transferred from the internet into RDS is free. The first gig of data transferred from RDS to the internet is free as well, and for the next 10 terabytes of data in a month it costs $0.09 per GB; the cost decreases for transfering more data than that. + +### Redshift + +Redshift is Amazon's data warehouse service; it is a distributed system (something like the Hadoop framework) which lets you store huge amounts of data and get queries. The difference between this service and RDS is its high capacity and ability to work with big data (terabytes and petabytes). You can use simple SQL queries on Redshift as well. + +Redshift works on a distributed framework--data is distributed on different nodes (servers) connected on a cluster. Simply put, queries on a distributed system run in parallel on all the nodes and then the results get collected from each node and get summarized. + +#### Redshift benefits + +Redshift is highly scalable, meaning in theory (depending on the query, network structure and design, service specification, etc.) the speed of getting query out of 1 terabyte of data and 1 petabyte of data can match by scaling up (adding more cluster to) the system. + +When you create a table on Redshift, you can choose one of three distribution styles: EVEN, KEY, or ALL. + + * EVEN means the table rows will get distributed over all the nodes evenly. Then queries involving that table get distributed over the cluster and run in parallel, summarized at the end. Per Amazon's documentation, "[EVEN distribution is appropriate when a table does not participate in joins][19]". + + * ALL means that on each node there will be a copy of this table, so if you query for a join on that table, the table is already there on all the nodes and there is no need for copying the required data across the network from node to node. The problem is "[ALL distribution multiplies the storage required by the number of nodes in the cluster, and so it takes much longer to load, update, or insert data into multiple tables][19]". + + * In the KEY style, distribution rows of the table are distributed based on the values in one column, in an attempt to keep the rows with the same value of that column in the same node. Physically storing matching values on the same nodes make joining on that specific column faster in parallel systems, see more information [here][19]. + + + + +#### Redshift cost + +Redshift has two types of instances: Dense Compute or Dense Storage. Dense Compute is optimized for fast querying and it is cost effective for less than 500GB of data in size (~$5,500/TB/Year for a three-year contract with partial upfront). + +Dense Storage is optimized for high size storage (~$1,000/TB/Year for a three-year contract with partial upfront) and is cost effective for +500GB, but it is slower. You can find more general pricing [here][20]. + +You can also save a large amount of data on S3 and use [Amazon Redshift Spectrum][21] to run SQL query on that data. For Redshift Spectrum, AWS charges you by the number of bytes scanned by Redshift Spectrum per query; and $5 per terabyte of data scanned (10 megabyte minimum per query). + +### Elastic MapReduce (EMR) + +EMR is suitable for setting up Hadoop clusters with Spark and other distributed type applications. A Hadoop cluster can be used as a compute engine or a (distributed) storage system. However, if the data is so big that you need a distributed system to handle it, Redshift is more suitable and way cheaper than storing in EMR. + +There are three types of [nodes][22] on a cluster: + + * The master node (you only have one) is responsible for managing the cluster. It distributes the workloads to the core and task nodes, tracks the status of tasks, and monitors the health of the cluster. + * Core nodes run tasks and store the data. + * Task nodes can only run tasks. + + + +#### EMR benefits + +Since you can set EMR to install Apache Spark, this service is good for for cleaning, reformatting, and analyzing big data. You can use EMR on-demand, meaning you can set it to grab the code and data from a source (e.g. S3 for the code, and S3 or RDS for the data), run the task on the cluster, and store the results somewhere (again s3, RDS, or Redshift) and terminate the cluster. + +By using the service in such a way, you can reduce the cost of your cluster significantly. In my opinion, EMR is one of the most useful AWS services for data scientists. + +To setup an EMR cluster, you need to first configure applications you want to have on the cluster. Note that different versions of EMR come with different versions of the applications. For example, if you configure EMR version 5.10.0 to install Spark, the default version of the Spark for this version is 2.2.0. So if your code works only on Spark 1.6, you need to run EMR on the 4.x version. EMR will set up the network and configures all the nodes on the cluster along with needed tools. + +An EMR cluster comes with one master instance and a number of core nodes (slave instances). You can choose the number of core nodes, and can even select to have no core node and only use the master server for your work. Like other services, you can choose the computational power of the servers and the storage size available on each node. You can use autoscale option for your core nodes, meaning you can add rules to the system to add/remove core node (up to a maximum number you choose) if needed while running your code. See [Using Automatic Scaling in Amazon EMR][23] for more information on auto scaling. + +#### EMR pricing + +EMR pricing is based on the computational power you choose for different instances (master, core and task nodes). Basically, it is the cost of the EC2 servers plus the cost of EMR. You can find detailed pricing [here][24]. + +![EMRpricing][25] + +### Conclusion + +I have developed many end-to-end data-driven products (including reporting, machine learning models, and product health checking systems) for our company using Python and Spark on AWS, which later became good sources of income for the company. + +Experience working with cloud services, especially a well-known one like AWS, is a huge plus in your data scientist career. Many companies depend on these services now and use them constantly, so you being familiar with these services will give them the confidence that you need less training to get on board. With more and more people moving into data science, you want your resume to stand out as much as possible. + +Do you have cloud tips to add? [Let us know][26]. + +-------------------------------------------------------------------------------- + +via: https://www.dataquest.io/blog/introduction-to-aws-for-data-scientists/ + +作者：[Read More][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.dataquest.io/blog/author/armin/ +[1]:/blog/content/images/2018/01/sky-690293_1920.jpg +[2]:http://www.computerweekly.com/feature/A-history-of-cloud-computing +[3]:https://www.forbes.com/sites/bobevans1/2017/07/28/ibm-beats-amazon-in-12-month-cloud-revenue-15-1-billion-to-14-5-billion/#53c3e14c39d6 +[4]:https://aws.amazon.com/blogs/aws/new-per-second-billing-for-ec2-instances-and-ebs-volumes/ +[5]:https://aws.amazon.com/ec2/pricing/on-demand/ +[6]:https://aws.amazon.com/ec2/spot/pricing/ +[7]:https://aws.amazon.com/ebs/pricing/ +[8]:https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/storage-optimized-instances.html +[9]:/blog/content/images/2018/01/Ec2cost.png +[10]:https://boto3.readthedocs.io +[11]:https://aws.amazon.com/s3/ +[12]:https://aws.amazon.com/message/41926/ +[13]:https://venturebeat.com/2017/02/28/aws-is-investigating-s3-issues-affecting-quora-slack-trello/ +[14]:https://aws.amazon.com/s3/pricing/ +[15]:https://aws.amazon.com/rds/aurora/ +[16]:https://aws.amazon.com/rds/postgresql/pricing/ +[17]:/blog/content/images/2018/01/RDSpricing.png +[18]:https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Storage.html +[19]:http://docs.aws.amazon.com/redshift/latest/dg/c_choosing_dist_sort.html +[20]:https://aws.amazon.com/redshift/pricing/ +[21]:https://aws.amazon.com/redshift/spectrum/ +[22]:http://docs.aws.amazon.com/emr/latest/DeveloperGuide/emr-nodes.html +[23]:https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-automatic-scaling.html +[24]:https://aws.amazon.com/emr/pricing/ +[25]:/blog/content/images/2018/01/EMRpricing.png +[26]:https://twitter.com/dataquestio diff --git a/sources/tech/20180130 Linux Kernel 4.15 An Unusual Release Cycle.md b/sources/tech/20180130 Linux Kernel 4.15 An Unusual Release Cycle.md new file mode 100644 index 0000000000..062cd3c3ca --- /dev/null +++ b/sources/tech/20180130 Linux Kernel 4.15 An Unusual Release Cycle.md @@ -0,0 +1,62 @@ +Linux Kernel 4.15: 'An Unusual Release Cycle' +============================================================ + + +![Linux](https://www.linux.com/sites/lcom/files/styles/rendered_file/public/background-penguin.png?itok=g8NBQs24 "Linux") +Linus Torvalds released version 4.15 of the Linux Kernel on Sunday, a week later than originally scheduled. Learn about key updates in this latest release.[Creative Commons Zero][1]Pixabay + +Linus Torvalds [released version 4.15 of the Linux Kernel][7] on Sunday, again, and for a second version in a row, a week later than scheduled. The culprits for the late release were the Meltdown and Spectre bugs, as these two vulnerabilities forced developers to submit major patches well into what should have been the last cycle. Torvalds was not comfortable rushing the release, so he gave it another week. + +Unsurprisingly, the first big bunch of patches worth mentioning were those designed to sidestep [Meltdown and Spectre][8]. To avoid Meltdown, a problem that affects Intel chips, [developers have implemented  _Page Table Isolation_  (PTI)][9] for the x86 architecture. If for any reason you want to turn this off, you can use the `pti=off` kernel boot option. + +Spectre v2 affects both Intel and AMD chips and, to avoid it, [the kernel now comes with the  _retpoline_  mechanism][10]. Retpoline requires a version of GCC that supports the `-mindirect-branch=thunk-extern` functionality. As with PTI, the Spectre-inhibiting mechanism can be turned of. To do so, use the `spectre_v2=off` option at boot time. Although developers are working to address Spectre v1, at the moment of writing there is still not a solution, so there is no patch for this bug in 4.15. + +The solution for Meltdown on ARM has also been pushed to the next development cycle, but there is [a remedy for the bug on PowerPC with the  _RFI flush of L1-D cache_ feature][11] included in this release. + +An interesting side affect of all of the above is that new kernels now come with a  _/sys/devices/system/cpu/vulnerabilities/_  virtual directory. This directory shows the vulnerabilities affecting your CPU and the remedies being currently applied. + +The issues with buggy chips (and the manufacturers that keep things like this secret) has revived the call for the development of viable open source alternatives. This brings us to the partial support for [RISC-V][12] chips that has now been merged into the mainline kernel. RISC-V is an open instruction set architecture that allows manufacturers to create their own implementation of RISC-V chips, and it has resulted in several open sourced chips. While RISC-V chips are currently used mainly in embedded devices, powering things like smart hard disks or Arduino-like development boards, RISC-V proponents argue that the architecture is also well-suited for use on personal computers and even in multi-node supercomputers. + +[The support for RISC-V][13], as mentioned above, is still incomplete, and includes the architecture code but no device drivers. This means that, although a Linux kernel will run on RISC-V, there is no significant way to actually interact with the underlying hardware. That said, RISC-V is not vulnerable to any of the bugs that have dogged other closed architectures, and development for its support is progressing at a brisk pace, as [the RISC-V Foundation has the support of some of the industries biggest heavyweights][14]. + +### Other stuff that's new in kernel 4.15 + +Torvalds has often declared he likes things boring. Fortunately for him, he says, apart from the Spectre and Meltdown messes, most of the other things that happened in 4.15 were very much run of the mill, such as incremental improvements for drivers, support for new devices, and so on. However, there were a few more things worth pointing out: + +* [AMD got support for Secure Encrypted Virtualization][3]. This allows the kernel to fence off the memory a virtual machine is using by encrypting it. The encrypted memory can only be decrypted by the virtual machine that is using it. Not even the hypervisor can see inside it. This means that data being worked on by VMs in the cloud, for example, is safe from being spied on by any other process outside the VM. + +* AMD GPUs get a substantial boost thanks to [the inclusion of  _display code_][4] . This gives mainline support to Radeon RX Vega and Raven Ridge cards and also implements HDMI/DP audio for AMD cards. + +* Raspberry Pi aficionados will be glad to know that [the 7'' touchscreen is now natively supported][5], which is guaranteed to lead to hundreds of fun projects. + +To find out more, you can check out the write-ups at [Kernel Newbies][15] and [Phoronix][16]. + + _Learn more about Linux through the free ["Introduction to Linux" ][6]course from The Linux Foundation and edX._ + +-------------------------------------------------------------------------------- + +via: https://www.linux.com/blog/intro-to-linux/2018/1/linux-kernel-415-unusual-release-cycle + +作者：[PAUL BROWN ][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.linux.com/users/bro66 +[1]:https://www.linux.com/licenses/category/creative-commons-zero +[2]:https://www.linux.com/files/images/background-penguinpng +[3]:https://git.kernel.org/linus/33e63acc119d15c2fac3e3775f32d1ce7a01021b +[4]:https://git.kernel.org/torvalds/c/f6705bf959efac87bca76d40050d342f1d212587 +[5]:https://git.kernel.org/linus/2f733d6194bd58b26b705698f96b0f0bd9225369 +[6]:https://training.linuxfoundation.org/linux-courses/system-administration-training/introduction-to-linux +[7]:https://lkml.org/lkml/2018/1/28/173 +[8]:https://meltdownattack.com/ +[9]:https://git.kernel.org/linus/5aa90a84589282b87666f92b6c3c917c8080a9bf +[10]:https://git.kernel.org/linus/76b043848fd22dbf7f8bf3a1452f8c70d557b860 +[11]:https://git.kernel.org/linus/aa8a5e0062ac940f7659394f4817c948dc8c0667 +[12]:https://riscv.org/ +[13]:https://git.kernel.org/torvalds/c/b293fca43be544483b6488d33ad4b3ed55881064 +[14]:https://riscv.org/membership/ +[15]:https://kernelnewbies.org/Linux_4.15 +[16]:https://www.phoronix.com/scan.php?page=search&q=Linux+4.15 \ No newline at end of file diff --git a/sources/tech/20180130 Mitigating known security risks in open source libraries.md b/sources/tech/20180130 Mitigating known security risks in open source libraries.md new file mode 100644 index 0000000000..adb1491e7d --- /dev/null +++ b/sources/tech/20180130 Mitigating known security risks in open source libraries.md @@ -0,0 +1,249 @@ +Mitigating known security risks in open source libraries +============================================================ + +>Fixing vulnerable open source packages. + + +![Machine](https://d3tdunqjn7n0wj.cloudfront.net/360x240/machine-2881186_1920-aa3ebed0567d4ab0a107baa640661e35.jpg) + +Machine (source: [Skitterphoto][9]) + + +This is an excerpt from [Securing Open Source Libraries][13], by Guy Podjarny.  +[Read the preceding chapter][14] or [view the full report][15]. + + +### Fixing Vulnerable Packages + +Finding out if you’re using vulnerable packages is an important step, but it’s not the real goal. The real goal is to fix those issues! + +This chapter focuses on all you should know about fixing vulnerable packages, including remediation options, tooling, and various nuances. Note that SCA tools traditionally focused on finding or preventing vulnerabilities, and most put little emphasis on fix beyond providing advisory information or logging an issue. Therefore, you may need to implement some of these remediations yourself, at least until more SCA solutions expand to include them. + +There are several ways to fix vulnerable packages, but upgrading is the best choice. If that is not possible, patching offers a good alternative. The following sections discuss each of these options, and we will later take a look at what you can do in situations where neither of these solutions is possible. + +### Upgrading + +As I’ve previously stated, a vulnerability is a type of bug, and the best way to address a bug is to use a newer version where it is fixed. And so, the best way to fix a vulnerable dependency is to upgrade to a newer version. Statistically, most disclosed vulnerabilities are eventually fixed. In npm, 59% of reported vulnerabilities have a fix. In Maven, 90% are remediable, while that portion is 85% in RubyGems.[1][4] In other words, more often than not, there is a version of your library where the vulnerability is fixed. + +Finding a vulnerable package requires knowledge of which versions are vulnerable. This means that, at the very least, every tool that finds issues can tell which versions are vulnerable, allowing you to look for newer versions of the library and upgrade. Most tools also take the minor extra step of determining the minimal fixed version, and noting it in the advisory. + +Upgrading is therefore the best way to make a vulnerability go away. It’s technically easy (update a manifest or lock file), and it’s something dev teams are very accustomed to doing. That said, upgrading still holds some complexity. + +### Major Upgrades + +While most issues are fixed, very often the fix is only applied to the latest and greatest version of the library. If you’re still using an older version of the library, upgrading may mean switching to a new major version. Major upgrades are typically not backward compatible, introducing more risk and requiring more dev effort. + +Another reason for fixing an issue only in the next major version is that sometimes fixing a vulnerability means reducing functionality. For instance, fixing a certain [XSS vulnerability in a jQuery 2.x codebase][5] requires a change to the way certain selectors are interpreted. The jQuery team determined too many people are relying on this functionality to deem this a non-breaking change, and so only fixed the vulnerability in their 3.x stream. + +For these reasons, a major upgrade can often be difficult, but if you can accept it, it’s still the best way to fix a vulnerability. + +### Indirect Dependency Upgrade + +If you’re consuming a dependency directly, upgrading is relatively straightforward. But what happens when one of your dependencies is the one who pulled in the vulnerable package? Most dependencies are in fact indirect dependencies (a.k.a. transitive dependencies), making upgrades a bit more complex. + +The cleanest way to perform an indirect upgrade is through a direct one. If your app uses `A@1`, which uses a vulnerable `B@1`, it’s possible that upgrading to `A@2` will trigger a downstream upgrade to `B@2` and fix the issue. Applying such an upgrade is easy (it’s essentially a direct upgrade), but discovering  _which_  upgrade to do (and whether one even exists) is time consuming. While not common, some SCA tools can determine and advise on the  _direct_  upgrades you need to make to fix an  _indirect_  vulnerability. If your tooling doesn’t support it, you’ll need to do the searching manually. + +Old vulnerabilities in indirect libraries can often be fixed with a direct upgrade, but such upgrades are frequently unavailable for new issues. When a new vulnerability is disclosed, even if the offending package releases a fix right away, it takes a while for the dependency chain to catch up. If you can’t find a path to an indirect upgrade for a newly disclosed flaw, be sure to recheck frequently as one may show up soon. Once again, some SCA tools will do this monitoring for you and alert you when new remediations are available. + + +![The direct vulnerable EJS can be upgraded, but indirect instance cannot currently be upgraded](https://d3ansictanv2wj.cloudfront.net/sosl_0301-d3ce5b0bf64893e26ee74627bfba5300.png) +Figure 1-1. The direct vulnerable EJS can be upgraded, but indirect instance cannot currently be upgraded + +### Conflicts + +Another potential obstacle to upgrading is a conflict. Many languages, such as Ruby and Python, require dependencies to be global, and clients such as Ruby’s bundler and Python’s pip determine the mix of library versions that can co-exist. As a result, upgrading one library may trigger a conflict with another. While developers are adept at handling such conflicts, there are times when such issues simply cannot be resolved. + +On the positive side, global dependency managers, such as Ruby’s bundler, allow the parent app to add a constraint. For instance, if a downstream `B@1` gem is vulnerable, you can add `B@^2` to your Gemfile, and have bundler sort out the surrounding impact. Adding such constraints is a safe and legitimate solution, as long as your ecosystem tooling can figure out a conflict-free combination of libraries. + +### Is a Newer Version Always Safer? + +The conversation about upgrading begs a question: can a vulnerability also be fixed by downgrading? + +For the most part, the answer is no. Vulnerabilities are bugs, and bugs are typically fixed in a newer version, not an older one. In general, maintaining a good upgrade cadence and keeping your dependencies up to date is a good preventative measure to reduce the risk of vulnerabilities. + +However, in certain cases, code changes or (more often) new features are the ones that trigger a vulnerability. In those cases, it’s indeed possible that downgrading will fix the discovered flaw. The advisory should give you the information you need about which versions are affected by the vulnerability. That said, note that downgrading a package puts you at higher risk of being exposed to new issues, and can make it harder to upgrade when that happens. I suggest you see downgrading as a temporary and rarely used remediation path. + +### There Is No Fixed Version + +Last on the list of reasons preventing you from upgrading to a safe version is such a version not existing in the first place! + +While most vulnerabilities are fixed, many remain unfixed. This is sometimes a temporary situation—for instance, when a vulnerability was made public without waiting for a fix to be released. Other times, it may be a more long-term scenario, as many repositories fall into a poor maintenance state, and don’t fix reported issues nor accept community patches. + +In the following sections I’ll discuss some options for when you cannot upgrade a vulnerability away. + +### Patching + +Despite all the complexity it may involve, upgrading is the best way to fix an issue. However, if you cannot upgrade, patching the vulnerability is the next best option. + +Patching means taking a library as is, including its vulnerabilities, and then modifying it to fix a vulnerability it holds. Patching should apply the minimal set of changes to the library, so as to keep its functionality unharmed and only address the issue at hand. + +Patching inevitably holds a certain amount of risk. When you use a package downloaded millions of time a month, you have some assurance that bugs in it will be discovered, reported, and often fixed. When you download that package and modify it, your version of the code will not be quite as battle tested. + +Patching is therefore an exercise in risk management. What presents a greater risk: having the vulnerability, or applying the patch? For well-managed patches, especially for ones small in scope, I believe it’s almost always better to have a patch than a vulnerability. + +It’s worth noting that patching application dependencies is a relatively new concept, but an old hat in the operating system world. When dealing with operating system dependencies, we’re accustomed to consuming a feed of fixes by running `apt-get upgrade` or an equivalent command, often remaining unaware of which issues we fixed. What most don’t know is that many of the fixes you pull down are in fact back-ported versions of the original OS author code changes, created and tested by Canonical, RedHat, and the like. A safe registry that feeds you the non-vulnerable variants of your dependencies doesn’t exist yet in the application libraries world, but patching is sometimes doable in other ways. + +### Sourcing Patches + +To create a patch, you first need to have a fix for the vulnerability! You could write one yourself, but patches are more often sourced from existing community fixes. + +The first place to look for a patch is a new version of the vulnerable package. Most often the vulnerability  _was_  fixed by the maintainers of the library, but that fix may be in an out-of-reach indirect dependency, or perhaps was only fitted back into the latest major version. Those fixes can be extracted from the original repo and stored into their own patch file, as well as back-ported into older versions if need be. + +Another common source for patches are external pull requests (PRs). Open source maintenance is a complicated topic, and it’s not uncommon for repos to go inactive. In such repos, you may find community pull requests that fix a vulnerability, have been commented on and perhaps vetted by others, but are not merged and published into the main stream. Such PRs are a good starting point—if not the full solution—for creating a patch. For instance, an XSS issue in the popular JavaScript Markdown parsing library marked had an [open fix PR][6] for nearly a year before it was incorporated into a new release. During this period, you could use the fix PR code to patch the issue in your apps. + +Snyk maintains its own set of patches in its [open source database][7]. Most of those patches are captures or back-ports of original fixes, a few are packaged pull requests, and even fewer are written by the Snyk security research team. + +### Depend on GitHub Hash + +In very specific cases, you may be able to patch without storing any code changes. This is only possible if the vulnerable dependency is a direct dependency of your app, and the public repo holding the package has a commit that fixes the issue (often a pull request, as mentioned before). + +If that’s the case, most package managers allow you to change your manifest file to point to the GitHub commit instead of naming your package and version. Git hashes are immutable, so you’ll know exactly what you’re getting, even if the pull request evolved. However, the commit may be deleted, introducing certain reliability concerns. + +### Fork and Patch + +When patching a vulnerability in a direct dependency, assuming you don’t want to depend on an external commit or have none to use, you can create one of your own. Doing so typically means forking the GitHub repository to a user you control, and patching it. Once done, you can modify your manifest to point to your fixed repository. + +Forking is a fairly common way of fixing different bugs in dependencies, and also carries some nice reliability advantages, as the code you use is now in your own control. It has the downside of breaking off the normal version stream of the dependency, but it’s a decent short-term solution to vulnerabilities in direct dependencies. Unfortunately, forking is not a viable option for patching indirect dependencies. + +### Static Patching at Build Time + +Another opportunity to patch a dependency is during build time. This type of patching is more complicated, as it requires: + +1. Storing a patch in a file (often a  _.patch_  file, or an alternative JAR file with the issue fixed) + +2. Installing the dependencies as usual + +3. Determining where the dependency you’d like to patch was installed + +4. Applying the patch by modifying or swapping out the risky code + +These steps are not trivial, but they’re also usually doable using package manager commands. If a vulnerability is worth fixing, and there are no easier means to fix it, this approach should be considered. + +This is a classic problem for tools to address, as patches can be reused and their application can be repeated. However, at the time of this writing, Snyk is the only SCA tool that maintains patches in its DB and lets you apply them in your pipeline. I predict over time more and more tools will adopt this approach. + +### Dynamic Patching at Boot Time + +In certain programming languages, classes can also be modified at runtime, a technique often referred to as "monkey patching." Monkey patching can be used to fix vulnerabilities, though that practice has not become the norm in any ecosystem. The most prevalent use of monkey patching to fix vulnerabilities is in Ruby on Rails, where the Rails team has often released patches for vulnerabilities in the libraries it maintains. + +### Other Remediation Paths + +So far, I’ve stated upgrades are the best way to address a vulnerability, and patching the second best. However, what should you do when you cannot (or will not) upgrade nor patch? + +In those cases, you have no choice but to dig deeper. You need to understand the vulnerability better, and how it plays into your application. If it indeed puts your application at notable risk, there are a few steps you can take. + +### Removal + +Removing a dependency is a very effective way of fixing its vulnerabilities. Unfortunately, you’ll be losing its functionality at the same time. + +Dropping a dependency is often hard, as it by definition requires changes to your actual code. That said, such removal may turn out to be easy—for instance, when a dependency was used for convenience and can be rewritten instead, or when a comparable alternative exists in the ecosystem. + +Easy or hard, removing a dependency should always be considered an option, and weighed against the risk of keeping it. + +### External Mitigation + +If you can’t fix the vulnerable code, you can try to block attacks that attempt to exploit it instead. Introducing a rule in a web app firewall, modifying the parts of your app that accept related user input, or even blocking a port are all potential ways to mitigate a vulnerability. + +Whether you can mitigate and how to do so depends on the specific vulnerability and application, and in many cases such protection is impossible or high risk. That said, the most trivially exploited vulnerabilities, such as the March 2017 Struts2 RCE and ImageTragick, are often the ones most easily identified and blocked, so this approach is definitely worth exploring. + +###### Tip + +### Protecting Against Unknown Vulnerabilities + +Once you’re aware of a known vulnerability, your best move is to fix it, and external mitigation is a last resort. However, security controls that protect against unknown vulnerabilities, ranging from web app firewalls to sandboxed processes to ensuring least privilege, can often protect you from known vulnerabilities as well. + +### Log Issue + +Last but not least, even if you choose not to remediate the issue, the least you can do is create an issue for it. Beyond its risk management advantages, logging the issue will remind you to re-examine the remediation options over time—for instance, looking for newly available upgrades or patches that can help. + +If you have a security operations team, make sure to make them aware of vulnerabilities you are not solving right now. This information can prove useful when they triage suspicious behavior on the network, as such behavior may come down to this security hole being exploited. + +### Remediation Process + +Beyond the specific techniques, there are few broader guidelines when it comes to remediating issues. + +### Ignoring Issues + +If you choose not to fix an issue, or to fix it through a custom path, you’ll need to tell your SCA tool you did. Otherwise, the tool will continue to indicate this problem. + +All OSS security tools support ignoring a vulnerability, but have slightly different capabilities. You should consider the following, and try to note that in your tool of choice: + +* Are you ignoring the issue because it doesn’t affect you (perhaps you’ve mitigated it another way) or because you’ve accepted the risk? This may reflect differently in your top-level reports. + +* Do you want to mute the issue indefinitely, or just "snooze" it? Ignoring temporarily is common for low-severity issues that don’t yet have an upgrade, where you’re comfortable taking the risk for a bit and anticipate an upgrade will show up soon. + +* Do you want to ignore all instances of this known vulnerability (perhaps it doesn’t apply to your system), or only certain vulnerable paths (which, after a careful vetting process, you’ve determined to be non-exploitable)? + +Properly tagging the reason for muting an alert helps manage these vulnerabilities over time and across projects, and reduces the chance of an issue being wrongfully ignored and slipping through the cracks. + +### Fix All Vulnerable Paths + +For all the issues you’re not ignoring, remember that remediation has to be done for  _every vulnerable path_ . + +This is especially true for upgrades, as every path must be assessed for upgrade separately, but also applies to patches in many ecosystems. + +### Track Remediations Over Time + +As already mentioned, a fix is typically issued for the vulnerable package first, and only later propagates through the dependency chain as other libraries upgrade to use the newer (and safer) version. Similarly, community or author code contributions are created constantly, addressing issues that weren’t previously fixable. + +Therefore, it’s worth tracking remediation options over time. For ignored issues, periodically check if an easy fix is now available. For patched issues, track potential updates you can switch to. Certain SCA tools automate this tracking and notify you (or open automated pull requests) when such new remediations are available. + +### Invest in Making Fixing Easy + +The unfortunate reality is that new vulnerabilities in libraries are discovered all the time. This is a fact of life—code will have bugs, some of those bugs are security bugs (vulnerabilities), and some of those are disclosed. Therefore, you and your team should expect to get a constant stream of vulnerability notifications, which you need to act on. + +If fixing these vulnerabilities isn’t easy, your team will not do it. Fixing these issues competes with many priorities, and its oh-so-easy to put off this invisible risk. If each alert requires a lot of time to triage and determine a fix for, the ensuing behavior would likely be to either put it off or try to convince yourself it’s not a real problem. + +In the world of operating systems, fixing has become the default action. In fact, "patching your servers" means taking in a feed of fixes, often without ever knowing which vulnerabilities we fix. We should strive to achieve at least this level of simplicity when dealing with vulnerable app dependencies too. + +Part of this effort is on tooling providers. SCA tools should let you fix vulnerabilities with a click or proactive pull requests, or patch them with a single command like `apt-get upgrade` does on servers. The other part of the effort is on you. Consider it a high priority to make vulnerability remediation easy, choose priority, choose your tools accordingly, and put in the effort to enrich or adapt those tools to fit your workflow. + +### Summary + +You should always keep in mind that finding these vulnerabilities isn’t the goal—fixing them is. Because fixing vulnerabilities is something your team will need to do often, defining the processes and tools to get that done is critical. + +A great way to get started with remediation is to find vulnerabilities that can be fixed with a non-breaking upgrade, and get those upgrades done. While not entirely risk-free, these upgrades should be backward compatible, and getting these security holes fixed gets you off to a very good start. + +[1][8]Stats based on vulnerabilities curated in the Snyk vulnerability DB. + + +This is an excerpt from [Securing Open Source Libraries][16], by Guy Podjarny.  +[Read the preceding chapter][17] or [view the full report][18]. + + + +------------------------------------- + +作者简介： + +Guy Podjarny (Guypo) is a web performance researcher/evangelist and Akamai's Web CTO, focusing primarily on Mobile and Front-End performance. As a researcher, Guy frequently runs large scale tests, exploring performance in the real world and matching it to how browsers behave, and was one of the first to highlight the performance implications of Responsive Web Design. Guy is also the author of Mobitest, a free mobile measurement tool, and contributes to various open source tools. Guy was previously the co-founder and CTO of blaze.io, ac... + +-------------------------------------------------------------------------------- + +via: https://www.oreilly.com/ideas/mitigating-known-security-risks-in-open-source-libraries + +作者：[ Guy Podjarny][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.oreilly.com/people/4dda0-guy-podjarny +[1]:https://www.safaribooksonline.com/home/?utm_source=newsite&utm_medium=content&utm_campaign=lgen&utm_content=security-post-safari-right-rail-cta +[2]:https://www.safaribooksonline.com/home/?utm_source=newsite&utm_medium=content&utm_campaign=lgen&utm_content=security-post-safari-right-rail-cta +[3]:https://www.safaribooksonline.com/home/?utm_source=newsite&utm_medium=content&utm_campaign=lgen&utm_content=security-post-safari-right-rail-cta +[4]:https://www.oreilly.com/ideas/mitigating-known-security-risks-in-open-source-libraries#id-xJ0u4SBFphz +[5]:https://snyk.io/vuln/npm:jquery:20150627 +[6]:https://github.com/chjj/marked/pull/592 +[7]:https://github.com/snyk/vulnerabilitydb +[8]:https://www.oreilly.com/ideas/mitigating-known-security-risks-in-open-source-libraries#id-xJ0u4SBFphz-marker +[9]:https://pixabay.com/en/machine-mill-industry-steam-2881186/ +[10]:https://www.oreilly.com/ideas/mitigating-known-security-risks-in-open-source-libraries +[11]:https://www.oreilly.com/people/4dda0-guy-podjarny +[12]:https://www.oreilly.com/people/4dda0-guy-podjarny +[13]:https://www.safaribooksonline.com/library/view/securing-open-source/9781491996980/?utm_source=oreilly&utm_medium=newsite&utm_campaign=fixing-vulnerable-open-source-packages +[14]:https://www.oreilly.com/ideas/finding-vulnerable-open-source-packages?utm_source=oreilly&utm_medium=newsite&utm_campaign=fixing-vulnerable-open-source-packages +[15]:https://www.safaribooksonline.com/library/view/securing-open-source/9781491996980/?utm_source=oreilly&utm_medium=newsite&utm_campaign=fixing-vulnerable-open-source-packages +[16]:https://www.safaribooksonline.com/library/view/securing-open-source/9781491996980/?utm_source=oreilly&utm_medium=newsite&utm_campaign=fixing-vulnerable-open-source-packages +[17]:https://www.oreilly.com/ideas/finding-vulnerable-open-source-packages?utm_source=oreilly&utm_medium=newsite&utm_campaign=fixing-vulnerable-open-source-packages +[18]:https://www.safaribooksonline.com/library/view/securing-open-source/9781491996980/?utm_source=oreilly&utm_medium=newsite&utm_campaign=fixing-vulnerable-open-source-packages +[19]:https://pixabay.com/en/machine-mill-industry-steam-2881186/ \ No newline at end of file diff --git a/sources/tech/20180130 Python - Memcached- Efficient Caching in Distributed Applications - Real Python.md b/sources/tech/20180130 Python - Memcached- Efficient Caching in Distributed Applications - Real Python.md new file mode 100644 index 0000000000..647a5968e6 --- /dev/null +++ b/sources/tech/20180130 Python - Memcached- Efficient Caching in Distributed Applications - Real Python.md @@ -0,0 +1,239 @@ +Python + Memcached: Efficient Caching in Distributed Applications – Real Python +====== + +When writing Python applications, caching is important. Using a cache to avoid recomputing data or accessing a slow database can provide you with a great performance boost. + +Python offers built-in possibilities for caching, from a simple dictionary to a more complete data structure such as [`functools.lru_cache`][2]. The latter can cache any item using a [Least-Recently Used algorithm][3] to limit the cache size. + +Those data structures are, however, by definition local to your Python process. When several copies of your application run across a large platform, using a in-memory data structure disallows sharing the cached content. This can be a problem for large-scale and distributed applications. + +![](https://files.realpython.com/media/python-memcached.97e1deb2aa17.png) + +Therefore, when a system is distributed across a network, it also needs a cache that is distributed across a network. Nowadays, there are plenty of network servers that offer caching capability—we already covered [how to use Redis for caching with Django][4]. + +As you’re going to see in this tutorial, [memcached][5] is another great option for distributed caching. After a quick introduction to basic memcached usage, you’ll learn about advanced patterns such as “cache and set” and using fallback caches to avoid cold cache performance issues. + +### Installing memcached + +Memcached is [available for many platforms][6]: + + * If you run **Linux** , you can install it using `apt-get install memcached` or `yum install memcached`. This will install memcached from a pre-built package but you can alse build memcached from source, [as explained here][6]. + * For **macOS** , using [Homebrew][7] is the simplest option. Just run `brew install memcached` after you’ve installed the Homebrew package manager. + * On **Windows** , you would have to compile memcached yourself or find [pre-compiled binaries][8]. + + + +Once installed, memcached can simply be launched by calling the `memcached` command: +``` +$ memcached + +``` + +Before you can interact with memcached from Python-land you’ll need to install a memcached client library. You’ll see how to do this in the next section, along with some basic cache access operations. + +### Storing and Retrieving Cached Values Using Python + +If you never used memcached, it is pretty easy to understand. It basically provides a giant network-available dictionary. This dictionary has a few properties that are different from a classical Python dictionnary, mainly: + + * Keys and values have to be bytes + * Keys and values are automatically deleted after an expiration time + + + +Therefore, the two basic operations for interacting with memcached are `set` and `get`. As you might have guessed, they’re used to assign a value to a key or to get a value from a key, respectively. + +My preferred Python library for interacting with memcached is [`pymemcache`][9]—I recommend using it. You can simply [install it using pip][10]: +``` +$ pip install pymemcache + +``` + +The following code shows how you can connect to memcached and use it as a network-distributed cache in your Python applications: +``` +>>> from pymemcache.client import base + +# Don't forget to run `memcached' before running this next line: +>>> client = base.Client(('localhost', 11211)) + +# Once the client is instantiated, you can access the cache: +>>> client.set('some_key', 'some value') + +# Retrieve previously set data again: +>>> client.get('some_key') +'some value' + +``` + +memcached network protocol is really simple an its implementation extremely fast, which makes it useful to store data that would be otherwise slow to retrieve from the canonical source of data or to compute again: + +While straightforward enough, this example allows storing key/value tuples across the network and accessing them through multiple, distributed, running copies of your application. This is simplistic, yet powerful. And it’s a great first step towards optimizing your application. + +### Automatically Expiring Cached Data + +When storing data into memcached, you can set an expiration time—a maximum number of seconds for memcached to keep the key and value around. After that delay, memcached automatically removes the key from its cache. + +What should you set this cache time to? There is no magic number for this delay, and it will entirely depend on the type of data and application that you are working with. It could be a few seconds, or it might be a few hours. + +Cache invalidation, which defines when to remove the cache because it is out of sync with the current data, is also something that your application will have to handle. Especially if presenting data that is too old or or stale is to be avoided. + +Here again, there is no magical recipe; it depends on the type of application you are building. However, there are several outlying cases that should be handled—which we haven’t yet covered in the above example. + +A caching server cannot grow infinitely—memory is a finite resource. Therefore, keys will be flushed out by the caching server as soon as it needs more space to store other things. + +Some keys might also be expired because they reached their expiration time (also sometimes called the “time-to-live” or TTL.) In those cases the data is lost, and the canonical data source must be queried again. + +This sounds more complicated than it really is. You can generally work with the following pattern when working with memcached in Python: +``` +from pymemcache.client import base + + +def do_some_query(): + # Replace with actual querying code to a database, + # a remote REST API, etc. + return 42 + + +# Don't forget to run `memcached' before running this code +client = base.Client(('localhost', 11211)) +result = client.get('some_key') + +if result is None: + # The cache is empty, need to get the value + # from the canonical source: + result = do_some_query() + + # Cache the result for next time: + client.set('some_key', result) + +# Whether we needed to update the cache or not, +# at this point you can work with the data +# stored in the `result` variable: +print(result) + +``` + +> **Note:** Handling missing keys is mandatory because of normal flush-out operations. It is also obligatory to handle the cold cache scenario, i.e. when memcached has just been started. In that case, the cache will be entirely empty and the cache needs to be fully repopulated, one request at a time. + +This means you should view any cached data as ephemeral. And you should never expect the cache to contain a value you previously wrote to it. + +### Warming Up a Cold Cache + +Some of the cold cache scenarios cannot be prevented, for example a memcached crash. But some can, for example migrating to a new memcached server. + +When it is possible to predict that a cold cache scenario will happen, it is better to avoid it. A cache that needs to be refilled means that all of the sudden, the canonical storage of the cached data will be massively hit by all cache users who lack a cache data (also known as the [thundering herd problem][11].) + +pymemcache provides a class named `FallbackClient` that helps in implementing this scenario as demonstrated here: +``` +from pymemcache.client import base +from pymemcache import fallback + + +def do_some_query(): + # Replace with actual querying code to a database, + # a remote REST API, etc. + return 42 + + +# Set `ignore_exc=True` so it is possible to shut down +# the old cache before removing its usage from +# the program, if ever necessary. +old_cache = base.Client(('localhost', 11211), ignore_exc=True) +new_cache = base.Client(('localhost', 11212)) + +client = fallback.FallbackClient((new_cache, old_cache)) + +result = client.get('some_key') + +if result is None: + # The cache is empty, need to get the value + # from the canonical source: + result = do_some_query() + + # Cache the result for next time: + client.set('some_key', result) + +print(result) + +``` + +The `FallbackClient` queries the old cache passed to its constructor, respecting the order. In this case, the new cache server will always be queried first, and in case of a cache miss, the old one will be queried—avoiding a possible return-trip to the primary source of data. + +If any key is set, it will only be set to the new cache. After some time, the old cache can be decommissioned and the `FallbackClient` can be replaced directed with the `new_cache` client. + +### Check And Set + +When communicating with a remote cache, the usual concurrency problem comes back: there might be several clients trying to access the same key at the same time. memcached provides a check and set operation, shortened to CAS, which helps to solve this problem. + +The simplest example is an application that wants to count the number of users it has. Each time a visitor connects, a counter is incremented by 1. Using memcached, a simple implementation would be: +``` +def on_visit(client): + result = client.get('visitors') + if result is None: + result = 1 + else: + result += 1 + client.set('visitors', result) + +``` + +However, what happens if two instances of the application try to update this counter at the same time? + +The first call `client.get('visitors')` will return the same number of visitors for both of them, let’s say it’s 42. Then both will add 1, compute 43, and set the number of visitors to 43. That number is wrong, and the result should be 44, i.e. 42 + 1 + 1. + +To solve this concurrency issue, the CAS operation of memcached is handy. The following snippet implements a correct solution: +``` +def on_visit(client): + while True: + result, cas = client.gets('visitors') + if result is None: + result = 1 + else: + result += 1 + if client.cas('visitors', result, cas): + break + +``` + +The `gets` method returns the value, just like the `get` method, but it also returns a CAS value. + +What is in this value is not relevant, but it is used for the next method `cas` call. This method is equivalent to the `set` operation, except that it fails if the value has changed since the `gets` operation. In case of success, the loop is broken. Otherwise, the operation is restarted from the beginning. + +In the scenario where two instances of the application try to update the counter at the same time, only one succeeds to move the counter from 42 to 43. The second instance gets a `False` value returned by the `client.cas` call, and have to retry the loop. It will retrieve 43 as value this time, will increment it to 44, and its `cas` call will succeed, thus solving our problem. + +Incrementing a counter is interesting as an example to explain how CAS works because it is simplistic. However, memcached also provides the `incr` and `decr` methods to increment or decrement an integer in a single request, rather than doing multiple `gets`/`cas` calls. In real-world applications `gets` and `cas` are used for more complex data type or operations + +Most remote caching server and data store provide such a mechanism to prevent concurrency issues. It is critical to be aware of those cases to make proper use of their features. + +### Beyond Caching + +The simple techniques illustrated in this article showed you how easy it is to leverage memcached to speed up the performances of your Python application. + +Just by using the two basic “set” and “get” operations you can often accelerate data retrieval or avoid recomputing results over and over again. With memcached you can share the cache accross a large number of distributed nodes. + +Other, more advanced patterns you saw in this tutorial, like the Check And Set (CAS) operation allow you to update data stored in the cache concurrently across multiple Python threads or processes while avoiding data corruption. + +If you are interested into learning more about advanced techniques to write faster and more scalable Python applications, check out [Scaling Python][12]. It covers many advanced topics such as network distribution, queuing systems, distributed hashing, and code profiling. + +-------------------------------------------------------------------------------- + +via: https://realpython.com/blog/python/python-memcache-efficient-caching/ + +作者：[Julien Danjou][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://realpython.com/team/jdanjou/ +[1]:https://realpython.com/blog/categories/python/ +[2]:https://docs.python.org/3/library/functools.html#functools.lru_cache +[3]:https://en.wikipedia.org/wiki/Cache_replacement_policies#Least_Recently_Used_(LRU) +[4]:https://realpython.com/blog/python/caching-in-django-with-redis/ +[5]:http://memcached.org +[6]:https://github.com/memcached/memcached/wiki/Install +[7]:https://brew.sh/ +[8]:https://commaster.net/content/installing-memcached-windows +[9]:https://pypi.python.org/pypi/pymemcache +[10]:https://realpython.com/learn/python-first-steps/#11-pythons-power-packagesmodules +[11]:https://en.wikipedia.org/wiki/Thundering_herd_problem +[12]:https://scaling-python.com diff --git a/sources/tech/20180130 Quick Look at the Arch Based Indie Linux Distribution- MagpieOS.md b/sources/tech/20180130 Quick Look at the Arch Based Indie Linux Distribution- MagpieOS.md new file mode 100644 index 0000000000..a850a8fd33 --- /dev/null +++ b/sources/tech/20180130 Quick Look at the Arch Based Indie Linux Distribution- MagpieOS.md @@ -0,0 +1,78 @@ +Quick Look at the Arch Based Indie Linux Distribution: MagpieOS +====== +Most of the Linux distros that are in use today are either created and developed in the US or Europe. A young developer from Bangladesh wants to change all that. + +### Who is Rizwan? + +[Rizwan][1] is a computer science student from Bangladesh. He is currently studying to become a profession Python programmer. He started using Linux back in 2015. Working with Linux inspired him to create this own Linux distribution. He also wants to let the rest of the world know that Bangladesh is upgrading to Linux. + +He has also worked on creating a [live version of Linux From Scratch][2]. + +## ![MagpieOS Linux][3] + +### What is MagpieOS? + +Rizwan's new distro is named MagpieOS. [MagpieOS][4] is very simple. It is basically Arch with the GNOME3 desktop environment. MagpieOS also includes a custom repo with icons and themes (claimed to be) not available on other Arch-based distros or AUR. + +Here is a list of the software included with MagpieOS: Firefox, LibreOffice, Uget, Bleachbit, Notepadqq, SUSE Studio Image Writer, Pamac Package Manager, Gparted, Gimp, Rhythmbox, Simple Screen Recorder, all default GNOME software including Totem Video Player, and a new set of custom wallpaper. + +Currently, MagpieOS only supported the GNOME desktop environment. Rizwan picked it because it is his favorite. However, he plans to add more desktop environments in the future. + +Unfortunately, MagpieOS does not support the Bangla language or any other local languages. It supports GNOME's default language like English, Hindi etc. + +Rizwan named his distro MagpieOS because the [magpie][5] is the official bird of Bangladesh. + +## ![MagpieOS Linux][6] + +### Why Arch? + +Like most people, Rizwan started his Linux journey by using [Ubuntu][7]. In the beginning, he was happy with it. However, sometimes the software he wanted to install was not available in the repos and he had to hunt through Google looking for the correct PPA. He decided to switch to [Arch][8] because Arch has many packages that were not available on Ubuntu. Rizwan also liked the fact that Arch is a rolling release and would always be up-to-date. + +The problem with Arch is that it is complicated and time-consuming to install. So, Rizwan tried out several Arch-based distros and was not happy with any of them. He didn't like [Manjaro][9] because they did not have permission to use Arch's repos. Also, Arch repo mirrors are faster than Manjaro's and have more software. He liked [Antergos][10], but to install you need a constant internet connection. If your connection fails during installation, you have to start over. + +Because of these issues, Rizwan decided to create a simple distro that would give him and others an Arch install without all the hassle. He also hopes to get developers from his home country to switch from Ubuntu to Arch by using his distro. + +### How to Help Rizwan with MagpieOS + +If you are interested in helping Rizwan develop MagpieOS, you can contact him via the [MagpieOS website][4]. You can also check out the project's [GitHub page][11]. Rizwan said that he is not looking for financial support at the moment. + +## ![MagpieOS Linux][12] + +### Final Thoughts + +I installed MagpieOS to give it a quick once-over. It uses the [Calamares installer][13], which means installing it was relatively quick and painless. After I rebooted, I was greeted by an audio message welcoming me to MagpieOS. + +To be honest, it was the first time I have heard a post-install greeting. (Windows 10 might have one, but I'm not sure.) There was also a Mac OS-esque application dock at the bottom of the screen. Other than that, it felt like any other GNOME 3 desktop I have used. + +Considering that it's an indie project at the nascent stage, I won't recommend it using as your main OS. But if you are a distrohopper, you can surely give it a try. + +That being said, this is a good first try for a student seeking to put his country on the technological map. All the best, Rizwan. + +Have you already heard of MagpieOS? What is your favorite region or locally made Linux distro? Please let us know in the comments below. + +If you found this article interesting, please take a minute to share it on social media. + +-------------------------------------------------------------------------------- + +via: https://itsfoss.com/magpieos/ + +作者：[John Paul][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]: https://itsfoss.com/author/john/ +[1]:https://twitter.com/Linux_Saikat +[2]:https://itsfoss.com/linux-from-scratch-live-cd/ +[3]:https://itsfoss.com/wp-content/uploads/2018/01/magpieos1.jpg +[4]:http://www.magpieos.net +[5]:https://en.wikipedia.org/wiki/Magpie +[6]:https://itsfoss.com/wp-content/uploads/2018/01/magpieos2.jpg +[7]:https://www.ubuntu.com +[8]:https://www.archlinux.org +[9]:http://manjaro.org +[10]:https://antergos.com +[11]:https://github.com/Rizwan-Hasan/MagpieOS +[12]:https://itsfoss.com/wp-content/uploads/2018/01/magpieos3.png +[13]:https://calamares.io diff --git a/sources/tech/20180130 Reckoning The Spectre And Meltdown Performance Hit.md b/sources/tech/20180130 Reckoning The Spectre And Meltdown Performance Hit.md new file mode 100644 index 0000000000..d7140a21cf --- /dev/null +++ b/sources/tech/20180130 Reckoning The Spectre And Meltdown Performance Hit.md @@ -0,0 +1,85 @@ +Reckoning The Spectre And Meltdown Performance Hit For HPC +============================================================ + +![](https://3s81si1s5ygj3mzby34dq6qf-wpengine.netdna-ssl.com/wp-content/uploads/2015/03/intel-chip-logo-bw-200x147.jpg) + +While no one has yet created an exploit to take advantage of the Spectre and Meltdown speculative execution vulnerabilities that were exposed by Google six months ago and that were revealed in early January, it is only a matter of time. The [patching frenzy has not settled down yet][2], and a big concern is not just whether these patches fill the security gaps, but at what cost they do so in terms of application performance. + +To try to ascertain the performance impact of the Spectre and Meltdown patches, most people have relied on comments from Google on the negligible nature of the performance hit on its own applications and some tests done by Red Hat on a variety of workloads, [which we profiled in our initial story on the vulnerabilities][3]. This is a good starting point, but what companies really need to do is profile the performance of their applications before and after applying the patches – and in such a fine-grained way that they can use the data to debug the performance hit and see if there is any remediation they can take to alleviate the impact. + +In the meantime, we are relying on researchers and vendors to figure out the performance impacts. Networking chip maker Mellanox Technologies, always eager to promote the benefits of the offload model of its switch and network interface chips, has run some tests to show the effects of the Spectre and Meltdown patches on high performance networking for various workloads and using various networking technologies, including its own Ethernet and InfiniBand devices and Intel’s OmniPath. Some HPC researchers at the University of Buffalo have also done some preliminary benchmarking of selected HPC workloads to see the effect on compute and network performance. This is a good starting point, but is far from a complete picture of the impact that might be seen on HPC workloads after organization deploy the Spectre and Meltdown patches to their systems. + +To recap, here is what Red Hat found out when it tested the initial Spectre and Meltdown patches running its Enterprise Linux 7 release on servers using Intel’s “Haswell” Xeon E5 v3, “Broadwell” Xeon E5 v4, and “Skylake” Xeon SP processors: + +* **Measurable, 8 percent to 19 percent:** Highly cached random memory, with buffered I/O, OLTP database workloads, and benchmarks with high kernel-to-user space transitions are impacted between 8 percent and 19 percent. Examples include OLTP Workloads (TPC), sysbench, pgbench, netperf (< 256 byte), and fio (random I/O to NvME). + +* **Modest, 3 percent to 7 percent:** Database analytics, Decision Support System (DSS), and Java VMs are impacted less than the Measurable category. These applications may have significant sequential disk or network traffic, but kernel/device drivers are able to aggregate requests to moderate level of kernel-to-user transitions. Examples include SPECjbb2005, Queries/Hour and overall analytic timing (sec). + +* **Small, 2 percent to 5 percent:** HPC CPU-intensive workloads are affected the least with only 2 percent to 5 percent performance impact because jobs run mostly in user space and are scheduled using CPU pinning or NUMA control. Examples include Linpack NxN on X86 and SPECcpu2006. + +* **Minimal impact:** Linux accelerator technologies that generally bypass the kernel in favor of user direct access are the least affected, with less than 2% overhead measured. Examples tested include DPDK (VsPERF at 64 byte) and OpenOnload (STAC-N). Userspace accesses to VDSO like get-time-of-day are not impacted. We expect similar minimal impact for other offloads. + +And just to remind you, according to Red Hat containerized applications running atop Linux do not incur an extra Spectre or Meltdown penalty compared to applications running on bare metal because they are implemented as generic Linux processes themselves. But applications running inside virtual machines running atop hypervisors, Red Hat does expect that, thanks to the increase in the frequency of user-to-kernel transitions, the performance hit will be higher. (How much has not yet been revealed.) + +Gilad Shainer, the vice president of marketing for the InfiniBand side of the Mellanox house, shared some initial performance data from the company’s labs with regard to the Spectre and Meltdown patches. ([The presentation is available online here.][4]) + +In general, Shainer tells  _The Next Platform_ , the offload model that Mellanox employs in its InfiniBand switches (RDMA is a big component of this) and in its Ethernet (The RoCE clone of RDMA is used here) are a very big deal given the fact that the network drivers bypass the operating system kernels. The exploits take advantage, in one of three forms, of the porous barrier between the kernel and user spaces in the operating systems, so anything that is kernel heavy will be adversely affected. This, says Shainer, includes the TCP/IP protocol that underpins Ethernet as well as the OmniPath protocol, which by its nature tries to have the CPUs in the system do a lot of the network processing. Intel and others who have used an onload model have contended that this allows for networks to be more scalable, and clearly there are very scalable InfiniBand and OmniPath networks, with many thousands of nodes, so both approaches seem to work in production. + +Here are the feeds and speeds on the systems that Mellanox tested on two sets of networking tests. For the comparison of Ethernet with RoCE added and standard TCP over Ethernet, the hardware was a two-socket server using Intel’s Xeon E5-2697A v4 running at 2.60 GHz. This machine was configured with Red Hat Enterprise Linux 7.4, with kernel versions 3.10.0-693.11.6.el7.x86_64 and 3.10.0-693.el7.x86_64\. (Those numbers  _are_  different – there is an  _11.6_  in the middle of the second one.) The machines were equipped with ConnectX-5 server adapters with firmware 16.22.0170 and the MLNX_OFED_LINUX-4.3-0.0.5.0 driver. The workload that was tested was not a specific HPC application, but rather a very low level, homegrown interconnect benchmark that is used to stress switch chips and NICs to see their peak  _sustained_  performance, as distinct from peak  _theoretical_ performance, which is the absolute ceiling. This particular test was run on a two-node cluster, passing data from one machine to the other. + +Here is how the performance stacked up before and after the Spectre and Meltdown patches were added to the systems: + + [![](https://3s81si1s5ygj3mzby34dq6qf-wpengine.netdna-ssl.com/wp-content/uploads/2018/01/mellanox-spectre-meltdown-roce-versus-tcp.jpg)][5] + +As you can see, at this very low level, there is no impact on network performance between two machines supporting RoCE on Ethernet, but running plain vanilla TCP without an offload on top of Ethernet, there are some big performance hits. Interestingly, on this low-level test, the impact was greatest on small message sizes in the TCP stack and then disappeared as the message sizes got larger. + +On a separate round of tests pitting InfiniBand from Mellanox against OmniPath from Intel, the server nodes were configured with a pair of Intel Xeon SP Gold 6138 processors running at 2 GHz, also with Red Hat Enterprise Linux 7.4 with the 3.10.0-693.el7.x86_64 and 3.10.0-693.11.6.el7.x86_64          kernel versions. The OmniPath adapter uses the IntelOPA-IFS.RHEL74-x86_64.10.6.1.0.2 driver and the Mellanox ConnectX-5 adapter uses the MLNX_OFED 4.2 driver. + +Here is how the InfiniBand and OmniPath protocols did on the tests before and after the patches: + + [![](https://3s81si1s5ygj3mzby34dq6qf-wpengine.netdna-ssl.com/wp-content/uploads/2018/01/mellanox-spectre-meltdown-infiniband-versus-omnipath.jpg)][6] + +Again, thanks to the offload model and the fact that this was a low level benchmark that did not hit the kernel very much (and some HPC applications might cross that boundary and therefore invoke the Spectre and Meltdown performance penalties), there was no real effect on the two-node cluster running InfiniBand. With the OmniPath system, the impact was around 10 percent for small message sizes, and then grew to 25 percent or so once the message sizes transmitted reached 512 bytes. + +We have no idea what the performance implications are for clusters of more than two machines using the Mellanox approach. It would be interesting to see if the degradation compounds or doesn’t. + +### Early HPC Performance Tests + +While such low level benchmarks provide some initial guidance on what the effect might be of the Spectre and Meltdown patches on HPC performance, what you really need is a benchmark run of real HPC applications running on clusters of various sizes, both before and after the Spectre and Meltdown patches are applied to the Linux nodes. A team of researchers led by Nikolay Simakov at the Center For Computational Research at SUNY Buffalo fired up some HPC benchmarks and a performance monitoring tool derived from the National Science Foundation’s Extreme Digital (XSEDE) program to see the effect of the Spectre and Meltdown patches on how much work they could get done as gauged by wall clock time to get that work done. + +The paper that Simakov and his team put together on the initial results [is found here][7]. The tool that was used to monitor the performance of the systems was called XD Metrics on Demand, or XDMoD, and it was open sourced and is available for anyone to use. (You might consider [Open XDMoD][8] for your own metrics to determine the performance implications of the Spectre and Meltdown patches.) The benchmarks tested by the SUNY Buffalo researchers included the NAMD molecular dynamics and NWChem computational chemistry applications, as well as the HPC Challenge suite, which itself includes the STREAM memory bandwidth test and the NASA Parallel Benchmarks (NPB), the Interconnect MPI Benchmarks (IMB). The researchers also tested the IOR file reading and the MDTest metadata benchmark tests from Lawrence Livermore National Laboratory. The IOR and MDTest benchmarks were run in local mode and in conjunction with a GPFS parallel file system running on an external 3 PB storage cluster. (The tests with a “.local” suffix in the table are run on storage in the server nodes themselves.) + +SUNY Buffalo has an experimental cluster with two-socket machines based on Intel “Nehalem” Xeon L5520 processors, which have eight cores and which are, by our reckoning, very long in the tooth indeed in that they are nearly nine years old. Each node has 24 GB of main memory and has 40 Gb/sec QDR InfiniBand links cross connecting them together. The systems are running the latest CentOS 7.4.1708 release, without and then with the patches applied. (The same kernel patches outlined above in the Mellanox test.) Simakov and his team ran each benchmark on a single node configuration and then ran the benchmark on a two node configuration, and it shows the difference between running a low-level benchmark and actual applications when doing tests. Take a look at the table of results: + + [![](https://3s81si1s5ygj3mzby34dq6qf-wpengine.netdna-ssl.com/wp-content/uploads/2018/01/suny-buffalo-spectre-meltdown-test-table.jpg)][9] + +The before runs of each application tested were done on around 20 runs, and the after was done on around 50 runs. For the core HPC applications – NAMD, NWChem, and the elements of HPCC – the performance degradation was between 2 percent and 3 percent, consistent with what Red Hat told people to expect back in the first week that the Spectre and Meltdown vulnerabilities were revealed and the initial patches were available. However, moving on to two-node configurations, where network overhead was taken into account, the performance impact ranged from 5 percent to 11 percent. This is more than you would expect based on the low level benchmarks that Mellanox has done. Just to make things interesting, on the IOR and MDTest benchmarks, moving from one to two nodes actually lessened the performance impact; running the IOR test on the local disks resulted in a smaller performance hit then over the network for a single node, but was not as low as for a two-node cluster running out to the GPFS file system. + +There is a lot of food for thought in this data, to say the least. + +What we want to know – and what the SUNY Buffalo researchers are working on – is what happens to performance on these HPC applications when the cluster is scaled out. + +“We will know that answer soon,” Simakov tells  _The Next Platform_ . “But there are only two scenarios that are possible. Either it is going to get worse or it is going to stay about the same as a two-node cluster. We think that it will most likely stay the same, because all of the MPI communication happens through the shared memory on a single node, and when you get to two nodes, you get it into the network fabric and at that point, you are probably paying all of the extra performance penalties.” + +We will update this story with data on larger scale clusters as soon as Simakov and his team provide the data. + +-------------------------------------------------------------------------------- + +via: https://www.nextplatform.com/2018/01/30/reckoning-spectre-meltdown-performance-hit-hpc/ + +作者：[Timothy Prickett Morgan][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.nextplatform.com/author/tpmn/ +[1]:https://www.nextplatform.com/author/tpmn/ +[2]:https://www.nextplatform.com/2018/01/18/datacenters-brace-spectre-meltdown-impact/ +[3]:https://www.nextplatform.com/2018/01/08/cost-spectre-meltdown-server-taxes/ +[4]:http://www.mellanox.com/related-docs/presentations/2018/performance/Spectre-and-Meltdown-Performance.pdf?homepage +[5]:https://3s81si1s5ygj3mzby34dq6qf-wpengine.netdna-ssl.com/wp-content/uploads/2018/01/mellanox-spectre-meltdown-roce-versus-tcp.jpg +[6]:https://3s81si1s5ygj3mzby34dq6qf-wpengine.netdna-ssl.com/wp-content/uploads/2018/01/mellanox-spectre-meltdown-infiniband-versus-omnipath.jpg +[7]:https://arxiv.org/pdf/1801.04329.pdf +[8]:http://open.xdmod.org/7.0/index.html +[9]:https://3s81si1s5ygj3mzby34dq6qf-wpengine.netdna-ssl.com/wp-content/uploads/2018/01/suny-buffalo-spectre-meltdown-test-table.jpg \ No newline at end of file diff --git a/sources/tech/20180130 Refreshing old computers with Linux.md b/sources/tech/20180130 Refreshing old computers with Linux.md new file mode 100644 index 0000000000..0a9f49c1d4 --- /dev/null +++ b/sources/tech/20180130 Refreshing old computers with Linux.md @@ -0,0 +1,104 @@ +Refreshing old computers with Linux +============================================================ + +### A middle school's Tech Stewardship program is now an elective class for science and technology students. + + +![Refreshing old computers with Linux](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/idea_innovation_kid_education.png?itok=3lRp6gFa "Refreshing old computers with Linux") +Image by : opensource.com + +It's nearly impossible to enter a school these days without seeing an abundance of technology. Despite this influx of computers into education, funding inequity forces school systems to make difficult choices. Some educators see things as they are and wonder, "Why?" while others see problems as opportunities and think, "Why not?" + +​[Andrew Dobbie ][31]is one of those visionaries who saw his love of Linux and computer reimaging as a unique learning opportunity for his students. + +Andrew teaches sixth grade at Centennial Senior Public School in Brampton, Ontario, Canada, and is a[Google Certified Innovator][16]. Andrew said, "Centennial Senior Public School hosts a special regional science & technology program that invites students from throughout the region to spend three years learning Ontario curriculum through the lens of science and technology." However, the school's students were in danger of falling prey to the digital divide that's exacerbated by hardware and software product lifecycles and inadequate funding. + +![Tech Stewardship students working on a computer](https://opensource.com/sites/default/files/u128651/techstewardship_students.jpg "Tech Stewardship students working on a computer") + +Image courtesy of [Affordable Tech for All][6] + +Although there was a school-wide need for access to computers in the classrooms, Andrew and his students discovered that dozens of old computers were being shipped out of the school because they were too old and slow to keep up with the latest proprietary operating systems or function on the school's network. + +Andrew saw this problem as a unique learning opportunity for his students and created the [Tech Stewardship][17] program. He works in partnership with two other teachers, Mike Doiu and Neil Lyons, and some students, who "began experimenting with open source operating systems like [Lubuntu][18] and [CubLinux][19] to help develop a solution to our in-class computer problem," he says. + +The sixth-grade students deployed the reimaged computers into classrooms throughout the school. When they exhausted the school's supply of surplus computers, they sourced more free computers from a local nonprofit organization called [Renewed Computer Technology Ontario][20]. In all, the Tech Stewardship program has provided more than 200 reimaged computers for students to use in classrooms throughout the school. + + +![Tech Stewardship students](https://opensource.com/sites/default/files/u128651/techstewardship_class.jpg "Tech Stewardship students") + +Image courtesy of [Affordable Tech for All][7] + +The Tech Stewardship program is now an elective class for the school's science and technology students in grades six, seven, and eight. Not only are the students learning about computer reimaging, they're also giving back to their local communities through this open source outreach program. + +### A broad impact + +The Tech Stewardship program is linked directly to the school's curriculum, especially in social studies by teaching the [United Nations' Sustainable Development Goals][21] (SDGs). The program is a member of [Teach SDGs][22], and Andrew serves as a Teach SDGs ambassador. Also, as a Google Certified Innovator, Andrew partners with Google and the [EdTechTeam][23], and Tech Stewardship has participated in Ontario's [Bring it Together][24] conference for educational technology. + +Andrew's students also serve as mentors to their fellow students. In one instance, a group of girls taught a grade 3 class about effective use of Google Drive and helped these younger students to make the best use of their Linux computers. Andrew said, "outreach and extension of learning beyond the classroom at Centennial is a major goal of the Tech Stewardship program." + +### What the students say + +Linux and open source are an integral part of the program. A girl named Ashna says, "In grade 6, Mr. Dobbie had shown us how to reimage a computer into Linux to use it for educational purposes. Since then, we have been learning more and growing." Student Shradhaa says, "At the very beginning, we didn't even know how to reimage with Linux. Mr. Dobbie told us to write steps for how to reimage Linux devices, and using those steps we are trying to reimage the computers." + + +![Tech Stewardship student upgrading memory](https://opensource.com/sites/default/files/u128651/techstewardship_upgrading-memory.jpg "Tech Stewardship student upgrading memory") + +Image courtesy of [Affordable Tech for All][8] + +The students were quick to add that Tech Stewardship has become a portal for discussion about being advocates for the change they want to see in the world. Through their hands-on activity, students learn to support the United Nations Sustainable Development goals. They also learn lessons far beyond the curriculum itself. For example, a student named Areez says he has learned how to find other resources, including donations, that allow the project to expand, since the class work upfitting older computers doesn't produce an income stream. + +Another student, Harini, thinks the Tech Stewardship program has demonstrated to other students what is possible and how one small initiative can change the world. After learning about the program, 40 other schools and individuals are reimaging computers with Linux. Harini says, "The more people who use them for educational purposes, the more outstanding the future will become since those educated people will lead out new, amazing lives with jobs." + +Joshua, another student in the program, sees it this way: "I thought of it as just a fun experience, but as it went on, we continued learning and understanding how what we were doing was making such a big impact on the world!" Later, he says, "a school reached out to us and asked us if we could reimage some computers for them. We went and completed the task. Then it continued to grow, as people from Europe came to see how we were fixing broken computers and started doing it when they went back." + +Andrew Dobbie is keen to share his experience with schools and interested individuals. You can contact him on [Twitter][25] or through his [website][26]. + + +### About the author + + [![](https://opensource.com/sites/default/files/styles/profile_pictures/public/donw2-crop.jpg?itok=OqOYd3A8)][27] Don Watkins - Educator, education technology specialist,  entrepreneur, open source advocate. M.A. in Educational Psychology, MSED in Educational Leadership, Linux system administrator, CCNA, virtualization using Virtual Box. Follow me at [@Don_Watkins .][13][More about me][14] + +-------------------------------------------------------------------------------- + +via: https://opensource.com/article/18/1/new-linux-computers-classroom + +作者：[Don Watkins ][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://opensource.com/users/don-watkins +[1]:https://opensource.com/resources/what-is-linux?intcmp=70160000000h1jYAAQ&utm_source=intcallout&utm_campaign=linuxcontent +[2]:https://opensource.com/resources/what-are-linux-containers?intcmp=70160000000h1jYAAQ&utm_source=intcallout&utm_campaign=linuxcontent +[3]:https://developers.redhat.com/promotions/linux-cheatsheet/?intcmp=70160000000h1jYAAQ&utm_source=intcallout&utm_campaign=linuxcontent +[4]:https://developers.redhat.com/cheat-sheet/advanced-linux-commands-cheatsheet?intcmp=70160000000h1jYAAQ&utm_source=intcallout&utm_campaign=linuxcontent +[5]:https://opensource.com/tags/linux?intcmp=70160000000h1jYAAQ&utm_source=intcallout&utm_campaign=linuxcontent +[6]:https://photos.google.com/share/AF1QipPnm-q9OIQnrzDD4n7oWIBBIE7RQ6BI9lv486RaU5lKBrs88pq3gPKM8VAgY0prkw?key=cS1RdEZ3ZHdXLWp0bUwzMEk3UnFQRkUwbWl1dWhn +[7]:https://photos.google.com/share/AF1QipPnm-q9OIQnrzDD4n7oWIBBIE7RQ6BI9lv486RaU5lKBrs88pq3gPKM8VAgY0prkw?key=cS1RdEZ3ZHdXLWp0bUwzMEk3UnFQRkUwbWl1dWhn +[8]:https://photos.google.com/share/AF1QipPnm-q9OIQnrzDD4n7oWIBBIE7RQ6BI9lv486RaU5lKBrs88pq3gPKM8VAgY0prkw?key=cS1RdEZ3ZHdXLWp0bUwzMEk3UnFQRkUwbWl1dWhn +[9]:https://opensource.com/file/384581 +[10]:https://opensource.com/file/384591 +[11]:https://opensource.com/file/384586 +[12]:https://opensource.com/article/18/1/new-linux-computers-classroom?rate=bK5X7pRc5y9TyY6jzOZeLDW6ehlWmNPXuP38DYsQ-6I +[13]:https://twitter.com/Don_Watkins +[14]:https://opensource.com/users/don-watkins +[15]:https://opensource.com/user/15542/feed +[16]:https://edutrainingcenter.withgoogle.com/certification_innovator +[17]:https://sites.google.com/view/mrdobbie/tech-stewardship +[18]:https://lubuntu.net/ +[19]:https://en.wikipedia.org/wiki/Cub_Linux +[20]:http://www.rcto.ca/ +[21]:http://www.un.org/sustainabledevelopment/sustainable-development-goals/ +[22]:http://www.teachsdgs.org/ +[23]:https://www.edtechteam.com/team/ +[24]:http://bringittogether.ca/ +[25]:https://twitter.com/A_Dobbie11 +[26]:http://bit.ly/linuxresources +[27]:https://opensource.com/users/don-watkins +[28]:https://opensource.com/users/don-watkins +[29]:https://opensource.com/users/don-watkins +[30]:https://opensource.com/article/18/1/new-linux-computers-classroom#comments +[31]:https://twitter.com/A_Dobbie11 +[32]:https://opensource.com/tags/education +[33]:https://opensource.com/tags/linux \ No newline at end of file diff --git a/sources/tech/20180130 Trying Other Go Versions.md b/sources/tech/20180130 Trying Other Go Versions.md new file mode 100644 index 0000000000..731747d19a --- /dev/null +++ b/sources/tech/20180130 Trying Other Go Versions.md @@ -0,0 +1,112 @@ +Trying Other Go Versions +============================================================ + +While I generally use the current release of Go, sometimes I need to try a different version. For example, I need to check that all the examples in my [Guide to JSON][2] work with [both the supported releases of Go][3](1.8.6 and 1.9.3 at time of writing) along with go1.10rc1. + +I primarily use the current version of Go, updating it when new versions are released. I try out other versions as needed following the methods described in this article. + +### Trying Betas and Release Candidates[¶][4] + +When [go1.8beta2 was released][5], a new tool for trying the beta and release candidates was also released that allowed you to `go get` the beta. It allowed you to easily run the beta alongside your Go installation by getting the beta with: + +``` +go get golang.org/x/build/version/go1.8beta2 +``` + +This downloads and builds a small program that will act like the `go` tool for that specific version. The full release can then be downloaded and installed with: + +``` +go1.8beta2 download +``` + +This downloads the release from [https://golang.org/dl][6] and installs it into `$HOME/sdk` or `%USERPROFILE%\sdk`. + +Now you can use `go1.8beta2` as if it were the normal Go command. + +This method works for [all the beta and release candidates][7] released after go1.8beta2. + +### Trying a Specific Release[¶][8] + +While only beta and release candidates are provided, they can easily be adapted to work with any released version. For example, to use go1.9.2: + +``` +package main + +import ( + "golang.org/x/build/version" +) + +func main() { + version.Run("go1.9.2") +} +``` + +Replace `go1.9.2` with the release you want to run and build/install as usual. + +Since the program I use to build my [Guide to JSON][9] calls `go` itself (for each example), I build this as `go` and prepend the directory to my `PATH` so it will use this one instead of my normal version. + +### Trying Any Release[¶][10] + +This small program can be extended so you can specify the release to use instead of having to maintain binaries for each version. + +``` +package main + +import ( + "fmt" + "os" + + "golang.org/x/build/version" +) + +func main() { + if len(os.Args) < 2 { + fmt.Printf("USAGE: %v [commands as normal]\n", + os.Args[0]) + os.Exit(1) + } + + v := os.Args[1] + os.Args = append(os.Args[0:1], os.Args[2:]...) + + version.Run("go" + v) +} +``` + +I have this installed as `gov` and run it like `gov 1.8.6 version`, using the version I want to run. + +### Trying a Source Build (e.g., tip)[¶][11] + +I also use this same infrastructure to manage source builds of Go, such as tip. There’s just a little trick to it: + +* use the directory `$HOME/sdk/go` (e.g., `$HOME/sdk/gotip`) + +* [build as normal][1] + +* `touch $HOME/sdk/go/.unpacked-success` This is an empty file used as a sentinel to indicate the download and unpacking was successful. + +(On Windows, replace `$HOME/sdk` with `%USERPROFILE%\sdk`) + + +-------------------------------------------------------------------------------- + +via: https://pocketgophers.com/trying-other-versions/ + +作者：[Nathan Kerr ][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:nathan@pocketgophers.com +[1]:https://golang.org/doc/install/source +[2]:https://pocketgophers.com/guide-to-json/ +[3]:https://pocketgophers.com/when-should-you-upgrade-go/ +[4]:https://pocketgophers.com/trying-other-versions/#trying-betas-and-release-candidates +[5]:https://groups.google.com/forum/#!topic/golang-announce/LvfYP-Wk1s0 +[6]:https://golang.org/dl +[7]:https://godoc.org/golang.org/x/build/version#pkg-subdirectories +[8]:https://pocketgophers.com/trying-other-versions/#trying-a-specific-release +[9]:https://pocketgophers.com/guide-to-json/ +[10]:https://pocketgophers.com/trying-other-versions/#trying-any-release +[11]:https://pocketgophers.com/trying-other-versions/#trying-a-source-build-e-g-tip \ No newline at end of file diff --git a/sources/tech/20180130 Use of du - df commands (with examples).md b/sources/tech/20180130 Use of du - df commands (with examples).md new file mode 100644 index 0000000000..ac284b0025 --- /dev/null +++ b/sources/tech/20180130 Use of du - df commands (with examples).md @@ -0,0 +1,112 @@ +translating---geekpi + +Use of du & df commands (with examples) +====== +In this article I will discuss du & df commands. Both du & df commands are important utilities of Linux system & shows disk usage of Linux filesystem. Here we will share usage of both commands with some examples. + +**(Recommended Read:[Files transfer using scp & rsync commands][1])** + + **(Also Read:[Cloning Disks using dd & cat commands for Linux systems][2])** + +### du COMMAND + +du command (short for disk usage) is useful command which is used to find disk usage for files & directories. du command when used with various options provides results in many formats. + +Some of the examples are mentioned below:- + + **1- To find out summary of disk usage for a directory with all its sub-directories** + +``` + $ du /home +``` + +![du command][4] + +Output of the command shows all the files & directories in /home with block size. + +**2- Disk usage with file/directory sizes in human readable format I.e. in kb, mb etc** + +``` + $ du -h /home +``` + +![du command][6] + +**3- Total disk size of a directory** + +``` + $ du -s /home +``` + +![du command][8] + +It will total size of /home directory. + +### df COMMAND + +df command (short for disk filesystem) is used to show disk utilization for a Linux system. + +Some examples are shared below. + + **1- To display information of device name, total blocks, total disk space, used disk space, available disk space and mount points on a file system.** + +``` + $ df +``` + + +![df command][10] + +**2- Information in human readable format** + +``` + $ df -h +``` + +![df command][12] + +Above command displays information in human readable format. + +**3- Display information of a particular partition** + +``` + $ df -hT /etc +``` + +![df command][14] + +Using -hT with a target directory will show information of /etc/ in human readable format. + +Though there are many more options that can be used with du & df commands, but these should get you started. If you don't find what you are looking for here then you can always refer to man pages for the concerned command. + +Also, read my other posts [**HERE**][15] where i have shared some other important & frequently used Linux. + +And as always your comments/queries are really appreciated, so please leave your comments/queries down below & I will get back to you. + + +-------------------------------------------------------------------------------- + +via: http://linuxtechlab.com/du-df-commands-examples/ + +作者：[SHUSAIN][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://linuxtechlab.com/author/shsuain/ +[1]:http://linuxtechlab.com/files-transfer-scp-rsync-commands/ +[2]:http://linuxtechlab.com/linux-cloning-disks-using-dd-cat-commands/ +[3]:https://i1.wp.com/linuxtechlab.com/wp-content/plugins/a3-lazy-load/assets/images/lazy_placeholder.gif?resize=453%2C162 +[4]:https://i2.wp.com/linuxtechlab.com/wp-content/uploads/2017/02/du1.jpg?resize=453%2C162 +[5]:https://i1.wp.com/linuxtechlab.com/wp-content/plugins/a3-lazy-load/assets/images/lazy_placeholder.gif?resize=491%2C163 +[6]:https://i1.wp.com/linuxtechlab.com/wp-content/uploads/2017/02/du2.jpg?resize=491%2C163 +[7]:https://i1.wp.com/linuxtechlab.com/wp-content/plugins/a3-lazy-load/assets/images/lazy_placeholder.gif?resize=584%2C61 +[8]:https://i0.wp.com/linuxtechlab.com/wp-content/uploads/2017/02/du3.jpg?resize=584%2C61 +[9]:https://i1.wp.com/linuxtechlab.com/wp-content/plugins/a3-lazy-load/assets/images/lazy_placeholder.gif?resize=638%2C157 +[10]:https://i0.wp.com/linuxtechlab.com/wp-content/uploads/2017/02/df1.jpg?resize=638%2C157 +[11]:https://i1.wp.com/linuxtechlab.com/wp-content/plugins/a3-lazy-load/assets/images/lazy_placeholder.gif?resize=641%2C149 +[12]:https://i0.wp.com/linuxtechlab.com/wp-content/uploads/2017/02/df2.jpg?resize=641%2C149 +[13]:https://i1.wp.com/linuxtechlab.com/wp-content/plugins/a3-lazy-load/assets/images/lazy_placeholder.gif?resize=638%2C62 +[14]:https://i0.wp.com/linuxtechlab.com/wp-content/uploads/2017/02/df3-1.jpg?resize=638%2C62 +[15]:http://linuxtechlab.com/tips-tricks/ diff --git a/sources/tech/20180130 tmux - A Powerful Terminal Multiplexer For Heavy Command-Line Linux User.md b/sources/tech/20180130 tmux - A Powerful Terminal Multiplexer For Heavy Command-Line Linux User.md new file mode 100644 index 0000000000..4adaa7a2bc --- /dev/null +++ b/sources/tech/20180130 tmux - A Powerful Terminal Multiplexer For Heavy Command-Line Linux User.md @@ -0,0 +1,259 @@ +tmux – A Powerful Terminal Multiplexer For Heavy Command-Line Linux User +====== +tmux stands for terminal multiplexer, it allows users to create/enable multiple terminals (vertical & horizontal) in single window, this can be accessed and controlled easily from single window when you are working with different issues. + +It uses a client-server model, which allows you to share sessions between users, also you can attach terminals to a tmux session back. We can easily move or rearrange the virtual console as per the need. Terminal sessions can freely rebound from one virtual console to another. + +tmux depends on libevent and ncurses libraries. tmux offers status-line at the bottom of the screen which display information about your current tmux session suc[]h as current window number, window name, username, hostname, current time, and current date. + +When tmux is started it creates a new session with a single window and displays it on screen. It allows users to create Any number of windows in the same session. + +Many of us says it's similar to screen but i'm not since this offers wide range of configuration options. + +**Make a note:** `Ctrl+b` is the default prefix in tmux so, to perform any action in tumx, you have to type the prefix first then required options. + +**Suggested Read :** [List Of Terminal Emulator For Linux][1] + +### tmux Features + + * Create any number of windows + * Create any number of panes in the single window + * It allows vertical and horizontal splits + * Detach and Re-attach window + * Server-client architecture which allows users to share sessions between users + * tmux offers wide range of configuration hacks + + + +**Suggested Read :** +**(#)** [tmate - Instantly Share Your Terminal Session To Anyone In Seconds][2] +**(#)** [Teleconsole - A Tool To Share Your Terminal Session Instantly To Anyone In Seconds][3] + +### How to Install tmux Command + +tmux command is pre-installed by default in most of the Linux systems. If no, follow the below procedure to get installed. + +For **`Debian/Ubuntu`** , use [APT-GET Command][4] or [APT Command][5] to install tmux. +``` +$ sudo apt install tmux + +``` + +For **`RHEL/CentOS`** , use [YUM Command][6] to install tmux. +``` +$ sudo yum install tmux + +``` + +For **`Fedora`** , use [DNF Command][7] to install tmux. +``` +$ sudo dnf install tmux + +``` + +For **`Arch Linux`** , use [Pacman Command][8] to install tmux. +``` +$ sudo pacman -S tmux + +``` + +For **`openSUSE`** , use [Zypper Command][9] to install tmux. +``` +$ sudo zypper in tmux + +``` + +### How to Use tmux + +kick start the tmux session by running following command on terminal. When tmux is started it creates a new session with a single window and will automatically login to your default shell with your user account. +``` +$ tmux + +``` + +[![][10]![][10]][11] + +You will get similar to above screenshot like us. tmux comes with status bar which display an information's about current sessions details, date, time, etc.,. + +The status bar information's are below: + + * **`0 :`** It is indicating the session number which was created by the tmux server. By default it starts with 0. + * **`0:username@host: :`** 0 is indicating the session number. Username and Hostname which is holding the current window. + * **`~ :`** It is indicating the current directory (We are in the Home directory) + * **`* :`** This indicate that the window is active now. + * **`Hostname :`** This shows fully qualified hostname of the server + * **`Date& Time:`** It shows current date and time + + + +### How to Split Window + +tmux allows users to split window vertically and horizontally. Let 's see how to do that. + +Press `**(Ctrl+b), %**` to split the pane vertically. +[![][10]![][10]][13] + +Press `**(Ctrl+b), "**` to split the pane horizontally. +[![][10]![][10]][14] + +### How to Move Between Panes + +Lets say, we have created few panes and want to move between them. How to do that? If you don 't know how to do, then there is no purpose to use tmux. Use the following control keys to perform the actions. There are many ways to move between panes. + +Press `(Ctrl+b), Left arrow` - To Move Left + +Press `(Ctrl+b), Right arrow` - To Move Right + +Press `(Ctrl+b), Up arrow` - To Move Up + +Press `(Ctrl+b), Down arrow` - To Move Down + +Press `(Ctrl+b), {` - To Move Left + +Press `(Ctrl+b), }` - To Move Right + +Press `(Ctrl+b), o` - Switch to next pane (left-to-right, top-down) + +Press `(Ctrl+b), ;` - Move to the previously active pane. + +For testing purpose, we are going to move between panes. Now, we are in the `pane2` which shows `lsb_release -a` command output. +[![][10]![][10]][15] + +And we are going to move to `pane0` which shows `uname -a` command output. +[![][10]![][10]][16] + +### How to Open/Create New Window + +You can open any number of windows within one terminal. Terminal window can be split vertically & horizontally which is called `panes`. Each pane will contain its own, independently running terminal instance. + +Press `(Ctrl+b), c` to create a new window. + +Press `(Ctrl+b), n` move to the next window. + +Press `(Ctrl+b), p` to move to the previous window. + +Press `(Ctrl+b), (0-9)` to immediately move to a specific window. + +Press `(Ctrl+b), l` Move to the previously selected window. + +I have two windows, first window has three panes which contains operating system distribution information, top command output & kernal information. +[![][10]![][10]][17] + +And second window has two panes which contains Linux distributions logo information. Use the following commands perform the action. +[![][10]![][10]][18] + +Press `(Ctrl+b), w` Choose the current window interactively. +[![][10]![][10]][19] + +### How to Zoom Panes + +You are working in some pane which is very small and you want to zoom it out for further work. To do use the following key binds. + +Currently we have three panes and i'm working in `pane1` which shows system activity using **Top** command and am going to zoom that. +[![][10]![][10]][17] + +When you zoom a pane, it will hide all other panes and display only the zoomed pane in the window. +[![][10]![][10]][20] + +Press `(Ctrl+b), z` to zoom the pane and press it again, to bring the zoomed pane back. + +### Display Pane Information + +To know about pane number and it's size, run the following command. + +Press `(Ctrl+b), q` to briefly display pane indexes. +[![][10]![][10]][21] + +### Display Window Information + +To know about window number, layout size, number of panes associated with the window and it's size, etc., run the following command. + +Just run `tmux list-windows` to view window information. +[![][10]![][10]][22] + +### How to Resize Panes + +You may want to resize the panes to fit your requirement. You have to press `(Ctrl+b), :` then type the following details on the `yellow` color bar in the bottom of the page. +[![][10]![][10]][23] + +In the previous section we have print pane index which shows panes size as well. To test this we are going to increase `10 cells UPward`. See the following output that has increased the pane1 & pane2 size from `55x21` to `55x31`. +[![][10]![][10]][24] + +**Syntax:** `(Ctrl+b), :` then type `resize-pane [options] [cells size]` + +`(Ctrl+b), :` then type `resize-pane -D 10` to resize the current pane Down for 10 cells. + +`(Ctrl+b), :` then type `resize-pane -U 10` to resize the current pane UPward for 10 cells. + +`(Ctrl+b), :` then type `resize-pane -L 10` to resize the current pane Left for 10 cells. + +`(Ctrl+b), :` then type `resize-pane -R 10` to resize the current pane Right for 10 cells. + +### Detaching and Re-attaching tmux Session + +One of the most powerful features of tmux is the ability to detach and reattach session whenever you need. + +Run a long running process and press `Ctrl+b` followed by `d` to detach your tmux session safely by leaving the running process. + +**Suggested Read :** [How To Keep A Process/Command Running After Disconnecting SSH Session][25] + +Now, run a long running process. For demonstration purpose, we are going to move this server backup to another remote server for disaster recovery (DR) purpose. + +You will get similar output like below after detached tmux session. +``` +[detached (from session 0)] + +``` + +Run the following command to list the available tmux sessions. +``` +$ tmux ls +0: 3 windows (created Tue Jan 30 06:17:47 2018) [109x45] + +``` + +Now, re-attach the tmux session using an appropriate session ID as follow. +``` +$ tmux attach -t 0 + +``` + +### How to Close Panes & Window + +Just type `exit` or hit `Ctrl-d` in the corresponding pane to close it. It's similar to terminal close. To close window, press `(Ctrl+b), &`. + +-------------------------------------------------------------------------------- + +via: https://www.2daygeek.com/tmux-a-powerful-terminal-multiplexer-emulator-for-linux/ + +作者：[Magesh Maruthamuthu][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.2daygeek.com/author/magesh/ +[1]:https://www.2daygeek.com/category/terminal-emulator/ +[2]:https://www.2daygeek.com/tmate-instantly-share-your-terminal-session-to-anyone-in-seconds/ +[3]:https://www.2daygeek.com/teleconsole-share-terminal-session-instantly-to-anyone-in-seconds/ +[4]:https://www.2daygeek.com/apt-get-apt-cache-command-examples-manage-packages-debian-ubuntu-systems/ +[5]:https://www.2daygeek.com/apt-command-examples-manage-packages-debian-ubuntu-systems/ +[6]:https://www.2daygeek.com/yum-command-examples-manage-packages-rhel-centos-systems/ +[7]:https://www.2daygeek.com/dnf-command-examples-manage-packages-fedora-system/ +[8]:https://www.2daygeek.com/pacman-command-examples-manage-packages-arch-linux-system/ +[9]:https://www.2daygeek.com/zypper-command-examples-manage-packages-opensuse-system/ +[10]:data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7 +[11]:https://www.2daygeek.com/wp-content/uploads/2018/01/tmux-a-powerful-terminal-multiplexer-emulator-for-linux-1.png +[13]:https://www.2daygeek.com/wp-content/uploads/2018/01/tmux-a-powerful-terminal-multiplexer-emulator-for-linux-2.png +[14]:https://www.2daygeek.com/wp-content/uploads/2018/01/tmux-a-powerful-terminal-multiplexer-emulator-for-linux-3.png +[15]:https://www.2daygeek.com/wp-content/uploads/2018/01/tmux-a-powerful-terminal-multiplexer-emulator-for-linux-4.png +[16]:https://www.2daygeek.com/wp-content/uploads/2018/01/tmux-a-powerful-terminal-multiplexer-emulator-for-linux-5.png +[17]:https://www.2daygeek.com/wp-content/uploads/2018/01/tmux-a-powerful-terminal-multiplexer-emulator-for-linux-8.png +[18]:https://www.2daygeek.com/wp-content/uploads/2018/01/tmux-a-powerful-terminal-multiplexer-emulator-for-linux-6.png +[19]:https://www.2daygeek.com/wp-content/uploads/2018/01/tmux-a-powerful-terminal-multiplexer-emulator-for-linux-7.png +[20]:https://www.2daygeek.com/wp-content/uploads/2018/01/tmux-a-powerful-terminal-multiplexer-emulator-for-linux-9.png +[21]:https://www.2daygeek.com/wp-content/uploads/2018/01/tmux-a-powerful-terminal-multiplexer-emulator-for-linux-10.png +[22]:https://www.2daygeek.com/wp-content/uploads/2018/01/tmux-a-powerful-terminal-multiplexer-emulator-for-linux-14.png +[23]:https://www.2daygeek.com/wp-content/uploads/2018/01/tmux-a-powerful-terminal-multiplexer-emulator-for-linux-11.png +[24]:https://www.2daygeek.com/wp-content/uploads/2018/01/tmux-a-powerful-terminal-multiplexer-emulator-for-linux-13.png +[25]:https://www.2daygeek.com/how-to-keep-a-process-command-running-after-disconnecting-ssh-session/ diff --git a/sources/tech/20180131 10 things I love about Vue.md b/sources/tech/20180131 10 things I love about Vue.md new file mode 100644 index 0000000000..55b9007ce1 --- /dev/null +++ b/sources/tech/20180131 10 things I love about Vue.md @@ -0,0 +1,128 @@ +10 things I love about Vue +============================================================ + +![](https://cdn-images-1.medium.com/max/1600/1*X4ipeKVYzmY2M3UPYgUYuA.png) + +I love Vue. When I first looked at it in 2016, perhaps I was coming from a perspective of JavaScript framework fatigue. I’d already had experience with Backbone, Angular, React, among others and I wasn’t overly enthusiastic to try a new framework. It wasn’t until I read a comment on hacker news describing Vue as the ‘new jquery’ of JavaScript, that my curiosity was piqued. Until that point, I had been relatively content with React — it is a good framework based on solid design principles centred around view templates, virtual DOM and reacting to state, and Vue also provides these great things. In this blog post, I aim to explore why Vue is the framework for me. I choose it above any other that I have tried. Perhaps you will agree with some of my points, but at the very least I hope to give you some insight into what it is like to develop modern JavaScript applications with Vue. + +1\. Minimal Template Syntax + +The template syntax which you are given by default from Vue is minimal, succinct and extendable. Like many parts of Vue, it’s easy to not use the standard template syntax and instead use something like JSX (there is even an official page of documentation about how to do this), but I don’t know why you would want to do that to be honest. For all that is good about JSX, there are some valid criticisms: by blurring the line between JavaScript and HTML, it makes it a bit too easy to start writing complex code in your template which should instead be separated out and written elsewhere in your JavaScript view code. + +Vue instead uses standard HTML to write your templates, with a minimal template syntax for simple things such as iteratively creating elements based on the view data. + +``` + + + + + +``` + + +I also like the short-bindings provided by Vue, ‘:’ for binding data variables into your template and ‘@’ for binding to events. It’s a small thing, but it feels nice to type and keeps your components succinct. + +2\. Single File Components + +When most people write Vue, they do so using ‘single file components’. Essentially it is a file with the suffix .vue containing up to 3 parts (the css, html and javascript) for each component. + +This coupling of technologies feels right. It makes it easy to understand each component in a single place. It also has the nice side effect of encouraging you to keep your code short for each component. If the JavaScript, CSS and HTML for your component is taking up too many lines then it might be time to modularise further. + +When it comes to the . + +* Linked stylesheet: We write styles of all the elements in a separate file with .css extension. This file is called Stylesheet. + +Let’s have a look at how we defined the inline style of the “div” until now: + +``` +
+``` + +We can write this same style inside `` like this: + +``` +div{ + width:550px; +} +``` + +In embedded styling, the styles we write are separate from the elements. So we need a way to relate the element and its style. The first word “div” does exactly that. It lets the browser know that whatever style is inside the curly braces `{…}` belongs to the “div” element. Since this phrase determines which element to apply the style to, it’s called a selector. + +The way we write style remains same: property(width) and value(550px) separated by a colon(:) and ended by a semicolon(;). + +Let’s remove inline style from our “div” and “img” element and write it inside the ` +``` + +``` +
+

Bat Letter

+ +

+ After all the battles we faught together, after all the difficult times we saw together, after all the good and bad moments we've been through, I think it's time I let you know how I feel about you. +

+``` + +``` +

You are the light of my life

+

+ You complete my darkness with your light. I love: +

+
    +
  • the way you see good in the worse
    • +
  • the way you handle emotionally difficult situations
    • +
  • the way you look at Justice
    • +
+

+ I have learned a lot from you. You have occupied a special place in my heart over the time. +

+

I have a confession to make

+

+ It feels like my chest does have a heart. You make my heart beat. Your smile brings smile on my face, your pain brings pain to my heart. +

+

+ I don't show my emotions, but I think this man behind the mask is falling for you. +

+

I love you Superman.

+

+ Your not-so-secret-lover,
+ Batman +

+
+``` + +Save and refresh, and the result should remain the same. + +There is one big problem though — what if there is more than one “div” and “img” element in our HTML file? The styles that we defined for div and img inside the “style” element will apply to every div and img on the page. + +If you add another div in your code in the future, then that div will also become 550px wide. We don’t want that. + +We want to apply our styles to the specific div and img that we are using right now. To do this, we need to give our div and img element unique ids. Here’s how you can give an id to an element using its “id” attribute: + +``` +
+``` + +and here’s how to use this id in our embedded style as a selector: + +``` +#letter-container{ + ... +} +``` + +Notice the “#” symbol. It indicates that it is an id, and the styles inside {…} should apply to the element with that specific id only. + +Let’s apply this to our code: + +``` + +``` + +``` +
+

Bat Letter

+ +

+ After all the battles we faught together, after all the difficult times we saw together, after all the good and bad moments we've been through, I think it's time I let you know how I feel about you. +

+``` + +``` +

You are the light of my life

+

+ You complete my darkness with your light. I love: +

+
    +
  • the way you see good in the worse
    • +
  • the way you handle emotionally difficult situations
    • +
  • the way you look at Justice
    • +
+

+ I have learned a lot from you. You have occupied a special place in my heart over the time. +

+

I have a confession to make

+

+ It feels like my chest does have a heart. You make my heart beat. Your smile brings smile on my face, your pain brings pain to my heart. +

+

+ I don't show my emotions, but I think this man behind the mask is falling for you. +

+

I love you Superman.

+

+ Your not-so-secret-lover,
+ Batman +

+
+``` + +Our HTML is ready with embedded styling. + +However, you can see that as we include more styles, the will get bigger. This can quickly clutter our main html file. So let’s go one step further and use linked styling by copying the content inside our style tag to a new file. + +Create a new file in the project root directory and save it as style.css: + +``` +#letter-container{ + width:550px; +} +#header-bat-logo{ + width:100%; +} +``` + +We don’t need to write `` in our CSS file. + +We need to link our newly created CSS file to our HTML file using the ``tag in our html file. Here’s how we can do that: + +``` + +``` + +We use the link element to include external resources inside your HTML document. It is mostly used to link Stylesheets. The three attributes that we are using are: + +* rel: Relation. What relationship the linked file has to the document. The file with the .css extension is called a stylesheet, and so we keep rel=“stylesheet”. + +* type: the Type of the linked file; it’s “text/css” for a CSS file. + +* href: Hypertext Reference. Location of the linked file. + +There is no at the end of the link element. So, is also a self-closing tag. + +``` + +``` + +If only getting a Girlfriend was so easy :D + +Nah, that’s not gonna happen, let’s move on. + +Here’s the content of our loveletter.html: + +``` + +
+

Bat Letter

+ +

+ After all the battles we faught together, after all the difficult times we saw together, after all the good and bad moments we've been through, I think it's time I let you know how I feel about you. +

+

You are the light of my life

+

+ You complete my darkness with your light. I love: +

+
    +
  • the way you see good in the worse
    • +
  • the way you handle emotionally difficult situations
    • +
  • the way you look at Justice
    • +
+

+ I have learned a lot from you. You have occupied a special place in my heart over the time. +

+

I have a confession to make

+

+ It feels like my chest does have a heart. You make my heart beat. Your smile brings smile on my face, your pain brings pain to my heart. +

+

+ I don't show my emotions, but I think this man behind the mask is falling for you. +

+

I love you Superman.

+

+ Your not-so-secret-lover,
+ Batman +

+
+``` + +and our style.css: + +``` +#letter-container{ + width:550px; +} +#header-bat-logo{ + width:100%; +} +``` + +Save both the files and refresh, and your output in the browser should remain the same. + +### A Few Formalities + +Our love letter is almost ready to deliver to Batman, but there are a few formal pieces remaining. + +Like any other programming language, HTML has also gone through many versions since its birth year(1990). The current version of HTML is HTML5. + +So, how would the browser know which version of HTML you are using to code your page? To tell the browser that you are using HTML5, you need to include `` at top of the page. For older versions of HTML, this line used to be different, but you don’t need to learn that because we don’t use them anymore. + +Also, in previous HTML versions, we used to encapsulate the entire document inside `` tag. The entire file was divided into two major sections: Head, inside ``, and Body, inside ``. This is not required in HTML5, but we still do this for compatibility reasons. Let’s update our code with ``, ``, `` and ``: + +``` + + + + + + +
+

Bat Letter

+ +

+ After all the battles we faught together, after all the difficult times we saw together, after all the good and bad moments we've been through, I think it's time I let you know how I feel about you. +

+

You are the light of my life

+

+ You complete my darkness with your light. I love: +

+
    +
  • the way you see good in the worse
    • +
  • the way you handle emotionally difficult situations
    • +
  • the way you look at Justice
    • +
+

+ I have learned a lot from you. You have occupied a special place in my heart over the time. +

+

I have a confession to make

+

+ It feels like my chest does have a heart. You make my heart beat. Your smile brings smile on my face, your pain brings pain to my heart. +

+

+ I don't show my emotions, but I think this man behind the mask is falling for you. +

+

I love you Superman.

+

+ Your not-so-secret-lover,
+ Batman +

+
+ + +``` + +The main content goes inside `` and meta information goes inside ``. So we keep the div inside `` and load the stylesheets inside ``. + +Save and refresh, and your HTML page should display the same as earlier. + +### Title in HTML + +This is the last change. I promise. + +You might have noticed that the title of the tab is displaying the path of the HTML file: + + +![](https://cdn-images-1.medium.com/max/1000/1*PASKm4ji29hbcZXVSP8afg.jpeg) + +We can use `` tag to define a title for our HTML file. The title tag also, like the link tag, goes inside head. Let’s put “Bat Letter” in our title: + +``` +<!DOCTYPE html> +<html> +<head> + <title>Bat Letter + + + +
+

Bat Letter

+ +

+ After all the battles we faught together, after all the difficult times we saw together, after all the good and bad moments we've been through, I think it's time I let you know how I feel about you. +

+

You are the light of my life

+

+ You complete my darkness with your light. I love: +

+
    +
  • the way you see good in the worse
    • +
  • the way you handle emotionally difficult situations
    • +
  • the way you look at Justice
    • +
+

+ I have learned a lot from you. You have occupied a special place in my heart over the time. +

+

I have a confession to make

+

+ It feels like my chest does have a heart. You make my heart beat. Your smile brings smile on my face, your pain brings pain to my heart. +

+

+ I don't show my emotions, but I think this man behind the mask is falling for you. +

+

I love you Superman.

+

+ Your not-so-secret-lover,
+ Batman +

+
+ + +``` + +Save and refresh, and you will see that instead of the file path, “Bat Letter” is now displayed on the tab. + +Batman’s Love Letter is now complete. + +Congratulations! You made Batman’s Love Letter in HTML. + + +![](https://cdn-images-1.medium.com/max/1000/1*qC8qtrYtxAB6cJfm9aVOOQ.jpeg) + +### What we learned + +We learned the following new concepts: + +* The structure of an HTML document + +* How to write elements in HTML (

) + +* How to write styles inside the element using the style attribute (this is called inline styling, avoid this as much as you can) + +* How to write styles of an element inside (this is called embedded styling) + +* How to write styles in a separate file and link to it in HTML using (this is called a linked stylesheet) + +* What is a tag name, attribute, opening tag, and closing tag + +* How to give an id to an element using id attribute + +* Tag selectors and id selectors in CSS + +We learned the following HTML tags: + +*

: for paragraphs + +*
: for line breaks + +*

    ,
  • : to display lists + +*
    : for grouping elements of our letter + +*

    ,

    : for heading and sub heading + +* : to insert an image + +* , : for bold and italic text styling + +*