diff --git a/sources/tech/20190110 Why isn-t open source hot among computer science students.md b/20180110 Why isn-t open source hot among computer science students.md similarity index 100% rename from sources/tech/20190110 Why isn-t open source hot among computer science students.md rename to 20180110 Why isn-t open source hot among computer science students.md diff --git a/published/20170802 Creating SWAP partition using FDISK - FALLOCATE commands.md b/published/20170802 Creating SWAP partition using FDISK - FALLOCATE commands.md new file mode 100644 index 0000000000..09e1d1fbaf --- /dev/null +++ b/published/20170802 Creating SWAP partition using FDISK - FALLOCATE commands.md @@ -0,0 +1,113 @@ +使用 fdisk 和 fallocate 命令创建交换分区 +====== + +交换分区在物理内存（RAM）被填满时用来保持内存中的内容。当 RAM 被耗尽，Linux 会将内存中不活动的页移动到交换空间中，从而空出内存给系统使用。虽然如此，但交换空间不应被认为是物理内存的替代品。 + +大多数情况下，建议交换内存的大小为物理内存的 1 到 2 倍。也就是说如果你有 8GB 内存, 那么交换空间大小应该介于8-16 GB。 + +若系统中没有配置交换分区，当内存耗尽后，系统可能会杀掉正在运行中的进程/应用，从而导致系统崩溃。在本文中，我们将学会如何为 Linux 系统添加交换分区，我们有两个办法： + +- 使用 fdisk 命令 +- 使用 fallocate 命令 + +### 第一个方法（使用 fdisk 命令） + +通常，系统的第一块硬盘会被命名为 `/dev/sda`，而其中的分区会命名为 `/dev/sda1` 、 `/dev/sda2`。 本文我们使用的是一块有两个主分区的硬盘，两个分区分别为 `/dev/sda1`、 `/dev/sda2`，而我们使用 `/dev/sda3` 来做交换分区。 + +首先创建一个新分区， + +``` +$ fdisk /dev/sda +``` + +按 `n` 来创建新分区。系统会询问你从哪个柱面开始，直接按回车键使用默认值即可。然后系统询问你到哪个柱面结束, 这里我们输入交换分区的大小（比如 1000MB）。这里我们输入 `+1000M`。 + +![swap][2] + +现在我们创建了一个大小为 1000MB 的磁盘了。但是我们并没有设置该分区的类型，我们按下 `t` 然后回车，来设置分区类型。 + +现在我们要输入分区编号，这里我们输入 `3`，然后输入磁盘分类号，交换分区的分区类型为 `82` （要显示所有可用的分区类型，按下 `l` ) ，然后再按下 `w` 保存磁盘分区表。 + +![swap][4] + +再下一步使用 `mkswap` 命令来格式化交换分区： + +``` +$ mkswap /dev/sda3 +``` + +然后激活新建的交换分区： + +``` +$ swapon /dev/sda3 +``` + +然而我们的交换分区在重启后并不会自动挂载。要做到永久挂载，我们需要添加内容到 `/etc/fstab` 文件中。打开 `/etc/fstab` 文件并输入下面行： + +``` +$ vi /etc/fstab + +/dev/sda3 swap swap default 0 0 +``` + +保存并关闭文件。现在每次重启后都能使用我们的交换分区了。 + +### 第二种方法（使用 fallocate 命令） + +我推荐用这种方法因为这个是最简单、最快速的创建交换空间的方法了。`fallocate` 是最被低估和使用最少的命令之一了。 `fallocate` 命令用于为文件预分配块/大小。 + +使用 `fallocate` 创建交换空间，我们首先在 `/` 目录下创建一个名为 `swap_space` 的文件。然后分配 2GB 到 `swap_space` 文件： + +``` +$ fallocate -l 2G /swap_space +``` + +我们运行下面命令来验证文件大小： + +``` +$ ls -lh /swap_space +``` + +然后更改文件权限，让 `/swap_space` 更安全： + +``` +$ chmod 600 /swap_space +``` + +这样只有 root 可以读写该文件了。我们再来格式化交换分区（LCTT 译注：虽然这个 `swap_space` 是个文件，但是我们把它当成是分区来挂载）： + +``` +$ mkswap /swap_space +``` + +然后启用交换空间： + +``` +$ swapon -s +``` + +每次重启后都要重新挂载磁盘分区。因此为了使之持久化，就像上面一样，我们编辑 `/etc/fstab` 并输入下面行： + +``` +/swap_space swap swap sw 0 0 +``` + +保存并退出文件。现在我们的交换分区会一直被挂载了。我们重启后可以在终端运行 `free -m` 来检查交换分区是否生效。 + +我们的教程至此就结束了，希望本文足够容易理解和学习，如果有任何疑问欢迎提出。 + +-------------------------------------------------------------------------------- + +via: http://linuxtechlab.com/create-swap-using-fdisk-fallocate/ + +作者：[Shusain][a] +译者：[lujun9972](https://github.com/lujun9972) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://linuxtechlab.com/author/shsuain/ +[1]:https://i1.wp.com/linuxtechlab.com/wp-content/plugins/a3-lazy-load/assets/images/lazy_placeholder.gif?resize=668%2C211 +[2]:https://i0.wp.com/linuxtechlab.com/wp-content/uploads/2017/02/fidsk.jpg?resize=668%2C211 +[3]:https://i1.wp.com/linuxtechlab.com/wp-content/plugins/a3-lazy-load/assets/images/lazy_placeholder.gif?resize=620%2C157 +[4]:https://i0.wp.com/linuxtechlab.com/wp-content/uploads/2017/02/fidsk-swap-select.jpg?resize=620%2C157 diff --git a/translated/tech/20170918 Linux fmt command - usage and examples.md b/published/20170918 Linux fmt command - usage and examples.md similarity index 68% rename from translated/tech/20170918 Linux fmt command - usage and examples.md rename to published/20170918 Linux fmt command - usage and examples.md index e9b1d8921a..5724d9abb2 100644 --- a/translated/tech/20170918 Linux fmt command - usage and examples.md +++ b/published/20170918 Linux fmt command - usage and examples.md @@ -1,13 +1,13 @@ -Linux fmt 命令 - 用法与案例 +Linux 的 fmt 命令用法与案例 ====== -有时你会发现需要格式化某个文本文件中的内容。比如，该文本文件每行一个单词，而人物是把所有的单词都放在同一行。当然，你可以手工来做，但没人喜欢手工做这么耗时的工作。而且，这只是一个例子 - 事实上的任务可能千奇百怪。 +有时你会发现需要格式化某个文本文件中的内容。比如，该文本文件每行一个单词，而任务是把所有的单词都放在同一行。当然，你可以手工来做，但没人喜欢手工做这么耗时的工作。而且，这只是一个例子 - 事实上的任务可能千奇百怪。 好在，有一个命令可以满足至少一部分的文本格式化的需求。这个工具就是 `fmt`。本教程将会讨论 `fmt` 的基本用法以及它提供的一些主要功能。文中所有的命令和指令都在 Ubuntu 16.04LTS 下经过了测试。 ### Linux fmt 命令 -fmt 命令是一个简单的文本格式化工具，任何人都能在命令行下运行它。它的基本语法为： +`fmt` 命令是一个简单的文本格式化工具，任何人都能在命令行下运行它。它的基本语法为： ``` fmt [-WIDTH] [OPTION]... [FILE]... @@ -15,15 +15,13 @@ fmt [-WIDTH] [OPTION]... [FILE]... 它的 man 页是这么说的： -``` -重新格式化文件FILE(s)中的每一个段落,将结果写到标准输出. 选项 -WIDTH 是 --width=DIGITS 形式的缩写 -``` +> 重新格式化文件中的每一个段落，将结果写到标准输出。选项 `-WIDTH` 是 `--width=DIGITS` 形式的缩写。 -下面这些问答方式的例子应该能让你对 fmt 的用法有很好的了解。 +下面这些问答方式的例子应该能让你对 `fmt` 的用法有很好的了解。 -### Q1。如何使用 fmt 来将文本内容格式成同一行？ +### Q1、如何使用 fmt 来将文本内容格式成同一行？ -使用 `fmt` 命令的基本格式（省略任何选项）就能做到这一点。你只需要将文件名作为参数传递给它。 +使用 `fmt` 命令的基本形式（省略任何选项）就能做到这一点。你只需要将文件名作为参数传递给它。 ``` fmt [file-name] @@ -33,9 +31,9 @@ fmt [file-name] [![format contents of file in single line][1]][2] -你可以看到文件中多行内容都被格式化成同一行了。请注意，这并不会修改原文件（也就是 file1）。 +你可以看到文件中多行内容都被格式化成同一行了。请注意，这并不会修改原文件（file1）。 -### Q2。如何修改最大行宽？ +### Q2、如何修改最大行宽？ 默认情况下，`fmt` 命令产生的输出中的最大行宽为 75。然而，如果你想的话，可以用 `-w` 选项进行修改，它接受一个表示新行宽的数字作为参数值。 @@ -47,7 +45,7 @@ fmt -w [n] [file-name] [![change maximum line width][3]][4] -### Q3。如何让 fmt 突出显示第一行？ +### Q3、如何让 fmt 突出显示第一行？ 这是通过让第一行的缩进与众不同来实现的，你可以使用 `-t` 选项来实现。 @@ -57,7 +55,7 @@ fmt -t [file-name] [![make fmt highlight the first line][5]][6] -### Q4。如何使用 fmt 拆分长行？ +### Q4、如何使用 fmt 拆分长行？ fmt 命令也能用来对长行进行拆分，你可以使用 `-s` 选项来应用该功能。 @@ -69,9 +67,9 @@ fmt -s [file-name] [![make fmt split long lines][7]][8] -### Q5。如何在单词与单词之间，行与行之间用空格分开？ +### Q5、如何在单词与单词之间，句子之间用空格分开？ -fmt 命令提供了一个 `-u` 选项，这会在单词与单词之间用单个空格分开，行与行之间用两个空格分开。你可以这样用： +fmt 命令提供了一个 `-u` 选项，这会在单词与单词之间用单个空格分开，句子之间用两个空格分开。你可以这样用： ``` fmt -u [file-name] @@ -81,7 +79,7 @@ fmt -u [file-name] ### 总结 -没错，fmt 提供的功能不多，但不代表它的应用就不广泛。因为你永远不知道什么时候会用到它。在本教程中，我们已经讲解了 `fmt` 提供的主要选项。若想了解更多细节，请查看该工具的 [man 页 ][9]。 +没错，`fmt` 提供的功能不多，但不代表它的应用就不广泛。因为你永远不知道什么时候会用到它。在本教程中，我们已经讲解了 `fmt` 提供的主要选项。若想了解更多细节，请查看该工具的 [man 页][9]。 -------------------------------------------------------------------------------- @@ -90,7 +88,7 @@ via: https://www.howtoforge.com/linux-fmt-command/ 作者：[Himanshu Arora][a] 译者：[lujun9972](https://github.com/lujun9972) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 diff --git a/published/20170919 What Are Bitcoins.md b/published/20170919 What Are Bitcoins.md new file mode 100644 index 0000000000..49a58ef9d1 --- /dev/null +++ b/published/20170919 What Are Bitcoins.md @@ -0,0 +1,76 @@ +比特币是什么？ +====== + +![what are bitcoins](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/what-are-bitcoins_orig.jpg) + +[比特币][1]Bitcoin 是一种数字货币或者说是电子现金，依靠点对点技术来完成交易。 由于使用点对点技术作为主要网络，比特币提供了一个类似于管制经济managed economy的社区。 这就是说，比特币消除了货币管理的集中式管理方式，促进了货币的社区管理。 大部分比特币数字现金的挖掘和管理软件也是开源的。 + +第一个比特币软件是由中本聪Satoshi Nakamoto开发的，基于开源的密码协议。 比特币最小单位被称为聪Satoshi，它基本上是一个比特币的百万分之一（0.00000001 BTC）。 + +人们不能低估比特币在数字经济中消除的界限。 例如，比特币消除了由中央机构对货币进行的管理控制，并将控制和管理提供给整个社区。 此外，比特币基于开放源代码密码协议的事实使其成为一个开放的领域，其中存在价值波动、通货紧缩和通货膨胀等严格的活动。 当许多互联网用户正在意识到他们在网上完成交易的隐私性时，比特币正在变得比以往更受欢迎。 但是，对于那些了解暗网及其工作原理的人们，可以确认有些人早就开始使用它了。 + +不利的一面是，比特币在匿名支付方面也非常安全，可能会对安全或个人健康构成威胁。 例如，暗网市场是进口药物甚至武器的主要供应商和零售商。 在暗网中使用比特币有助于这种犯罪活动。 尽管如此，如果使用得当，比特币有许多的好处，可以消除一些由于集中的货币代理管理导致的经济上的谬误。 另外，比特币允许在世界任何地方交换现金。 比特币的使用也可以减少货币假冒、印刷或贬值。 同时，依托对等网络作为骨干网络，促进交易记录的分布式权限，交易会更加安全。 + +比特币的其他优点包括： + +- 在网上商业世界里，比特币促进资金安全和完全控制。这是因为买家受到保护，以免商家可能想要为较低成本的服务额外收取钱财。买家也可以选择在交易后不分享个人信息。此外，由于隐藏了个人信息，也就保护了身份不被盗窃。 +- 对于主要的常见货币灾难，比如如丢失、冻结或损坏，比特币是一种替代品。但是，始终都建议对比特币进行备份并使用密码加密。 +- 使用比特币进行网上购物和付款时，收取的费用少或者不收取。这就提高了使用时的可承受性。 +- 与其他电子货币不同，商家也面临较少的欺诈风险，因为比特币交易是无法逆转的。即使在高犯罪率和高欺诈的时刻，比特币也是有用的，因为在公开的公共总账（区块链）上难以对付某个人。 +- 比特币货币也很难被操纵，因为它是开源的，密码协议是非常安全的。 +- 交易也可以随时随地进行验证和批准。这是数字货币提供的灵活性水准。 + +还可以阅读 - [Bitkey：专用于比特币交易的 Linux 发行版][2] + +### 如何挖掘比特币和完成必要的比特币管理任务的应用程序 + +在数字货币中，比特币挖矿和管理需要额外的软件。有许多开源的比特币管理软件，便于进行支付，接收付款，加密和备份比特币，还有很多的比特币挖掘软件。有些网站，比如：通过查看广告赚取免费比特币的 [Freebitcoin][4]，MoonBitcoin 是另一个可以免费注册并获得比特币的网站。但是，如果有空闲时间和相当多的人脉圈参与，会很方便。有很多提供比特币挖矿的网站，可以轻松注册然后开始挖矿。其中一个主要秘诀就是尽可能引入更多的人构建成一个大型的网络。 + +与比特币一起使用时需要的应用程序包括比特币钱包，使得人们可以安全的持有比特币。这就像使用实物钱包来保存硬通货币一样，而这里是以数字形式存在的。钱包可以在这里下载 —— [比特币-钱包][6]。其他类似的应用包括：与比特币钱包类似的[区块链][7]。 + +下面的屏幕截图分别显示了 Freebitco 和 MoonBitco 这两个挖矿网站。 + + [![freebitco bitcoin mining site](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/freebitco-bitcoin-mining-site_orig.jpg)][8] + + [![moonbitcoin bitcoin mining site](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/moonbitcoin-bitcoin-mining-site_orig.png)][9] + +获得比特币的方式多种多样。其中一些包括比特币挖矿机的使用，比特币在交易市场的购买以及免费的比特币在线采矿。比特币可以在 [MtGox][10]（LCTT 译注：本文比较陈旧，此交易所已经倒闭），[bitNZ][11]，[Bitstamp][12]，[BTC-E][13]，[VertEx][14] 等等这些网站买到，这些网站都提供了开源开源应用程序。这些应用包括：Bitminter、[5OMiner][15]，[BFG Miner][16] 等等。这些应用程序使用一些图形卡和处理器功能来生成比特币。在个人电脑上开采比特币的效率在很大程度上取决于显卡的类型和采矿设备的处理器。（LCTT 译注：目前个人挖矿已经几乎毫无意义了）此外，还有很多安全的在线存储用于备份比特币。这些网站免费提供比特币存储服务。比特币管理网站的例子包括：[xapo][17] , [BlockChain][18] 等。在这些网站上注册需要有效的电子邮件和电话号码进行验证。 Xapo 通过电话应用程序提供额外的安全性，无论何时进行新的登录都需要做请求验证。 + +### 比特币的缺点 + +使用比特币数字货币所带来的众多优势不容忽视。 但是，由于比特币还处于起步阶段，因此遇到了几个阻力点。 例如，大多数人没有完全意识到比特币数字货币及其工作方式。 缺乏意识可以通过教育和意识的创造来缓解。 比特币用户也面临波动，因为比特币的需求量高于可用的货币数量。 但是，考虑到更长的时间，很多人开始使用比特币的时候，波动性会降低。 + +### 改进点 + +基于[比特币技术][19]的起步，仍然有变化的余地使其更安全更可靠。 考虑到更长的时间，比特币货币将会发展到足以提供作为普通货币的灵活性。 为了让比特币成功，除了给出有关比特币如何工作及其好处的信息之外，还需要更多人了解比特币。 + +-------------------------------------------------------------------------------- + +via: http://www.linuxandubuntu.com/home/things-you-need-to-know-about-bitcoins + +作者：[LINUXANDUBUNTU][a] +译者：[Flowsnow](https://github.com/Flowsnow) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://www.linuxandubuntu.com/ +[1]:http://www.linuxandubuntu.com/home/bitkey-a-linux-distribution-dedicated-for-conducting-bitcoin-transactions +[2]:http://www.linuxandubuntu.com/home/bitkey-a-linux-distribution-dedicated-for-conducting-bitcoin-transactions +[3]:http://www.linuxandubuntu.com/home/things-you-need-to-know-about-bitcoins +[4]:https://freebitco.in/?r=2167375 +[5]:http://moonbit.co.in/?ref=c637809a5051 +[6]:https://bitcoin.org/en/choose-your-wallet +[7]:https://blockchain.info/wallet/ +[8]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/freebitco-bitcoin-mining-site_orig.jpg +[9]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/moonbitcoin-bitcoin-mining-site_orig.png +[10]:http://mtgox.com/ +[11]:https://en.bitcoin.it/wiki/BitNZ +[12]:https://www.bitstamp.net/ +[13]:https://btc-e.com/ +[14]:https://www.vertexinc.com/ +[15]:https://www.downloadcloud.com/bitcoin-miner-software.html +[16]:https://github.com/luke-jr/bfgminer +[17]:https://xapo.com/ +[18]:https://www.blockchain.com/ +[19]:https://en.wikipedia.org/wiki/Bitcoin diff --git a/translated/tech/20170924 Simulate System Loads.md b/published/20170924 Simulate System Loads.md similarity index 56% rename from translated/tech/20170924 Simulate System Loads.md rename to published/20170924 Simulate System Loads.md index 66b74be5c1..8c079664d9 100644 --- a/translated/tech/20170924 Simulate System Loads.md +++ b/published/20170924 Simulate System Loads.md @@ -1,71 +1,70 @@ -模拟系统负载的方法 +在 Linux 上简单模拟系统负载的方法 ====== + 系统管理员通常需要探索在不同负载对应用性能的影响。这意味着必须要重复地人为创造负载。当然，你可以通过专门的工具来实现，但有时你可能不想也无法安装新工具。 -每个 Linux 发行版中都自带有创建负载的工具。他们不如专门的工具那么灵活但它们是现成的，而且无需专门学习。 +每个 Linux 发行版中都自带有创建负载的工具。他们不如专门的工具那么灵活，但它们是现成的，而且无需专门学习。 ### CPU 下面命令会创建 CPU 负荷，方法是通过压缩随机数据并将结果发送到 `/dev/null`： + ``` cat /dev/urandom | gzip -9 > /dev/null - ``` 如果你想要更大的负荷，或者系统有多个核，那么只需要对数据进行压缩和解压就行了，像这样： + ``` cat /dev/urandom | gzip -9 | gzip -d | gzip -9 | gzip -d > /dev/null - ``` -按下 `CTRL+C` 来暂停进程。 +按下 `CTRL+C` 来终止进程。 -### RAM +### 内存占用 -下面命令会减少可用内存的总量。它是是通过在内存中创建文件系统然后往里面写文件来实现的。你可以使用任意多的内存，只需哟往里面写入更多的文件就行了。 +下面命令会减少可用内存的总量。它是通过在内存中创建文件系统然后往里面写文件来实现的。你可以使用任意多的内存，只需哟往里面写入更多的文件就行了。 + +首先，创建一个挂载点，然后将 ramfs 文件系统挂载上去： -首先，创建一个挂载点，然后将 `ramfs` 文件系统挂载上去： ``` mkdir z mount -t ramfs ramfs z/ - ``` 第二步，使用 `dd` 在该目录下创建文件。这里我们创建了一个 128M 的文件： + ``` dd if=/dev/zero of=z/file bs=1M count=128 - ``` 文件的大小可以通过下面这些操作符来修改： - + **bs=** 块大小。可以是任何数字后面接上 **B**( 表示字节 )，**K**( 表示 KB)，**M**( 表示 MB) 或者 **G**( 表示 GB)。 - + **count=** 要写多少个块 +- `bs=` 块大小。可以是任何数字后面接上 `B`（表示字节），`K`（表示 KB），`M`（ 表示 MB）或者 `G`（表示 GB）。 +- `count=` 要写多少个块。 +### 磁盘 I/O +创建磁盘 I/O 的方法是先创建一个文件，然后使用 `for` 循环来不停地拷贝它。 -### Disk +下面使用命令 `dd` 创建了一个全是零的 1G 大小的文件： -创建磁盘 I/O 的方法是先创建一个文件，然后使用 for 循环来不停地拷贝它。 - -下面使用命令 `dd` 创建了一个充满零的 1G 大小的文件： ``` dd if=/dev/zero of=loadfile bs=1M count=1024 - ``` -下面命令用 for 循环执行 10 次操作。每次都会拷贝 `loadfile` 来覆盖 `loadfile1`： +下面命令用 `for` 循环执行 10 次操作。每次都会拷贝 `loadfile` 来覆盖 `loadfile1`： + ``` for i in {1..10}; do cp loadfile loadfile1; done - ``` -通过修改 `{1。.10}` 中的第二个参数来调整运行时间的长短。 +通过修改 `{1..10}` 中的第二个参数来调整运行时间的长短。（LCTT 译注：你的 Linux 系统中的默认使用的 `cp` 命令很可能是 `cp -i` 的别名，这种情况下覆写会提示你输入 `y` 来确认，你可以使用 `-f` 参数的 `cp` 命令来覆盖此行为，或者直接用 `/bin/cp` 命令。） 若你想要一直运行，直到按下 `CTRL+C` 来停止，则运行下面命令： + ``` while true; do cp loadfile loadfile1; done - ``` -------------------------------------------------------------------------------- @@ -73,7 +72,7 @@ via: https://bash-prompt.net/guides/create-system-load/ 作者：[Elliot Cooper][a] 译者：[lujun9972](https://github.com/lujun9972) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 diff --git a/translated/tech/20170925 A Commandline Fuzzy Search Tool For Linux.md b/published/20170925 A Commandline Fuzzy Search Tool For Linux.md similarity index 55% rename from translated/tech/20170925 A Commandline Fuzzy Search Tool For Linux.md rename to published/20170925 A Commandline Fuzzy Search Tool For Linux.md index 9d16aaf1aa..d76309f820 100644 --- a/translated/tech/20170925 A Commandline Fuzzy Search Tool For Linux.md +++ b/published/20170925 A Commandline Fuzzy Search Tool For Linux.md @@ -1,34 +1,38 @@ -Pick - 一款 Linux 上的命令行模糊搜索工具 +Pick：一款 Linux 上的命令行模糊搜索工具 ====== -![](https：//www.ostechnix.com/wp-content/uploads/2017/09/search-720x340.jpg) -今天，我们要讲的是一款有趣的命令行工具，名叫 `Pick`。它允许用户通过 ncurses（3X） 界面来从一系列选项中进行选择，而且还支持模糊搜索的功能。当你想要选择某个名字中包含非英文字符的目录或文件时，这款工具就很有用了。你根本都无需学习如何输入非英文字符。借助 Pick，你可以很方便地进行搜索，选择，然后浏览该文件或进入该目录。你甚至无需输入任何字符来过滤文件/目录。这很适合那些有大量目录和文件的人来用。 +![](https://www.ostechnix.com/wp-content/uploads/2017/09/search-720x340.jpg) -### Pick - 一款 Linux 上的命令行模糊搜索工具 +今天，我们要讲的是一款有趣的命令行工具，名叫 Pick。它允许用户通过 ncurses(3X) 界面来从一系列选项中进行选择，而且还支持模糊搜索的功能。当你想要选择某个名字中包含非英文字符的目录或文件时，这款工具就很有用了。你根本都无需学习如何输入非英文字符。借助 Pick，你可以很方便地进行搜索、选择，然后浏览该文件或进入该目录。你甚至无需输入任何字符来过滤文件/目录。这很适合那些有大量目录和文件的人来用。 -#### 安装 Pick +### 安装 Pick + +对 Arch Linux 及其衍生品来说，Pick 放在 [AUR][1] 中。因此 Arch 用户可以使用类似 [Pacaur][2]，[Packer][3]，以及 [Yaourt][4] 等 AUR 辅助工具来安装它。 -对 **Arch Linux** 及其衍生品来说，pick 放在 [**AUR**][1] 中。因此 Arch 用户可以使用类似 [**Pacaur**][2]，[**Packer**][3]，以及 [**Yaourt**][4] 等 AUR 辅助工具来安装它。 ``` pacaur -S pick ``` 或者， + ``` packer -S pick ``` 或者， + ``` yaourt -S pick ``` -**Debian**，**Ubuntu**，**Linux Mint** 用户则可以通过运行下面命令来安装 Pick。 +Debian，Ubuntu，Linux Mint 用户则可以通过运行下面命令来安装 Pick。 + ``` sudo apt-get install pick ``` -其他的发行版则可以从[**这里 **][5] 下载最新的安装包，然后按照下面的步骤来安装。在写本指南时，其最新版为 1.9.0。 +其他的发行版则可以从[这里][5]下载最新的安装包，然后按照下面的步骤来安装。在写本指南时，其最新版为 1.9.0。 + ``` wget https://github.com/calleerlandsson/pick/releases/download/v1.9.0/pick-1.9.0.tar.gz tar -zxvf pick-1.9.0.tar.gz @@ -36,81 +40,87 @@ cd pick-1.9.0/ ``` 使用下面命令进行配置： + ``` ./configure ``` -最后，构建并安装 pick： +最后，构建并安装 Pick： + ``` make sudo make install ``` -#### 用法 +### 用法 通过将它与其他命令集成能够大幅简化你的工作。我这里会给出一些例子，让你理解它是怎么工作的。 让们先创建一堆目录。 + ``` mkdir -p abcd/efgh/ijkl/mnop/qrst/uvwx/yz/ ``` -现在，你想进入目录 `/ijkl/`。你有两种选择。可以使用 **cd** 命令： +现在，你想进入目录 `/ijkl/`。你有两种选择。可以使用 `cd` 命令： + ``` cd abcd/efgh/ijkl/ ``` -或者，创建一个[**快捷方式 **][6] 或者说别名指向这个目录，这样你可以迅速进入该目录。 +或者，创建一个[快捷方式][6] 或者说别名指向这个目录，这样你可以迅速进入该目录。 + +但，使用 `pick` 命令则问题变得简单的多。看下面这个例子。 -但，使用 "pick" 命令则问题变得简单的多。看下面这个例子。 ``` cd $(find . -type d | pick) ``` 这个命令会列出当前工作目录下的所有目录及其子目录，你可以用上下箭头选择你想进入的目录，然后按下回车就行了。 -**像这样：** +像这样： -[![][7]][8] +![][8] 而且，它还会根据你输入的内容过滤目录和文件。比如，当我输入 “or” 时会显示如下结果。 -[![][7]][9] +![][9] -这只是一个例子。你也可以将 “pick” 命令跟其他命令一起混用。 +这只是一个例子。你也可以将 `pick` 命令跟其他命令一起混用。 这是另一个例子。 + ``` find -type f | pick | xargs less ``` -该命令让你选择当前目录中的某个文件并用 less 来查看它。 +该命令让你选择当前目录中的某个文件并用 `less` 来查看它。 -[![][7]][10] +![][10] + +还想看其他例子？还有呢。下面命令让你选择当前目录下的文件或目录，并将之迁移到其他地方去，比如这里我们迁移到 `/home/sk/ostechnix`。 -还想看其他例子？还有呢。下面命令让你选择当前目录下的文件或目录，并将之迁移到其他地方去，比如这里我们迁移到 **/home/sk/ostechnix**。 ``` mv "$(find . -maxdepth 1 |pick)" /home/sk/ostechnix/ ``` -[![][7]][11] +![][11] 通过上下按钮选择要迁移的文件，然后按下回车就会把它迁移到 `/home/sk/ostechnix/` 目录中的。 -[![][7]][12] +![][12] -从上面的结果中可以看到，我把一个名叫 “abcd” 的目录移动到 "ostechnix" 目录中了。 +从上面的结果中可以看到，我把一个名叫 `abcd` 的目录移动到 `ostechnix` 目录中了。 -使用案例是无限的。甚至 Vim 编辑器上还有一个叫做 [**pick.vim**][13] 的插件让你在 Vim 中选择更加方便。 +使用方式是无限的。甚至 Vim 编辑器上还有一个叫做 [pick.vim][13] 的插件让你在 Vim 中选择更加方便。 要查看详细信息，请参阅它的 man 页。 + ``` man pick ``` -我们的讲解至此就结束了。希望这狂工具能给你们带来帮助。如果你觉得我们的指南有用的话，请将它分享到您的社交网络上，并向大家推荐 OSTechNix 博客。 - - +我们的讲解至此就结束了。希望这款工具能给你们带来帮助。如果你觉得我们的指南有用的话，请将它分享到您的社交网络上，并向大家推荐我们。 -------------------------------------------------------------------------------- @@ -118,7 +128,7 @@ via: https://www.ostechnix.com/pick-commandline-fuzzy-search-tool-linux/ 作者：[SK][a] 译者：[lujun9972](https://github.com/lujun9972) -校对：[校对者ID](https://github.com/校对者ID) +校对：[wxy](https://github.com/wxy) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 @@ -130,9 +140,9 @@ via: https://www.ostechnix.com/pick-commandline-fuzzy-search-tool-linux/ [5]:https://github.com/calleerlandsson/pick/releases/ [6]:https://www.ostechnix.com/create-shortcuts-frequently-used-directories-shell/ [7]:data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7 -[8]:http://www.ostechnix.com/wp-content/uploads/2017/09/sk@sk_001-3.png () -[9]:http://www.ostechnix.com/wp-content/uploads/2017/09/sk@sk_002-1.png () -[10]:http://www.ostechnix.com/wp-content/uploads/2017/09/sk@sk_004-1.png () -[11]:http://www.ostechnix.com/wp-content/uploads/2017/09/sk@sk_005.png () -[12]:http://www.ostechnix.com/wp-content/uploads/2017/09/sk@sk_006-1.png () +[8]:http://www.ostechnix.com/wp-content/uploads/2017/09/sk@sk_001-3.png +[9]:http://www.ostechnix.com/wp-content/uploads/2017/09/sk@sk_002-1.png +[10]:http://www.ostechnix.com/wp-content/uploads/2017/09/sk@sk_004-1.png +[11]:http://www.ostechnix.com/wp-content/uploads/2017/09/sk@sk_005.png +[12]:http://www.ostechnix.com/wp-content/uploads/2017/09/sk@sk_006-1.png [13]:https://github.com/calleerlandsson/pick.vim/ diff --git a/published/20170925 Linux Free Command Explained for Beginners (6 Examples).md b/published/20170925 Linux Free Command Explained for Beginners (6 Examples).md new file mode 100644 index 0000000000..6e810f9f28 --- /dev/null +++ b/published/20170925 Linux Free Command Explained for Beginners (6 Examples).md @@ -0,0 +1,140 @@ +6 个例子让初学者掌握 free 命令 +====== + +在 Linux 系统上，有时你可能想从命令行快速地了解系统的已使用和未使用的内存空间。如果你是一个 Linux 新手，有个好消息：有一条系统内置的命令可以显示这些信息：`free`。 + +在本文中，我们会讲到 free 命令的基本用法以及它所提供的一些重要的功能。文中提到的所有命令和用法都是在 Ubuntu 16.04LTS 上测试过的。 + +### Linux free 命令 + +让我们看一下 `free` 命令的语法： + +``` +free [options] +``` + +free 命令的 man 手册如是说： + +> `free` 命令显示了系统的可用和已用的物理内存及交换内存的总量，以及内核用到的缓存空间。这些信息是从 `/proc/meminfo` 中得到的。 + +接下来我们用问答的方式了解一下 `free` 命令是怎么工作的。 + +### Q1. 怎么用 free 命令查看已使用和未使用的内存？ + +这很容易，您只需不加任何参数地运行 `free` 这条命令就可以了： + +``` +free +``` + +这是 `free` 命令在我的系统上的输出： + +[![view used and available memory using free command][1]][2] + +这些列是什么意思呢？ + +[![Free command columns][3]][4] + +- `total` - 安装的内存的总量（等同于 `/proc/meminfo` 中的 `MemTotal` 和 `SwapTotal`） +- `used` - 已使用的内存（计算公式为：`used` = `total` - `free` - `buffers` - `cache`） +- `free` - 未被使用的内存（等同于 `/proc/meminfo` 中的 `MemFree` 和 `SwapFree`） +- `shared` - 通常是临时文件系统使用的内存（等同于 `/proc/meminfo` 中的 `Shmem`；自内核 2.6.32 版本可用，不可用则显示为 `0`） +- `buffers` - 内核缓冲区使用的内存（等同于 `/proc/meminfo` 中的 `Buffers`） +- `cache` - 页面缓存和 Slab 分配机制使用的内存（等同于 `/proc/meminfo` 中的 `Cached` 和 `Slab`） +- `buff/cache` - `buffers` 与 `cache` 之和 +- `available` - 在不计算交换空间的情况下，预计可以被新启动的应用程序所使用的内存空间。与 `cache` 或者 `free` 部分不同，这一列把页面缓存计算在内，并且不是所有的可回收的 slab 内存都可以真正被回收，因为可能有被占用的部分。（等同于 `/proc/meminfo` 中的 `MemAvailable`；自内核 3.14 版本可用，自内核 2.6.27 版本开始模拟；在其他版本上这个值与 `free` 这一列相同） + +### Q2. 如何更改显示的单位呢？ + +如果需要的话，你可以更改内存的显示单位。比如说，想要内存以兆为单位显示，你可以用 `-m` 这个参数： + +``` +free -m +``` + +[![free command display metrics change][5]][6] + +同样地，你可以用 `-b` 以字节显示、`-k` 以 KB 显示、`-m` 以 MB 显示、`-g` 以 GB 显示、`--tera` 以 TB 显示。 + +### Q3. 怎么显示可读的结果呢？ + +`free` 命令提供了 `-h` 这个参数使输出转化为可读的格式。 + +``` +free -h +``` + +用这个参数，`free` 命令会自己决定用什么单位显示内存的每个数值。例如： + +[![diplsy data fromm free command in human readable form][7]][8] + +### Q4. 怎么让 free 命令以一定的时间间隔持续运行？ + +您可以用 `-s` 这个参数让 `free` 命令以一定的时间间隔持续地执行。您需要传递给命令行一个数字参数，做为这个时间间隔的秒数。 + +例如，使 `free` 命令每隔 3 秒执行一次： + +``` +free -s 3 +``` + +如果您需要 `free` 命令只执行几次，您可以用 `-c` 这个参数指定执行的次数： + +``` +free -s 3 -c 5 +``` + +上面这条命令可以确保 `free` 命令每隔 3 秒执行一次，总共执行 5 次。 + +注：这个功能目前在 Ubuntu 系统上还存在 [问题][9]，所以并未测试。 + +### Q5. 怎么使 free 基于 1000 计算内存，而不是 1024？ + +如果您指定 `free` 用 MB 来显示内存（用 `-m` 参数），但又想基于 1000 来计算结果，可以用 `--sj` 这个参数来实现。下图展示了用与不用这个参数的结果： + +[![How to make free use power of 1000 \(not 1024\) while displaying memory figures][10]][11] + +### Q6. 如何使 free 命令显示每一列的总和？ + +如果您想要 `free` 命令显示每一列的总和，你可以用 `-t` 这个参数。 + +``` +free -t +``` + +如下图所示： + +[![How to make free display total of columns][12]][13] + +请注意 `Total` 这一行出现了。 + +### 总结 + +`free` 命令对于系统管理来讲是个极其有用的工具。它有很多参数可以定制化您的输出，易懂易用。我们在本文中也提到了很多有用的参数。练习完之后，请您移步至 [man 手册][14]了解更多内容。 + + +-------------------------------------------------------------------------------- + +via: https://www.howtoforge.com/linux-free-command/ + +作者：[Himanshu Arora][a] +译者：[jessie-pang](https://github.com/jessie-pang) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.howtoforge.com +[1]:https://www.howtoforge.com/images/linux_free_command/free-command-output.png +[2]:https://www.howtoforge.com/images/linux_free_command/big/free-command-output.png +[3]:https://www.howtoforge.com/images/linux_free_command/free-output-columns.png +[4]:https://www.howtoforge.com/images/linux_free_command/big/free-output-columns.png +[5]:https://www.howtoforge.com/images/linux_free_command/free-m-option.png +[6]:https://www.howtoforge.com/images/linux_free_command/big/free-m-option.png +[7]:https://www.howtoforge.com/images/linux_free_command/free-h.png +[8]:https://www.howtoforge.com/images/linux_free_command/big/free-h.png +[9]:https://bugs.launchpad.net/ubuntu/+source/procps/+bug/1551731 +[10]:https://www.howtoforge.com/images/linux_free_command/free-si-option.png +[11]:https://www.howtoforge.com/images/linux_free_command/big/free-si-option.png +[12]:https://www.howtoforge.com/images/linux_free_command/free-t-option.png +[13]:https://www.howtoforge.com/images/linux_free_command/big/free-t-option.png +[14]:https://linux.die.net/man/1/free diff --git a/published/20171011 What is a firewall.md b/published/20171011 What is a firewall.md new file mode 100644 index 0000000000..d854340ab6 --- /dev/null +++ b/published/20171011 What is a firewall.md @@ -0,0 +1,81 @@ +什么是防火墙？ +===== + +> 流行的防火墙是多数组织主要的边界防御。 + +![](https://images.techhive.com/images/article/2017/04/firewall-100716789-large.jpg) + +基于网络的防火墙已经在美国企业无处不在，因为它们证实了抵御日益增长的威胁的防御能力。 + +通过网络测试公司 NSS 实验室最近的一项研究发现，高达 80% 的美国大型企业运行着下一代防火墙。研究公司 IDC 评估防火墙和相关的统一威胁管理市场的营业额在 2015 是 76 亿美元，预计到 2020 年底将达到 127 亿美元。 + +**如果你想升级，这里是《[当部署下一代防火墙时要考虑什么》][1]** + +### 什么是防火墙？ + +防火墙作为一个边界防御工具，其监控流量——要么允许它、要么屏蔽它。 多年来，防火墙的功能不断增强，现在大多数防火墙不仅可以阻止已知的一些威胁、执行高级访问控制列表策略，还可以深入检查流量中的每个数据包，并测试包以确定它们是否安全。大多数防火墙都部署为用于处理流量的网络硬件，和允许终端用户配置和管理系统的软件。越来越多的软件版防火墙部署到高度虚拟化的环境中，以在被隔离的网络或 IaaS 公有云中执行策略。 + +随着防火墙技术的进步，在过去十年中创造了新的防火墙部署选择，所以现在对于部署防火墙的最终用户来说，有了更多选择。这些选择包括： + +### 有状态的防火墙 + +当防火墙首次创造出来时，它们是无状态的，这意味着流量所通过的硬件当单独地检查被监视的每个网络流量包时，屏蔽或允许是隔离的。从 1990 年代中后期开始，防火墙的第一个主要进展是引入了状态。有状态防火墙在更全面的上下文中检查流量，同时考虑到网络连接的工作状态和特性，以提供更全面的防火墙。例如，维持这个状态的防火墙可以允许某些流量访问某些用户，同时对其他用户阻塞同一流量。 + +### 基于代理的防火墙 + +这些防火墙充当请求数据的最终用户和数据源之间的网关。在传递给最终用户之前，所有的流量都通过这个代理过滤。这通过掩饰信息的原始请求者的身份来保护客户端不受威胁。 + +### Web 应用防火墙（WAF） + +这些防火墙位于特定应用的前面，而不是在更广阔的网络的入口或者出口上。基于代理的防火墙通常被认为是保护终端客户的，而 WAF 则被认为是保护应用服务器的。 + +### 防火墙硬件 + +防火墙硬件通常是一个简单的服务器，它可以充当路由器来过滤流量和运行防火墙软件。这些设备放置在企业网络的边缘，位于路由器和 Internet 服务提供商（ISP）的连接点之间。通常企业可能在整个数据中心部署十几个物理防火墙。 用户需要根据用户基数的大小和 Internet 连接的速率来确定防火墙需要支持的吞吐量容量。 + +### 防火墙软件 + +通常，终端用户部署多个防火墙硬件端和一个中央防火墙软件系统来管理该部署。 这个中心系统是配置策略和特性的地方，在那里可以进行分析，并可以对威胁作出响应。 + +### 下一代防火墙（NGFW） + +多年来，防火墙增加了多种新的特性，包括深度包检查、入侵检测和防御以及对加密流量的检查。下一代防火墙（NGFW）是指集成了许多先进的功能的防火墙。 + +#### 有状态的检测 + +阻止已知不需要的流量，这是基本的防火墙功能。 + +#### 反病毒 + +在网络流量中搜索已知病毒和漏洞，这个功能有助于防火墙接收最新威胁的更新，并不断更新以保护它们。 + +#### 入侵防御系统（IPS） + +这类安全产品可以部署为一个独立的产品，但 IPS 功能正逐步融入 NGFW。 虽然基本的防火墙技术可以识别和阻止某些类型的网络流量，但 IPS 使用更细粒度的安全措施，如签名跟踪和异常检测，以防止不必要的威胁进入公司网络。 这一技术的以前版本是入侵检测系统（IDS），其重点是识别威胁而不是遏制它们，已经被 IPS 系统取代了。 + +#### 深度包检测（DPI） + +DPI 可作为 IPS 的一部分或与其结合使用，但其仍然成为一个 NGFW 的重要特征，因为它提供细粒度分析流量的能力，可以具体到流量包头和流量数据。DPI 还可以用来监测出站流量，以确保敏感信息不会离开公司网络，这种技术称为数据丢失防御（DLP）。 + +#### SSL 检测 + +安全套接字层（SSL）检测是一个检测加密流量来测试威胁的方法。随着越来越多的流量进行加密，SSL 检测成为 NGFW 正在实施的 DPI 技术的一个重要组成部分。SSL 检测作为一个缓冲区，它在送到最终目的地之前解码流量以检测它。 + +#### 沙盒 + +这个是被卷入 NGFW 中的一个较新的特性，它指防火墙接收某些未知的流量或者代码，并在一个测试环境运行，以确定它是否存在问题的能力。 + +-------------------------------------------------------------------------------- + +via: https://www.networkworld.com/article/3230457/lan-wan/what-is-a-firewall-perimeter-stateful-inspection-next-generation.html + +作者：[Brandon Butler][a] +译者：[zjon](https://github.com/zjon) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.networkworld.com/author/Brandon-Butler/ +[1]:https://www.networkworld.com/article/3236448/lan-wan/what-to-consider-when-deploying-a-next-generation-firewall.html + + diff --git a/published/20180101 The mysterious case of the Linux Page Table Isolation patches.md b/published/20180101 The mysterious case of the Linux Page Table Isolation patches.md new file mode 100644 index 0000000000..f4b2d1568c --- /dev/null +++ b/published/20180101 The mysterious case of the Linux Page Table Isolation patches.md @@ -0,0 +1,139 @@ +关于 Linux 页面表隔离补丁的神秘情况 +===== + +**[本文勘误与补充][1]** + +_长文预警：_ 这是一个目前严格限制的、禁止披露的安全 bug（LCTT 译注：目前已经部分披露），它影响到目前几乎所有实现虚拟内存的 CPU 架构，需要硬件的改变才能完全解决这个 bug。通过软件来缓解这种影响的紧急开发工作正在进行中，并且最近在 Linux 内核中已经得以实现，并且，在 11 月份，在 NT 内核中也开始了一个类似的紧急开发。在最糟糕的情况下，软件修复会导致一般工作负载出现巨大的减速（LCTT 译注：外在表现为 CPU 性能下降）。这里有一个提示，攻击会影响虚拟化环境，包括 Amazon EC2 和 Google 计算引擎，以及另外的提示是，这种精确的攻击可能涉及一个新的 Rowhammer 变种（LCTT 译注：一个由 Google 安全团队提出的 DRAM 的安全漏洞，在文章的后面部分会简单介绍）。 + +我一般不太关心安全问题，但是，对于这个 bug 我有点好奇，而一般会去写这个主题的人似乎都很忙，要么就是知道这个主题细节的人会保持沉默。这让我在新年的第一天（元旦那天）花了几个小时深入去挖掘关于这个谜团的更多信息，并且我将这些信息片断拼凑到了一起。 + +注意，这是一件相互之间高度相关的事件，因此，它的主要描述都是猜测，除非过一段时间，它的限制禁令被取消。我所看到的，包括涉及到的供应商、许多争论和这种戏剧性场面，将在限制禁令取消的那一天出现。 + +### LWN + +这个事件的线索出现于 12 月 20 日 LWN 上的 [内核页面表的当前状况：页面隔离][2]这篇文章。从文章语气上明显可以看到这项工作的紧急程度，内核的核心开发者紧急加入了 [KAISER 补丁系列][3]的开发——它由奥地利的 [TU Graz][4] 的一组研究人员首次发表于去年 10 月份。 + +这一系列的补丁的用途从概念上说很简单：为了阻止运行在用户空间的进程在进程页面表中通过映射得到内核空间页面的各种攻击方式，它可以很好地阻止了从非特权的用户空间代码中识别到内核虚拟地址的攻击企图。 + +这个小组在描述 KAISER 的论文《[KASLR 已死：KASLR 永存][5]》摘要中特别指出，当用户代码在 CPU 上处于活动状态的时候，在内存管理硬件中删除所有内核地址空间的信息。 + +这个补丁集的魅力在于它触及到了核心，内核的全部基柱（以及与用户空间的接口），显然，它应该被最优先考虑。遍观 Linux 中内存管理方面的变化，通常某个变化的首次引入会发生在该改变被合并的很久之前，并且，通常会进行多次的评估、拒绝、以及因各种原因爆发争论的一系列过程。 + +而 KAISER（就是现在的 KPTI）系列（从引入到）被合并还不足三个月。 + +### ASLR 概述 + +从表面上看，这些补丁设计以确保地址空间布局随机化Address Space Layout Randomization（ASLR）仍然有效：这是一个现代操作系统的安全特性，它试图将更多的随机位引入到公共映射对象的地址空间中。 + +例如，在引用 `/usr/bin/python` 时，动态链接将对系统的 C 库、堆、线程栈、以及主要的可执行文件进行排布，去接受随机分配的地址范围： + +``` +$ bash -c ‘grep heap /proc/$$/maps’ +019de000-01acb000 rw-p 00000000 00:00 0                                  [heap] +$ bash -c 'grep heap /proc/$$/maps’ +023ac000-02499000 rw-p 00000000 00:00 0                                  [heap] +``` +注意两次运行的 bash 进程的堆（heap）的开始和结束偏移量上的变化。 + +如果一个缓存区管理的 bug 将导致攻击者可以去覆写一些程序代码指向的内存地址，而那个地址之后将在程序控制流中使用，这样这种攻击者就可以使控制流转向到一个包含他们所选择的内容的缓冲区上。而这个特性的作用是，对于攻击者来说，使用机器代码来填充缓冲区做他们想做的事情（例如，调用 `system()` C 库函数）将更困难，因为那个函数的地址在不同的运行进程上不同的。 + +这是一个简单的示例，ASLR 被设计用于去保护类似这样的许多场景，包括阻止攻击者了解有可能被用来修改控制流的程序数据的地址或者实现一个攻击。 + +KASLR 是应用到内核本身的一个 “简化的” ASLR：在每个重新引导的系统上，属于内核的地址范围是随机的，这样就使得，虽然被攻击者操控的控制流运行在内核模式上，但是，他们不能猜测到为实现他们的攻击目的所需要的函数和结构的地址，比如，定位当前进程的数据段，将活动的 UID 从一个非特权用户提升到 root 用户，等等。 + +### 坏消息：缓减这种攻击的软件运行成本过于贵重 + +之前的方式，Linux 将内核的内存映射到用户内存的同一个页面表中的主要原因是，当用户的代码触发一个系统调用、故障、或者产生中断时，就不需要改变正在运行的进程的虚拟内存布局。 + +因为它不需要去改变虚拟内存布局，进而也就不需要去清洗掉（flush）依赖于该布局的与 CPU 性能高度相关的缓存（LCTT 译注：意即如果清掉这些高速缓存，CPU 性能就会下降），而主要是通过 [转换查找缓冲器][6]Translation Lookaside Buffer（TLB）（LCTT 译注：TLB ，将虚拟地址转换为物理地址）。 + +随着页面表分割补丁的合并，内核每次开始运行时，需要将内核的缓存清掉，并且，每次用户代码恢复运行时都会这样。对于大多数工作负载，在每个系统调用中，TLB 的实际总损失将导致明显的变慢：[@grsecurity 测量的一个简单的案例][7]，在一个最新的 AMD CPU 上，Linux `du -s` 命令变慢了 50%。 + +### 34C3 + +在今年的 CCC 大会上，你可以找到 TU Graz 的另外一位研究人员，《[描述了一个纯 Javascript 的 ASLR 攻击][8]》，通过仔细地掌握 CPU 内存管理单元的操作时机，遍历了描述虚拟内存布局的页面表，来实现 ASLR 攻击。它通过高度精确的时间掌握和选择性回收的 CPU 缓存行的组合方式来实现这种结果，一个运行在 web 浏览器的 Javascript 程序可以找回一个 Javascript 对象的虚拟地址，使得可以利用浏览器内存管理 bug 进行接下来的攻击。（LCTT 译注：本文作者勘误说，上述链接 CCC 的讲演与 KAISER 补丁完全无关，是作者弄错了） + +因此，从表面上看，我们有一组 KAISER 补丁，也展示了解除 ASLR 化地址的技术，并且，这个展示使用的是 Javascript，它很快就可以在一个操作系统内核上进行重新部署。 + +### 虚拟内存概述 + +在通常情况下，当一些机器码尝试去加载、存储、或者跳转到一个内存地址时，现代的 CPU 必须首先去转换这个 _虚拟地址_ 到一个 _物理地址_ ，这是通过遍历一系列操作系统托管的数组（被称为页面表）的方式进行的，这些数组描述了虚拟地址和安装在这台机器上的物理内存之间的映射。 + +在现代操作系统中，虚拟内存可能是最重要的强大特性：它可以避免什么发生呢？例如，一个濒临死亡的进程崩溃了操作系统、一个 web 浏览器 bug 崩溃了你的桌面环境、或者一个运行在 Amazon EC2 中的虚拟机的变化影响了同一台主机上的另一个虚拟机。 + +这种攻击的原理是，利用 CPU 上维护的大量的缓存，通过仔细地操纵这些缓存的内容，它可以去推测内存管理单元的地址，以去访问页面表的不同层级，因为一个未缓存的访问将比一个缓存的访问花费更长的时间（以实时而言）。通过检测页面表上可访问的元素，它可能能够恢复在 MMU（LCTT 译注：存储器管理单元）忙于解决的虚拟地址中的大部分比特（bits）。 + +### 这种动机的证据，但是不用恐慌 + +我们找到了动机，但是到目前为止，我们并没有看到这项工作引进任何恐慌。总的来说，ASLR 并不能完全缓减这种风险，并且也是一道最后的防线：仅在这 6 个月的周期内，即便是一个没有安全意识的人也能看到一些关于解除（unmasking） ASLR 化的指针的新闻，并且，实际上这种事从 ASLR 出现时就有了。 + +单独的修复 ASLR 并不足于去描述这项工作高优先级背后的动机。 + +### 它是硬件安全 bug 的证据 + +通过阅读这一系列补丁，可以明确许多事情。 + +第一，正如 [@grsecurity 指出][9] 的，代码中的一些注释已经被编辑掉了（redacted），并且，描述这项工作的附加的主文档文件已经在 Linux 源代码树中看不到了。 + +通过检查代码，它以运行时补丁的方式构建，在系统引导时仅当内核检测到是受影响的系统时才会被应用，与对臭名昭著的 [Pentium F00F bug][10] 的缓解措施，使用完全相同的机制： + +![](http://78.media.tumblr.com/1c80c45e14c1e676b35cdd89cc9b557c/tumblr_inline_p1untxZBBD1rkm8fh_500.jpg) + +### 更多的线索：Microsoft 也已经实现了页面表的分割 + +通过对 FreeBSD 源代码的一个简单挖掘可以看出，目前，其它的自由操作系统没有实现页面表分割，但是，通过 [Alex Ioniscu 在 Twitter][11] 上的提示，这项工作已经不局限于 Linux 了：从 11 月起，公开的 NT 内核也已经实现了同样的技术。 + +### 猜测：Rowhammer + +对 TU Graz 研究人员的工作的进一步挖掘，我们找到这篇 《[当 rowhammer 仅敲一次][12]》，这是 12 月 4 日通告的一个 [新的 Rowhammer 攻击的变种][13]： + +> 在这篇论文中，我们提出了新的 Rowhammer 攻击和漏洞的原始利用方式，表明即便是组合了所有防御也没有效果。我们的新攻击技术，对一个位置的反复 “敲打”（hammering），打破了以前假定的触发 Rowhammer bug 的前提条件。 + +快速回顾一下，Rowhammer 是多数（全部？）种类的商业 DRAM 的一类根本性问题，比如，在普通的计算机中的内存上。通过精确操作内存中的一个区域，这可能会导致内存该区域存储的相关（但是逻辑上是独立的）内容被毁坏。效果是，Rowhammer 可能被用于去反转内存中的比特（bits），使未经授权的用户代码可以访问到，比如，这个比特位描述了系统中的其它代码的访问权限。 + +我发现在 Rowhammer 上，这项工作很有意思，尤其是它反转的位接近页面表分割补丁时，但是，因为 Rowhammer 攻击要求一个目标：你必须知道你尝试去反转的比特在内存中的物理地址，并且，第一步是得到的物理地址可能是一个虚拟地址，就像在 KASLR 中的解除（unmasking）工作。 + +### 猜测：它影响主要的云供应商 + +在我能看到的内核邮件列表中，除了该子系统维护者的名字之外，e-mail 地址属于 Intel、Amazon 和 Google 的雇员，这表示这两个大的云计算供应商对此特别感兴趣，这为我们提供了一个强大的线索，这项工作很大的可能是受虚拟化安全驱动的。 + +它可能会导致产生更多的猜测：虚拟机 RAM 和由这些虚拟机所使用的虚拟内存地址，最终表示为在主机上大量的相邻的数组，那些数组，尤其是在一个主机上只有两个租户的情况下，在 Xen 和 Linux 内核中是通过内存分配来确定的，这样可能会有（准确性）非常高的可预测行为。 + +### 最喜欢的猜测：这是一个提升特权的攻击 + +把这些综合到一起，我并不难预测，可能是我们在 2018 年会使用的这些存在提升特权的 bug 的发行版，或者类似的系统推动了如此紧急的进展，并且在补丁集的抄送列表中出现如此多的感兴趣者的名字。 + +最后的一个趣闻，虽然我在阅读补丁集的时候没有找到我要的东西，但是，在一些代码中标记，paravirtual 或者 HVM Xen 是不受此影响的。 + +### 吃瓜群众表示 2018 将很有趣 + +这些猜想是完全有可能的，它离实现很近，但是可以肯定的是，当这些事情被公开后，那将是一个非常令人激动的几个星期。 + +-------------------------------------------------------------------------------- + +via: http://pythonsweetness.tumblr.com/post/169166980422/the-mysterious-case-of-the-linux-page-table + +作者：[python sweetness][a] +译者：[qhwdw](https://github.com/qhwdw) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://pythonsweetness.tumblr.com/ +[1]:http://pythonsweetness.tumblr.com/post/169217189597/quiet-in-the-peanut-gallery +[2]:https://linux.cn/article-9201-1.html +[3]:https://lwn.net/Articles/738975/ +[4]:https://www.iaik.tugraz.at/content/research/sesys/ +[5]:https://gruss.cc/files/kaiser.pdf +[6]:https://en.wikipedia.org/wiki/Translation_lookaside_buffer +[7]:https://twitter.com/grsecurity/status/947439275460702208 +[8]:https://www.youtube.com/watch?v=ewe3-mUku94 +[9]:https://twitter.com/grsecurity/status/947147105684123649 +[10]:https://en.wikipedia.org/wiki/Pentium_F00F_bug +[11]:https://twitter.com/aionescu/status/930412525111296000 +[12]:https://www.tugraz.at/en/tu-graz/services/news-stories/planet-research/singleview/article/wenn-rowhammer-nur-noch-einmal-klopft/ +[13]:https://arxiv.org/abs/1710.00551 +[14]:http://pythonsweetness.tumblr.com/post/169166980422/the-mysterious-case-of-the-linux-page-table +[15]:http://pythonsweetness.tumblr.com/ + + diff --git a/published/20180112 City of Barcelona Kicks Out Microsoft in Favor of Linux and Open Source.md b/published/20180112 City of Barcelona Kicks Out Microsoft in Favor of Linux and Open Source.md new file mode 100644 index 0000000000..2103838406 --- /dev/null +++ b/published/20180112 City of Barcelona Kicks Out Microsoft in Favor of Linux and Open Source.md @@ -0,0 +1,62 @@ +巴塞罗那城放弃微软，转向 Linux 和开源 +============= + +> 概述：巴塞罗那城市管理署已为从其现存的来自微软和专有软件的系统转换到 Linux 和开源软件规划好路线图。 + +西班牙报纸 [El País][1] 日前报道，[巴塞罗那城][2]已在迁移其计算机系统至开源技术的进程中。 + +根据该新闻报道，巴塞罗那城计划首先用开源应用程序替换掉所有的用户端应用。所有的专有软件都会被替换，最后仅剩下 Windows，而最终它也会被一个 Linux 发行版替代。 + +![BarcelonaSave][image-1] + +### 巴塞罗那将会在 2019 年春季全面转换到开源 + +巴塞罗那城已经计划来年将其软件预算的 70% 投入到开源软件中。根据其城市议会技术和数字创新委员会委员 Francesca Bria 的说法，这一转换的过渡期将会在 2019 年春季本届城市管理署的任期结束前完成。 + +### 迁移旨在帮助 IT 人才 + +为了完成向开源的迁移，巴塞罗那城将会在中小企业中探索 IT 相关的项目。另外，城市管理署将吸纳 65 名新的开发者来构建软件以满足特定的需求。 + +设想中的一项重要项目，是开发一个在线的数字市场平台，小型企业将会利用其参加公开招标。 + +### Ubuntu 将成为替代的 Linux 发行版 + +由于巴塞罗那已经运行着一个 1000 台规模的基于 Ubuntu 桌面的试点项目，Ubuntu 可能会成为替代 Windows 的 Linux 发行版。新闻报道同时披露，Open-Xchange 将会替代 Outlook 邮件客户端和 Exchange 邮件服务器，而 Firefox 与 LibreOffice 将会替代 Internet Explorer 与微软 Office。 + +### 巴塞罗那市政当局成为首个参与「公共资产，公共代码Public Money, Public Code」运动的当局 + +凭借此次向开源项目迁移，巴塞罗那市政当局成为首个参与欧洲的「[公共资产，公共代码Public Money, Public Code](3)」运动的当局。 + +[欧洲自由软件基金会](4)发布了一封[公开信](5)，倡议公共筹资的软件应该是自由的，并发起了这项运动。已有超过 15,000 人和 100 家组织支持这一号召。你也可以支持一个，只需要[签署请愿书](6)并且为开源发出你的声音。 + +### 资金永远是一个理由 + +根据 Bria 的说法，从 Windows 到开源软件的迁移，就已开发的程序可以被部署在西班牙或世界上的其他地方当局而言，促进了重复利用。显然，这一迁移也是为了防止大量的金钱被花费在专有软件上。 + +### 你的想法如何？ + +对于开源社区来讲，巴塞罗那的迁移是一场已经赢得的战争，也是一个有利条件。当[慕尼黑选择回归微软的怀抱](7)时，这一消息是开源社区十分需要的。 + +你对巴塞罗那转向开源有什么开发？你有预见到其他欧洲城市也跟随这一变化吗？在评论中和我们分享你的观点吧。 + +*來源: [Open Source Observatory][8]* + +-------------------------------------------------------------------------------- +via: https://itsfoss.com/barcelona-open-source/ + +作者：[Derick Sullivan M. Lobga][a] +译者：[Purling Nayuki](https://github.com/PurlingNayuki) +校对：[wxy](https://github.com/wxy) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://itsfoss.com/author/derick/ +[1]:https://elpais.com/ccaa/2017/12/01/catalunya/1512145439_132556.html +[2]:https://en.wikipedia.org/wiki/Barcelona +[image-1]:https://4bds6hergc-flywheel.netdna-ssl.com/wp-content/uploads/2018/01/barcelona-city-animated.jpg +[3]:https://publiccode.eu/ +[4]:https://fsfe.org/ +[5]:https://publiccode.eu/openletter/ +[6]:https://creativecommons.org/2017/09/18/sign-petition-public-money-produce-public-code/ +[7]:https://itsfoss.com/munich-linux-failure/ +[8]:https://joinup.ec.europa.eu/news/public-money-public-code diff --git a/sources/talk/20180111 AI and machine learning bias has dangerous implications.md b/sources/talk/20180111 AI and machine learning bias has dangerous implications.md index 7a83ebb3a2..f42d82f316 100644 --- a/sources/talk/20180111 AI and machine learning bias has dangerous implications.md +++ b/sources/talk/20180111 AI and machine learning bias has dangerous implications.md @@ -1,5 +1,6 @@ AI and machine learning bias has dangerous implications ====== +translating ![](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/LAW_goodbadugly.png?itok=ZxaimUWU) diff --git a/sources/tech/20090211 Page Cache the Affair Between Memory and Files.md b/sources/tech/20090211 Page Cache the Affair Between Memory and Files.md deleted file mode 100644 index 98c546eb2a..0000000000 --- a/sources/tech/20090211 Page Cache the Affair Between Memory and Files.md +++ /dev/null @@ -1,76 +0,0 @@ -Translating by qhwdw [20090211 Page Cache, the Affair Between Memory and Files][1] -============================================================ - - -Previously we looked at how the kernel [manages virtual memory][2] for a user process, but files and I/O were left out. This post covers the important and often misunderstood relationship between files and memory and its consequences for performance. - -Two serious problems must be solved by the OS when it comes to files. The first one is the mind-blowing slowness of hard drives, and [disk seeks in particular][3], relative to memory. The second is the need to load file contents in physical memory once and share the contents among programs. If you use [Process Explorer][4] to poke at Windows processes, you'll see there are ~15MB worth of common DLLs loaded in every process. My Windows box right now is running 100 processes, so without sharing I'd be using up to ~1.5 GB of physical RAM just for common DLLs. No good. Likewise, nearly all Linux programs need [ld.so][5] and libc, plus other common libraries. - -Happily, both problems can be dealt with in one shot: the page cache, where the kernel stores page-sized chunks of files. To illustrate the page cache, I'll conjure a Linux program named render, which opens file scene.dat and reads it 512 bytes at a time, storing the file contents into a heap-allocated block. The first read goes like this: - -![Reading and the page cache](http://static.duartes.org/img/blogPosts/readFromPageCache.png) - -After 12KB have been read, render's heap and the relevant page frames look thus: - -![Non-mapped file read](http://static.duartes.org/img/blogPosts/nonMappedFileRead.png) - -This looks innocent enough, but there's a lot going on. First, even though this program uses regular read calls, three 4KB page frames are now in the page cache storing part of scene.dat. People are sometimes surprised by this, but all regular file I/O happens through the page cache. In x86 Linux, the kernel thinks of a file as a sequence of 4KB chunks. If you read a single byte from a file, the whole 4KB chunk containing the byte you asked for is read from disk and placed into the page cache. This makes sense because sustained disk throughput is pretty good and programs normally read more than just a few bytes from a file region. The page cache knows the position of each 4KB chunk within the file, depicted above as #0, #1, etc. Windows uses 256KB views analogous to pages in the Linux page cache. - -Sadly, in a regular file read the kernel must copy the contents of the page cache into a user buffer, which not only takes cpu time and hurts the [cpu caches][6], but also wastes physical memory with duplicate data. As per the diagram above, the scene.dat contents are stored twice, and each instance of the program would store the contents an additional time. We've mitigated the disk latency problem but failed miserably at everything else. Memory-mapped files are the way out of this madness: - -![Mapped file read](http://static.duartes.org/img/blogPosts/mappedFileRead.png) - -When you use file mapping, the kernel maps your program's virtual pages directly onto the page cache. This can deliver a significant performance boost: [Windows System Programming][7] reports run time improvements of 30% and up relative to regular file reads, while similar figures are reported for Linux and Solaris in [Advanced Programming in the Unix Environment][8]. You might also save large amounts of physical memory, depending on the nature of your application. - -As always with performance, [measurement is everything][9], but memory mapping earns its keep in a programmer's toolbox. The API is pretty nice too, it allows you to access a file as bytes in memory and does not require your soul and code readability in exchange for its benefits. Mind your [address space][10] and experiment with [mmap][11] in Unix-like systems, [CreateFileMapping][12] in Windows, or the many wrappers available in high level languages. When you map a file its contents are not brought into memory all at once, but rather on demand via [page faults][13]. The fault handler [maps your virtual pages][14] onto the page cache after [obtaining][15] a page frame with the needed file contents. This involves disk I/O if the contents weren't cached to begin with. - -Now for a pop quiz. Imagine that the last instance of our render program exits. Would the pages storing scene.dat in the page cache be freed immediately? People often think so, but that would be a bad idea. When you think about it, it is very common for us to create a file in one program, exit, then use the file in a second program. The page cache must handle that case. When you think more about it, why should the kernel ever get rid of page cache contents? Remember that disk is 5 orders of magnitude slower than RAM, hence a page cache hit is a huge win. So long as there's enough free physical memory, the cache should be kept full. It is therefore not dependent on a particular process, but rather it's a system-wide resource. If you run render a week from now and scene.dat is still cached, bonus! This is why the kernel cache size climbs steadily until it hits a ceiling. It's not because the OS is garbage and hogs your RAM, it's actually good behavior because in a way free physical memory is a waste. Better use as much of the stuff for caching as possible. - -Due to the page cache architecture, when a program calls [write()][16] bytes are simply copied to the page cache and the page is marked dirty. Disk I/O normally does not happen immediately, thus your program doesn't block waiting for the disk. On the downside, if the computer crashes your writes will never make it, hence critical files like database transaction logs must be [fsync()][17]ed (though one must still worry about drive controller caches, oy!). Reads, on the other hand, normally block your program until the data is available. Kernels employ eager loading to mitigate this problem, an example of which is read ahead where the kernel preloads a few pages into the page cache in anticipation of your reads. You can help the kernel tune its eager loading behavior by providing hints on whether you plan to read a file sequentially or randomly (see [madvise()][18], [readahead()][19], [Windows cache hints][20] ). Linux [does read-ahead][21] for memory-mapped files, but I'm not sure about Windows. Finally, it's possible to bypass the page cache using [O_DIRECT][22] in Linux or [NO_BUFFERING][23] in Windows, something database software often does. - -A file mapping may be private or shared. This refers only to updates made to the contents in memory: in a private mapping the updates are not committed to disk or made visible to other processes, whereas in a shared mapping they are. Kernels use the copy on write mechanism, enabled by page table entries, to implement private mappings. In the example below, both render and another program called render3d (am I creative or what?) have mapped scene.dat privately. Render then writes to its virtual memory area that maps the file: - -![The Copy-On-Write mechanism](http://static.duartes.org/img/blogPosts/copyOnWrite.png) - -The read-only page table entries shown above do not mean the mapping is read only, they're merely a kernel trick to share physical memory until the last possible moment. You can see how 'private' is a bit of a misnomer until you remember it only applies to updates. A consequence of this design is that a virtual page that maps a file privately sees changes done to the file by other programs as long as the page has only been read from. Once copy-on-write is done, changes by others are no longer seen. This behavior is not guaranteed by the kernel, but it's what you get in x86 and makes sense from an API perspective. By contrast, a shared mapping is simply mapped onto the page cache and that's it. Updates are visible to other processes and end up in the disk. Finally, if the mapping above were read-only, page faults would trigger a segmentation fault instead of copy on write. - -Dynamically loaded libraries are brought into your program's address space via file mapping. There's nothing magical about it, it's the same private file mapping available to you via regular APIs. Below is an example showing part of the address spaces from two running instances of the file-mapping render program, along with physical memory, to tie together many of the concepts we've seen. - -![Mapping virtual memory to physical memory](http://static.duartes.org/img/blogPosts/virtualToPhysicalMapping.png) - -This concludes our 3-part series on memory fundamentals. I hope the series was useful and provided you with a good mental model of these OS topics. - --------------------------------------------------------------------------------- - -via:https://manybutfinite.com/post/page-cache-the-affair-between-memory-and-files/ - -作者：[Gustavo Duarte][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:http://duartes.org/gustavo/blog/about/ -[1]:https://manybutfinite.com/post/page-cache-the-affair-between-memory-and-files/ -[2]:https://manybutfinite.com/post/how-the-kernel-manages-your-memory -[3]:https://manybutfinite.com/post/what-your-computer-does-while-you-wait -[4]:http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx -[5]:http://ld.so -[6]:https://manybutfinite.com/post/intel-cpu-caches -[7]:http://www.amazon.com/Windows-Programming-Addison-Wesley-Microsoft-Technology/dp/0321256190/ -[8]:http://www.amazon.com/Programming-Environment-Addison-Wesley-Professional-Computing/dp/0321525949/ -[9]:https://manybutfinite.com/post/performance-is-a-science -[10]:https://manybutfinite.com/post/anatomy-of-a-program-in-memory -[11]:http://www.kernel.org/doc/man-pages/online/pages/man2/mmap.2.html -[12]:http://msdn.microsoft.com/en-us/library/aa366537(VS.85).aspx -[13]:http://lxr.linux.no/linux+v2.6.28/mm/memory.c#L2678 -[14]:http://lxr.linux.no/linux+v2.6.28/mm/memory.c#L2436 -[15]:http://lxr.linux.no/linux+v2.6.28/mm/filemap.c#L1424 -[16]:http://www.kernel.org/doc/man-pages/online/pages/man2/write.2.html -[17]:http://www.kernel.org/doc/man-pages/online/pages/man2/fsync.2.html -[18]:http://www.kernel.org/doc/man-pages/online/pages/man2/madvise.2.html -[19]:http://www.kernel.org/doc/man-pages/online/pages/man2/readahead.2.html -[20]:http://msdn.microsoft.com/en-us/library/aa363858(VS.85).aspx#caching_behavior -[21]:http://lxr.linux.no/linux+v2.6.28/mm/filemap.c#L1424 -[22]:http://www.kernel.org/doc/man-pages/online/pages/man2/open.2.html -[23]:http://msdn.microsoft.com/en-us/library/cc644950(VS.85).aspx \ No newline at end of file diff --git a/sources/tech/20100419 10 Tools To Add Some Spice To Your UNIX-Linux Shell Scripts.md b/sources/tech/20100419 10 Tools To Add Some Spice To Your UNIX-Linux Shell Scripts.md new file mode 100644 index 0000000000..d350bd07b8 --- /dev/null +++ b/sources/tech/20100419 10 Tools To Add Some Spice To Your UNIX-Linux Shell Scripts.md @@ -0,0 +1,383 @@ +10 Tools To Add Some Spice To Your UNIX/Linux Shell Scripts +====== +There are some misconceptions that shell scripts are only for a CLI environment. You can efficiently use various tools to write GUI and network (socket) scripts under KDE or Gnome desktops. Shell scripts can make use of some of the GUI widget (menus, warning boxes, progress bars, etc.). You can always control the final output, cursor position on the screen, various output effects, and more. With the following tools, you can build powerful, interactive, user-friendly UNIX / Linux bash shell scripts. + +Creating GUI application is not an expensive task but a task that takes time and patience. Luckily, both UNIX and Linux ships with plenty of tools to write beautiful GUI scripts. The following tools are tested on FreeBSD and Linux operating systems but should work under other UNIX like operating systems. + +### 1. notify-send Command + +The notify-send command allows you to send desktop notifications to the user via a notification daemon from the command line. This is useful to inform the desktop user about an event or display some form of information without getting in the user's way. You need to install the following package on a Debian/Ubuntu Linux using [apt command][1]/[apt-get command][2]: +`$ sudo apt-get install libnotify-bin` +CentOS/RHEL user try the following [yum command][3]: +`$ sudo yum install libnotify` +Fedora Linux user type the following dnf command: +`$ sudo dnf install libnotify` +In this example, send simple desktop notification from the command line, enter: +``` +### send some notification ## +notify-send "rsnapshot done :)" +``` + +Sample outputs: +![Fig:01: notify-send in action ][4] +Here is another code with additional options: +``` +.... +alert=18000 +live=$(lynx --dump http://money.rediff.com/ | grep 'BSE LIVE' | awk '{ print $5}' | sed 's/,//g;s/\.[0-9]*//g') +[ $notify_counter -eq 0 ] && [ $live -ge $alert ] && { notify-send -t 5000 -u low -i "BSE Sensex touched 18k"; notify_counter=1; } +... +``` + +Sample outputs: +![Fig.02: notify-send with timeouts and other options][5] +Where, + + * -t 5000: Specifies the timeout in milliseconds ( 5000 milliseconds = 5 seconds) + * -u low : Set the urgency level (i.e. low, normal, or critical). + * -i gtk-dialog-info : Set an icon filename or stock icon to display (you can set path as -i /path/to/your-icon.png). + + + +For more information on use of the notify-send utility, please refer to the notify-send man page, viewable by typing man notify-send from the command line: +``` +man notify-send +``` + +### #2: tput Command + +The tput command is used to set terminal features. With tput you can set: + + * Move the cursor around the screen. + * Get information about terminal. + * Set colors (background and foreground). + * Set bold mode. + * Set reverse mode and much more. + + + +Here is a sample code: +``` +#!/bin/bash + +# clear the screen +tput clear + +# Move cursor to screen location X,Y (top left is 0,0) +tput cup 3 15 + +# Set a foreground colour using ANSI escape +tput setaf 3 +echo "XYX Corp LTD." +tput sgr0 + +tput cup 5 17 +# Set reverse video mode +tput rev +echo "M A I N - M E N U" +tput sgr0 + +tput cup 7 15 +echo "1. User Management" + +tput cup 8 15 +echo "2. Service Management" + +tput cup 9 15 +echo "3. Process Management" + +tput cup 10 15 +echo "4. Backup" + +# Set bold mode +tput bold +tput cup 12 15 +read -p "Enter your choice [1-4] " choice + +tput clear +tput sgr0 +tput rc +``` + + +Sample outputs: +![Fig.03: tput in action][6] +For more detail concerning the tput command, see the following man page: +``` +man 5 terminfo +man tput +``` + +### #3: setleds Command + +The setleds command allows you to set the keyboard leds. In this example, set NumLock on: +``` +setleds -D +num +``` + +To turn it off NumLock, enter: +``` +setleds -D -num +``` + + * -caps : Clear CapsLock. + * +caps : Set CapsLock. + * -scroll : Clear ScrollLock. + * +scroll : Set ScrollLock. + + + +See setleds command man page for more information and options: +`man setleds` + +### #4: zenity Command + +The [zenity commadn will display GTK+ dialogs box][7], and return the users input. This allows you to present information, and ask for information from the user, from all manner of shell scripts. Here is a sample GUI client for the whois directory service for given domain name: + +```shell +#!/bin/bash +# Get domain name +_zenity="/usr/bin/zenity" +_out="/tmp/whois.output.$$" +domain=$(${_zenity} --title "Enter domain" \ + --entry --text "Enter the domain you would like to see whois info" ) + +if [ $? -eq 0 ] +then + # Display a progress dialog while searching whois database + whois $domain | tee >(${_zenity} --width=200 --height=100 \ + --title="whois" --progress \ + --pulsate --text="Searching domain info..." \ + --auto-kill --auto-close \ + --percentage=10) >${_out} + + # Display back output + ${_zenity} --width=800 --height=600 \ + --title "Whois info for $domain" \ + --text-info --filename="${_out}" +else + ${_zenity} --error \ + --text="No input provided" +fi +``` + +Sample outputs: +![Fig.04: zenity in Action][8] +See the zenity man page for more information and all other supports GTK+ widgets: +``` +zenity --help +man zenity +``` + +### #5: kdialog Command + +kdialog is just like zenity but it is designed for KDE desktop / qt apps. You can display dialogs using kdialog. The following will display message on screen: +``` +kdialog --dontagain myscript:nofilemsg --msgbox "File: '~/.backup/config' not found." +``` + +Sample outputs: +![Fig.05: Suppressing the display of a dialog ][9] + +See [shell scripting with KDE Dialogs][10] tutorial for more information. + +### #6: Dialog + +[Dialog is an application used in shell scripts][11] which displays text user interface widgets. It uses the curses or ncurses library. Here is a sample code: +``` +#!/bin/bash +dialog --title "Delete file" \ +--backtitle "Linux Shell Script Tutorial Example" \ +--yesno "Are you sure you want to permanently delete \"/tmp/foo.txt\"?" 7 60 + +# Get exit status +# 0 means user hit [yes] button. +# 1 means user hit [no] button. +# 255 means user hit [Esc] key. +response=$? +case $response in + 0) echo "File deleted.";; + 1) echo "File not deleted.";; + 255) echo "[ESC] key pressed.";; +esac +``` + +See the dialog man page for details: +`man dialog` + +#### A Note About Other User Interface Widgets Tools + +UNIX and Linux comes with lots of other tools to display and control apps from the command line, and shell scripts can make use of some of the KDE / Gnome / X widget set: + + * **gmessage** - a GTK-based xmessage clone. + * **xmessage** - display a message or query in a window (X-based /bin/echo) + * **whiptail** - display dialog boxes from shell scripts + * **python-dialog** - Python module for making simple Text/Console-mode user interfaces + + + +### #7: logger command + +The logger command writes entries in the system log file such as /var/log/messages. It provides a shell command interface to the syslog system log module: +``` +logger "MySQL database backup failed." +tail -f /var/log/messages +logger -t mysqld -p daemon.error "Database Server failed" +tail -f /var/log/syslog +``` + +Sample outputs: +``` +Apr 20 00:11:45 vivek-desktop kernel: [38600.515354] CPU0: Temperature/speed normal +Apr 20 00:12:20 vivek-desktop mysqld: Database Server failed +``` + +See howto [write message to a syslog / log file][12] for more information. Alternatively, you can see the logger man page for details: +`man logger` + +### #8: setterm Command + +The setterm command can set various terminal attributes. In this example, force screen to turn black in 15 minutes. Monitor standby will occur at 60 minutes: +``` +setterm -blank 15 -powersave powerdown -powerdown 60 +``` + +In this example show underlined text for xterm window: +``` +setterm -underline on; +echo "Add Your Important Message Here" +setterm -underline off +``` + +Another useful option is to turn on or off cursor: +``` +setterm -cursor off +``` + +Turn it on: +``` +setterm -cursor on +``` + +See the setterm command man page for details: +`man setterm` + +### #9: smbclient: Sending Messages To MS-Windows Workstations + +The smbclient command can talk to an SMB/CIFS server. It can send a message to selected users or all users on MS-Windows systems: +``` +smbclient -M WinXPPro </dev/tcp/localhost/25) &>/dev/null && echo "TCP port 25 open" || echo "TCP port 25 close" +``` + +You can use [bash loop and find out open ports][14] with the snippets: +``` +echo "Scanning TCP ports..." +for p in {1..1023} +do + (echo >/dev/tcp/localhost/$p) >/dev/null 2>&1 && echo "$p open" +done +``` + + +Sample outputs: +``` +Scanning TCP ports... +22 open +53 open +80 open +139 open +445 open +631 open +``` + +In this example, your bash script act as an HTTP client: +``` +#!/bin/bash +exec 3<> /dev/tcp/${1:-www.cyberciti.biz}/80 + +printf "GET / HTTP/1.0\r\n" >&3 +printf "Accept: text/html, text/plain\r\n" >&3 +printf "Accept-Language: en\r\n" >&3 +printf "User-Agent: nixCraft_BashScript v.%s\r\n" "${BASH_VERSION}" >&3 +printf "\r\n" >&3 + +while read LINE <&3 +do + # do something on $LINE + # or send $LINE to grep or awk for grabbing data + # or simply display back data with echo command + echo $LINE +done +``` + +See the bash man page for more information: +`man bash` + +### A Note About GUI Tools and Cronjob + +You need to request local display/input service using export DISPLAY=[user's machine]:0 command if you are [using cronjob][15] to call your scripts. For example, call /home/vivek/scripts/monitor.stock.sh as follows which uses zenity tool: +`@hourly DISPLAY=:0.0 /home/vivek/scripts/monitor.stock.sh` + +Have a favorite UNIX tool to spice up shell script? Share it in the comments below. + +### about the author + +The author is the creator of nixCraft and a seasoned sysadmin and a trainer for the Linux operating system/Unix shell scripting. He has worked with global clients and in various industries, including IT, education, defense and space research, and the nonprofit sector. Follow him on [Twitter][16], [Facebook][17], [Google+][18]. + +-------------------------------------------------------------------------------- + +via: https://www.cyberciti.biz/tips/spice-up-your-unix-linux-shell-scripts.html + +作者：[Vivek Gite][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.cyberciti.biz +[1]:https://www.cyberciti.biz/faq/ubuntu-lts-debian-linux-apt-command-examples/ (See Linux/Unix apt command examples for more info) +[2]:https://www.cyberciti.biz/tips/linux-debian-package-management-cheat-sheet.html (See Linux/Unix apt-get command examples for more info) +[3]:https://www.cyberciti.biz/faq/rhel-centos-fedora-linux-yum-command-howto/ (See Linux/Unix yum command examples for more info) +[4]:https://www.cyberciti.biz/media/new/tips/2010/04/notify-send.png (notify-send: Shell Script Get Or Send Desktop Notifications ) +[5]:https://www.cyberciti.biz/media/new/tips/2010/04/notify-send-with-icons-timeout.png (Linux / UNIX: Display Notifications From Your Shell Scripts With notify-send) +[6]:https://www.cyberciti.biz/media/new/tips/2010/04/tput-options.png (Linux / UNIX Script Colours and Cursor Movement With tput) +[7]:https://bash.cyberciti.biz/guide/Zenity:_Shell_Scripting_with_Gnome +[8]:https://www.cyberciti.biz/media/new/tips/2010/04/zenity-outputs.png (zenity: Linux / UNIX display Dialogs Boxes From The Shell Scripts) +[9]:https://www.cyberciti.biz/media/new/tips/2010/04/KDialog.png (Kdialog: Suppressing the display of a dialog ) +[10]:http://techbase.kde.org/Development/Tutorials/Shell_Scripting_with_KDE_Dialogs +[11]:https://bash.cyberciti.biz/guide/Bash_display_dialog_boxes +[12]:https://www.cyberciti.biz/tips/howto-linux-unix-write-to-syslog.html +[13]:https://www.cyberciti.biz/tips/freebsd-sending-a-message-to-windows-workstation.html +[14]:https://www.cyberciti.biz/faq/bash-for-loop/ +[15]:https://www.cyberciti.biz/faq/how-do-i-add-jobs-to-cron-under-linux-or-unix-oses/ +[16]:https://twitter.com/nixcraft +[17]:https://facebook.com/nixcraft +[18]:https://plus.google.com/+CybercitiBiz diff --git a/sources/tech/20121211 Python Nmon Analyzer- moving away from excel macros.md b/sources/tech/20121211 Python Nmon Analyzer- moving away from excel macros.md deleted file mode 100644 index 3591e379d5..0000000000 --- a/sources/tech/20121211 Python Nmon Analyzer- moving away from excel macros.md +++ /dev/null @@ -1,101 +0,0 @@ -translating by lujun9972 -Python Nmon Analyzer: moving away from excel macros -====== -[Nigel's monitor][1], dubbed "Nmon", is a fantastic tool for monitoring, recording and analyzing a Linux/*nix system's performance over time. Nmon was originally developed by IBM and Open Sourced in the summer of 2009. By now Nmon is available on just about every linux platfrom and architecture. It provides a great real-time command line visualization of current system statistics, such as CPU, RAM, Network and Disk I/O. However, Nmon's greatest feature is the capability to record system performance snapshots over time. -For example: `nmon -f -s 1`. -![nmon CPU and Disk utilization][2] -This will create a log file starting of with some system metadata(Section AAA - BBBV), followed by timed snapshots of all monitored system attributes, such as CPU and Memory usage. This produces a file that is hard to directly interpret with a spreadsheet application, hence the birth of the [Nmon_Analyzer][3] excel macro. This tool is great, if you have access to Windows/Mac with Microsoft Office installed. If not there is also the Nmon2rrd tool, which generates RRD input files to generate your graphs. This is a very rigid approach and slightly painful. Now to provide a more flexible tool, I am introducing the pyNmonAnalyzer, which aims to provide a customization solution for generating organized CSV files and simple HTML reports with [matplotlib][4] based graphs. - -### Getting Started: - -System requirements: -As the name indicates you will need python. Additionally pyNmonAnalyzer depends on matplotlib and numpy. If you are on a debian-derivative system these are the packages you'll need to install: -``` -$> sudo apt-get install python-numpy python-matplotlib - -``` - -##### Getting pyNmonAnalyzer: - -Either clone the git repository: -``` -$> git clone git@github.com:madmaze/pyNmonAnalyzer.git - -``` - -or - -Download the current release here: [pyNmonAnalyzer-0.1.zip][5] - -Next we need an an Nmon file, if you do not already have one, either use the example provided in the release or record a sample: `nmon -F test.nmon -s 1 -c 120`, this will record 120 snapshots at 1 second intervals to test.nmon. - -Lets have a look at the basic help output: -``` -$> ./pyNmonAnalyzer.py -h -usage: pyNmonAnalyzer.py [-h] [-x] [-d] [-o OUTDIR] [-c] [-b] [-r CONFFNAME] - input_file - -nmonParser converts Nmon monitor files into time-sorted -CSV/Spreadsheets for easier analysis, without the use of the -MS Excel Macro. Also included is an option to build an HTML -report with graphs, which is configured through report.config. - -positional arguments: - input_file Input NMON file - -optional arguments: - -h, --help show this help message and exit - -x, --overwrite overwrite existing results (Default: False) - -d, --debug debug? (Default: False) - -o OUTDIR, --output OUTDIR - Output dir for CSV (Default: ./data/) - -c, --csv CSV output? (Default: False) - -b, --buildReport report output? (Default: False) - -r CONFFNAME, --reportConfig CONFFNAME - Report config file, if none exists: we will write the - default config file out (Default: ./report.config) - -``` - -There are 2 main options of using this tool - - 1. Turn the nmon file into a set of separate CSV file - 2. Generate an HTML report with matplotlib graphs - - - -The following command does both: -``` -$> ./pyNmonAnalyzer.py -c -b test.nmon - -``` - -This will create a directory called ./data in which you will find a folder of CSV files ("./data/csv/"), a folder of PNG graphs ("./data/img/") and an HTML report ("./data/report.html"). - -By default the HTML report will include graphs for CPU, Disk Busy, Memory utilization and Network transfers. This is all defined in a self explanitory configuration file, "report.config". At the moment this is not yet very flexible as CPU and MEM are not configurable besides on or off, but one of the next steps will be to refine the plotting approach and to expose more flexibility with which graphs plot which data points. - -### Report Example: - -[![pyNmonAnalyzer Graph output][6] -**Click to see the full Report**][7] - -Currently these reports are very bare bones and only prints out basic labeled graphs, but development is on-going. Currently in development is a wizard that will make adjusting the configurations easier. Please do let me know if you have any suggestions, find any bugs or have feature requests. - --------------------------------------------------------------------------------- - -via: https://matthiaslee.com/python-nmon-analyzer-moving-away-from-excel-macros/ - -作者：[Matthias Lee][a] -译者：[lujun9972](https://github.com/lujun9972) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://matthiaslee.com/ -[1]:http://nmon.sourceforge.net/ -[2]:https://matthiaslee.com//content/images/2015/06/nmon_cpudisk.png -[3]:http://www.ibm.com/developerworks/wikis/display/WikiPtype/nmonanalyser -[4]:http://matplotlib.org/ -[5]:https://github.com/madmaze/pyNmonAnalyzer/blob/master/release/pyNmonAnalyzer-0.1.zip?raw=true -[6]:https://matthiaslee.com//content/images/2017/04/teaser-short_0.png (pyNmonAnalyzer Graph output) -[7]:http://matthiaslee.com/pub/pyNmonAnalyzer/data/report.html diff --git a/sources/tech/20140210 Three steps to learning GDB.md b/sources/tech/20140210 Three steps to learning GDB.md new file mode 100644 index 0000000000..3e94e3d77f --- /dev/null +++ b/sources/tech/20140210 Three steps to learning GDB.md @@ -0,0 +1,113 @@ +Translating by Torival Three steps to learning GDB +============================================================ + +Debugging C programs used to scare me a lot. Then I was writing my [operating system][2] and I had so many bugs to debug! I was extremely fortunate to be using the emulator qemu, which lets me attach a debugger to my operating system. The debugger is called `gdb`. + +I’m going to explain a couple of small things you can do with `gdb`, because I found it really confusing to get started. We’re going to set a breakpoint and examine some memory in a tiny program. + +### 1\. Set breakpoints + +If you’ve ever used a debugger before, you’ve probably set a breakpoint. + +Here’s the program that we’re going to be “debugging” (though there aren’t any bugs): + +``` +#include +void do_thing() { + printf("Hi!\n"); +} +int main() { + do_thing(); +} + +``` + +Save this as `hello.c`. We can debug it with gdb like this: + +``` +bork@kiwi ~> gcc -g hello.c -o hello +bork@kiwi ~> cat +bork@kiwi ~> gdb ./hello +``` + +This compiles `hello.c` with debugging symbols (so that gdb can do better work), and gives us kind of scary prompt that just says + +`(gdb)` + +We can then set a breakpoint using the `break` command, and then `run` the program. + +``` +(gdb) break do_thing +Breakpoint 1 at 0x4004f8 +(gdb) run +Starting program: /home/bork/hello + +Breakpoint 1, 0x00000000004004f8 in do_thing () +``` + +This stops the program at the beginning of `do_thing`. + +We can find out where we are in the call stack with `where`: (thanks to [@mgedmin][3] for the tip) + +``` +(gdb) where +#0 do_thing () at hello.c:3 +#1 0x08050cdb in main () at hello.c:6 +(gdb) +``` + +### 2\. Look at some assembly code + +We can look at the assembly code for our function using the `disassemble`command! This is cool. This is x86 assembly. I don’t understand it very well, but the line that says `callq` is what does the `printf` function call. + +``` +(gdb) disassemble do_thing +Dump of assembler code for function do_thing: + 0x00000000004004f4 <+0>: push %rbp + 0x00000000004004f5 <+1>: mov %rsp,%rbp +=> 0x00000000004004f8 <+4>: mov $0x40060c,%edi + 0x00000000004004fd <+9>: callq 0x4003f0 + 0x0000000000400502 <+14>: pop %rbp + 0x0000000000400503 <+15>: retq +``` + +You can also shorten `disassemble` to `disas` + +### 3\. Examine some memory! + +The main thing I used `gdb` for when I was debugging my kernel was to examine regions of memory to make sure they were what I thought they were. The command for examining memory is `examine`, or `x` for short. We’re going to use `x`. + +From looking at that assembly above, it seems like `0x40060c` might be the address of the string we’re printing. Let’s check! + +``` +(gdb) x/s 0x40060c +0x40060c: "Hi!" +``` + +It is! Neat! Look at that. The `/s` part of `x/s` means “show it to me like it’s a string”. I could also have said “show me 10 characters” like this: + +``` +(gdb) x/10c 0x40060c +0x40060c: 72 'H' 105 'i' 33 '!' 0 '\000' 1 '\001' 27 '\033' 3 '\003' 59 ';' +0x400614: 52 '4' 0 '\000' +``` + +You can see that the first four characters are ‘H’, ‘i’, and ‘!’, and ‘\0’ and then after that there’s more unrelated stuff. + +I know that gdb does lots of other stuff, but I still don’t know it very well and `x`and `break` got me pretty far. You can read the [documentation for examining memory][4]. + +-------------------------------------------------------------------------------- + +via: https://jvns.ca/blog/2014/02/10/three-steps-to-learning-gdb/ + +作者：[Julia Evans ][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://jvns.ca +[1]:https://jvns.ca/categories/spytools +[2]:http://jvns.ca/blog/categories/kernel +[3]:https://twitter.com/mgedmin +[4]:https://ftp.gnu.org/old-gnu/Manuals/gdb-5.1.1/html_chapter/gdb_9.html#SEC56 diff --git a/sources/tech/20150615 Let-s Build A Simple Interpreter. Part 1..md b/sources/tech/20150615 Let-s Build A Simple Interpreter. Part 1..md index 4c0d541a5a..9a815f2852 100644 --- a/sources/tech/20150615 Let-s Build A Simple Interpreter. Part 1..md +++ b/sources/tech/20150615 Let-s Build A Simple Interpreter. Part 1..md @@ -1,3 +1,4 @@ +// Translating by Linchenguang.... Let’s Build A Simple Interpreter. Part 1. ====== diff --git a/sources/tech/20160810 How does gdb work.md b/sources/tech/20160810 How does gdb work.md new file mode 100644 index 0000000000..4159f1c65a --- /dev/null +++ b/sources/tech/20160810 How does gdb work.md @@ -0,0 +1,218 @@ +How does gdb work? +============================================================ + +Hello! Today I was working a bit on my [ruby stacktrace project][1] and I realized that now I know a couple of things about how gdb works internally. + +Lately I’ve been using gdb to look at Ruby programs, so we’re going to be running gdb on a Ruby program. This really means the Ruby interpreter. First, we’re going to print out the address of a global variable: `ruby_current_thread`: + +### getting a global variable + +Here’s how to get the address of the global `ruby_current_thread`: + +``` +$ sudo gdb -p 2983 +(gdb) p & ruby_current_thread +$2 = (rb_thread_t **) 0x5598a9a8f7f0 + +``` + +There are a few places a variable can live: on the heap, the stack, or in your program’s text. Global variables are part of your program! You can think of them as being allocated at compile time, kind of. It turns out we can figure out the address of a global variable pretty easily! Let’s see how `gdb` came up with `0x5598a9a8f7f0`. + +We can find the approximate region this variable lives in by looking at a cool file in `/proc` called `/proc/$pid/maps`. + +``` +$ sudo cat /proc/2983/maps | grep bin/ruby +5598a9605000-5598a9886000 r-xp 00000000 00:32 323508 /home/bork/.rbenv/versions/2.1.6/bin/ruby +5598a9a86000-5598a9a8b000 r--p 00281000 00:32 323508 /home/bork/.rbenv/versions/2.1.6/bin/ruby +5598a9a8b000-5598a9a8d000 rw-p 00286000 00:32 323508 /home/bork/.rbenv/versions/2.1.6/bin/ruby + +``` + +So! There’s this starting address `5598a9605000` That’s  _like_  `0x5598a9a8f7f0`, but different. How different? Well, here’s what I get when I subtract them: + +``` +(gdb) p/x 0x5598a9a8f7f0 - 0x5598a9605000 +$4 = 0x48a7f0 + +``` + +“What’s that number?”, you might ask? WELL. Let’s look at the **symbol table**for our program with `nm`. + +``` +sudo nm /proc/2983/exe | grep ruby_current_thread +000000000048a7f0 b ruby_current_thread + +``` + +What’s that we see? Could it be `0x48a7f0`? Yes it is! So!! If we want to find the address of a global variable in our program, all we need to do is look up the name of the variable in the symbol table, and then add that to the start of the range in `/proc/whatever/maps`, and we’re done! + +So now we know how gdb does that. But gdb does so much more!! Let’s skip ahead to… + +### dereferencing pointers + +``` +(gdb) p ruby_current_thread +$1 = (rb_thread_t *) 0x5598ab3235b0 + +``` + +The next thing we’re going to do is **dereference** that `ruby_current_thread`pointer. We want to see what’s in that address! To do that, gdb will run a bunch of system calls like this: + +``` +ptrace(PTRACE_PEEKTEXT, 2983, 0x5598a9a8f7f0, [0x5598ab3235b0]) = 0 + +``` + +You remember this address `0x5598a9a8f7f0`? gdb is asking “hey, what’s in that address exactly”? `2983` is the PID of the process we’re running gdb on. It’s using the `ptrace` system call which is how gdb does everything. + +Awesome! So we can dereference memory and figure out what bytes are at what memory addresses. Some useful gdb commands to know here are `x/40w variable` and `x/40b variable` which will display 40 words / bytes at a given address, respectively. + +### describing structs + +The memory at an address looks like this. A bunch of bytes! + +``` +(gdb) x/40b ruby_current_thread +0x5598ab3235b0: 16 -90 55 -85 -104 85 0 0 +0x5598ab3235b8: 32 47 50 -85 -104 85 0 0 +0x5598ab3235c0: 16 -64 -55 115 -97 127 0 0 +0x5598ab3235c8: 0 0 2 0 0 0 0 0 +0x5598ab3235d0: -96 -83 -39 115 -97 127 0 0 + +``` + +That’s useful, but not that useful! If you are a human like me and want to know what it MEANS, you need more. Like this: + +``` +(gdb) p *(ruby_current_thread) +$8 = {self = 94114195940880, vm = 0x5598ab322f20, stack = 0x7f9f73c9c010, + stack_size = 131072, cfp = 0x7f9f73d9ada0, safe_level = 0, raised_flag = 0, + last_status = 8, state = 0, waiting_fd = -1, passed_block = 0x0, + passed_bmethod_me = 0x0, passed_ci = 0x0, top_self = 94114195612680, + top_wrapper = 0, base_block = 0x0, root_lep = 0x0, root_svar = 8, thread_id = + 140322820187904, + +``` + +GOODNESS. That is a lot more useful. How does gdb know that there are all these cool fields like `stack_size`? Enter DWARF. DWARF is a way to store extra debugging data about your program, so that debuggers like gdb can do their job better! It’s generally stored as part of a binary. If I run `dwarfdump` on my Ruby binary, I get some output like this: + +(I’ve redacted it heavily to make it easier to understand) + +``` +DW_AT_name "rb_thread_struct" +DW_AT_byte_size 0x000003e8 +DW_TAG_member + DW_AT_name "self" + DW_AT_type <0x00000579> + DW_AT_data_member_location DW_OP_plus_uconst 0 +DW_TAG_member + DW_AT_name "vm" + DW_AT_type <0x0000270c> + DW_AT_data_member_location DW_OP_plus_uconst 8 +DW_TAG_member + DW_AT_name "stack" + DW_AT_type <0x000006b3> + DW_AT_data_member_location DW_OP_plus_uconst 16 +DW_TAG_member + DW_AT_name "stack_size" + DW_AT_type <0x00000031> + DW_AT_data_member_location DW_OP_plus_uconst 24 +DW_TAG_member + DW_AT_name "cfp" + DW_AT_type <0x00002712> + DW_AT_data_member_location DW_OP_plus_uconst 32 +DW_TAG_member + DW_AT_name "safe_level" + DW_AT_type <0x00000066> + +``` + +So. The name of the type of `ruby_current_thread` is `rb_thread_struct`. It has size `0x3e8` (or 1000 bytes), and it has a bunch of member items. `stack_size` is one of them, at an offset of 24, and it has type 31\. What’s 31? No worries! We can look that up in the DWARF info too! + +``` +< 1><0x00000031> DW_TAG_typedef + DW_AT_name "size_t" + DW_AT_type <0x0000003c> +< 1><0x0000003c> DW_TAG_base_type + DW_AT_byte_size 0x00000008 + DW_AT_encoding DW_ATE_unsigned + DW_AT_name "long unsigned int" + +``` + +So! `stack_size` has type `size_t`, which means `long unsigned int`, and is 8 bytes. That means that we can read the stack size! + +How that would break down, once we have the DWARF debugging data, is: + +1. Read the region of memory that `ruby_current_thread` is pointing to + +2. Add 24 bytes to get to `stack_size` + +3. Read 8 bytes (in little-endian format, since we’re on x86) + +4. Get the answer! + +Which in this case is 131072 or 128 kb. + +To me, this makes it a lot more obvious what debugging info is **for** – if we didn’t have all this extra metadata about what all these variables meant, we would have no idea what the bytes at address `0x5598ab3235b0` meant. + +This is also why you can install debug info for a program separately from your program – gdb doesn’t care where it gets the extra debug info from. + +### DWARF is confusing + +I’ve been reading a bunch of DWARF info recently. Right now I’m using libdwarf which hasn’t been the best experience – the API is confusing, you initialize everything in a weird way, and it’s really slow (it takes 0.3 seconds to read all the debugging data out of my Ruby program which seems ridiculous). I’ve been told that libdw from elfutils is better. + +Also, I casually remarked that you can look at `DW_AT_data_member_location` to get the offset of a struct member! But I looked up on Stack Overflow how to actually do that and I got [this answer][2]. Basically you start with a check like: + +``` +dwarf_whatform(attrs[i], &form, &error); + if (form == DW_FORM_data1 || form == DW_FORM_data2 + form == DW_FORM_data2 || form == DW_FORM_data4 + form == DW_FORM_data8 || form == DW_FORM_udata) { + +``` + +and then it keeps GOING. Why are there 8 million different `DW_FORM_data` things I need to check for? What is happening? I have no idea. + +Anyway my impression is that DWARF is a large and complicated standard (and possibly the libraries people use to generate DWARF are subtly incompatible?), but it’s what we have, so that’s what we work with! + +I think it’s really cool that I can write code that reads DWARF and my code actually mostly works. Except when it crashes. I’m working on that. + +### unwinding stacktraces + +In an earlier version of this post, I said that gdb unwinds stacktraces using libunwind. It turns out that this isn’t true at all! + +Someone who’s worked on gdb a lot emailed me to say that they actually spent a ton of time figuring out how to unwind stacktraces so that they can do a better job than libunwind does. This means that if you get stopped in the middle of a weird program with less debug info than you might hope for that’s done something strange with its stack, gdb will try to figure out where you are anyway. Thanks <3 + +### other things gdb does + +The few things I’ve described here (reading memory, understanding DWARF to show you structs) aren’t everything gdb does – just looking through Brendan Gregg’s [gdb example from yesterday][3], we see that gdb also knows how to + +* disassemble assembly + +* show you the contents of your registers + +and in terms of manipulating your program, it can + +* set breakpoints and step through a program + +* modify memory (!! danger !!) + +Knowing more about how gdb works makes me feel a lot more confident when using it! I used to get really confused because gdb kind of acts like a C REPL sometimes – you type `ruby_current_thread->cfp->iseq`, and it feels like writing C code! But you’re not really writing C at all, and it was easy for me to run into limitations in gdb and not understand why. + +Knowing that it’s using DWARF to figure out the contents of the structs gives me a better mental model and have more correct expectations! Awesome. + +-------------------------------------------------------------------------------- + +via: https://jvns.ca/blog/2016/08/10/how-does-gdb-work/ + +作者：[ Julia Evans][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://jvns.ca/ +[1]:http://jvns.ca/blog/2016/06/12/a-weird-system-call-process-vm-readv/ +[2]:https://stackoverflow.com/questions/25047329/how-to-get-struct-member-offset-from-dwarf-info +[3]:http://www.brendangregg.com/blog/2016-08-09/gdb-example-ncurses.html diff --git a/sources/tech/20170319 ftrace trace your kernel functions.md b/sources/tech/20170319 ftrace trace your kernel functions.md new file mode 100644 index 0000000000..0ff3fd6416 --- /dev/null +++ b/sources/tech/20170319 ftrace trace your kernel functions.md @@ -0,0 +1,284 @@ +ftrace: trace your kernel functions! +============================================================ + +Hello! Today we’re going to talk about a debugging tool we haven’t talked about much before on this blog: ftrace. What could be more exciting than a new debugging tool?! + +Better yet, ftrace isn’t new! It’s been around since Linux kernel 2.6, or about 2008. [here’s the earliest documentation I found with some quick Gooogling][10]. So you might be able to use it even if you’re debugging an older system! + +I’ve known that ftrace exists for about 2.5 years now, but hadn’t gotten around to really learning it yet. I’m supposed to run a workshop tomorrow where I talk about ftrace, so today is the day we talk about it! + +### what’s ftrace? + +ftrace is a Linux kernel feature that lets you trace Linux kernel function calls. Why would you want to do that? Well, suppose you’re debugging a weird problem, and you’ve gotten to the point where you’re staring at the source code for your kernel version and wondering what **exactly** is going on. + +I don’t read the kernel source code very often when debugging, but occasionally I do! For example this week at work we had a program that was frozen and stuck spinning inside the kernel. Looking at what functions were being called helped us understand better what was happening in the kernel and what systems were involved (in that case, it was the virtual memory system)! + +I think ftrace is a bit of a niche tool (it’s definitely less broadly useful and harder to use than strace) but that it’s worth knowing about. So let’s learn about it! + +### first steps with ftrace + +Unlike strace and perf, ftrace isn’t a **program** exactly – you don’t just run `ftrace my_cool_function`. That would be too easy! + +If you read [Debugging the kernel using Ftrace][11] it starts out by telling you to `cd /sys/kernel/debug/tracing` and then do various filesystem manipulations. + +For me this is way too annoying – a simple example of using ftrace this way is something like + +``` +cd /sys/kernel/debug/tracing +echo function > current_tracer +echo do_page_fault > set_ftrace_filter +cat trace + +``` + +This filesystem interface to the tracing system (“put values in these magic files and things will happen”) seems theoretically possible to use but really not my preference. + +Luckily, team ftrace also thought this interface wasn’t that user friendly and so there is an easier-to-use interface called **trace-cmd**!!! trace-cmd is a normal program with command line arguments. We’ll use that! I found an intro to trace-cmd on LWN at [trace-cmd: A front-end for Ftrace][12]. + +### getting started with trace-cmd: let’s trace just one function + +First, I needed to install `trace-cmd` with `sudo apt-get install trace-cmd`. Easy enough. + +For this first ftrace demo, I decided I wanted to know when my kernel was handling a page fault. When Linux allocates memory, it often does it lazily (“you weren’t  _really_  planning to use that memory, right?“). This means that when an application tries to actually write to memory that it allocated, there’s a page fault and the kernel needs to give the application physical memory to use. + +Let’s start `trace-cmd` and make it trace the `do_page_fault` function! + +``` +$ sudo trace-cmd record -p function -l do_page_fault + plugin 'function' +Hit Ctrl^C to stop recording + +``` + +I ran it for a few seconds and then hit `Ctrl+C`. Awesome! It created a 2.5MB file called `trace.dat`. Let’s see what’s that file! + +``` +$ sudo trace-cmd report + chrome-15144 [000] 11446.466121: function: do_page_fault + chrome-15144 [000] 11446.467910: function: do_page_fault + chrome-15144 [000] 11446.469174: function: do_page_fault + chrome-15144 [000] 11446.474225: function: do_page_fault + chrome-15144 [000] 11446.474386: function: do_page_fault + chrome-15144 [000] 11446.478768: function: do_page_fault + CompositorTileW-15154 [001] 11446.480172: function: do_page_fault + chrome-1830 [003] 11446.486696: function: do_page_fault + CompositorTileW-15154 [001] 11446.488983: function: do_page_fault + CompositorTileW-15154 [001] 11446.489034: function: do_page_fault + CompositorTileW-15154 [001] 11446.489045: function: do_page_fault + +``` + +This is neat – it shows me the process name (chrome), process ID (15144), CPU (000), and function that got traced. + +By looking at the whole report, (`sudo trace-cmd report | grep chrome`) I can see that we traced for about 1.5 seconds and in that time Chrome had about 500 page faults. Cool! We have done our first ftrace! + +### next ftrace trick: let’s trace a process! + +Okay, but just seeing one function is kind of boring! Let’s say I want to know everything that’s happening for one program. I use a static site generator called Hugo. What’s the kernel doing for Hugo? + +Hugo’s PID on my computer right now is 25314, so I recorded all the kernel functions with: + +``` +sudo trace-cmd record --help # I read the help! +sudo trace-cmd record -p function -P 25314 # record for PID 25314 + +``` + +`sudo trace-cmd report` printed out 18,000 lines of output. If you’re interested, you can see [all 18,000 lines here][13]. + +18,000 lines is a lot so here are some interesting excerpts. + +This looks like what happens when the `clock_gettime` system call runs. Neat! + +``` + compat_SyS_clock_gettime + SyS_clock_gettime + clockid_to_kclock + posix_clock_realtime_get + getnstimeofday64 + __getnstimeofday64 + arch_counter_read + __compat_put_timespec + +``` + +This is something related to process scheduling: + +``` + cpufreq_sched_irq_work + wake_up_process + try_to_wake_up + _raw_spin_lock_irqsave + do_raw_spin_lock + _raw_spin_lock + do_raw_spin_lock + walt_ktime_clock + ktime_get + arch_counter_read + walt_update_task_ravg + exiting_task + +``` + +Being able to see all these function calls is pretty cool, even if I don’t quite understand them. + +### “function graph” tracing + +There’s another tracing mode called `function_graph`. This is the same as the function tracer except that it instruments both entering  _and_  exiting a function. [Here’s the output of that tracer][14] + +``` +sudo trace-cmd record -p function_graph -P 25314 + +``` + +Again, here’s a snipped (this time from the futex code) + +``` + | futex_wake() { + | get_futex_key() { + | get_user_pages_fast() { + 1.458 us | __get_user_pages_fast(); + 4.375 us | } + | __might_sleep() { + 0.292 us | ___might_sleep(); + 2.333 us | } + 0.584 us | get_futex_key_refs(); + | unlock_page() { + 0.291 us | page_waitqueue(); + 0.583 us | __wake_up_bit(); + 5.250 us | } + 0.583 us | put_page(); ++ 24.208 us | } + +``` + +We see in this example that `get_futex_key` gets called right after `futex_wake`. Is that what really happens in the source code? We can check!! [Here’s the definition of futex_wake in Linux 4.4][15] (my kernel version). + +I’ll save you a click: it looks like this: + +``` +static int +futex_wake(u32 __user *uaddr, unsigned int flags, int nr_wake, u32 bitset) +{ + struct futex_hash_bucket *hb; + struct futex_q *this, *next; + union futex_key key = FUTEX_KEY_INIT; + int ret; + WAKE_Q(wake_q); + + if (!bitset) + return -EINVAL; + + ret = get_futex_key(uaddr, flags & FLAGS_SHARED, &key, VERIFY_READ); + +``` + +So the first function called in `futex_wake` really is `get_futex_key`! Neat! Reading the function trace was definitely an easier way to find that out than by reading the kernel code, and it’s nice to see how long all of the functions took. + +### How to know what functions you can trace + +If you run `sudo trace-cmd list -f` you’ll get a list of all the functions you can trace. That’s pretty simple but it’s important. + +### one last thing: events! + +So, now we know how to trace functions in the kernel! That’s really cool! + +There’s one more class of thing we can trace though! Some events don’t correspond super well to function calls. For example, you might want to knowwhen a program is scheduled on or off the CPU! You might be able to figure that out by peering at function calls, but I sure can’t. + +So the kernel also gives you a few events so you can see when a few important things happen. You can see a list of all these events with `sudo cat /sys/kernel/debug/tracing/available_events` + +I looked at all the sched_switch events. I’m not exactly sure what sched_switch is but it’s something to do with scheduling I guess. + +``` +sudo cat /sys/kernel/debug/tracing/available_events +sudo trace-cmd record -e sched:sched_switch +sudo trace-cmd report + +``` + +The output looks like this: + +``` + 16169.624862: Chrome_ChildIOT:24817 [112] S ==> chrome:15144 [120] + 16169.624992: chrome:15144 [120] S ==> swapper/3:0 [120] + 16169.625202: swapper/3:0 [120] R ==> Chrome_ChildIOT:24817 [112] + 16169.625251: Chrome_ChildIOT:24817 [112] R ==> chrome:1561 [112] + 16169.625437: chrome:1561 [112] S ==> chrome:15144 [120] + +``` + +so you can see it switching from PID 24817 -> 15144 -> kernel -> 24817 -> 1561 -> 15114\. (all of these events are on the same CPU) + +### how does ftrace work? + +ftrace is a dynamic tracing system. This means that when I start ftracing a kernel function, the **function’s code gets changed**. So – let’s suppose that I’m tracing that `do_page_fault` function from before. The kernel will insert some extra instructions in the assembly for that function to notify the tracing system every time that function gets called. The reason it can add extra instructions is that Linux compiles in a few extra NOP instructions into every function, so there’s space to add tracing code when needed. + +This is awesome because it means that when I’m not using ftrace to trace my kernel, it doesn’t affect performance at all. When I do start tracing, the more functions I trace, the more overhead it’ll have. + +(probably some of this is wrong, but this is how I think ftrace works anyway) + +### use ftrace more easily: brendan gregg’s tools & kernelshark + +As we’ve seen in this post, you need to think quite a lot about what individual kernel functions / events do to use ftrace directly. This is cool, but it’s also a lot of work! + +Brendan Gregg (our linux debugging tools hero) has repository of tools that use ftrace to give you information about various things like IO latency. They’re all in his [perf-tools][16] repository on GitHub. + +The tradeoff here is that they’re easier to use, but you’re limited to things that Brendan Gregg thought of & decided to make a tool for. Which is a lot of things! :) + +Another tool for visualizing the output of ftrace better is [kernelshark][17]. I haven’t played with it much yet but it looks useful. You can install it with `sudo apt-get install kernelshark`. + +### a new superpower + +I’m really happy I took the time to learn a little more about ftrace today! Like any kernel tool, it’ll work differently between different kernel versions, but I hope that you find it useful one day. + +### an index of ftrace articles + +Finally, here’s a list of a bunch of ftrace articles I found. Many of them are on LWN (Linux Weekly News), which is a pretty great source of writing on Linux. (you can buy a [subscription][18]!) + +* [Debugging the kernel using Ftrace - part 1][1] (Dec 2009, Steven Rostedt) + +* [Debugging the kernel using Ftrace - part 2][2] (Dec 2009, Steven Rostedt) + +* [Secrets of the Linux function tracer][3] (Jan 2010, Steven Rostedt) + +* [trace-cmd: A front-end for Ftrace][4] (Oct 2010, Steven Rostedt) + +* [Using KernelShark to analyze the real-time scheduler][5] (2011, Steven Rostedt) + +* [Ftrace: The hidden light switch][6] (2014, Brendan Gregg) + +* the kernel documentation: (which is quite useful) [Documentation/ftrace.txt][7] + +* documentation on events you can trace [Documentation/events.txt][8] + +* some docs on ftrace design for linux kernel devs (not as useful, but interesting) [Documentation/ftrace-design.txt][9] + +-------------------------------------------------------------------------------- + +via: https://jvns.ca/blog/2017/03/19/getting-started-with-ftrace/ + +作者：[Julia Evans ][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://jvns.ca +[1]:https://lwn.net/Articles/365835/ +[2]:https://lwn.net/Articles/366796/ +[3]:https://lwn.net/Articles/370423/ +[4]:https://lwn.net/Articles/410200/ +[5]:https://lwn.net/Articles/425583/ +[6]:https://lwn.net/Articles/608497/ +[7]:https://raw.githubusercontent.com/torvalds/linux/v4.4/Documentation/trace/ftrace.txt +[8]:https://raw.githubusercontent.com/torvalds/linux/v4.4/Documentation/trace/events.txt +[9]:https://raw.githubusercontent.com/torvalds/linux/v4.4/Documentation/trace/ftrace-design.txt +[10]:https://lwn.net/Articles/290277/ +[11]:https://lwn.net/Articles/365835/ +[12]:https://lwn.net/Articles/410200/ +[13]:https://gist.githubusercontent.com/jvns/e5c2d640f7ec76ed9ed579be1de3312e/raw/78b8425436dc4bb5bb4fa76a4f85d5809f7d1ef2/trace-cmd-report.txt +[14]:https://gist.githubusercontent.com/jvns/f32e9b06bcd2f1f30998afdd93e4aaa5/raw/8154d9828bb895fd6c9b0ee062275055b3775101/function_graph.txt +[15]:https://github.com/torvalds/linux/blob/v4.4/kernel/futex.c#L1313-L1324 +[16]:https://github.com/brendangregg/perf-tools +[17]:https://lwn.net/Articles/425583/ +[18]:https://lwn.net/subscribe/Info diff --git a/sources/tech/20170526 Creating a YUM repository from ISO - Online repo.md b/sources/tech/20170526 Creating a YUM repository from ISO - Online repo.md index cd21bb951a..ac11acc6e5 100644 --- a/sources/tech/20170526 Creating a YUM repository from ISO - Online repo.md +++ b/sources/tech/20170526 Creating a YUM repository from ISO - Online repo.md @@ -1,3 +1,5 @@ +translating---geekpi + Creating a YUM repository from ISO & Online repo ====== diff --git a/sources/tech/20170628 Notes on BPF and eBPF.md b/sources/tech/20170628 Notes on BPF and eBPF.md new file mode 100644 index 0000000000..25a7456649 --- /dev/null +++ b/sources/tech/20170628 Notes on BPF and eBPF.md @@ -0,0 +1,152 @@ +translating by qhwdw Notes on BPF & eBPF +============================================================ + +Today it was Papers We Love, my favorite meetup! Today [Suchakra Sharma][6]([@tuxology][7] on twitter/github) gave a GREAT talk about the original BPF paper and recent work in Linux on eBPF. It really made me want to go write eBPF programs! + +The paper is [The BSD Packet Filter: A New Architecture for User-level Packet Capture][8] + +I wanted to write some notes on the talk here because I thought it was super super good. + +To start, here are the [slides][9] and a [pdf][10]. The pdf is good because there are links at the end and in the PDF you can click the links. + +### what’s BPF? + +Before BPF, if you wanted to do packet filtering you had to copy all the packets into userspace and then filter them there (with “tap”). + +this had 2 problems: + +1. if you filter in userspace, it means you have to copy all the packets into userspace, copying data is expensive + +2. the filtering algorithms people were using were inefficient + +The solution to problem #1 seems sort of obvious, move the filtering logic into the kernel somehow. Okay. (though the details of how that’s done isn’t obvious, we’ll talk about that in a second) + +But why were the filtering algorithms inefficient! Well!! + +If you run `tcpdump host foo` it actually runs a relatively complicated query, which you could represent with this tree: + +![](https://jvns.ca/images/bpf-1.png) + +Evaluating this tree is kind of expensive. so the first insight is that you can actually represent this tree in a simpler way, like this: + +![](https://jvns.ca/images/bpf-2.png) + +Then if you have `ether.type = IP` and `ip.src = foo` you automatically know that the packet matches `host foo`, you don’t need to check anything else. So this data structure (they call it a “control flow graph” or “CFG”) is a way better representation of the program you actually want to execute to check matches than the tree we started with. + +### How BPF works in the kernel + +The main important here is that packets are just arrays of bytes. BPF programs run on these arrays of bytes. They’re not allowed to have loops but they  _can_  have smart stuff to figure out the length of the IP header (IPv6 & IPv4 are different lengths!) and then find the TCP port based on that length + +``` +x = ip_header_length +port = *(packet_start + x + port_offset) + +``` + +(it looks different from that but it’s basically the same). There’s a nice description of the virtual machine in the paper/slides so I won’t explain it. + +When you run `tcpdump host foo` this is what happens, as far as I understand + +1. convert `host foo` into an efficient DAG of the rules + +2. convert that DAG into a BPF program (in BPF bytecode) for the BPF virtual machine + +3. Send the BPF bytecode to the Linux kernel, which verifies it + +4. compile the BPF bytecode program into native code. For example [here’s the JIT code for ARM][1] and for [x86][2] + +5. when packets come in, Linux runs the native code to decide if that packet should be filtered or not. It’l often run only 100-200 CPU instructions for each packet that needs to be processed, which is super fast! + +### the present: eBPF + +But BPF has been around for a long time! Now we live in the EXCITING FUTURE which is eBPF. I’d heard about eBPF a bunch before but I felt like this helped me put the pieces together a little better. (i wrote this [XDP & eBPF post][11]back in April when I was at netdev) + +some facts about eBPF: + +* eBPF programs have their own bytecode language, and are compiled from that bytecode language into native code in the kernel, just like BPF programs + +* eBPF programs run in the kernel + +* eBPF programs can’t access arbitrary kernel memory. Instead the kernel provides functions to get at some restricted subset of things. + +* they  _can_  communicate with userspace programs through BPF maps + +* there’s a `bpf` syscall as of Linux 3.18 + +### kprobes & eBPF + +You can pick a function (any function!) in the Linux kernel and execute a program that you write every time that function happens. This seems really amazing and magical. + +For example! There’s this [BPF program called disksnoop][12] which tracks when you start/finish writing a block to disk. Here’s a snippet from the code: + +``` +BPF_HASH(start, struct request *); +void trace_start(struct pt_regs *ctx, struct request *req) { + // stash start timestamp by request ptr + u64 ts = bpf_ktime_get_ns(); + start.update(&req, &ts); +} +... +b.attach_kprobe(event="blk_start_request", fn_name="trace_start") +b.attach_kprobe(event="blk_mq_start_request", fn_name="trace_start") + +``` + +This basically declares a BPF hash (which the program uses to keep track of when the request starts / finishes), a function called `trace_start` which is going to be compiled into BPF bytecode, and attaches `trace_start` to the `blk_start_request` kernel function. + +This is all using the `bcc` framework which lets you write Python-ish programs that generate BPF code. You can find it (it has tons of example programs) at[https://github.com/iovisor/bcc][13] + +### uprobes & eBPF + +So I sort of knew you could attach eBPF programs to kernel functions, but I didn’t realize you could attach eBPF programs to userspace functions! That’s really exciting. Here’s [an example of counting malloc calls in Python using an eBPF program][14]. + +### things you can attach eBPF programs to + +* network cards, with XDP (which I wrote about a while back) + +* tc egress/ingress (in the network stack) + +* kprobes (any kernel function) + +* uprobes (any userspace function apparently ?? like in any C program with symbols.) + +* probes that were built for dtrace called “USDT probes” (like [these mysql probes][3]). Here’s an [example program using dtrace probes][4] + +* [the JVM][5] + +* tracepoints (not sure what that is yet) + +* seccomp / landlock security things + +* a bunch more things + +### this talk was super cool + +There are a bunch of great links in the slides and in [LINKS.md][15] in the iovisor repository. It is late now but soon I want to actually write my first eBPF program! + +-------------------------------------------------------------------------------- + +via: https://jvns.ca/blog/2017/06/28/notes-on-bpf---ebpf/ + +作者：[Julia Evans ][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://jvns.ca/ +[1]:https://github.com/torvalds/linux/blob/v4.10/arch/arm/net/bpf_jit_32.c#L512 +[2]:https://github.com/torvalds/linux/blob/v3.18/arch/x86/net/bpf_jit_comp.c#L189 +[3]:https://dev.mysql.com/doc/refman/5.7/en/dba-dtrace-ref-query.html +[4]:https://github.com/iovisor/bcc/blob/master/examples/tracing/mysqld_query.py +[5]:http://blogs.microsoft.co.il/sasha/2016/03/31/probing-the-jvm-with-bpfbcc/ +[6]:http://suchakra.in/ +[7]:https://twitter.com/tuxology +[8]:http://www.vodun.org/papers/net-papers/van_jacobson_the_bpf_packet_filter.pdf +[9]:https://speakerdeck.com/tuxology/the-bsd-packet-filter +[10]:http://step.polymtl.ca/~suchakra/PWL-Jun28-MTL.pdf +[11]:https://jvns.ca/blog/2017/04/07/xdp-bpf-tutorial/ +[12]:https://github.com/iovisor/bcc/blob/0c8c179fc1283600887efa46fe428022efc4151b/examples/tracing/disksnoop.py +[13]:https://github.com/iovisor/bcc +[14]:https://github.com/iovisor/bcc/blob/00f662dbea87a071714913e5c7382687fef6a508/tests/lua/test_uprobes.lua +[15]:https://github.com/iovisor/bcc/blob/master/LINKS.md diff --git a/sources/tech/20170829 How To Set Up PF Firewall on FreeBSD to Protect a Web Server.md b/sources/tech/20170829 How To Set Up PF Firewall on FreeBSD to Protect a Web Server.md new file mode 100644 index 0000000000..45ce0c0a7a --- /dev/null +++ b/sources/tech/20170829 How To Set Up PF Firewall on FreeBSD to Protect a Web Server.md @@ -0,0 +1,333 @@ +How To Set Up PF Firewall on FreeBSD to Protect a Web Server +====== + +I am a new FreeBSD server user and moved from netfilter on Linux. How do I setup a firewall with PF on FreeBSD server to protect a web server with single public IP address and interface? + + +PF is an acronym for packet filter. It was created for OpenBSD but has been ported to FreeBSD and other operating systems. It is a stateful packet filtering engine. This tutorial will show you how to set up a firewall with PF on FreeBSD 10.x and 11.x server to protect your web server. + + +## Step 1 - Turn on PF firewall + +You need to add the following three lines to /etc/rc.conf file: +``` +# echo 'pf_enable="YES"' >> /etc/rc.conf +# echo 'pf_rules="/usr/local/etc/pf.conf"' >> /etc/rc.conf +# echo 'pflog_enable="YES"' >> /etc/rc.conf +# echo 'pflog_logfile="/var/log/pflog"' >> /etc/rc.conf +``` +Where, + + 1. **pf_enable="YES"** - Turn on PF service. + 2. **pf_rules="/usr/local/etc/pf.conf"** - Read PF rules from this file. + 3. **pflog_enable="YES"** - Turn on logging support for PF. + 4. **pflog_logfile="/var/log/pflog"** - File where pflogd should store the logfile i.e. store logs in /var/log/pflog file. + + + +[![How To Set Up a Firewall with PF on FreeBSD to Protect a Web Server][1]][1] + +## Step 2 - Creating firewall rules in /usr/local/etc/pf.conf + +Type the following command: +``` +# vi /usr/local/etc/pf.conf +``` +Append the following PF rulesets : +``` +# vim: set ft=pf +# /usr/local/etc/pf.conf + +## Set your public interface ## +ext_if="vtnet0" + +## Set your server public IP address ## +ext_if_ip="172.xxx.yyy.zzz" + +## Set and drop these IP ranges on public interface ## +martians = "{ 127.0.0.0/8, 192.168.0.0/16, 172.16.0.0/12, \ + 10.0.0.0/8, 169.254.0.0/16, 192.0.2.0/24, \ + 0.0.0.0/8, 240.0.0.0/4 }" + +## Set http(80)/https (443) port here ## +webports = "{http, https}" + +## enable these services ## +int_tcp_services = "{domain, ntp, smtp, www, https, ftp, ssh}" +int_udp_services = "{domain, ntp}" + +## Skip loop back interface - Skip all PF processing on interface ## +set skip on lo + +## Sets the interface for which PF should gather statistics such as bytes in/out and packets passed/blocked ## +set loginterface $ext_if + +## Set default policy ## +block return in log all +block out all + +# Deal with attacks based on incorrect handling of packet fragments +scrub in all + +# Drop all Non-Routable Addresses +block drop in quick on $ext_if from $martians to any +block drop out quick on $ext_if from any to $martians + +## Blocking spoofed packets +antispoof quick for $ext_if + +# Open SSH port which is listening on port 22 from VPN 139.xx.yy.zz Ip only +# I do not allow or accept ssh traffic from ALL for security reasons +pass in quick on $ext_if inet proto tcp from 139.xxx.yyy.zzz to $ext_if_ip port = ssh flags S/SA keep state label "USER_RULE: Allow SSH from 139.xxx.yyy.zzz" +## Use the following rule to enable ssh for ALL users from any IP address # +## pass in inet proto tcp to $ext_if port ssh +### [ OR ] ### +## pass in inet proto tcp to $ext_if port 22 + +# Allow Ping-Pong stuff. Be a good sysadmin +pass inet proto icmp icmp-type echoreq + +# All access to our Nginx/Apache/Lighttpd Webserver ports +pass proto tcp from any to $ext_if port $webports + +# Allow essential outgoing traffic +pass out quick on $ext_if proto tcp to any port $int_tcp_services +pass out quick on $ext_if proto udp to any port $int_udp_services + +# Add custom rules below +``` + +Save and close the file. PR [welcome here to improve rulesets][2]. To check for syntax error, run: +`# service pf check` +OR +`/etc/rc.d/pf check` +OR +`# pfctl -n -f /usr/local/etc/pf.conf ` + +## Step 3 - Start PF firewall + +The commands are as follows. Be careful you might be disconnected from your server over ssh based session: + +### Start PF + +`# service pf start` + +### Stop PF + +`# service pf stop` + +### Check PF for syntax error + +`# service pf check` + +### Restart PF + +`# service pf restart` + +### See PF status + +`# service pf status` +Sample outputs: +``` +Status: Enabled for 0 days 00:02:18 Debug: Urgent + +Interface Stats for vtnet0 IPv4 IPv6 + Bytes In 19463 0 + Bytes Out 18541 0 + Packets In + Passed 244 0 + Blocked 3 0 + Packets Out + Passed 136 0 + Blocked 12 0 + +State Table Total Rate + current entries 1 + searches 395 2.9/s + inserts 4 0.0/s + removals 3 0.0/s +Counters + match 19 0.1/s + bad-offset 0 0.0/s + fragment 0 0.0/s + short 0 0.0/s + normalize 0 0.0/s + memory 0 0.0/s + bad-timestamp 0 0.0/s + congestion 0 0.0/s + ip-option 0 0.0/s + proto-cksum 0 0.0/s + state-mismatch 0 0.0/s + state-insert 0 0.0/s + state-limit 0 0.0/s + src-limit 0 0.0/s + synproxy 0 0.0/s + map-failed 0 0.0/s +``` + + +### Command to start/stop/restart pflog service + +Type the following commands: +``` +# service pflog start +# service pflog stop +# service pflog restart +``` + +## Step 4 - A quick introduction to pfctl command + +You need to use the pfctl command to see PF ruleset and parameter configuration including status information from the packet filter. Let us see all common commands: + +### Show PF rules information + +`# pfctl -s rules` +Sample outputs: +``` +block return in log all +block drop out all +block drop in quick on ! vtnet0 inet from 172.xxx.yyy.zzz/24 to any +block drop in quick inet from 172.xxx.yyy.zzz/24 to any +pass in quick on vtnet0 inet proto tcp from 139.aaa.ccc.ddd to 172.xxx.yyy.zzz/24 port = ssh flags S/SA keep state label "USER_RULE: Allow SSH from 139.aaa.ccc.ddd" +pass inet proto icmp all icmp-type echoreq keep state +pass out quick on vtnet0 proto tcp from any to any port = domain flags S/SA keep state +pass out quick on vtnet0 proto tcp from any to any port = ntp flags S/SA keep state +pass out quick on vtnet0 proto tcp from any to any port = smtp flags S/SA keep state +pass out quick on vtnet0 proto tcp from any to any port = http flags S/SA keep state +pass out quick on vtnet0 proto tcp from any to any port = https flags S/SA keep state +pass out quick on vtnet0 proto tcp from any to any port = ftp flags S/SA keep state +pass out quick on vtnet0 proto tcp from any to any port = ssh flags S/SA keep state +pass out quick on vtnet0 proto udp from any to any port = domain keep state +pass out quick on vtnet0 proto udp from any to any port = ntp keep state +``` + +#### Show verbose output for each rule + +`# pfctl -v -s rules` + +#### Add rule numbers with verbose output for each rule + +`# pfctl -vvsr show` + +#### Show state + +``` +# pfctl -s state +# pfctl -s state | more +# pfctl -s state | grep 'something' +``` + +### How to disable PF from the CLI + +`# pfctl -d ` + +### How to enable PF from the CLI + +`# pfctl -e ` + +### How to flush ALL PF rules/nat/tables from the CLI + +`# pfctl -F all` +Sample outputs: +``` +rules cleared +nat cleared +0 tables deleted. +2 states cleared +source tracking entries cleared +pf: statistics cleared +pf: interface flags reset +``` + +#### How to flush only the PF RULES from the CLI + +`# pfctl -F rules ` + +#### How to flush only queue's from the CLI + +`# pfctl -F queue ` + +#### How to flush all stats that are not part of any rule from the CLI + +`# pfctl -F info` + +#### How to clear all counters from the CLI + +`# pfctl -z clear ` + +## Step 5 - See PF log + +PF logs are in binary format. To see them type: +`# tcpdump -n -e -ttt -r /var/log/pflog` +Sample outputs: +``` +Aug 29 15:41:11.757829 rule 0/(match) block in on vio0: 86.47.225.151.55806 > 45.FOO.BAR.IP.23: S 757158343:757158343(0) win 52206 [tos 0x28] +Aug 29 15:41:44.193309 rule 0/(match) block in on vio0: 5.196.83.88.25461 > 45.FOO.BAR.IP.26941: S 2224505792:2224505792(0) ack 4252565505 win 17520 (DF) [tos 0x24] +Aug 29 15:41:54.628027 rule 0/(match) block in on vio0: 45.55.13.94.50217 > 45.FOO.BAR.IP.465: S 3941123632:3941123632(0) win 65535 +Aug 29 15:42:11.126427 rule 0/(match) block in on vio0: 87.250.224.127.59862 > 45.FOO.BAR.IP.80: S 248176545:248176545(0) win 28200 (DF) +Aug 29 15:43:04.953537 rule 0/(match) block in on vio0: 77.72.82.22.47218 > 45.FOO.BAR.IP.7475: S 1164335542:1164335542(0) win 1024 +Aug 29 15:43:05.122156 rule 0/(match) block in on vio0: 77.72.82.22.47218 > 45.FOO.BAR.IP.7475: R 1164335543:1164335543(0) win 1200 +Aug 29 15:43:37.302410 rule 0/(match) block in on vio0: 94.130.12.27.18080 > 45.FOO.BAR.IP.64857: S 683904905:683904905(0) ack 4000841729 win 16384 +Aug 29 15:44:46.574863 rule 0/(match) block in on vio0: 77.72.82.22.47218 > 45.FOO.BAR.IP.7677: S 3451987887:3451987887(0) win 1024 +Aug 29 15:44:46.819754 rule 0/(match) block in on vio0: 77.72.82.22.47218 > 45.FOO.BAR.IP.7677: R 3451987888:3451987888(0) win 1200 +Aug 29 15:45:21.194752 rule 0/(match) block in on vio0: 185.40.4.130.55910 > 45.FOO.BAR.IP.80: S 3106068642:3106068642(0) win 1024 +Aug 29 15:45:32.999219 rule 0/(match) block in on vio0: 185.40.4.130.55910 > 45.FOO.BAR.IP.808: S 322591763:322591763(0) win 1024 +Aug 29 15:46:30.157884 rule 0/(match) block in on vio0: 77.72.82.22.47218 > 45.FOO.BAR.IP.6511: S 2412580953:2412580953(0) win 1024 [tos 0x28] +Aug 29 15:46:30.252023 rule 0/(match) block in on vio0: 77.72.82.22.47218 > 45.FOO.BAR.IP.6511: R 2412580954:2412580954(0) win 1200 [tos 0x28] +Aug 29 15:49:44.337015 rule 0/(match) block in on vio0: 189.219.226.213.22640 > 45.FOO.BAR.IP.23: S 14807:14807(0) win 14600 [tos 0x28] +Aug 29 15:49:55.161572 rule 0/(match) block in on vio0: 5.196.83.88.25461 > 45.FOO.BAR.IP.40321: S 1297217585:1297217585(0) ack 1051525121 win 17520 (DF) [tos 0x24] +Aug 29 15:49:59.735391 rule 0/(match) block in on vio0: 36.7.147.209.2545 > 45.FOO.BAR.IP.3389: SWE 3577047469:3577047469(0) win 8192 (DF) [tos 0x2 (E)] +Aug 29 15:50:00.703229 rule 0/(match) block in on vio0: 36.7.147.209.2546 > 45.FOO.BAR.IP.3389: SWE 1539382950:1539382950(0) win 8192 (DF) [tos 0x2 (E)] +Aug 29 15:51:33.880334 rule 0/(match) block in on vio0: 45.55.22.21.53510 > 45.FOO.BAR.IP.2362: udp 14 +Aug 29 15:51:34.006656 rule 0/(match) block in on vio0: 77.72.82.22.47218 > 45.FOO.BAR.IP.6491: S 151489102:151489102(0) win 1024 [tos 0x28] +Aug 29 15:51:34.274654 rule 0/(match) block in on vio0: 77.72.82.22.47218 > 45.FOO.BAR.IP.6491: R 151489103:151489103(0) win 1200 [tos 0x28] +Aug 29 15:51:36.393019 rule 0/(match) block in on vio0: 60.191.38.78.4249 > 45.FOO.BAR.IP.8000: S 3746478095:3746478095(0) win 29200 (DF) +Aug 29 15:51:57.213051 rule 0/(match) block in on vio0: 24.137.245.138.7343 > 45.FOO.BAR.IP.5358: S 14134:14134(0) win 14600 +Aug 29 15:52:37.852219 rule 0/(match) block in on vio0: 122.226.185.125.51128 > 45.FOO.BAR.IP.23: S 1715745381:1715745381(0) win 5840 (DF) +Aug 29 15:53:31.309325 rule 0/(match) block in on vio0: 189.218.148.69.377 > 45.FOO.BAR.IP5358: S 65340:65340(0) win 14600 [tos 0x28] +Aug 29 15:53:31.809570 rule 0/(match) block in on vio0: 13.93.104.140.53184 > 45.FOO.BAR.IP.1433: S 39854048:39854048(0) win 1024 +Aug 29 15:53:32.138231 rule 0/(match) block in on vio0: 13.93.104.140.53184 > 45.FOO.BAR.IP.1433: R 39854049:39854049(0) win 1200 +Aug 29 15:53:41.459088 rule 0/(match) block in on vio0: 77.72.82.22.47218 > 45.FOO.BAR.IP.6028: S 168338703:168338703(0) win 1024 +Aug 29 15:53:41.789732 rule 0/(match) block in on vio0: 77.72.82.22.47218 > 45.FOO.BAR.IP.6028: R 168338704:168338704(0) win 1200 +Aug 29 15:54:34.993594 rule 0/(match) block in on vio0: 212.47.234.50.5102 > 45.FOO.BAR.IP.5060: udp 408 (DF) [tos 0x28] +Aug 29 15:54:57.987449 rule 0/(match) block in on vio0: 51.15.69.145.5100 > 45.FOO.BAR.IP.5060: udp 406 (DF) [tos 0x28] +Aug 29 15:55:07.001743 rule 0/(match) block in on vio0: 190.83.174.214.58863 > 45.FOO.BAR.IP.23: S 757158343:757158343(0) win 27420 +Aug 29 15:55:51.269549 rule 0/(match) block in on vio0: 142.217.201.69.26112 > 45.FOO.BAR.IP.22: S 757158343:757158343(0) win 22840 +Aug 29 15:58:41.346028 rule 0/(match) block in on vio0: 169.1.29.111.29765 > 45.FOO.BAR.IP.23: S 757158343:757158343(0) win 28509 +Aug 29 15:59:11.575927 rule 0/(match) block in on vio0: 187.160.235.162.32427 > 45.FOO.BAR.IP.5358: S 22445:22445(0) win 14600 [tos 0x28] +Aug 29 15:59:37.826598 rule 0/(match) block in on vio0: 94.74.81.97.54656 > 45.FOO.BAR.IP.3128: S 2720157526:2720157526(0) win 1024 [tos 0x28] +Aug 29 15:59:37.991171 rule 0/(match) block in on vio0: 94.74.81.97.54656 > 45.FOO.BAR.IP.3128: R 2720157527:2720157527(0) win 1200 [tos 0x28] +Aug 29 16:01:36.990050 rule 0/(match) block in on vio0: 182.18.8.28.23299 > 45.FOO.BAR.IP.445: S 1510146048:1510146048(0) win 16384 +``` + +To see live log run: +`# tcpdump -n -e -ttt -i pflog0` +For more info the [PF FAQ][3], [FreeBSD HANDBOOK][4] and the following man pages: +``` +# man tcpdump +# man pfctl +# man pf +``` + +## about the author: + +The author is the creator of nixCraft and a seasoned sysadmin and a trainer for the Linux operating system/Unix shell scripting. He has worked with global clients and in various industries, including IT, education, defense and space research, and the nonprofit sector. Follow him on [Twitter][5], [Facebook][6], [Google+][7]. + +-------------------------------------------------------------------------------- + +via: https://www.cyberciti.biz/faq/how-to-set-up-a-firewall-with-pf-on-freebsd-to-protect-a-web-server/ + +作者：[Vivek Gite][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.cyberciti.biz +[1]:https://www.cyberciti.biz/media/new/faq/2017/08/howto-setup-a-firewall-with-pf-on-freebsd.001.jpeg +[2]:https://github.com/nixcraft/pf.conf/blob/master/pf.conf +[3]:https://www.openbsd.org/faq/pf/ +[4]:https://www.freebsd.org/doc/handbook/firewalls.html +[5]:https://twitter.com/nixcraft +[6]:https://facebook.com/nixcraft +[7]:https://plus.google.com/+CybercitiBiz diff --git a/sources/tech/20170919 What Are Bitcoins.md b/sources/tech/20170919 What Are Bitcoins.md deleted file mode 100644 index c61b32b76a..0000000000 --- a/sources/tech/20170919 What Are Bitcoins.md +++ /dev/null @@ -1,82 +0,0 @@ -translating by Flowsnow - -What Are Bitcoins? -====== - -![what are bitcoins](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/what-are-bitcoins_orig.jpg) - - **[Bitcoin][1]** is a digital currency or electronic cash the relies on peer to peer technology for completing transactions. Since peer to peer technology is used as the major network, bitcoins provide a community like managed economy. This is to mean, bitcoins eliminate the centralized authority way of managing currency and promotes community management of currency. Most Also of the software related to bitcoin mining and managing of bitcoin digital cash is open source. - -​The first Bitcoin software was developed by Satoshi Nakamoto and it's based on open source cryptographic protocol. Bitcoins smallest unit is known as the Satoshi which is basically one-hundredth millionth of a single bitcoin (0.00000001 BTC). - -One cannot underestimate the boundaries BITCOINS eliminate in the digital economy. For instance, the BITCOIN eliminates governed controls over currency by a centralised agency and offers control and management to the community as a whole. Furthermore, the fact that the BITCOIN is based on an open source cryptographic protocol makes it an open place where there are scrupulous activities such as fluctuating value, deflation and inflation among others. While many internet users are becoming aware of the privacy they should exercise to complete some online transactions, bitcoin is gaining more popularity than ever before. However, for those who know about the dark web and how it works can acknowledge that some people began using it long ago. - -On the downside, the bitcoin is also very secure in making anonymous payments which may be a threat to security or personal health. For instance, the dark web markets are the major suppliers and retailers of imported drugs and even weapons. The use of BITCOINs in the dark web facilitates a safe network for such criminal activities. Despite that, if put to good use, bitcoin has many benefits that can eliminate some of the economic fallacy as a result of centralized agency management of currency. In addition, the bitcoin allows for instance exchange of cash anywhere in the world. The use of bitcoins also mitigates counterfeiting, printing, or devaluation over time. Also, while relying on peer to peer network as its backbone, it promotes the distributed authority of transaction records making it safe to make exchanges. - -Other advantages of the bitcoin include; - -* In the online business world, bitcoin promotes money security and total control. This is because buyers are protected against merchants who may want to charge extra for a lower cost service. The buyer can also choose not to share personal information after making a transaction. Besides, identity theft protection is achieved as a result of backed up hiding personal information. - -* Bitcoins are provided alternatives to major common currency catastrophes such as getting lost, frozen or damaged. However, it is recommended to always make a backup of your bitcoins and encrypt them with a password. - -* In making online purchases and payments using bitcoins, there is a small fee or zero transaction fee charged. This promotes affordability of use. - -* Merchants also face fewer risks that could result from fraud as bitcoin transactions cannot be reversed, unlike other currencies in electronic form. Bitcoins also prove useful even in moments of high crime rate and fraud since it is difficult to con someone over an open public ledger (Blockchain). - -* Bitcoin currency is also hard to be manipulated as it is open source and the cryptographic protocol is very secure. - -* Transactions can also be verified and approved, anywhere, anytime. This is the level of flexibility offered by this digital currency. - -Also Read - [Bitkey A Linux Distribution Dedicated To Bitcoin Transactions][2] - -### How To Mine Bitcoins and The Applications to Accomplish Necessary Bitcoin Management Tasks - -​In the digital currency, BITCOIN mining and management requires additional software. There are numerous open source bitcoin management software that make it easy to make payments, receive payments, encrypt and backup of your bitcoins and also bitcoin mining software. There are sites such as; [Freebitcoin][4] where one earns free bitcoins by viewing ads, [MoonBitcoin][5] is another site that one can sign up for free and earn bitcoins. However, it is convenient if one has spare time and a sizable network of friends participating in the same. There are many sites offering bitcoin mining and one can easily sign up and start mining. One of the major secrets is referring as many people as you can to create a large network. - -Applications required for use with bitcoins include the bitcoin wallet which allows one to safely keep bitcoins. This is just like the physical wallet using to keep hard cash but in a digital form. The wallet can be downloaded here - [Bitcoin - Wallet][6] . Other similar applications include; the [Blockchain][7] which works similar to the Bitcoin Wallet. - -The screenshots below show the Freebitco and MoonBitco mining sites respectively. - - [![freebitco bitcoin mining site](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/freebitco-bitcoin-mining-site_orig.jpg)][8] - [![moonbitcoin bitcoin mining site](http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/moonbitcoin-bitcoin-mining-site_orig.png)][9] - -​There are various ways of acquiring the bitcoin currency. Some of them include the use of bitcoin mining rigs, purchasing of bitcoins in exchange markets and doing free bitcoin mining online. Purchasing of bitcoins can be done at; [MtGox][10] , [bitNZ][11] , [Bitstamp][12] , [BTC-E][13] , [VertEx][14] , etc.. Several mining open source applications are available online. These applications include; Bitminter, [5OMiner][15] , [BFG Miner][16] among others. These applications make use of some graphics card and processor features to generate bitcoins. The efficiency of mining bitcoins on a pc largely depends on the type of graphics card and the processor of the mining rig. Besides, there are many secure online storages for backing up bitcoins. These sites provide bitcoin storage services free of charge. Examples of bitcoin managing sites include; [xapo][17] , [BlockChain][18] etc. signing up on these sites require a valid email and phone number for verification. Xapo offers additional security through the phone application by requesting for verification whenever a new sign in is made. - -### Disadvantages Of Bitcoins - -​The numerous advantages ripped from using bitcoins digital currency cannot be overlooked. However, as it is still in its infancy stage, the bitcoin currency meets several points of resistance. For instance, the majority of individual are not fully aware of the bitcoin digital currency and how it works. The lack of awareness can be mitigated through education and creation of awareness. Bitcoin users also face volatility as the demand for bitcoins is higher than the available amount of coins. However, given more time, volatility will be lowered as when many people will start using bitcoins. - -### Improvements Can be Made - -​Based on the infancy of the [bitcoin technology][19] , there is still room for changes to make it more secure and reliable. Given more time, the bitcoin currency will be developed enough to provide flexibility as a common currency. For the bitcoin to succeed, many people need to be made aware of it besides being given information on how it works and its benefits. - --------------------------------------------------------------------------------- - -via: http://www.linuxandubuntu.com/home/things-you-need-to-know-about-bitcoins - -作者：[LINUXANDUBUNTU][a] -译者：[译者ID](https://github.com/译者ID) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:http://www.linuxandubuntu.com/ -[1]:http://www.linuxandubuntu.com/home/bitkey-a-linux-distribution-dedicated-for-conducting-bitcoin-transactions -[2]:http://www.linuxandubuntu.com/home/bitkey-a-linux-distribution-dedicated-for-conducting-bitcoin-transactions -[3]:http://www.linuxandubuntu.com/home/things-you-need-to-know-about-bitcoins -[4]:https://freebitco.in/?r=2167375 -[5]:http://moonbit.co.in/?ref=c637809a5051 -[6]:https://bitcoin.org/en/choose-your-wallet -[7]:https://blockchain.info/wallet/ -[8]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/freebitco-bitcoin-mining-site_orig.jpg -[9]:http://www.linuxandubuntu.com/uploads/2/1/1/5/21152474/moonbitcoin-bitcoin-mining-site_orig.png -[10]:http://mtgox.com/ -[11]:https://en.bitcoin.it/wiki/BitNZ -[12]:https://www.bitstamp.net/ -[13]:https://btc-e.com/ -[14]:https://www.vertexinc.com/ -[15]:https://www.downloadcloud.com/bitcoin-miner-software.html -[16]:https://github.com/luke-jr/bfgminer -[17]:https://xapo.com/ -[18]:https://www.blockchain.com/ -[19]:https://en.wikipedia.org/wiki/Bitcoin diff --git a/sources/tech/20171224 My first Rust macro.md b/sources/tech/20171224 My first Rust macro.md new file mode 100644 index 0000000000..a8002e050b --- /dev/null +++ b/sources/tech/20171224 My first Rust macro.md @@ -0,0 +1,145 @@ +My first Rust macro +============================================================ + +Last night I wrote a Rust macro for the first time!! The most striking thing to me about this was how **easy** it was – I kind of expected it to be a weird hard finicky thing, and instead I found that I could go from “I don’t know how macros work but I think I could do this with a macro” to “wow I’m done” in less than an hour. + +I used [these examples][2] to figure out how to write my macro. + +### what’s a macro? + +There’s more than one kind of macro in Rust – + +* macros defined using `macro_rules` (they have an exclamation mark and you call them like functions – `my_macro!()`) + +* “syntax extensions” / “procedural macros” like `#[derive(Debug)]` (you put these like annotations on your functions) + +* built-in macros like `println!` + +[Macros in Rust][3] and [Macros in Rust part II][4] seems like a nice overview of the different kinds with examples + +I’m not actually going to try to explain what a macro **is**, instead I will just show you what I used a macro for yesterday and hopefully that will be interesting. I’m going to be talking about `macro_rules!`, I don’t understand syntax extension/procedural macros yet. + +### compiling the `get_stack_trace` function for 30 different Ruby versions + +I’d written some functions that got the stack trace out of a running Ruby program (`get_stack_trace`). But the function I wrote only worked for Ruby 2.2.0 – here’s what it looked like. Basically it imported some structs from `bindings::ruby_2_2_0` and then used them. + +``` +use bindings::ruby_2_2_0::{rb_control_frame_struct, rb_thread_t, RString}; +fn get_stack_trace(pid: pid_t) -> Vec { + // some code using rb_control_frame_struct, rb_thread_t, RString +} + +``` + +Let’s say I wanted to instead have a version of `get_stack_trace` that worked for Ruby 2.1.6. `bindings::ruby_2_2_0` and `bindings::ruby_2_1_6` had basically all the same structs in them. But `bindings::ruby_2_1_6::rb_thread_t` wasn’t the **same** as `bindings::ruby_2_2_0::rb_thread_t`, it just had the same name and most of the same struct members. + +So I could implement a working function for Ruby 2.1.6 really easily! I just need to basically replace `2_2_0` for `2_1_6`, and then the compiler would generate different code (because `rb_thread_t` is different). Here’s a sketch of what the Ruby 2.1.6 version would look like: + +``` +use bindings::ruby_2_1_6::{rb_control_frame_struct, rb_thread_t, RString}; +fn get_stack_trace(pid: pid_t) -> Vec { + // some code using rb_control_frame_struct, rb_thread_t, RString +} + +``` + +### what I wanted to do + +I basically wanted to write code like this, to generate a `get_stack_trace` function for every Ruby version. The code inside `get_stack_trace` would be the same in every case, it’s just the `use bindings::ruby_2_1_3` that needed to be different + +``` +pub mod ruby_2_1_3 { + use bindings::ruby_2_1_3::{rb_control_frame_struct, rb_thread_t, RString}; + fn get_stack_trace(pid: pid_t) -> Vec { + // insert code here + } +} +pub mod ruby_2_1_4 { + use bindings::ruby_2_1_4::{rb_control_frame_struct, rb_thread_t, RString}; + fn get_stack_trace(pid: pid_t) -> Vec { + // same code + } +} +pub mod ruby_2_1_5 { + use bindings::ruby_2_1_5::{rb_control_frame_struct, rb_thread_t, RString}; + fn get_stack_trace(pid: pid_t) -> Vec { + // same code + } +} +pub mod ruby_2_1_6 { + use bindings::ruby_2_1_6::{rb_control_frame_struct, rb_thread_t, RString}; + fn get_stack_trace(pid: pid_t) -> Vec { + // same code + } +} + +``` + +### macros to the rescue! + +This really repetitive thing was I wanted to do was a GREAT fit for macros. Here’s what using `macro_rules!` to do this looked like! + +``` +macro_rules! ruby_bindings( + ($ruby_version:ident) => ( + pub mod $ruby_version { + use bindings::$ruby_version::{rb_control_frame_struct, rb_thread_t, RString}; + fn get_stack_trace(pid: pid_t) -> Vec { + // insert code here + } + } +)); + +``` + +I basically just needed to put my code in and insert `$ruby_version` in the places I wanted it to go in. So simple! I literally just looked at an example, tried the first thing I thought would work, and it worked pretty much right away. + +(the [actual code][5] is more lines and messier but the usage of macros is exactly as simple in this example) + +I was SO HAPPY about this because I’d been worried getting this to work would be hard but instead it was so easy!! + +### dispatching to the right code + +Then I wrote some super simple dispatch code to call the right code depending on which Ruby version was running! + +``` + let version = get_api_version(pid); + let stack_trace_function = match version.as_ref() { + "2.1.1" => stack_trace::ruby_2_1_1::get_stack_trace, + "2.1.2" => stack_trace::ruby_2_1_2::get_stack_trace, + "2.1.3" => stack_trace::ruby_2_1_3::get_stack_trace, + "2.1.4" => stack_trace::ruby_2_1_4::get_stack_trace, + "2.1.5" => stack_trace::ruby_2_1_5::get_stack_trace, + "2.1.6" => stack_trace::ruby_2_1_6::get_stack_trace, + "2.1.7" => stack_trace::ruby_2_1_7::get_stack_trace, + "2.1.8" => stack_trace::ruby_2_1_8::get_stack_trace, + // and like 20 more versions + _ => panic!("OH NO OH NO OH NO"), + }; + +``` + +### it works! + +I tried out my prototype, and it totally worked! The same program could get stack traces out the running Ruby program for all of the ~10 different Ruby versions I tried – it figured which Ruby version was running, called the right code, and got me stack traces!! + +Previously I’d compile a version for Ruby 2.2.0 but then if I tried to use it for any other Ruby version it would crash, so this was a huge improvement. + +There are still more issues with this approach that I need to sort out. The two main ones right now are: firstly the ruby binary that ships with Debian doesn’t have symbols and I need the address of the current thread, and secondly it’s still possible that `#ifdefs` will ruin my day. + +-------------------------------------------------------------------------------- + +via: https://jvns.ca/blog/2017/12/24/my-first-rust-macro/ + +作者：[Julia Evans ][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://jvns.ca +[1]:https://jvns.ca/categories/ruby-profiler +[2]:https://gist.github.com/jfager/5936197 +[3]:https://www.ncameron.org/blog/macros-in-rust-pt1/ +[4]:https://www.ncameron.org/blog/macros-in-rust-pt2/ +[5]:https://github.com/jvns/ruby-stacktrace/blob/b0b92863564e54da59ea7f066aff5bb0d92a4968/src/lib.rs#L249-L393 diff --git a/sources/tech/20180102 Best open source tutorials in 2017.md b/sources/tech/20180102 Best open source tutorials in 2017.md index e9d9d7b9ad..7612772b49 100644 --- a/sources/tech/20180102 Best open source tutorials in 2017.md +++ b/sources/tech/20180102 Best open source tutorials in 2017.md @@ -1,3 +1,4 @@ +Translating zjon Best open source tutorials in 2017 ====== ![](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/osdc-lead-teacher-learner.png?itok=rMJqBN5G) diff --git a/sources/tech/20180102 How To Find (Top-10) Largest Files In Linux.md b/sources/tech/20180102 How To Find (Top-10) Largest Files In Linux.md new file mode 100644 index 0000000000..7e5d8c82a5 --- /dev/null +++ b/sources/tech/20180102 How To Find (Top-10) Largest Files In Linux.md @@ -0,0 +1,189 @@ +How To Find (Top-10) Largest Files In Linux +====== +When you are running out of disk space in system, you may prefer to check with df command or du command or ncdu command but all these will tell you only current directory files and doesn't shows the system wide files. + +You have to spend huge amount of time to get the largest files in the system using the above commands, that to you have to navigate to each and every directory to achieve this. + +It's making you to face trouble and this is not the right way to do it. + +If so, what would be the suggested way to get top 10 largest files in Linux? + +I have spend a lot of time with google but i didn't found this. Everywhere i could see an article which list the top 10 files in the current directory. So, i want to make this article useful for people whoever looking to get the top 10 largest files in the system. + +In this tutorial, we are going to teach you how to find top 10 largest files in Linux system using below four methods. + +### Method-1 : + +There is no specific command available in Linux to do this, hence we are using more than one command (all together) to get this done. +``` +# find / -type f -print0 | xargs -0 du -h | sort -rh | head -n 10 + +1.4G /swapfile +1.1G /home/magi/ubuntu-17.04-desktop-amd64.iso +564M /home/magi/.gdfuse/magi/cache/0B5nso_FPaZFqTFU0XzkzUlJUZzA +378M /home/magi/.gdfuse/magi/cache/0B5nso_FPaZFqeldzUmhPeC03Zm8 +377M /home/magi/.gdfuse/magi/cache/0B5nso_FPaZFqRGd4V0VrOXM4YVU +100M /usr/lib/x86_64-linux-gnu/libOxideQtCore.so.0 +93M /usr/lib/firefox/libxul.so +84M /var/lib/snapd/snaps/core_3604.snap +84M /var/lib/snapd/snaps/core_3440.snap +84M /var/lib/snapd/snaps/core_3247.snap + +``` + +**Details :** +**`find`** : It 's a command, Search for files in a directory hierarchy. +**`/`** : Check in the whole system (starting from / directory) +**`-type`** : File is of type + +**`f`** : Regular file +**`-print0`** : Print the full file name on the standard output, followed by a null character +**`|`** : Control operator that send the output of one program to another program for further processing. + +**`xargs`** : It 's a command, which build and execute command lines from standard input. +**`-0`** : Input items are terminated by a null character instead of by whitespace +**`du -h`** : It 's a command to calculate disk usage with human readable format + +**`sort`** : It 's a command, Sort lines of text files +**`-r`** : Reverse the result of comparisons +**`-h`** : Print the output with human readable format + +**`head`** : It 's a command, Output the first part of files +**`n -10`** : Print the first 10 files. + +### Method-2 : + +This is an another way to find or check top 10 largest files in Linux system. Here also, we are putting few commands together to achieve this. +``` +# find / -type f -exec du -Sh {} + | sort -rh | head -n 10 + +1.4G /swapfile +1.1G /home/magi/ubuntu-17.04-desktop-amd64.iso +564M /home/magi/.gdfuse/magi/cache/0B5nso_FPaZFqTFU0XzkzUlJUZzA +378M /home/magi/.gdfuse/magi/cache/0B5nso_FPaZFqeldzUmhPeC03Zm8 +377M /home/magi/.gdfuse/magi/cache/0B5nso_FPaZFqRGd4V0VrOXM4YVU +100M /usr/lib/x86_64-linux-gnu/libOxideQtCore.so.0 +93M /usr/lib/firefox/libxul.so +84M /var/lib/snapd/snaps/core_3604.snap +84M /var/lib/snapd/snaps/core_3440.snap +84M /var/lib/snapd/snaps/core_3247.snap + +``` + +**Details :** +**`find`** : It 's a command, Search for files in a directory hierarchy. +**`/`** : Check in the whole system (starting from / directory) +**`-type`** : File is of type + +**`f`** : Regular file +**`-exec`** : This variant of the -exec action runs the specified command on the selected files +**`du`** : It 's a command to estimate file space usage. + +**`-S`** : Do not include size of subdirectories +**`-h`** : Print sizes in human readable format +**`{}`** : Summarize disk usage of each FILE, recursively for directories. + +**`|`** : Control operator that send the output of one program to another program for further processing. +**`sort`** : It 's a command, Sort lines of text files +**`-r`** : Reverse the result of comparisons + +**`-h`** : Compare human readable numbers +**`head`** : It 's a command, Output the first part of files +**`n -10`** : Print the first 10 files. + +### Method-3 : + +It 's an another method to find or search top 10 largest files in Linux system. +``` +# find / -type f -print0 | xargs -0 du | sort -n | tail -10 | cut -f2 | xargs -I{} du -sh {} + +84M /var/lib/snapd/snaps/core_3247.snap +84M /var/lib/snapd/snaps/core_3440.snap +84M /var/lib/snapd/snaps/core_3604.snap +93M /usr/lib/firefox/libxul.so +100M /usr/lib/x86_64-linux-gnu/libOxideQtCore.so.0 +377M /home/magi/.gdfuse/magi/cache/0B5nso_FPaZFqRGd4V0VrOXM4YVU +378M /home/magi/.gdfuse/magi/cache/0B5nso_FPaZFqeldzUmhPeC03Zm8 +564M /home/magi/.gdfuse/magi/cache/0B5nso_FPaZFqTFU0XzkzUlJUZzA +1.1G /home/magi/ubuntu-17.04-desktop-amd64.iso +1.4G /swapfile + +``` + +**Details :** +**`find`** : It 's a command, Search for files in a directory hierarchy. +**`/`** : Check in the whole system (starting from / directory) +**`-type`** : File is of type + +**`f`** : Regular file +**`-print0`** : Print the full file name on the standard output, followed by a null character +**`|`** : Control operator that send the output of one program to another program for further processing. + +**`xargs`** : It 's a command, which build and execute command lines from standard input. +**`-0`** : Input items are terminated by a null character instead of by whitespace +**`du`** : It 's a command to estimate file space usage. + +**`sort`** : It 's a command, Sort lines of text files +**`-n`** : Compare according to string numerical value +**`tail -10`** : It 's a command, output the last part of files (last 10 files) + +**`cut`** : It 's a command, remove sections from each line of files +**`-f2`** : Select only these fields value. +**`-I{}`** : Replace occurrences of replace-str in the initial-arguments with names read from standard input. + +**`-s`** : Display only a total for each argument +**`-h`** : Print sizes in human readable format +**`{}`** : Summarize disk usage of each FILE, recursively for directories. + +### Method-4 : + +It 's an another method to find or search top 10 largest files in Linux system. +``` +# find / -type f -ls | sort -k 7 -r -n | head -10 | column -t | awk '{print $7,$11}' + +1494845440 /swapfile +1085984380 /home/magi/ubuntu-17.04-desktop-amd64.iso +591003648 /home/magi/.gdfuse/magi/cache/0B5nso_FPaZFqTFU0XzkzUlJUZzA +395770383 /home/magi/.gdfuse/magi/cache/0B5nso_FPaZFqeldzUmhPeC03Zm8 +394891761 /home/magi/.gdfuse/magi/cache/0B5nso_FPaZFqRGd4V0VrOXM4YVU +103999072 /usr/lib/x86_64-linux-gnu/libOxideQtCore.so.0 +97356256 /usr/lib/firefox/libxul.so +87896064 /var/lib/snapd/snaps/core_3604.snap +87793664 /var/lib/snapd/snaps/core_3440.snap +87089152 /var/lib/snapd/snaps/core_3247.snap + +``` + +**Details :** +**`find`** : It 's a command, Search for files in a directory hierarchy. +**`/`** : Check in the whole system (starting from / directory) +**`-type`** : File is of type + +**`f`** : Regular file +**`-ls`** : List current file in ls -dils format on standard output. +**`|`** : Control operator that send the output of one program to another program for further processing. + +**`sort`** : It 's a command, Sort lines of text files +**`-k`** : start a key at POS1 +**`-r`** : Reverse the result of comparisons + +**`-n`** : Compare according to string numerical value +**`head`** : It 's a command, Output the first part of files +**`-10`** : Print the first 10 files. + +**`column`** : It 's a command, formats its input into multiple columns. +**`-t`** : Determine the number of columns the input contains and create a table. +**`awk`** : It 's a command, Pattern scanning and processing language +**`'{print $7,$11}'`** : Print only mentioned column. + +-------------------------------------------------------------------------------- + +via: https://www.2daygeek.com/how-to-find-search-check-print-top-10-largest-biggest-files-in-linux/ + +作者：[Magesh Maruthamuthu][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.2daygeek.com/author/magesh/ diff --git a/sources/tech/20180104 How does gdb call functions.md b/sources/tech/20180104 How does gdb call functions.md new file mode 100644 index 0000000000..a62b30ea31 --- /dev/null +++ b/sources/tech/20180104 How does gdb call functions.md @@ -0,0 +1,252 @@ +How does gdb call functions? +============================================================ + +(previous gdb posts: [how does gdb work? (2016)][4] and [three things you can do with gdb (2014)][5]) + +I discovered this week that you can call C functions from gdb! I thought this was cool because I’d previously thought of gdb as mostly a read-only debugging tool. + +I was really surprised by that (how does that WORK??). As I often do, I asked [on Twitter][6] how that even works, and I got a lot of really useful answers! My favorite answer was [Evan Klitzke’s example C code][7] showing a way to do it. Code that  _works_  is very exciting! + +I believe (through some stracing & experiments) that that example C code is different from how gdb actually calls functions, so I’ll talk about what I’ve figured out about what gdb does in this post and how I’ve figured it out. + +There is a lot I still don’t know about how gdb calls functions, and very likely some things in here are wrong. + +### What does it mean to call a C function from gdb? + +Before I get into how this works, let’s talk quickly about why I found it surprising / nonobvious. + +So, you have a running C program (the “target program”). You want to run a function from it. To do that, you need to basically: + +* pause the program (because it is already running code!) + +* find the address of the function you want to call (using the symbol table) + +* convince the program (the “target program”) to jump to that address + +* when the function returns, restore the instruction pointer and registers to what they were before + +Using the symbol table to figure out the address of the function you want to call is pretty straightforward – here’s some sketchy (but working!) Rust code that I’ve been using on Linux to do that. This code uses the [elf crate][8]. If I wanted to find the address of the `foo` function in PID 2345, I’d run `elf_symbol_value("/proc/2345/exe", "foo")`. + +``` +fn elf_symbol_value(file_name: &str, symbol_name: &str) -> Result> { + // open the ELF file + let file = elf::File::open_path(file_name).ok().ok_or("parse error")?; + // loop over all the sections & symbols until you find the right one! + let sections = &file.sections; + for s in sections { + for sym in file.get_symbols(&s).ok().ok_or("parse error")? { + if sym.name == symbol_name { + return Ok(sym.value); + } + } + } + None.ok_or("No symbol found")? +} + +``` + +This won’t totally work on its own, you also need to look at the memory maps of the file and add the symbol offset to the start of the place that file is mapped. But finding the memory maps isn’t so hard, they’re in `/proc/PID/maps`. + +Anyway, this is all to say that finding the address of the function to call seemed straightforward to me but that the rest of it (change the instruction pointer? restore the registers? what else?) didn’t seem so obvious! + +### You can’t just jump + +I kind of said this already but – you can’t just find the address of the function you want to run and then jump to that address. I tried that in gdb (`jump foo`) and the program segfaulted. Makes sense! + +### How you can call C functions from gdb + +First, let’s see that this is possible. I wrote a tiny C program that sleeps for 1000 seconds and called it `test.c`: + +``` +#include + +int foo() { + return 3; +} +int main() { + sleep(1000); +} + +``` + +Next, compile and run it: + +``` +$ gcc -o test test.c +$ ./test + +``` + +Finally, let’s attach to the `test` program with gdb: + +``` +$ sudo gdb -p $(pgrep -f test) +(gdb) p foo() +$1 = 3 +(gdb) quit + +``` + +So I ran `p foo()` and it ran the function! That’s fun. + +### Why is this useful? + +a few possible uses for this: + +* it lets you treat gdb a little bit like a C REPL, which is fun and I imagine could be useful for development + +* utility functions to display / navigate complex data structures quickly while debugging in gdb (thanks [@invalidop][1]) + +* [set an arbitrary process’s namespace while it’s running][2] (featuring a not-so-surprising appearance from my colleague [nelhage][3]!) + +* probably more that I don’t know about + +### How it works + +I got a variety of useful answers on Twitter when I asked how calling functions from gdb works! A lot of them were like “well you get the address of the function from the symbol table” but that is not the whole story!! + +One person pointed me to this nice 2 part series on how gdb works that they’d written: [Debugging with the natives, part 1][9] and [Debugging with the natives, part 2][10]. Part 1 explains approximately how calling functions works (or could work – figuring out what gdb **actually** does isn’t trivial, but I’ll try my best!). + +The steps outlined there are: + +1. Stop the process + +2. Create a new stack frame (far away from the actual stack) + +3. Save all the registers + +4. Set the registers to the arguments you want to call your function with + +5. Set the stack pointer to the new stack frame + +6. Put a trap instruction somewhere in memory + +7. Set the return address to that trap instruction + +8. Set the instruction pointer register to the address of the function you want to call + +9. Start the process again! + +I’m not going to go through how gdb does all of these (I don’t know!) but here are a few things I’ve learned about the various pieces this evening. + +**Create a stack frame** + +If you’re going to run a C function, most likely it needs a stack to store variables on! You definitely don’t want it to clobber your current stack. Concretely – before gdb calls your function (by setting the instruction pointer to it and letting it go), it needs to set the **stack pointer** to… something. + +There was some speculation on Twitter about how this works: + +> i think it constructs a new stack frame for the call right on top of the stack where you’re sitting! + +and: + +> Are you certain it does that? It could allocate a pseudo stack, then temporarily change sp value to that location. You could try, put a breakpoint there and look at the sp register address, see if it’s contiguous to your current program register? + +I did an experiment where (inside gdb) I ran:` + +``` +(gdb) p $rsp +$7 = (void *) 0x7ffea3d0bca8 +(gdb) break foo +Breakpoint 1 at 0x40052a +(gdb) p foo() +Breakpoint 1, 0x000000000040052a in foo () +(gdb) p $rsp +$8 = (void *) 0x7ffea3d0bc00 + +``` + +This seems in line with the “gdb constructs a new stack frame for the call right on top of the stack where you’re sitting” theory, since the stack pointer (`$rsp`) goes from being `...bca8` to `..bc00` – stack pointers grow downward, so a `bc00`stack pointer is **after** a `bca8` pointer. Interesting! + +So it seems like gdb just creates the new stack frames right where you are. That’s a bit surprising to me! + +**change the instruction pointer** + +Let’s see whether gdb changes the instruction pointer! + +``` +(gdb) p $rip +$1 = (void (*)()) 0x7fae7d29a2f0 <__nanosleep_nocancel+7> +(gdb) b foo +Breakpoint 1 at 0x40052a +(gdb) p foo() +Breakpoint 1, 0x000000000040052a in foo () +(gdb) p $rip +$3 = (void (*)()) 0x40052a + +``` + +It does! The instruction pointer changes from `0x7fae7d29a2f0` to `0x40052a` (the address of the `foo` function). + +I stared at the strace output and I still don’t understand **how** it changes, but that’s okay. + +**aside: how breakpoints are set!!** + +Above I wrote `break foo`. I straced gdb while running all of this and understood almost nothing but I found ONE THING that makes sense to me!! + +Here are some of the system calls that gdb uses to set a breakpoint. It’s really simple! It replaces one instruction with `cc` (which [https://defuse.ca/online-x86-assembler.htm][11] tells me means `int3` which means `send SIGTRAP`), and then once the program is interrupted, it puts the instruction back the way it was. + +I was putting a breakpoint on a function `foo` with the address `0x400528`. + +This `PTRACE_POKEDATA` is how gdb changes the code of running programs. + +``` +// change the 0x400528 instructions +25622 ptrace(PTRACE_PEEKTEXT, 25618, 0x400528, [0x5d00000003b8e589]) = 0 +25622 ptrace(PTRACE_POKEDATA, 25618, 0x400528, 0x5d00000003cce589) = 0 +// start the program running +25622 ptrace(PTRACE_CONT, 25618, 0x1, SIG_0) = 0 +// get a signal when it hits the breakpoint +25622 ptrace(PTRACE_GETSIGINFO, 25618, NULL, {si_signo=SIGTRAP, si_code=SI_KERNEL, si_value={int=-1447215360, ptr=0x7ffda9bd3f00}}) = 0 +// change the 0x400528 instructions back to what they were before +25622 ptrace(PTRACE_PEEKTEXT, 25618, 0x400528, [0x5d00000003cce589]) = 0 +25622 ptrace(PTRACE_POKEDATA, 25618, 0x400528, 0x5d00000003b8e589) = 0 + +``` + +**put a trap instruction somewhere** + +When gdb runs a function, it **also** puts trap instructions in a bunch of places! Here’s one of them (per strace). It’s basically replacing one instruction with `cc` (`int3`). + +``` +5908 ptrace(PTRACE_PEEKTEXT, 5810, 0x7f6fa7c0b260, [0x48f389fd89485355]) = 0 +5908 ptrace(PTRACE_PEEKTEXT, 5810, 0x7f6fa7c0b260, [0x48f389fd89485355]) = 0 +5908 ptrace(PTRACE_POKEDATA, 5810, 0x7f6fa7c0b260, 0x48f389fd894853cc) = 0 + +``` + +What’s `0x7f6fa7c0b260`? Well, I looked in the process’s memory maps, and it turns it’s somewhere in `/lib/x86_64-linux-gnu/libc-2.23.so`. That’s weird! Why is gdb putting trap instructions in libc? + +Well, let’s see what function that’s in. It turns out it’s `__libc_siglongjmp`. The other functions gdb is putting traps in are `__longjmp`, `____longjmp_chk`, `dl_main`, and `_dl_close_worker`. + +Why? I don’t know! Maybe for some reason when our function `foo()` returns, it’s calling `longjmp`, and that is how gdb gets control back? I’m not sure. + +### how gdb calls functions is complicated! + +I’m going to stop there (it’s 1am!), but now I know a little more! + +It seems like the answer to “how does gdb call a function?” is definitely not that simple. I found it interesting to try to figure a little bit of it out and hopefully you have too! + +I still have a lot of unanswered questions about how exactly gdb does all of these things, but that’s okay. I don’t really need to know the details of how this works and I’m happy to have a slightly improved understanding. + +-------------------------------------------------------------------------------- + +via: https://jvns.ca/blog/2018/01/04/how-does-gdb-call-functions/ + +作者：[Julia Evans ][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://jvns.ca/ +[1]:https://twitter.com/invalidop/status/949161146526781440 +[2]:https://github.com/baloo/setns/blob/master/setns.c +[3]:https://github.com/nelhage +[4]:https://jvns.ca/blog/2016/08/10/how-does-gdb-work/ +[5]:https://jvns.ca/blog/2014/02/10/three-steps-to-learning-gdb/ +[6]:https://twitter.com/b0rk/status/948060808243765248 +[7]:https://github.com/eklitzke/ptrace-call-userspace/blob/master/call_fprintf.c +[8]:https://cole14.github.io/rust-elf +[9]:https://www.cl.cam.ac.uk/~srk31/blog/2016/02/25/#native-debugging-part-1 +[10]:https://www.cl.cam.ac.uk/~srk31/blog/2017/01/30/#native-debugging-part-2 +[11]:https://defuse.ca/online-x86-assembler.htm diff --git a/sources/tech/20180104 How to Change Your Linux Console Fonts.md b/sources/tech/20180104 How to Change Your Linux Console Fonts.md deleted file mode 100644 index 302f8459b4..0000000000 --- a/sources/tech/20180104 How to Change Your Linux Console Fonts.md +++ /dev/null @@ -1,88 +0,0 @@ -translating by lujun9972 -How to Change Your Linux Console Fonts -====== -![](https://www.linux.com/sites/lcom/files/styles/rendered_file/public/font-size_0.png?itok=d97vmyYa) - -I try to be a peaceful soul, but some things make that difficult, like tiny console fonts. Mark my words, friends, someday your eyes will be decrepit and you won't be able to read those tiny fonts you coded into everything, and then you'll be sorry, and I will laugh. - -Fortunately, Linux fans, you can change your console fonts. As always, the ever-changing Linux landscape makes this less than straightforward, and font management on Linux is non-existent, so we'll muddle along as best we can. In this article, I'll show what I've found to be the easiest approach. - -### What is the Linux Console? - -Let us first clarify what we're talking about. When I say Linux console, I mean TTY1-6, the virtual terminals that you access from your graphical desktop with Ctrl+Alt+F1 through F6. To get back to your graphical environment, press Alt+F7. (This is no longer universal, however, and your Linux distribution may have it mapped differently. You may have more or fewer TTYs, and your graphical session may not be at F7. For example, Fedora puts the default graphical session at F2, and an extra one at F1.) I think it is amazingly cool that we can have both X and console sessions running at the same time. - -The Linux console is part of the kernel, and does not run in an X session. This is the same console you use on headless servers that have no graphical environments. I call the terminals in a graphical session X terminals, and terminal emulators is my catch-all name for both console and X terminals. - -But that's not all. The Linux console has come a long way from the early ANSI days, and thanks to the Linux framebuffer, it has Unicode and limited graphics support. There are also a number of console multimedia applications that we will talk about in a future article. - -### Console Screenshots - -The easy way to get console screenshots is from inside a virtual machine. Then you can use your favorite graphical screen capture program from the host system. You may also make screen captures from your console with [fbcat][1] or [fbgrab][2]. `fbcat` creates a portable pixmap format (PPM) image; this is a highly portable uncompressed image format that should be readable on any operating system, and of course you can convert it to whatever format you want. `fbgrab` is a wrapper script to `fbcat` that creates a PNG file. There are multiple versions of `fbgrab` written by different people floating around. Both have limited options and make only a full-screen capture. - -`fbcat` needs root permissions, and must redirect to a file. Do not specify a file extension, but only the filename: -``` -$ sudo fbcat > Pictures/myfile - -``` - -After cropping in GIMP, I get Figure 1. - -It would be nice to have a little padding on the left margin, so if any of you excellent readers know how to do this, please tell us in the comments. - -`fbgrab` has a few more options that you can read about in `man fbgrab`, such as capturing a different console, and time delay. This example makes a screen grab just like `fbcat`, except you don't have to explicitly redirect: -``` -$ sudo fbgrab Pictures/myOtherfile - -``` - -### Finding Fonts - -As far as I know, there is no way to list your installed kernel fonts other than looking in the directories they are stored in: `/usr/share/consolefonts/` (Debian/etc.), `/lib/kbd/consolefonts/` (Fedora), `/usr/share/kbd/consolefonts` (openSUSE)...you get the idea. - -### Changing Fonts - -Readable fonts are not a new concept. Embrace the old! Readability matters. And so does configurability, which sometimes gets lost in the rush to the new-shiny. - -On Debian/Ubuntu/etc. systems you can run `sudo dpkg-reconfigure console-setup` to set your console font, then run the `setupcon` command in your console to activate the changes. `setupcon` is part of the `console-setup` package. If your Linux distribution doesn't include it, there might be a package for you at [openSUSE][3]. - -You can also edit `/etc/default/console-setup` directly. This example sets the Terminus Bold font at 32 points, which is my favorite, and restricts the width to 80 columns. -``` -ACTIVE_CONSOLES="/dev/tty[1-6]" -CHARMAP="UTF-8" -CODESET="guess" -FONTFACE="TerminusBold" -FONTSIZE="16x32" -SCREEN_WIDTH="80" - -``` - -The FONTFACE and FONTSIZE values come from the font's filename, `TerminusBold32x16.psf.gz`. Yes, you have to know to reverse the order for FONTSIZE. Computers are so much fun. Run `setupcon` to apply the new configuration. You can see the whole character set for your active font with `showconsolefont`. Refer to `man console-setup` for complete options. - -### Systemd - -Systemd is different from `console-setup`, and you don't need to install anything, except maybe some extra font packages. All you do is edit `/etc/vconsole.conf` and then reboot. On my Fedora and openSUSE systems I had to install some extra Terminus packages to get the larger sizes as the installed fonts only went up to 16 points, and I wanted 32. This is the contents of `/etc/vconsole.conf` on both systems: -``` -KEYMAP="us" -FONT="ter-v32b" - -``` - -Come back next week to learn some more cool console hacks, and some multimedia console applications. - -Learn more about Linux through the free ["Introduction to Linux" ][4]course from The Linux Foundation and edX. - --------------------------------------------------------------------------------- - -via: https://www.linux.com/learn/intro-to-linux/2018/1/how-change-your-linux-console-fonts - -作者：[Carla Schroder][a] -译者：[lujun9972](https://github.com/lujun9972) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://www.linux.com/users/cschroder -[1]:http://jwilk.net/software/fbcat -[2]:https://github.com/jwilk/fbcat/blob/master/fbgrab -[3]:https://software.opensuse.org/package/console-setup -[4]:https://training.linuxfoundation.org/linux-courses/system-administration-training/introduction-to-linux diff --git a/sources/tech/20180109 Profiler adventures resolving symbol addresses is hard.md b/sources/tech/20180109 Profiler adventures resolving symbol addresses is hard.md new file mode 100644 index 0000000000..971f575f5f --- /dev/null +++ b/sources/tech/20180109 Profiler adventures resolving symbol addresses is hard.md @@ -0,0 +1,163 @@ +Profiler adventures: resolving symbol addresses is hard! +============================================================ + +The other day I posted [How does gdb call functions?][1]. In that post I said: + +> Using the symbol table to figure out the address of the function you want to call is pretty straightforward + +Unsurprisingly, it turns out that figuring out the address in memory corresponding to a given symbol is actually not really that straightforward. This is actually something I’ve been doing in my profiler, and I think it’s interesting, so I thought I’d write about it! + +Basically the problem I’ve been trying to solve is – I have a symbol (like `ruby_api_version`), and I want to figure out which address that symbol is mapped to in my target process’s memory (so that I can get the data in it, like the Ruby process’s Ruby version). So far I’ve run into (and fixed!) 3 issues when trying to do this: + +1. When binaries are loaded into memory, they’re loaded at a random address (so I can’t just read the symbol table) + +2. The symbol I want isn’t necessary in the “main” binary (`/proc/PID/exe`, sometimes it’s in some other dynamically linked library) + +3. I need to look at the ELF program header to adjust which address I look at for the symbol + +I’ll start with some background, and then explain these 3 things! (I actually don’t know what gdb does) + +### what’s a symbol? + +Most binaries have functions and variables in them. For instance, Perl has a global variable called `PL_bincompat_options` and a function called `Perl_sv_catpv_mg`. + +Sometimes binaries need to look up functions from another binary (for example, if the binary is a dynamically linked library, you need to look up its functions by name). Also sometimes you’re debugging your code and you want to know what function an address corresponds to. + +Symbols are how you look up functions / variables in a binary. They’re in a section called the “symbol table”. The symbol table is basically an index for your binary! Sometimes they’re missing (“stripped”). There are a lot of binary formats, but this post is just about the usual binary format on Linux: ELF. + +### how do you get the symbol table of a binary? + +A thing that I learned today (or at least learned and then forgot) is that there are 2 possible sections symbols can live in: `.symtab` and `.dynsym`. `.dynsym` is the “dynamic symbol table”. According to [this page][2], the dynsym is a smaller version of the symtab that only contains global symbols. + +There are at least 3 ways to read the symbol table of a binary on Linux: you can use nm, objdump, or readelf. + +* **read the .symtab**: `nm $FILE`, `objdump --syms $FILE`, `readelf -a $FILE` + +* **read the .dynsym**: `nm -D $FILE`, `objdump --dynamic-syms $FILE`, `readelf -a $FILE` + +`readelf -a` is the same in both cases because `readelf -a` just shows you everything in an ELF file. It’s my favorite because I don’t need to guess where the information I want is, I can just print out everything and then use grep. + +Here’s an example of some of the symbols in `/usr/bin/perl`. You can see that each symbol has a **name**, a **value**, and a **type**. The value is basically the offset of the code/data corresponding to that symbol in the binary. (except some symbols have value 0\. I think that has something to do with dynamic linking but I don’t understand it so we’re not going to get into it) + +``` +$ readelf -a /usr/bin/perl +... + Num: Value Size Type Ndx Name + 523: 00000000004d6590 49 FUNC 14 Perl_sv_catpv_mg + 524: 0000000000543410 7 FUNC 14 Perl_sv_copypv + 525: 00000000005a43e0 202 OBJECT 16 PL_bincompat_options + 526: 00000000004e6d20 2427 FUNC 14 Perl_pp_ucfirst + 527: 000000000044a8c0 1561 FUNC 14 Perl_Gv_AMupdate +... + +``` + +### the question we want to answer: what address is a symbol mapped to? + +That’s enough background! + +Now – suppose I’m a debugger, and I want to know what address the `ruby_api_version` symbol is mapped to. Let’s use readelf to look at the relevant Ruby binary! + +``` +readelf -a ~/.rbenv/versions/2.1.6/bin/ruby | grep ruby_api_version + 365: 00000000001f9180 12 OBJECT GLOBAL DEFAULT 15 ruby_api_version + +``` + +Neat! The offset of `ruby_api_version` is `0x1f9180`. We’re done, right? Of course not! :) + +### Problem 1: ASLR (Address space layout randomization) + +Here’s the first issue: when Linux loads a binary into memory (like `~/.rbenv/versions/2.1.6/bin/ruby`), it doesn’t just load it at the `0` address. Instead, it usually adds a random offset. Wikipedia’s article on ASLR explains why: + +> Address space layout randomization (ASLR) is a memory-protection process for operating systems (OSes) that guards against buffer-overflow attacks by randomizing the location where system executables are loaded into memory. + +We can see this happening in practice: I started `/home/bork/.rbenv/versions/2.1.6/bin/ruby` 3 times and every time the process gets mapped to a different place in memory. (`0x56121c86f000`, `0x55f440b43000`, `0x56163334a000`) + +Here we’re meeting our good friend `/proc/$PID/maps` – this file contains a list of memory maps for a process. The memory maps tell us every address range in the process’s virtual memory (it turns out virtual memory isn’t contiguous! Instead process get a bunch of possibly-disjoint memory maps!). This file is so useful! You can find the address of the stack, the heap, every dynamically loaded library, anonymous memory maps, and probably more. + +``` +$ cat /proc/(pgrep -f 2.1.6)/maps | grep 'bin/ruby' +56121c86f000-56121caf0000 r-xp 00000000 00:32 323508 /home/bork/.rbenv/versions/2.1.6/bin/ruby +56121ccf0000-56121ccf5000 r--p 00281000 00:32 323508 /home/bork/.rbenv/versions/2.1.6/bin/ruby +56121ccf5000-56121ccf7000 rw-p 00286000 00:32 323508 /home/bork/.rbenv/versions/2.1.6/bin/ruby +$ cat /proc/(pgrep -f 2.1.6)/maps | grep 'bin/ruby' +55f440b43000-55f440dc4000 r-xp 00000000 00:32 323508 /home/bork/.rbenv/versions/2.1.6/bin/ruby +55f440fc4000-55f440fc9000 r--p 00281000 00:32 323508 /home/bork/.rbenv/versions/2.1.6/bin/ruby +55f440fc9000-55f440fcb000 rw-p 00286000 00:32 323508 /home/bork/.rbenv/versions/2.1.6/bin/ruby +$ cat /proc/(pgrep -f 2.1.6)/maps | grep 'bin/ruby' +56163334a000-5616335cb000 r-xp 00000000 00:32 323508 /home/bork/.rbenv/versions/2.1.6/bin/ruby +5616337cb000-5616337d0000 r--p 00281000 00:32 323508 /home/bork/.rbenv/versions/2.1.6/bin/ruby +5616337d0000-5616337d2000 rw-p 00286000 00:32 323508 /home/bork/.rbenv/versions/2.1.6/bin/ruby + +``` + +Okay, so in the last example we see that our binary is mapped at `0x56163334a000`. If we combine this with the knowledge that `ruby_api_version` is at `0x1f9180`, then that means that we just need to look that the address `0x1f9180 + 0x56163334a000` to find our variable, right? + +Yes! In this case, that works. But in other cases it won’t! So that brings us to problem 2. + +### Problem 2: dynamically loaded libraries + +Next up, I tried running system Ruby: `/usr/bin/ruby`. This binary has basically no symbols at all! Disaster! In particular it does not have a `ruby_api_version`symbol. + +But when I tried to print the `ruby_api_version` variable with gdb, it worked!!! Where was gdb finding my symbol? I found the answer with the help of our good friend: `/proc/PID/maps` + +It turns out that `/usr/bin/ruby` dynamically loads a library called `libruby-2.3`. You can see it in the memory maps here: + +``` +$ cat /proc/(pgrep -f /usr/bin/ruby)/maps | grep libruby +7f2c5d789000-7f2c5d9f1000 r-xp 00000000 00:14 /usr/lib/x86_64-linux-gnu/libruby-2.3.so.2.3.0 +7f2c5d9f1000-7f2c5dbf0000 ---p 00268000 00:14 /usr/lib/x86_64-linux-gnu/libruby-2.3.so.2.3.0 +7f2c5dbf0000-7f2c5dbf6000 r--p 00267000 00:14 /usr/lib/x86_64-linux-gnu/libruby-2.3.so.2.3.0 +7f2c5dbf6000-7f2c5dbf7000 rw-p 0026d000 00:14 /usr/lib/x86_64-linux-gnu/libruby-2.3.so.2.3.0 + +``` + +And if we read it with `readelf`, we find the address of that symbol! + +``` +readelf -a /usr/lib/x86_64-linux-gnu/libruby-2.3.so.2.3.0 | grep ruby_api_version + 374: 00000000001c72f0 12 OBJECT GLOBAL DEFAULT 13 ruby_api_version + +``` + +So in this case the address of the symbol we want is `0x7f2c5d789000` (the start of the libruby-2.3 memory map) plus `0x1c72f0`. Nice! But we’re still not done. There is (at least) one more mystery! + +### Problem 3: the `vaddr` offset in the ELF program header + +This one I just figured out today so it’s the one I have the shakiest understanding of. Here’s what happened. + +I was running system ruby on Ubuntu 14.04: Ruby 1.9.3\. And my usual code (find the libruby map, get its address, get the symbol offset, add them up) wasn’t working!!! I was confused. + +But I’d asked Julian if he knew of any weird stuff I need to worry about a while back and he said “well, you should read the code for `dlsym`, you’re trying to do basically the same thing”. So I decided to, instead of randomly guessing, go read the code for `dlsym`. + +The man page for `dlsym` says “dlsym, dlvsym - obtain address of a symbol in a shared object or executable”. Perfect!! + +[Here’s the dlsym code from musl I read][3]. (musl is like glibc, but, different. Maybe easier to read? I don’t understand it that well.) + +The dlsym code says (on line 1468) `return def.dso->base + def.sym->st_value;` That sounds like what I’m doing!! But what’s `dso->base`? It looks like `base = map - addr_min;`, and `addr_min = ph->p_vaddr;`. (there’s also some stuff that makes sure `addr_min` is aligned with the page size which I should maybe pay attention to.) + +So the code I want is something like `map_base - ph->p_vaddr + sym->st_value`. + +I looked up this `vaddr` thing in the ELF program header, subtracted it from my calculation, and voilà! It worked!!! + +### there are probably more problems! + +I imagine I will discover even more ways that I am calculating the symbol address wrong. It’s interesting that such a seemingly simple thing (“what’s the address of this symbol?”) is so complicated! + +It would be nice to be able to just call `dlsym` and have it do all the right calculations for me, but I think I can’t because the symbol is in a different process. Maybe I’m wrong about that though! I would like to be wrong about that. If you know an easier way to do all this I would very much like to know! + +-------------------------------------------------------------------------------- + +via: https://jvns.ca/blog/2018/01/09/resolving-symbol-addresses/ + +作者：[Julia Evans ][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://jvns.ca +[1]:https://jvns.ca/blog/2018/01/04/how-does-gdb-call-functions/ +[2]:https://blogs.oracle.com/ali/inside-elf-symbol-tables +[3]:https://github.com/esmil/musl/blob/194f9cf93da8ae62491b7386edf481ea8565ae4e/src/ldso/dynlink.c#L1451 diff --git a/sources/tech/20180111 BASH drivers, start your engines.md b/sources/tech/20180111 BASH drivers, start your engines.md index 7126bea3e0..e5f8631e39 100644 --- a/sources/tech/20180111 BASH drivers, start your engines.md +++ b/sources/tech/20180111 BASH drivers, start your engines.md @@ -1,4 +1,4 @@ -BASH drivers, start your engines +Translating by Torival BASH drivers, start your engines ====== ![](http://www.thelinuxrain.com/content/01-articles/201-bash-drivers-start-your-engines/headimage.jpg) diff --git a/sources/tech/20180111 How to Install Snipe-IT Asset Management Software on Debian 9.md b/sources/tech/20180111 How to Install Snipe-IT Asset Management Software on Debian 9.md new file mode 100644 index 0000000000..80412f03f3 --- /dev/null +++ b/sources/tech/20180111 How to Install Snipe-IT Asset Management Software on Debian 9.md @@ -0,0 +1,374 @@ +How to Install Snipe-IT Asset Management Software on Debian 9 +====== + +Snipe-IT is a free and open source IT assets management web application that can be used for tracking licenses, accessories, consumables, and components. It is written in PHP language and uses MySQL to store its data. It is a cross-platform application that works on all the major operating system like, Linux, Windows and Mac OS X. It easily integrates with Active Directory, LDAP and supports two-factor authentication with Google Authenticator. + +In this tutorial, we will learn how to install Snipe-IT on Debian 9 server. + +### Requirements + + * A server running Debian 9. + * A non-root user with sudo privileges. + + + +### Getting Started + +Before installing any packages, it is recommended to update the system package with the latest version. You can do this by running the following command: + +``` +sudo apt-get update -y +sudo apt-get upgrade -y +``` + +Next, restart the system to apply all the updates. Then install other required packages with the following command: + +``` +sudo apt-get install git curl unzip wget -y +``` + +Once all the packages are installed, you can proceed to the next step. + +### Install LAMP Server + +Snipe-IT runs on Apache web server, so you will need to install LAMP (Apache, MariaDB, PHP) to your system. + +First, install Apache, PHP and other PHP libraries with the following command: + +``` +sudo apt-get install apache2 libapache2-mod-php php php-pdo php-mbstring php-tokenizer php-curl php-mysql php-ldap php-zip php-fileinfo php-gd php-dom php-mcrypt php-bcmath -y +``` + +Once all the packages are installed, start Apache service and enable it to start on boot with the following command: + +``` +sudo systemctl start apache2 +sudo systemctl enable apache2 +``` + +### Install and Configure MariaDB + +Snipe-IT uses MariaDB to store its data. So you will need to install MariaDB to your system. By default, the latest version of the MariaDB is not available in the Debian 9 repository. So you will need to install MariaDB repository to your system. + +First, add the APT key with the following command: + +``` +sudo apt-get install software-properties-common -y +sudo apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0xcbcb082a1bb943db +``` + +Next, add the MariaDB repository using the following command: + +``` +sudo add-apt-repository 'deb [arch=amd64,i386,ppc64el] http://nyc2.mirrors.digitalocean.com/mariadb/repo/10.1/debian stretch main' +``` + +Next, update the repository with the following command: + +``` +sudo apt-get update -y +``` + +Once the repository is updated, you can install MariaDB with the following command: + +``` +sudo apt-get install mariadb-server mariadb-client -y +``` + +Next, start the MariaDB service and enable it to start on boot time with the following command: + +``` +sudo systemctl start mysql +sudo systemctl start mysql +``` + +You can check the status of MariaDB server with the following command: + +``` +sudo systemctl status mysql +``` + +If everything is fine, you should see the following output: +``` +? mariadb.service - MariaDB database server + Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled) + Active: active (running) since Mon 2017-12-25 08:41:25 EST; 29min ago + Process: 618 ExecStartPost=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS) + Process: 615 ExecStartPost=/etc/mysql/debian-start (code=exited, status=0/SUCCESS) + Process: 436 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= || VAR=`/usr/bin/galera_recovery`; [ $? -eq 0 ] && systemc + Process: 429 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS) + Process: 418 ExecStartPre=/usr/bin/install -m 755 -o mysql -g root -d /var/run/mysqld (code=exited, status=0/SUCCESS) + Main PID: 574 (mysqld) + Status: "Taking your SQL requests now..." + Tasks: 27 (limit: 4915) + CGroup: /system.slice/mariadb.service + ??574 /usr/sbin/mysqld + +Dec 25 08:41:07 debian systemd[1]: Starting MariaDB database server... +Dec 25 08:41:14 debian mysqld[574]: 2017-12-25 8:41:14 140488893776448 [Note] /usr/sbin/mysqld (mysqld 10.1.26-MariaDB-0+deb9u1) starting as p +Dec 25 08:41:25 debian systemd[1]: Started MariaDB database server. + +``` + +Next, secure your MariaDB by running the following script: + +``` +sudo mysql_secure_installation +``` + +Answer all the questions as shown below: +``` +Set root password? [Y/n] n +Remove anonymous users? [Y/n] y +Disallow root login remotely? [Y/n] y +Remove test database and access to it? [Y/n] y +Reload privilege tables now? [Y/n] y + +``` + +Once MariaDB is secured, log in to MariaDB shell with the following command: + +``` +mysql -u root -p +``` + +Enter your root password when prompt, then create a database for Snipe-IT with the following command: + +``` +MariaDB [(none)]> create database snipeitdb character set utf8; +``` + +Next, create a user for Snipe-IT and grant all privileges to the Snipe-IT with the following command: + +``` +MariaDB [(none)]> GRANT ALL PRIVILEGES ON snipeitdb.* TO 'snipeit'@'localhost' IDENTIFIED BY 'password'; +``` + +Next, flush the privileges with the following command: + +``` +MariaDB [(none)]> flush privileges; +``` + +Finally, exit from the MariaDB console using the following command: + +``` +MariaDB [(none)]> quit +``` + +### Install Snipe-IT + +You can download the latest version of the Snipe-IT from Git repository with the following command: + +``` +git clone https://github.com/snipe/snipe-it snipe-it +``` + +Next, move the downloaded directory to the apache root directory with the following command: + +``` +sudo mv snipe-it /var/www/ +``` + +Next, you will need to install Composer to your system. You can install it with the following command: + +``` +curl -sS https://getcomposer.org/installer | php +sudo mv composer.phar /usr/local/bin/composer +``` + +Next, change the directory to snipe-it and Install PHP dependencies using Composer with the following command: + +``` +cd /var/www/snipe-it +sudo composer install --no-dev --prefer-source +``` +Next, generate the "APP_Key" with the following command: + +``` +sudo php artisan key:generate +``` + +You should see the following output: +``` +************************************** +* Application In Production! * +************************************** + + Do you really wish to run this command? (yes/no) [no]: + > yes + +Application key [base64:uWh7O0/TOV10asWpzHc0DH1dOxJHprnZw2kSOnbBXww=] set successfully. + +``` + +Next, you will need to populate MySQL with Snipe-IT's default database schema. You can do this by running the following command: + +``` +sudo php artisan migrate +``` + +Type yes, when prompted to confirm that you want to perform the migration: +``` +************************************** +* Application In Production! * +************************************** + + Do you really wish to run this command? (yes/no) [no]: + > yes + +Migration table created successfully. + +``` + +Next, copy sample .env file and make some changes in it: + +``` +sudo cp .env.example .env +sudo nano .env +``` + +Change the following lines: +``` +APP_URL=http://example.com +APP_TIMEZONE=US/Eastern +APP_LOCALE=en + +# -------------------------------------------- +# REQUIRED: DATABASE SETTINGS +# -------------------------------------------- +DB_CONNECTION=mysql +DB_HOST=localhost +DB_DATABASE=snipeitdb +DB_USERNAME=snipeit +DB_PASSWORD=password +DB_PREFIX=null +DB_DUMP_PATH='/usr/bin' + +``` + +Save and close the file when you are finished. + +Next, provide the appropriate ownership and file permissions with the following command: + +``` +sudo chown -R www-data:www-data storage public/uploads +sudo chmod -R 755 storage public/uploads +``` + +### Configure Apache For Snipe-IT + +Next, you will need to create an apache virtual host directive for Snipe-IT. You can do this by creating `snipeit.conf` file inside `/etc/apache2/sites-available` directory: + +``` +sudo nano /etc/apache2/sites-available/snipeit.conf +``` + +Add the following lines: +``` + +ServerAdmin webmaster@example.com + + Require all granted + AllowOverride All + + DocumentRoot /var/www/snipe-it/public + ServerName example.com + ErrorLog /var/log/apache2/snipeIT.error.log + CustomLog /var/log/apache2/access.log combined + + +``` + +Save and close the file when you are finished. Then, enable virtual host with the following command: + +``` +sudo a2ensite snipeit.conf +``` + +Next, enable PHP mcrypt, mbstring module and Apache rewrite module with the following command: + +``` +sudo phpenmod mcrypt +sudo phpenmod mbstring +sudo a2enmod rewrite +``` + +Finally, restart apache web server to apply all the changes: + +``` +sudo systemctl restart apache2 +``` + +### Configure Firewall + +By default, Snipe-IT runs on port 80, so you will need to allow port 80 through the firewall. By default, UFW firewall is not installed in Debian 9, so you will need to install it first. You can install it by just running the following command: + +``` +sudo apt-get install ufw -y +``` + +Once UFW is installed, enable it to start on boot time with the following command: + +``` +sudo ufw enable +``` + +Next, allow port 80 using the following command: + +``` +sudo ufw allow 80 +``` + +Next, reload the UFW firewall rule with the following command: + +``` +sudo ufw reload +``` + +### Access Snipe-IT + +Everything is now installed and configured, it's time to access Snipe-IT web interface. + +Open your web browser and type the URL, you will be redirected to the following page: + +[![Snipe-IT Checks the system][2]][3] + +The above page will do a system check to make sure your configuration looks correct. Next, click on the **Create Database Table** button you should see the following page: + +[![Create database table][4]][5] + +Here, click on the **Create User** page, you should see the following page: + +[![Create user][6]][7] + +Here, provide your Site name, Domain name, Admin username, and password, then click on the **Save User** button, you should see the Snipe-IT default dashboard as below: + +[![Snipe-IT Dashboard][8]][9] + +### Conclusion + +In the above tutorial, we have learned to install Snipe-IT on Debian 9 server. We have also learned to configure Snipe-IT through web interface.I hope you have now enough knowledge to deploy Snipe-IT in your production environment. For more information you can refer Snipe-IT [Documentation Page][10]. + + +-------------------------------------------------------------------------------- + +via: https://www.howtoforge.com/tutorial/how-to-install-snipe-it-on-debian-9/ + +作者：[Hitesh Jethva][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.howtoforge.com +[1]:/cdn-cgi/l/email-protection +[2]:https://www.howtoforge.com/images/how_to_install_snipe_it_on_debian_9/Screenshot-of-snipeit-page1.png +[3]:https://www.howtoforge.com/images/how_to_install_snipe_it_on_debian_9/big/Screenshot-of-snipeit-page1.png +[4]:https://www.howtoforge.com/images/how_to_install_snipe_it_on_debian_9/Screenshot-of-snipeit-page2.png +[5]:https://www.howtoforge.com/images/how_to_install_snipe_it_on_debian_9/big/Screenshot-of-snipeit-page2.png +[6]:https://www.howtoforge.com/images/how_to_install_snipe_it_on_debian_9/Screenshot-of-snipeit-page3.png +[7]:https://www.howtoforge.com/images/how_to_install_snipe_it_on_debian_9/big/Screenshot-of-snipeit-page3.png +[8]:https://www.howtoforge.com/images/how_to_install_snipe_it_on_debian_9/Screenshot-of-snipeit-page4.png +[9]:https://www.howtoforge.com/images/how_to_install_snipe_it_on_debian_9/big/Screenshot-of-snipeit-page4.png +[10]:https://snipe-it.readme.io/docs diff --git a/sources/tech/20180111 Multimedia Apps for the Linux Console.md b/sources/tech/20180111 Multimedia Apps for the Linux Console.md new file mode 100644 index 0000000000..6cdd3ef857 --- /dev/null +++ b/sources/tech/20180111 Multimedia Apps for the Linux Console.md @@ -0,0 +1,112 @@ +Translating by Yinr + +Multimedia Apps for the Linux Console +====== + +![](https://www.linux.com/sites/lcom/files/styles/rendered_file/public/multimedia.jpg?itok=v-XrnKRB) +The Linux console supports multimedia, so you can enjoy music, movies, photos, and even read PDF files. + +When last we met, we learned that the Linux console supports multimedia. Yes, really! You can enjoy music, movies, photos, and even read PDF files without being in an X session with MPlayer, fbi, and fbgs. And, as a bonus, you can enjoy a Matrix-style screensaver for the console, CMatrix. + +You will probably have make some tweaks to your system to make this work. The examples used here are for Ubuntu Linux 16.04. + +### MPlayer + +You're probably familiar with the amazing and versatile MPlayer, which supports almost every video and audio format, and runs on nearly everything, including Linux, Android, Windows, Mac, Kindle, OS/2, and AmigaOS. Using MPLayer in your console will probably require some tweaking, depending on your Linux distribution. To start, try playing a video: +``` +$ mplayer [video name] + +``` + +If it works, then hurrah, and you can invest your time in learning useful MPlayer options, such as controlling the size of the video screen. However, some Linux distributions are managing the framebuffer differently than in the olden days, and you may have to adjust some settings to make it work. This is how to make it work on recent Ubuntu releases. + +First, add yourself to the video group. + +Second, verify that `/etc/modprobe.d/blacklist-framebuffer.conf` has this line: `#blacklist vesafb`. It should already be commented out, and if it isn't then comment it. All the other module lines should be un-commented, which prevents them from loading. Side note: if you want to dig more deeply into managing your framebuffer, the module for your video card may give better performance. + +Add these two modules to the end of `/etc/initramfs-tools/modules`, `vesafb` and `fbcon`, then rebuild the initramfs image: +``` +$ sudo nano /etc/initramfs-tools/modules + # List of modules that you want to include in your initramfs. + # They will be loaded at boot time in the order below. + fbcon + vesafb + +$ sudo update-initramfs -u + +``` + +[fbcon][1] is the Linux framebuffer console. It runs on top of the framebuffer and adds graphical features. It requires a framebuffer device, which is supplied by the `vesafb` module. + +Now you must edit your GRUB2 configuration. In `/etc/default/grub` you should see a line like this: +``` +GRUB_CMDLINE_LINUX_DEFAULT="quiet splash" + +``` + +It may have some other options, but it should be there. Add `vga=789`: +``` +GRUB_CMDLINE_LINUX_DEFAULT="quiet splash vga=789" + +``` + +Reboot and enter your console (Ctrl+Alt+F1), and try playing a video. This command selects the `fbdev2` video device; I haven't learned yet how to know which one to use, but I had to use it to play the video. The default screen size is 320x240, so I scaled it to 960: +``` +$ mplayer -vo fbdev2 -vf scale -zoom -xy 960 AlienSong_mp4.mov +``` + +And behold Figure 1. It's grainy because I have a low-fi copy of this video, not because MPlayer is making it grainy. + +MPLayer plays CDs, DVDs, network streams, and has a giant batch of playback options, which I shall leave as your homework to explore. + +### fbi Image Viewer + +`fbi`, the framebuffer image viewer, comes in the [fbida][2] package on most Linuxes. It has native support for the common image file formats, and uses `convert` (from Image Magick), if it is installed, for other formats. Its simplest use is to view a single image file: +``` +$ fbi filename + +``` + +Use the arrow keys to scroll a large image, + and - to zoom, and r and l to rotate 90 degress right and left. Press the Escape key to close the image. You can play a slideshow by giving `fbi` a list of files: +``` +$ fbi --list file-list.txt + +``` + +`fbi` supports autozoom. With `-a` `fbi` controls the zoom factor. `--autoup` and `--autodown` tell `fbi` to only zoom up or down. Control the blend time between images with `--blend [time]`, in milliseconds. Press the k and j keys to jump behind and ahead in your file list. + +`fbi` has commands for creating file lists from images you have viewed, and for exporting your commands to a file, and a host of other cool options. Check out `man fbi` for complete options. + +### CMatrix Console Screensaver + +The Matrix screensaver is still my favorite (Figure 2), second only to the bouncing cow. [CMatrix][3] runs on the console. Simply type `cmatrix` to start it, and Ctrl+C stops it. Run `cmatrix -s` to launch it in screensaver mode, which exits on any keypress. `-C` changes the color. Your choices are green, red, blue, yellow, white, magenta, cyan, and black. + +CMatrix supports asynchronous key presses, which means you can change options while it's running. + +`-B` is all bold text, and `-B` is partially bold. + +### fbgs PDF Viewer + +It seems that the addiction to PDF documents is pandemic and incurable, though PDFs are better than they used to be, with live hyperlinks, copy-paste, and good text search. The `fbgs` console PDF viewer is part of the `fbida` package. Options include page size, resolution, page selections, and most `fbi` options, with the exceptions listed in `man fbgs`. The main option I use is page size; you get `-l`, `xl`, and `xxl` to choose from: +``` +$ fbgs -xl annoyingpdf.pdf + +``` + +Learn more about Linux through the free ["Introduction to Linux" ][4]course from The Linux Foundation and edX. + +-------------------------------------------------------------------------------- + +via: https://www.linux.com/learn/intro-to-linux/2018/1/multimedia-apps-linux-console + +作者：[Carla Schroder][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.linux.com/users/cschroder +[1]:https://www.mjmwired.net/kernel/Documentation/fb/fbcon.txt +[2]:https://www.kraxel.org/blog/linux/fbida/ +[3]:http://www.asty.org/cmatrix/ +[4]:https://training.linuxfoundation.org/linux-courses/system-administration-training/introduction-to-linux diff --git a/sources/tech/20180111 The Fold Command Tutorial With Examples For Beginners.md b/sources/tech/20180111 The Fold Command Tutorial With Examples For Beginners.md new file mode 100644 index 0000000000..0d0623bb7a --- /dev/null +++ b/sources/tech/20180111 The Fold Command Tutorial With Examples For Beginners.md @@ -0,0 +1,114 @@ +translating by Flowsnow + +The Fold Command Tutorial With Examples For Beginners +====== + +![](https://www.ostechnix.com/wp-content/uploads/2018/01/Fold-Command-2-720x340.png) + +Have you ever found yourself in a situation where you want to fold or break the output of a command to fit within a specific width? I have find myself in this situation few times while running VMs, especially the servers with no GUI. Just in case, if you ever wanted to limit the output of a command to a particular width, look nowhere! Here is where **fold** command comes in handy! The fold command wraps each line in an input file to fit a specified width and prints it to the standard output. + +In this brief tutorial, we are going to see the usage of fold command with practical examples. + +### The Fold Command Tutorial With Examples + +Fold command is the part of GNU coreutils package, so let us not bother about installation. + +The typical syntax of fold command: +``` +fold [OPTION]... [FILE]... +``` + +Allow me to show you some examples, so you can get a better idea about fold command. I have a file named **linux.txt** with some random lines. + +[![][1]][2] + +To wrap each line in the above file to default width, run: +``` +fold linux.txt +``` + +**80** columns per line is the default width. Here is the output of above command: + +[![][1]][3] + +As you can see in the above output, fold command has limited the output to a width of 80 characters. + +Of course, we can specify your preferred width, for example 50, like below: +``` +fold -w50 linux.txt +``` + +Sample output would be: + +[![][1]][4] + +Instead of just displaying output, we can also write the output to a new file as shown below: +``` +fold -w50 linux.txt > linux1.txt +``` + +The above command will wrap the lines of **linux.txt** to a width of 50 characters, and writes the output to new file named **linux1.txt**. + +Let us check the contents of the new file: +``` +cat linux1.txt +``` + +[![][1]][5] + +Did you closely notice the output of the previous commands? Some words are broken between lines. To overcome this issue, we can use -s flag to break the lines at spaces. + +The following command wraps each line in a given file to width "50" and breaks the line at spaces: +``` +fold -w50 -s linux.txt +``` + +Sample output: + +[![][1]][6] + +See? Now, the output is much clear. This command puts each space separated word in a new line and words with length > 50 are wrapped. + +In all above examples, we limited the output width by columns. However, we can enforce the width of the output to the number of bytes specified using **-b** option. The following command breaks the output at 20 bytes. +``` +fold -b20 linux.txt +``` + +Sample output: + +[![][1]][7] + +**Also read:** + ++ [The Uniq Command Tutorial With Examples For Beginners][8] + +For more details, refer the man pages. +``` +man fold +``` + +And, that's for now folks. You know now how to use fold command to limit the output of a command to fit in a specific width. I hope this was useful. We will be posting more useful guides everyday. Stay tuned! + +Cheers! + + + +-------------------------------------------------------------------------------- + +via: https://www.ostechnix.com/fold-command-tutorial-examples-beginners/ + +作者：[SK][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.ostechnix.com/author/sk/ +[1]:data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7 +[2]:http://www.ostechnix.com/wp-content/uploads/2018/01/fold-command-1.png +[3]:http://www.ostechnix.com/wp-content/uploads/2018/01/fold-command-2.png +[4]:http://www.ostechnix.com/wp-content/uploads/2018/01/fold-command-3-1.png +[5]:http://www.ostechnix.com/wp-content/uploads/2018/01/fold-command-4.png +[6]:http://www.ostechnix.com/wp-content/uploads/2018/01/fold-command-5-1.png +[7]:http://www.ostechnix.com/wp-content/uploads/2018/01/fold-command-6-1.png +[8]:https://www.ostechnix.com/uniq-command-tutorial-examples-beginners/ diff --git a/sources/tech/20180111 The open organization and inner sourcing movements can share knowledge.md b/sources/tech/20180111 The open organization and inner sourcing movements can share knowledge.md new file mode 100644 index 0000000000..272c1b03ae --- /dev/null +++ b/sources/tech/20180111 The open organization and inner sourcing movements can share knowledge.md @@ -0,0 +1,121 @@ +The open organization and inner sourcing movements can share knowledge +====== + +![](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/gov_collaborative_risk.png?itok=we8DKHuL) +Image by : opensource.com + +Red Hat is a company with roughly 11,000 employees. The IT department consists of roughly 500 members. Though it makes up just a fraction of the entire organization, the IT department is still sufficiently staffed to have many application service, infrastructure, and operational teams within it. Our purpose is "to enable Red Hatters in all functions to be effective, productive, innovative, and collaborative, so that they feel they can make a difference,"--and, more specifically, to do that by providing technologies and related services in a fashion that is as open as possible. + +Being open like this takes time, attention, and effort. While we always strive to be as open as possible, it can be difficult. For a variety of reasons, we don't always succeed. + +In this story, I'll explain a time when, in the rush to innovate, the Red Hat IT organization lost sight of its open ideals. But I'll also explore how returning to those ideals--and using the collaborative tactics of "inner source"--helped us to recover and greatly improve the way we deliver services. + +### About inner source + +Before I explain how inner source helped our team, let me offer some background on the concept. + +Inner source is the adoption of open source development practices between teams within an organization to promote better and faster delivery without requiring project resources be exposed to the world or openly licensed. It allows an organization to receive many of the benefits of open source development methods within its own walls. + +In this way, inner source aligns well with open organization strategies and principles; it provides a path for open, collaborative development. While the open organization defines its principles of openness broadly as transparency, inclusivity, adaptability, collaboration, and community--and covers how to use these open principles for communication, decision making, and many other topics--inner source is about the adoption of specific and tactical practices, processes, and patterns from open source communities to improve delivery. + +For instance, [the Open Organization Maturity Model][1] suggests that in order to be transparent, teams should, at minimum, share all project resources with the project team (though it suggests that it's generally better to share these resources with the entire organization). The common pattern in both inner source and open source development is to host all resources in a publicly available version control system, for source control management, which achieves the open organization goal of high transparency. + +Inner source aligns well with open organization strategies and principles. + +Another example of value alignment appears in the way open source communities accept contributions. In open source communities, source code is transparently available. Community contributions in the form of patches or merge requests are commonly accepted practices (even expected ones). This provides one example of how to meet the open organization's goal of promoting inclusivity and collaboration. + +### The challenge + +Early in 2014, Red Hat IT began its first steps toward making Amazon Web Services (AWS) a standard hosting offering for business critical systems. While teams within Red Hat IT had built several systems and services in AWS by this time, these were bespoke creations, and we desired to make deploying services to IT standards in AWS both simple and standardized. + +In order to make AWS cloud hosting meet our operational standards (while being scalable), the Cloud Enablement team within Red Hat IT decided that all infrastructure in AWS would be configured through code, rather than manually, and that everyone would use a standard set of tools. The Cloud Enablement team designed and built these standard tools; a separate group, the Platform Operations team, was responsible for provisioning and hosting systems and services in AWS using the tools. + +The Cloud Enablement team built a toolset, obtusely named "Template Util," based on AWS Cloud Formations configurations wrapped in a management layer to enforce certain configuration requirements and make stamping out multiple copies of services across environments easier. While the Template Util toolset technically met all our initial requirements, and we eventually provisioned the infrastructure for more than a dozen services with it, engineers in every team working with the tool found using it to be painful. Michael Johnson, one engineer using the tool, said "It made doing something relatively straightforward really complicated." + +Among the issues Template Util exhibited were: + + * Underlying cloud formations technologies implied constraints on application stack management at odds with how we managed our application systems. + * The tooling was needlessly complex and brittle in places, using multiple layered templating technologies and languages making syntax issues hard to debug. + * The code for the tool--and some of the data users needed to manipulate the tool--were kept in a repository that was difficult for most users to access. + * There was no standard process to contributing or accepting changes. + * The documentation was poor. + + + +As more engineers attempted to use the Template Util toolset, they found even more issues and limitations with the tools. Unhappiness continued to grow. To make matters worse, the Cloud Enablement team then shifted priorities to other deliverables without relinquishing ownership of the tool, so bug fixes and improvements to the tools were further delayed. + +The real, core issues here were our inability to build an inclusive community to collaboratively build shared tooling that met everyone's needs. Fear of losing "ownership," fear of changing requirements, and fear of seeing hard work abandoned all contributed to chronic conflict, which in turn led to poorer outcomes. + +### Crisis point + +By September 2015, more than a year after launching our first major service in AWS with the Template Util tool, we hit a crisis point. + +Many engineers refused to use the tools. That forced all of the related service provisioning work on a small set of engineers, further fracturing the community and disrupting service delivery roadmaps as these engineers struggled to deal with unexpected work. We called an emergency meeting and invited all the teams involved to find a solution. + +During the emergency meeting, we found that people generally thought we needed immediate change and should start the tooling effort over, but even the decision to start over wasn't unanimous. Many solutions emerged--sometimes multiple solutions from within a single team--all of which would require significant work to implement. While we couldn't reach a consensus on which solution to use during this meeting, we did reach an agreement to give proponents of different technologies two weeks to work together, across teams, to build their case with a prototype, which the community could then review. + +While we didn't reach a final and definitive decision, this agreement was the first point where we started to return to the open source ideals that guide our mission. By inviting all involved parties, we were able to be transparent and inclusive, and we could begin rebuilding our internal community. By making clear that we wanted to improve things and were open to new options, we showed our commitment to adaptability and meritocracy. Most importantly, the plan for building prototypes gave people a clear, return path to collaboration. + +When the community reviewed the prototypes, it determined that the clear leader was an Ansible-based toolset that would eventually become known, internally, as Ansicloud. (At the time, no one involved with this work had any idea that Red Hat would acquire Ansible the following month. It should also be noted that other teams within Red Hat have found tools based on Cloud Formation extremely useful, even when our specific Template Util tool did not find success.) + +This prototyping and testing phase didn't fix things overnight, though. While we had consensus on the general direction we needed to head, we still needed to improve the new prototype to the point at which engineers could use it reliably for production services. + +So over the next several months, a handful of engineers worked to further build and extend the Ansicloud toolset. We built three new production services. While we were sharing code, that sharing activity occurred at a low level of maturity. Some engineers had trouble getting access due to older processes. Other engineers headed in slightly different directions, with each engineer having to rediscover some of the core design issues themselves. + +### Returning to openness + +This led to a turning point: Building on top of the previous agreement, we focused on developing a unified vision and providing easier access. To do this, we: + + 1. created a list of specific goals for the project (both "must-haves" and "nice-to-haves"), + 2. created an open issue log for the project to avoid solving the same problem repeatedly, + 3. opened our code base so anyone in Red Hat could read or clone it, and + 4. made it easy for engineers to get trusted committer access + + + +Our agreement to collaborate, our finally unified vision, and our improved tool development methods spurred the growth of our community. Ansicloud adoption spread throughout the involved organizations, but this led to a new problem: The tool started changing more quickly than users could adapt to it, and improvements that different groups submitted were beginning to affect other groups in unanticipated ways. + +These issues resulted in our recent turn to inner source practices. While every open source project operates differently, we focused on adopting some best practices that seemed common to many of them. In particular: + + * We identified the business owner of the project and the core-contributor group of developers who would govern the development of the tools and decide what contributions to accept. While we want to keep things open, we can't have people working against each other or breaking each other's functionality. + * We developed a project README clarifying the purpose of the tool and specifying how to use it. We also created a CONTRIBUTING document explaining how to contribute, what sort of contributions would be useful, and what sort of tests a contribution would need to pass to be accepted. + * We began building continuous integration and testing services for the Ansicloud tool itself. This helped us ensure we could quickly and efficiently validate contributions technically, before the project accepted and merged them. + + + +With these basic agreements, documents, and tools available, we were back onto the path of open collaboration and successful inner sourcing. + +### Why it matters + +Why does inner source matter? + +From a developer community point of view, shifting from a traditional siloed development model to the inner source model has produced significant, quantifiable improvements: + + * Contributions to our tooling have grown 72% per week (by number of commits). + * The percentage of contributions from non-core committers has grown from 27% to 78%; the users of the toolset are driving its development. + * The contributor list has grown by 15%, primarily from new users of the tool set, rather than core committers, increasing our internal community. + + + +And the tools we've delivered through this project have allowed us to see dramatic improvements in our business outcomes. Using the Ansicloud tools, 54 new multi-environment application service deployments were created in 385 days (compared to 20 services in 1,013 days with the Template Util tools). We've gone from one new service deployment in a 50-day period to one every week--a seven-fold increase in the velocity of our delivery. + +What really matters here is that the improvements we saw were not aberrations. Inner source provides common, easily understood patterns that organizations can adopt to effectively promote collaboration (not to mention other open organization principles). By mirroring open source production practices, inner source can also mirror the benefits of open source code, which have been seen time and time again: higher quality code, faster development, and more engaged communities. + +This article is part of the [Open Organization Workbook project][2]. + +### about the author +Tom Benninger - Tom Benninger is a Solutions Architect, Systems Engineer, and continual tinkerer at Red Hat, Inc. Having worked with startups, small businesses, and larger enterprises, he has experience within a broad set of IT disciplines. His current area of focus is improving Application Lifecycle Management in the enterprise. He has a particular interest in how open source, inner source, and collaboration can help support modern application development practices and the adoption of DevOps, CI/CD, Agile,... + +-------------------------------------------------------------------------------- + +via: https://opensource.com/open-organization/18/1/open-orgs-and-inner-source-it + +作者：[Tom Benninger][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://opensource.com/users/tomben +[1]:https://opensource.com/open-organization/resources/open-org-maturity-model +[2]:https://opensource.com/open-organization/17/8/workbook-project-announcement diff --git a/sources/tech/20180112 Linux yes Command Tutorial for Beginners (with Examples).md b/sources/tech/20180112 Linux yes Command Tutorial for Beginners (with Examples).md new file mode 100644 index 0000000000..a4b4ff385c --- /dev/null +++ b/sources/tech/20180112 Linux yes Command Tutorial for Beginners (with Examples).md @@ -0,0 +1,96 @@ +Linux yes Command Tutorial for Beginners (with Examples) +====== + +Most of the Linux commands you encounter do not depend on other operations for users to unlock their full potential, but there exists a small subset of command line tool which you can say are useless when used independently, but become a must-have or must-know when used with other command line operations. One such tool is **yes** , and in this tutorial, we will discuss this command with some easy to understand examples. + +But before we do that, it's worth mentioning that all examples provided in this tutorial have been tested on Ubuntu 16.04 LTS. + +### Linux yes command + +The yes command in Linux outputs a string repeatedly until killed. Following is the syntax of the command: + +``` +yes [STRING]... +yes OPTION +``` + +And here's what the man page says about this tool: +``` +Repeatedly output a line with all specified STRING(s), or 'y'. +``` + +The following Q&A-type examples should give you a better idea about the usage of yes. + +### Q1. How yes command works? + +As the man page says, the yes command produces continuous output - 'y' by default, or any other string if specified by user. Here's a screenshot that shows the yes command in action: + +[![How yes command works][1]][2] + +I could only capture the last part of the output as the output frequency was so fast, but the screenshot should give you a good idea about what kind of output the tool produces. + +You can also provide a custom string for the yes command to use in output. For example: + +``` +yes HTF +``` + +[![Repeat word with yes command][3]][4] + +### Q2. Where yes command helps the user? + +That's a valid question. Reason being, from what yes does, it's difficult to imagine the usefulness of the tool. But you'll be surprised to know that yes can not only save your time, but also automate some mundane tasks. + +For example, consider the following scenario: + +[![Where yes command helps the user][5]][6] + +You can see that user has to type 'y' for each query. It's in situation like these where yes can help. For the above scenario specifically, you can use yes in the following way: + +``` +yes | rm -ri test +``` + +[![yes command in action][7]][8] + +So the command made sure user doesn't have to write 'y' each time when rm asked for it. Of course, one would argue that we could have simply removed the '-i' option from the rm command. That's right, I took this example as it's simple enough to make people understand the situations in which yes can be helpful. + +Another - and probably more relevant - scenario would be when you're using the fsck command, and don't want to enter 'y' each time system asks your permission before fixing errors. + +### Q3. Is there any use of yes when it's used alone? + +Yes, there's at-least one use: to tell how well a computer system handles high amount of loads. Reason being, the tool utilizes 100% processor for systems that have a single processor. In case you want to apply this test on a system with multiple processors, you need to run a yes process for each processor. + +### Q4. What command line options yes offers? + +The tool only offers generic command line options: --help and --version. As the names suggests. the former displays help information related to the command, while the latter one outputs version related information. + +[![What command line options yes offers][9]][10] + +### Conclusion + +So now you'd agree that there could be several scenarios where the yes command would be of help. There are no command line options unique to yes, so effectively, there's no learning curve associated with the tool. Just in case you need, here's the command's [man page][11]. + + +-------------------------------------------------------------------------------- + +via: https://www.howtoforge.com/linux-yes-command/ + +作者：[Himanshu Arora][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.howtoforge.com +[1]:https://www.howtoforge.com/images/command-tutorial/yes-def-output.png +[2]:https://www.howtoforge.com/images/command-tutorial/big/yes-def-output.png +[3]:https://www.howtoforge.com/images/command-tutorial/yes-custom-string.png +[4]:https://www.howtoforge.com/images/command-tutorial/big/yes-custom-string.png +[5]:https://www.howtoforge.com/images/command-tutorial/rm-ri-output.png +[6]:https://www.howtoforge.com/images/command-tutorial/big/rm-ri-output.png +[7]:https://www.howtoforge.com/images/command-tutorial/yes-in-action.png +[8]:https://www.howtoforge.com/images/command-tutorial/big/yes-in-action.png +[9]:https://www.howtoforge.com/images/command-tutorial/yes-help-version1.png +[10]:https://www.howtoforge.com/images/command-tutorial/big/yes-help-version1.png +[11]:https://linux.die.net/man/1/yes diff --git a/sources/tech/20180112 Top 5 Firefox extensions to install now.md b/sources/tech/20180112 Top 5 Firefox extensions to install now.md new file mode 100644 index 0000000000..6e11993b45 --- /dev/null +++ b/sources/tech/20180112 Top 5 Firefox extensions to install now.md @@ -0,0 +1,83 @@ +Top 5 Firefox extensions to install now +====== + +The right extensions can greatly enhance your browser's capabilities, but it's important to choose carefully. Here are five that are worth a look. + +![](https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/firefox_blue_lead.jpg?itok=gYaubJUv) + +The web browser has become a critical component of the computing experience for many users. Modern browsers have evolved into powerful and extensible platforms. As part of this, _extensions_ can add or modify their functionality. Extensions for Firefox are built using the WebExtensions API, a cross-browser development system. + +Which extensions should you install? Generally, that decision comes down to how you use your browser, your views on privacy, how much you trust extension developers, and other personal preferences. + +First, I'd like to point out that browser extensions often require the ability to read and/or change everything on the web pages you visit. You should consider the ramifications of this _very_ carefully. If an extension has modify access to all the web pages you visit, it could act as a key logger, intercept credit card information, track you online, insert advertisements, and perform a variety of other nefarious activities. + +That doesn't mean every extension will surreptitiously do these things, but you should carefully consider the installation source, the permissions involved, your risk profile, and other factors before you install any extension. Keep in mind you can use profiles to manage how an extension impacts your attack surface--for example, using a dedicated profile with no extensions to perform tasks such as online banking. + +With that in mind, here are five Firefox extensions that you may want to consider. + +### uBlock Origin + +![ublock origin ad blocker screenshot][2] + + +Ublock Origin blocks ads and malware while enabling users to define their own content filters. + +[uBlock Origin][3] is a fast, low-memory, wide-spectrum blocker that not only blocks ads but also lets you enforce your own content filtering. The default behavior of uBlock Origin is to block ads, trackers, and malware sites using multiple predefined filter lists. From there it allows you to arbitrarily add lists and rules, or even lock down to a default-deny mode. In addition to being powerful, this extension has proven to be efficient and performant. + +### Privacy Badger + +![privacy badger ad blocker][5] + + +Privacy Badger uses algorithms to seamlessly block ads and trackers that violate the principles of user consent. + +As its name indicates, [Privacy Badger][6] is a privacy-focused extension that blocks ads and third-party trackers. From the EFF: "Privacy Badger was born out of our desire to be able to recommend a single extension that would automatically analyze and block any tracker or ad that violated the principle of user consent; which could function well without any settings, knowledge, or configuration by the user; which is produced by an organization that is unambiguously working for its users rather than for advertisers; and which uses algorithmic methods to decide what is and isn't tracking." + +Why is Privacy Badger on this list when it may seem so similar to uBlock Origin? One reason is that it fundamentally works differently than uBlock Origin. Another is that a practice of defense in depth is a sound policy to follow. + +### LastPass + +![lastpass password manager screenshot][8] + + +LastPass is a user-friendly password manager plugin that supports two-factor authorization. + +This is likely a controversial addition for many. Whether you should use a password manager at all--and if you do, whether you should choose one that has a browser plugin--is a hotly debated topic, and the answer very much depends on your personal risk profile. I'd assert that most casual computer users should use one, because it's much better than the most common alternative: using the same weak password everywhere. + +[LastPass][9] is user-friendly, supports two-factor authentication, and is reasonably secure. The company has had a few security incidents in the past, but it responded well and is well-funded moving forward. Keep in mind that using a password manager isn't an all-or-nothing proposition. Many users choose to use it for the majority of their passwords, while keeping a few complicated, well-constructed passwords for important sites such as banking and multi-factor authentication in their head. + +### Xmarks Sync + +[Xmarks Sync][10] is a convenient extension that will sync your bookmarks, open tabs, profiles, and browser history across instances. If you have multiple machines, want to sync across desktop and mobile, or use multiple different browsers on the same machine, take a look at Xmarks Sync. (Note that this extension was recently acquired by LastPass.) + +### Awesome Screenshot Plus + +[Awesome Screenshot Plus][11] allows you to easily capture all or part of any web page, as well as add annotations and comments, blur sensitive information, and more. You can also share images using an optional online service. I've found this tool great for capturing parts of sites for debugging issues, discussing design, and sharing information. It's one of those tools you'll find yourself using more than you might have expected. + +I've found all five of these extensions useful, and I recommend them to others. That said, there are many browser extensions out there. I'm curious about which ones other Opensource.com community members currently use and recommend. Let me know in the comments. + +![Awesome Screenshot Plus screenshot][13] + + +Awesome Screenshot Plus allows you to easily capture all or part of any web page. + +-------------------------------------------------------------------------------- + +via: https://opensource.com/article/18/1/top-5-firefox-extensions + +作者：[Jeremy Garcia][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://opensource.com/users/jeremy-garcia +[2]:https://opensource.com/sites/default/files/ublock.png (ublock origin ad blocker screenshot) +[3]:https://addons.mozilla.org/en-US/firefox/addon/ublock-origin/ +[5]:https://opensource.com/sites/default/files/images/life-uploads/privacy_badger_1.0.1.png (privacy badger ad blocker screenshot) +[6]:https://www.eff.org/privacybadger +[8]:https://opensource.com/sites/default/files/images/life-uploads/lastpass4.jpg (lastpass password manager screenshot) +[9]:https://addons.mozilla.org/en-US/firefox/addon/lastpass-password-manager/ +[10]:https://addons.mozilla.org/en-US/firefox/addon/xmarks-sync/ +[11]:https://addons.mozilla.org/en-US/firefox/addon/screenshot-capture-annotate/ +[13]:https://opensource.com/sites/default/files/screenshot_from_2018-01-04_17-11-32.png (Awesome Screenshot Plus screenshot) diff --git a/sources/tech/20180114 Playing Quake 4 on Linux in 2018.md b/sources/tech/20180114 Playing Quake 4 on Linux in 2018.md new file mode 100644 index 0000000000..26dd305a4a --- /dev/null +++ b/sources/tech/20180114 Playing Quake 4 on Linux in 2018.md @@ -0,0 +1,80 @@ +Playing Quake 4 on Linux in 2018 +====== +A few months back [I wrote an article][1] outlining the various options Linux users now have for playing Doom 3, as well as stating which of the three contenders I felt to be the best option in 2017. Having already gone to the trouble of getting the original Doom 3 binary working on my modern Arch Linux system, it made me wonder just how much effort it would take to get the closed source Quake 4 port up and running again as well. + +### Getting it running + +[![][2]][3] [![][4]][5] + +Quake 4 was ported to Linux by Timothee Besset in 2005, although the binaries themselves were later taken down along with the rest of the id Software FTP server by ZeniMax. The original [Linux FAQ page][6] is still online though, and mirrors hosting the Linux installer still exist, such as [this one][7] ran by the fan website [Quaddicted][8]. Once downloaded this will give you a graphical installer which will install the game binary without any of the game assets. + +These will need to be taken from either the game discs of a retail Windows version as I did, or taken from an already installed Windows version of the game such as from [Steam][9]. Follow the steps in the Linux FAQ to the letter for best results. Please note that the [GOG.com][10] release of Quake 4 is unique in not supplying a valid CD key, something which is still required for the Linux port to launch. There are [ways to get around this][11], but we only condone these methods for legitimate purchasers. + +Like with Doom 3 I had to remove the libgcc_s.so.1, libSDL-1.2.id.so.0, and libstdc++.so.6 libraries that the game came with in the install directory in order to get it to run. I also ran into the same sound issue I had with Doom 3, meaning I had to modify the Quake4Config.cfg file located in the hidden ~/.quake4/q4base directory in the same fashion as before. However, this time I ran into a whole host of other issues that made me have to modify the configuration file as well. + +First off the language the game wanted to use would always default to Spanish, meaning I had to manually tell the game to use English instead. I also ran into a known issue on all platforms wherein the game would not properly recognize the available VRAM on modern graphics cards, and as such would force the game to use lower image quality settings. Quake 4 will also not render see-through surfaces unless anti-aliasing is enabled, although going beyond 8x caused the game not to load for me. + +Appending the following to the end of the Quake4Config.cfg file resolved all of my issues: + +``` +seta image_downSize "0" +seta image_downSizeBump "0" +seta image_downSizeSpecular "0" +seta image_filter "GL_LINEAR_MIPMAP_LINEAR" +seta image_ignoreHighQuality "0" +seta image_roundDown "0" +seta image_useCompression "0" +seta image_useNormalCompression "0" +seta image_anisotropy "16" +seta image_lodbias "0" +seta r_renderer "best" +seta r_multiSamples "8" +seta sys_lang "english" +seta s_alsa_pcm "hw:0,0" +seta com_allowConsole "1" +``` + +Please note that this will also set the game to use 8x anti-aliasing and restore the drop down console to how it worked in all of the previous Quake games. Similar to the Linux port of Doom 3 the Linux version of Quake 4 also does not support Creative EAX ADVANCED HD audio technology. Unlike Doom 3 though Quake 4 does seem to also feature an alternate method for surround sound, and widescreen support was thankfully patched into the game soon after its release. + +### Playing the game + +[![][12]][13] [![][14]][15] + +Over the years Quake 4 has gained something of a reputation as the black sheep of the Quake family, with many people complaining that the game's vehicle sections, squad mechanics, and general aesthetic made it feel too close to contemporary military shooters of the time. In the game's heart of hearts though it really does feel like a concerted sequel to Quake II, with some of developer Raven Software's own Star Trek: Voyager - Elite Force title thrown in for good measure. + +To me at least Quake 4 does stand as being one of the "Last of the Romans" in terms of being a first person shooter that embraced classic design ideals at a time when similar titles were not getting the support of major publishers. Most of the game still features the player moving between levels featuring fixed enemy placements, a wide variety of available weapons, traditional health packs, and an array of enemies each sporting unique attributes and skills. + +Quake 4 also offers a well made campaign that I found myself going back to on a higher skill level not long after I had already finished my first try at the game. Certain aspects like the vehicle sections do indeed drag the game down a bit, and the multiplayer aspect pails in comparison to its predecessor Quake III Arena, but overall I am quite pleased with what Raven Software was able to accomplish with the Doom 3 engine, especially when so few others tried. + +### Final thoughts + +If anyone ever needed a reason to be reminded of the value of video game source code releases, this is it. Most of the problems I encountered could have been easily sidestepped if Quake 4 source ports were available, but with the likes of John Carmack and Timothee Besset gone from id Software and the current climate at ZeniMax not looking too promising, it is doubtful that any such creations will ever materialize. Doom 3 source ports look to be the end of the road. + +Instead we are stuck using this cranky 32 bit binary with an obstructive CD Key check and a graphics system that freaks out at the sight of any modern video card sporting more than 512 MB of VRAM. The game itself has aged well, with graphics that still look great and dynamic lighting that is better than what is included with many modern titles. It is just a shame that it is now such a pain to get running, not just on Linux, but on any platform. + +-------------------------------------------------------------------------------- + +via: https://www.gamingonlinux.com/articles/playing-quake-4-on-linux-in-2018.11017 + +作者：[Hamish][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.gamingonlinux.com/profiles/6 +[1]:https://www.gamingonlinux.com/articles/playing-doom-3-on-linux-in-2017.10561 +[2]:https://www.gamingonlinux.com/uploads/articles/article_images/thumbs/20458196191515697921gol6.jpg +[3]:https://www.gamingonlinux.com/uploads/articles/article_images/20458196191515697921gol6.jpg +[4]:https://www.gamingonlinux.com/uploads/articles/article_images/thumbs/9405540721515697921gol6.jpg +[5]:https://www.gamingonlinux.com/uploads/articles/article_images/9405540721515697921gol6.jpg +[6]:http://zerowing.idsoftware.com/linux/quake4/Quake4FrontPage/ +[7]:https://www.quaddicted.com/files/idgames2/idstuff/quake4/linux/ +[8]:https://www.quaddicted.com/ +[9]:http://store.steampowered.com/app/2210/Quake_IV/ +[10]:https://www.gog.com/game/quake_4 +[11]:https://www.gog.com/forum/quake_series/quake_4_on_linux_no_cd_key/post31 +[12]:https://www.gamingonlinux.com/uploads/articles/article_images/thumbs/5043571471515951537gol6.jpg +[13]:https://www.gamingonlinux.com/uploads/articles/article_images/5043571471515951537gol6.jpg +[14]:https://www.gamingonlinux.com/uploads/articles/article_images/thumbs/6922853731515697921gol6.jpg +[15]:https://www.gamingonlinux.com/uploads/articles/article_images/6922853731515697921gol6.jpg diff --git a/sources/tech/20180114 What a GNU C Compiler Bug looks like.md b/sources/tech/20180114 What a GNU C Compiler Bug looks like.md new file mode 100644 index 0000000000..3b95d4089b --- /dev/null +++ b/sources/tech/20180114 What a GNU C Compiler Bug looks like.md @@ -0,0 +1,77 @@ +What a GNU C Compiler Bug looks like +====== +Back in December a Linux Mint user sent a [strange bug report][1] to the darktable mailing list. Apparently the GNU C Compiler (GCC) on his system exited with the following error message, breaking the build process: +``` +cc1: error: unrecognized command line option '-Wno-format-truncation' [-Werror] +cc1: all warnings being treated as errors +src/iop/CMakeFiles/colortransfer.dir/build.make:67: recipe for target 'src/iop/CMakeFiles/colortransfer.dir/introspection_colortransfer.c.o' failed make[2]: 0_sync_master.sh 1_add_new_article_manual.sh 1_add_new_article_newspaper.sh 2_start_translating.sh 3_continue_the_work.sh 4_finish.sh 5_pause.sh base.sh env format.test lctt.cfg parse_url_by_manual.sh parse_url_by_newspaper.py parse_url_by_newspaper.sh README.org reformat.sh [src/iop/CMakeFiles/colortransfer.dir/introspection_colortransfer.c.o] Error 1 CMakeFiles/Makefile2:6323: recipe for target 'src/iop/CMakeFiles/colortransfer.dir/all' failed + +make[1]: 0_sync_master.sh 1_add_new_article_manual.sh 1_add_new_article_newspaper.sh 2_start_translating.sh 3_continue_the_work.sh 4_finish.sh 5_pause.sh base.sh env format.test lctt.cfg parse_url_by_manual.sh parse_url_by_newspaper.py parse_url_by_newspaper.sh README.org reformat.sh [src/iop/CMakeFiles/colortransfer.dir/all] Error 2 + +``` + +`-Wno-format-truncation` is a rather new GCC feature which instructs the compiler to issue a warning if it can already deduce at compile time that calls to formatted I/O functions like `snprintf()` or `vsnprintf()` might result in truncated output. + +That's definitely neat, but Linux Mint 18.3 (just like Ubuntu 16.04 LTS) uses GCC 5.4.0, which doesn't support this feature. And darktable relies on a chain of CMake macros to make sure it doesn't use any flags the compiler doesn't know about: +``` +CHECK_COMPILER_FLAG_AND_ENABLE_IT(-Wno-format-truncation) + +``` + +So why did this even happen? I logged into one of my Ubuntu 16.04 installations and tried to reproduce the problem. Which wasn't hard, I just had to check out the git tree in question and build it. Boom, same error. + +### The solution + +It turns out that while `-Wformat-truncation` isn't a valid option for GCC 5.4.0 (it's not documented), this version silently accepts the negation under some circumstances (!): +``` + +sturmflut@hogsmeade:/tmp$ gcc -Wformat-truncation -o test test.c +gcc: error: unrecognized command line option '-Wformat-truncation' +sturmflut@hogsmeade:/tmp$ gcc -Wno-format-truncation -o test test.c +sturmflut@hogsmeade:/tmp$ + +``` + +(test.c just contains an empty main() method). + +Because darktable uses `CHECK_COMPILER_FLAG_AND_ENABLE_IT(-Wno-format-truncation)`, it is fooled into thinking this compiler version actually supports `-Wno-format-truncation` at all times. The simple test case used by the CMake macro doesn't fail, but the compiler later decides to no longer silently accept the invalid command line option for some reason. + +One of the cases which triggered this was when the source file under compilation had already generated some other warnings before. If I forced a serialized build using `make -j1` on a clean darktable checkout on this machine, `./src/iop/colortransfer.c` actually was the first file which caused any +compiler warnings at all, so this is why the process failed exactly there. + +The minimum test case to trigger this behavior in GCC 5.4.0 is a C file with a `main()` function with a parameter which has the wrong type, like this one: +``` + +int main(int argc, int argv) +{ +} + +``` + +Then add `-Wall` to make sure the compiler will treat this as a warning, and it fails: +``` + +sturmflut@hogsmeade:/tmp$ gcc -Wall -Wno-format-truncation -o test test.c +test.c:1:5: warning: second argument of 'main' should be 'char **' [-Wmain] + int main(int argc, int argv) + ^ +cc1: warning: unrecognized command line option '-Wno-format-truncation' + +``` + +If you omit `-Wall`, the compiler will not generate the first warning and also not complain about `-Wno-format-truncation`. + +I've never run into this before, but I guess Ubuntu 16.04 is going to stay with us for a while since it is the current LTS release until May 2018, and even after that it will still be supported until 2021. So this buggy GCC version will most likely also stay alive for quite a while. Which is why the check for this flag has been removed from the + +-------------------------------------------------------------------------------- + +via: http://www.lieberbiber.de/2018/01/14/what-a-gnu-compiler-bug-looks-like/ + +作者：[sturmflut][a] +译者：[译者ID](https://github.com/译者ID) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://www.lieberbiber.de/author/sturmflut/ +[1]:https://www.mail-archive.com/darktable-dev@lists.darktable.org/msg02760.html diff --git a/translated/tech/20090211 Page Cache the Affair Between Memory and Files.md b/translated/tech/20090211 Page Cache the Affair Between Memory and Files.md new file mode 100644 index 0000000000..644cb1c33b --- /dev/null +++ b/translated/tech/20090211 Page Cache the Affair Between Memory and Files.md @@ -0,0 +1,76 @@ +[页面缓存，内存和文件之间的那些事][1] +============================================================ + + +上一篇文章中我们学习了内核怎么为一个用户进程 [管理虚拟内存][2]，而忽略了文件和 I/O。这一篇文章我们将专门去讲这个重要的主题 —— 页面缓存。文件和内存之间的关系常常很不好去理解，而它们对系统性能的影响却是非常大的。 + +在面对文件时，有两个很重要的问题需要操作系统去解决。第一个是相对内存而言，慢的让人发狂的硬盘驱动器，[尤其是磁盘查找][3]。第二个是需要将文件内容一次性地加载到物理内存中，以便程序间共享文件内容。如果你在 Windows 中使用 [进程浏览器][4] 去查看它的进程，你将会看到每个进程中加载了大约 ~15MB 的公共 DLLs。我的 Windows 机器上现在大约运行着 100 个进程，因此，如果不共享的话，仅这些公共的 DLLs 就要使用高达 ~1.5 GB 的物理内存。如果是那样的话，那就太糟糕了。同样的，几乎所有的 Linux 进程都需要 [ld.so][5] 和 libc，加上其它的公共库，它们占用的内存数量也不是一个小数目。 + +幸运的是，所有的这些问题都用一个办法解决了：页面缓存 —— 保存在内存中的页面大小的文件块。为了用图去说明页面缓存，我捏造出一个名为 Render 的 Linux 程序，它打开了文件 scene.dat，并且一次读取 512 字节，并将文件内容存储到一个分配的堆块中。第一次读取的过程如下： + +![Reading and the page cache](http://static.duartes.org/img/blogPosts/readFromPageCache.png) + +读取完 12KB 的文件内容以后，Render 程序的堆和相关的页面帧如下图所示： + +![Non-mapped file read](http://static.duartes.org/img/blogPosts/nonMappedFileRead.png) + +它看起来很简单，其实这一过程做了很多的事情。首先，虽然这个程序使用了普通的读取调用，但是，已经有三个 4KB 的页面帧将文件 scene.dat 的一部分内容保存在了页面缓存中。虽然有时让人觉得很惊奇，但是，普通的文件 I/O 就是这样通过页面缓存来进行的。在 x86 架构的 Linux 中，内核将文件认为是一系列的 4KB 大小的块。如果你从文件中读取单个字节，包含这个字节的整个 4KB 块将被从磁盘中读入到页面缓存中。这是可以理解的，因为磁盘通常是持续吞吐的，并且程序读取的磁盘区域也不仅仅只保存几个字节。页面缓存知道文件中的每个 4KB 块的位置，在上图中用 #0、#1、等等来描述。Windows 也是类似的，使用 256KB 大小的页面缓存。 + +不幸的是，在一个普通的文件读取中，内核必须拷贝页面缓存中的内容到一个用户缓存中，它不仅花费 CPU 时间和影响 [CPU 缓存][6]，在复制数据时也浪费物理内存。如前面的图示，scene.dat 的内存被保存了两次，并且，程序中的每个实例都在另外的时间中去保存了内容。我们虽然解决了从磁盘中读取文件缓慢的问题，但是在其它的方面带来了更痛苦的问题。内存映射文件是解决这种痛苦的一个方法： + +![Mapped file read](http://static.duartes.org/img/blogPosts/mappedFileRead.png) + +当你使用文件映射时，内核直接在页面缓存上映射你的程序的虚拟页面。这样可以显著提升性能：[Windows 系统编程][7] 的报告指出，在相关的普通文件读取上运行时性能有多达 30% 的提升，在 [Unix 环境中的高级编程][8] 的报告中，文件映射在 Linux 和 Solaris 也有类似的效果。取决于你的应用程序类型的不同，通过使用文件映射，可以节约大量的物理内存。 + +对高性能的追求是永衡不变的目标，[测量是很重要的事情][9]，内存映射应该是程序员始终要使用的工具。而 API 提供了非常好用的实现方式，它允许你通过内存中的字节去访问一个文件，而不需要为了这种好处而牺牲代码可读性。在一个类 Unix 的系统中，可以使用 [mmap][11] 查看你的 [地址空间][10]，在 Windows 中，可以使用 [CreateFileMapping][12]，或者在高级编程语言中还有更多的可用封装。当你映射一个文件内容时，它并不是一次性将全部内容都映射到内存中，而是通过 [页面故障][13] 来按需映射的。在 [获取][15] 需要的文件内容的页面帧后，页面故障句柄在页面缓存上 [映射你的虚拟页面][14] 。如果一开始文件内容没有缓存，这还将涉及到磁盘 I/O。 + +假设我们的 Reader 程序是持续存在的实例，现在出现一个突发的状况。在页面缓存中保存着 scene.dat 内容的页面要立刻释放掉吗？这是一个人们经常要考虑的问题，但是，那样做并不是个好主意。你应该想到，我们经常在一个程序中创建一个文件，退出程序，然后，在第二个程序去使用这个文件。页面缓存正好可以处理这种情况。如果考虑更多的情况，内核为什么要清除页面缓存的内容？请记住，磁盘读取的速度要慢于内存 5 个数量级，因此，命中一个页面缓存是一件有非常大收益的事情。因此，只要有足够大的物理内存，缓存就应该始终完整保存。并且，这一原则适用于所有的进程。如果你现在运行 Render，一周后 scene.dat 的内容还在缓存中，那么应该恭喜你！这就是什么内核缓存越来越大，直至达到最大限制的原因。它并不是因为操作系统设计的太“垃圾”而浪费你的内存，其实这是一个非常好的行为，因为，释放物理内存才是一种“浪费”。（译者注：释放物理内存会导致页面缓存被清除，下次运行程序需要的相关数据，需要再次从磁盘上进行读取，会“浪费” CPU 和 I/O 资源）最好的做法是尽可能多的使用缓存。 + +由于页面缓存架构的原因，当程序调用 [write()][16] 时，字节只是被简单地拷贝到页面缓存中，并将这个页面标记为“赃”页面。磁盘 I/O 通常并不会立即发生，因此，你的程序并不会被阻塞在等待磁盘写入上。如果这时候发生了电脑死机，你的写入将不会被标记，因此，对于至关重要的文件，像数据库事务日志，必须要求 [fsync()][17]ed（仍然还需要去担心磁盘控制器的缓存失败问题），另一方面，读取将被你的程序阻塞，走到数据可用为止。内核采取预加载的方式来缓解这个矛盾，它一般提前预读取几个页面并将它加载到页面缓存中，以备你后来的读取。在你计划进行一个顺序或者随机读取时（请查看 [madvise()][18]、[readahead()][19]、[Windows cache hints][20] ），你可以通过提示（hint）帮助内核去调整这个预加载行为。Linux 会对内存映射的文件进行 [预读取][21]，但是，在 Windows 上并不能确保被内存映射的文件也会预读。当然，在 Linux 中它可能会使用 [O_DIRECT][22] 跳过预读取，或者，在 Windows 中使用 [NO_BUFFERING][23] 去跳过预读，一些数据库软件就经常这么做。 + +一个内存映射的文件可以是私有的，也可以是共享的。当然，这只是针对内存中内容的更新而言：在一个私有的内存映射文件上，更新并不会提交到磁盘或者被其它进程可见，然而，共享的内存映射文件，则正好相反，它的任何更新都会提交到磁盘上，并且对其它的进程可见。内核在写机制上使用拷贝，这是通过页面表条目来实现这种私有的映射。在下面的例子中，Render 和另一个被称为 render3d 都私有映射到 scene.dat 上。然后 Render 去写入映射的文件的虚拟内存区域： + +![The Copy-On-Write mechanism](http://static.duartes.org/img/blogPosts/copyOnWrite.png) + +上面展示的只读页面表条目并不意味着映射是只读的，它只是内核的一个用于去共享物理内存的技巧，直到尽可能的最后一刻之前。你可以认为“私有”一词用的有点不太恰当，你只需要记住，这个“私有”仅用于更新的情况。这种设计的重要性在于，要想看到被映射的文件的变化，其它程序只能读取它的虚拟页面。一旦“写时复制”发生，从其它地方是看不到这种变化的。但是，内核并不能保证这种行为，因为它是在 x86 中实现的，从 API 的角度来看，这是有意义的。相比之下，一个共享的映射只是将它简单地映射到页面缓存上。更新会被所有的进程看到并被写入到磁盘上。最终，如果上面的映射是只读的，页面故障将触发一个内存段失败而不是写到一个副本。 + +动态加载库是通过文件映射融入到你的程序的地址空间中的。这没有什么可奇怪的，它通过普通的 APIs 为你提供与私有文件映射相同的效果。下面的示例展示了 Reader 程序映射的文件的两个实例运行的地址空间的一部分，以及物理内存，尝试将我们看到的许多概念综合到一起。 + +![Mapping virtual memory to physical memory](http://static.duartes.org/img/blogPosts/virtualToPhysicalMapping.png) + +这是内存架构系列的第三部分的结论。我希望这个系列文章对你有帮助，对理解操作系统的这些主题提供一个很好的思维模型。 + +-------------------------------------------------------------------------------- + +via:https://manybutfinite.com/post/page-cache-the-affair-between-memory-and-files/ + +作者：[Gustavo Duarte][a] +译者：[qhwdw](https://github.com/qhwdw) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:http://duartes.org/gustavo/blog/about/ +[1]:https://manybutfinite.com/post/page-cache-the-affair-between-memory-and-files/ +[2]:https://manybutfinite.com/post/how-the-kernel-manages-your-memory +[3]:https://manybutfinite.com/post/what-your-computer-does-while-you-wait +[4]:http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx +[5]:http://ld.so +[6]:https://manybutfinite.com/post/intel-cpu-caches +[7]:http://www.amazon.com/Windows-Programming-Addison-Wesley-Microsoft-Technology/dp/0321256190/ +[8]:http://www.amazon.com/Programming-Environment-Addison-Wesley-Professional-Computing/dp/0321525949/ +[9]:https://manybutfinite.com/post/performance-is-a-science +[10]:https://manybutfinite.com/post/anatomy-of-a-program-in-memory +[11]:http://www.kernel.org/doc/man-pages/online/pages/man2/mmap.2.html +[12]:http://msdn.microsoft.com/en-us/library/aa366537(VS.85).aspx +[13]:http://lxr.linux.no/linux+v2.6.28/mm/memory.c#L2678 +[14]:http://lxr.linux.no/linux+v2.6.28/mm/memory.c#L2436 +[15]:http://lxr.linux.no/linux+v2.6.28/mm/filemap.c#L1424 +[16]:http://www.kernel.org/doc/man-pages/online/pages/man2/write.2.html +[17]:http://www.kernel.org/doc/man-pages/online/pages/man2/fsync.2.html +[18]:http://www.kernel.org/doc/man-pages/online/pages/man2/madvise.2.html +[19]:http://www.kernel.org/doc/man-pages/online/pages/man2/readahead.2.html +[20]:http://msdn.microsoft.com/en-us/library/aa363858(VS.85).aspx#caching_behavior +[21]:http://lxr.linux.no/linux+v2.6.28/mm/filemap.c#L1424 +[22]:http://www.kernel.org/doc/man-pages/online/pages/man2/open.2.html +[23]:http://msdn.microsoft.com/en-us/library/cc644950(VS.85).aspx \ No newline at end of file diff --git a/translated/tech/20121211 Python Nmon Analyzer- moving away from excel macros.md b/translated/tech/20121211 Python Nmon Analyzer- moving away from excel macros.md new file mode 100644 index 0000000000..c772ceff73 --- /dev/null +++ b/translated/tech/20121211 Python Nmon Analyzer- moving away from excel macros.md @@ -0,0 +1,100 @@ +Python 版的 Nmon 分析器：让你远离 excel 宏 +====== +[Nigel's monitor][1]，也叫做 "Nmon"，是一个很好的监控，记录和分析 Linux/*nix 系统性能随时间变化的工具。Nmon 最初由 IBM 开发并于 2009 年夏天开源。时至今日 Nmon 已经在所有 linux 平台和架构上都可用了。它提供了大量的实时工具来可视化当前系统统计信息，这些统计信息包括 CPU，RAM，网络和磁盘 I/O。然而，Nmon 最棒的特性是可以随着时间的推移记录系统性能快照。 +比如：`nmon -f -s 1`。 +![nmon CPU and Disk utilization][2] +会创建一个日志文件，该日志文件最开头是一些系统的元数据 T( 章节 AAA - BBBV)，后面是定时抓取的监控系统属性的快照，比如 CPU 和内存的使用情况。这个文件很难直接由电子表格应用来处理，因此诞生了 [Nmon_Analyzer][3] excel 宏。如果你用的是 Windows/Mac 并安装了 Microsoft Office，那么这个工具非常不错。如果没有这个环境那也可以使用 Nmon2rrd 工具，这个工具能将日志文件转换 RRD 输入文件，进而生成图形。这个过程很死板而且有点麻烦。现在出现了一个更灵活的工具，像你们介绍一下 pyNmonAnalyzer，它一个可定制化的解决方案来生成结构化的 CSV 文件和基于 [matplotlib][4] 生成图片的简单 HTML 报告。 + +### 入门介绍： + +系统需求： +从名字中就能看出我们需要有 python。此外 pyNmonAnalyzer 还依赖于 matplotlib 和 numpy。若你使用的是 debian 衍生的系统，则你需要先安装这些包： +``` +$> sudo apt-get install python-numpy python-matplotlib + +``` + +##### 获取 pyNmonAnalyzer： + +你可页克隆 git 仓库： +``` +$> git clone git@github.com:madmaze/pyNmonAnalyzer.git + +``` + +或者 + +直接从这里下载：[pyNmonAnalyzer-0.1.zip][5] + +接下来我们需要一个 Nmon 文件，如果没有的话，可以使用发行版中提供的实例或者自己录制一个样本：`nmon -F test.nmon -s 1 -c 120`，会录制每个 1 秒录制一次，供录制 120 个快照道 test.nmon 文件中 .nmon。 + +让我们来看看基本的帮助信息： +``` +$> ./pyNmonAnalyzer.py -h +usage: pyNmonAnalyzer.py [-h] [-x] [-d] [-o OUTDIR] [-c] [-b] [-r CONFFNAME] + input_file + +nmonParser converts Nmon monitor files into time-sorted +CSV/Spreadsheets for easier analysis, without the use of the +MS Excel Macro. Also included is an option to build an HTML +report with graphs, which is configured through report.config. + +positional arguments: + input_file Input NMON file + +optional arguments: + -h, --help show this help message and exit + -x, --overwrite overwrite existing results (Default: False) + -d, --debug debug? (Default: False) + -o OUTDIR, --output OUTDIR + Output dir for CSV (Default: ./data/) + -c, --csv CSV output? (Default: False) + -b, --buildReport report output? (Default: False) + -r CONFFNAME, --reportConfig CONFFNAME + Report config file, if none exists: we will write the + default config file out (Default: ./report.config) + +``` + +该工具有两个主要的选项 + + 1。将 nmon 文件传唤成一系列独立的 CSV 文件 + 2。使用 matplotlib 生成带图形的 HTML 报告 + + + +下面命令既会生成 CSV 文件，也会生成 HTML 报告： +``` +$> ./pyNmonAnalyzer.py -c -b test.nmon + +``` + +这会常见一个 `。/data` 目录，其中有一个存放 CSV 文件的目录 ("。/data/csv/")，一个存放 PNG 图片的目录 ("。/data/img/") 以及一个 HTML 报告 ("。/data/report.html")。 + +默认情况下，HTML 报告中会用图片展示 CPU，磁盘繁忙度，内存使用情况和网络传输情况。所有这些都定义在一个自解释的配置文件中 ("report.config")。目前这个工具 h 那不是特别的灵活，因为 CPU 和 MEM 除了 on 和 off 外，无法做其他的配置。不过下一步将会改进作图的方法并允许用户灵活地指定针对哪些数据使用哪种作图方法。 + +### 报告的例子： + +[![pyNmonAnalyzer Graph output][6] +**Click to see the full Report**][7] + +目前这些报告还十分的枯燥而且只能打印出基本的几种标记图表，不过它的功能还在不断的完善中。目前在开发的是一个向导来让配置调整变得更容易。如果有任何建议，找到任何 bug 或者有任何功能需求，欢迎与我交流。 + +-------------------------------------------------------------------------------- + +via: https://matthiaslee.com/python-nmon-analyzer-moving-away-from-excel-macros/ + +作者：[Matthias Lee][a] +译者：[lujun9972](https://github.com/lujun9972) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://matthiaslee.com/ +[1]:http://nmon.sourceforge.net/ +[2]:https://matthiaslee.com//content/images/2015/06/nmon_cpudisk.png +[3]:http://www.ibm.com/developerworks/wikis/display/WikiPtype/nmonanalyser +[4]:http://matplotlib.org/ +[5]:https://github.com/madmaze/pyNmonAnalyzer/blob/master/release/pyNmonAnalyzer-0.1.zip?raw=true +[6]:https://matthiaslee.com//content/images/2017/04/teaser-short_0.png (pyNmonAnalyzer Graph output) +[7]:http://matthiaslee.com/pub/pyNmonAnalyzer/data/report.html diff --git a/translated/tech/20161004 What happens when you start a process on Linux.md b/translated/tech/20161004 What happens when you start a process on Linux.md new file mode 100644 index 0000000000..5c97fe7dc4 --- /dev/null +++ b/translated/tech/20161004 What happens when you start a process on Linux.md @@ -0,0 +1,153 @@ +当你在 Linux 上启动一个进程时会发生什么？ +=========================================================== + + +本文是关于 fork 和 exec 是如何在 Unix 上工作的。你或许已经知道，也有人还不知道。几年前当我了解到这些时，我惊叹不已。 + +我们要做的是启动一个进程。我们已经在博客上讨论了很多关于**系统调用**的问题，每当你启动一个进程或者打开一个文件，这都是一个系统调用。所以你可能会认为有这样的系统调用： + +``` +start_process(["ls", "-l", "my_cool_directory"]) + +``` + +这是一个合理的想法，显然这是它在 DOS 或 Windows 中的工作原理。我想说的是，这并不是 Linux 上的工作原理。但是，我查阅了文档，确实有一个 [posix_spawn][2] 的系统调用基本上是这样做的，不过这不在本文的讨论范围内。 + +### fork 和 exec + +Linux 上的 `posix_spawn` 是通过两个系统调用实现的，分别是 `fork` 和 `exec`（实际上是 execve），这些都是人们常常使用的。尽管在 OS X 上，人们使用 `posix_spawn`，而 fork 和 exec 是不提倡的，但我们将讨论的是 Linux。 + +Linux 中的每个进程都存在于“进程树”中。你可以通过运行 `pstree` 命令查看进程树。树的根是 `init`，进程号是 1。每个进程（init 除外）都有一个父进程，一个进程都可以有很多子进程。 + +所以，假设我要启动一个名为 `ls` 的进程来列出一个目录。我是不是只要发起一个进程 `ls` 就好了呢？不是的。 + +我要做的是，创建一个子进程，这个子进程是我本身的一个克隆，然后这个子进程的“大脑”被替代，变成 `ls`。 + +开始是这样的： + +``` +my parent + |- me + +``` + +然后运行 `fork()`，生成一个子进程，是我自己的一份克隆： + +``` +my parent + |- me + |-- clone of me + +``` + +然后我让子进程运行 `exec("ls")`，变成这样： + +``` +my parent + |- me + |-- ls + +``` + +当 ls 命令结束后，我几乎又变回了我自己： + +``` +my parent + |- me + |-- ls (zombie) + +``` + +在这时 ls 其实是一个僵尸进程。这意味着它已经死了，但它还在等我，以防我需要检查它的返回值（使用 `wait` 系统调用）。一旦我获得了它的返回值，我将再次恢复独自一人的状态。 + +``` +my parent + |- me + +``` + +### fork 和 exec 的代码实现 + +如果你要编写一个 shell，这是你必须做的一个练习（这是一个非常有趣和有启发性的项目。Kamal 在 Github 上有一个很棒的研讨会：[https://github.com/kamalmarhubi/shell-workshop][3]） + +事实证明，有了 C 或 Python 的技能，你可以在几个小时内编写一个非常简单的 shell，例如 bash。（至少如果你旁边能有个人多少懂一点，如果没有的话用时会久一点。）我已经完成啦，真的很棒。 + +这就是 fork 和 exec 在程序中的实现。我写了一段 C 的伪代码。请记住，[fork 也可能会失败哦。][4] + +``` +int pid = fork(); +// 我要分身啦 +// “我”是谁呢？可能是子进程也可能是父进程 +if (pid == 0) { + // 我现在是子进程 + // 我的大脑将被替代，然后变成一个完全不一样的进程“ls” + exec(["ls"]) +} else if (pid == -1) { + // 天啊，fork 失败了，简直是灾难！ +} else { + // 我是父进程耶 + // 继续做一个酷酷的美男子吧 + // 需要的话，我可以等待子进程结束 +} + +``` + +### 上文提到的“大脑被替代“是什么意思呢？ + +进程有很多属性： + +* 打开的文件（包括打开的网络连接） + +* 环境变量 + +* 信号处理程序（在程序上运行 Ctrl + C 时会发生什么？） + +* 内存（你的“地址空间”） + +* 寄存器 + +* 可执行文件（/proc/$pid/exe） + +* cgroups 和命名空间（与 Linux 容器相关） + +* 当前的工作目录 + +* 运行程序的用户 + +* 其他我还没想到的 + +当你运行 `execve` 并让另一个程序替代你的时候，实际上几乎所有东西都是相同的！ 你们有相同的环境变量、信号处理程序和打开的文件等等。 + +唯一改变的是，内存、寄存器以及正在运行的程序，这可是件大事。 + +### 为何 fork 并非那么耗费资源（写入时复制） + +你可能会问：“如果我有一个使用了 2 GB 内存的进程，这是否意味着每次我启动一个子进程，所有 2 GB 的内存都要被复制一次？这听起来要耗费很多资源！“ + +事实上，Linux 为 fork() 调用实现了写入时复制（copy on write），对于新进程的 2 GB 内存来说，就像是“看看旧的进程就好了，是一样的！”。然后，当如果任一进程试图写入内存，此时系统才真正地复制一个内存的副本给该进程。如果两个进程的内存是相同的，就不需要复制了。 + +### 为什么你需要知道这么多 + +你可能会说，好吧，这些琐事听起来很厉害，但为什么这么重要？关于信号处理程序或环境变量的细节会被继承吗？这对我的日常编程有什么实际影响呢？ + +有可能哦！比如说，在 Kamal 的博客上有一个很有意思的 [bug][5]。它讨论了 Python 如何使信号处理程序忽略了 SIGPIPE。也就是说，如果你从 Python 里运行一个程序，默认情况下它会忽略 SIGPIPE！这意味着，程序从 Python 脚本和从 shell 启动的表现会**有所不同**。在这种情况下，它会造成一个奇怪的问题。 + +所以，你的程序的环境（环境变量、信号处理程序等）可能很重要，都是从父进程继承来的。知道这些，在调试时是很有用的。 + + +-------------------------------------------------------------------------------- + +via: https://jvns.ca/blog/2016/10/04/exec-will-eat-your-brain/ + +作者：[ Julia Evans][a] +译者：[jessie-pang](https://github.com/jessie-pang) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://jvns.ca +[1]:https://jvns.ca/categories/favorite +[2]:http://man7.org/linux/man-pages/man3/posix_spawn.3.html +[3]:https://github.com/kamalmarhubi/shell-workshop +[4]:https://rachelbythebay.com/w/2014/08/19/fork/ +[5]:http://kamalmarhubi.com/blog/2015/06/30/my-favourite-bug-so-far-at-the-recurse-center/ diff --git a/translated/tech/20170802 Creating SWAP partition using FDISK - FALLOCATE commands.md b/translated/tech/20170802 Creating SWAP partition using FDISK - FALLOCATE commands.md deleted file mode 100644 index 455ade916c..0000000000 --- a/translated/tech/20170802 Creating SWAP partition using FDISK - FALLOCATE commands.md +++ /dev/null @@ -1,117 +0,0 @@ -使用 FDISK 和 FALLOCATE 命令创建交换分区 -====== -交换分区在物理内存(RAM)被填满时用来保持内存中的内容. 当 RAM 被耗尽, Linux 会将内存中不活动的页移动到交换空间中，从而空出内存给系统使用. 虽然如此, 但交换空间不应被认为是可以用来替代物理内存/RAM的. - -大多数情况下, 建议交换内存的大小为物理内存的1到2倍. 也就是说如果你有8GB内存, 那么交换空间大小应该介于8-16 GB. - -若系统中没有配置交换分区, 当内存耗尽后，系统可能会杀掉正在运行中哦该的进程/应哟该从而导致系统崩溃. 在本文中, 我们将学会如何为Linux系统添加交换分区,我们有两个办法: - -+ **使用 fdisk 命令** -+ **使用 fallocate 命令** - - - -### 第一个方法(使用 Fdisk 命令) - -通常, 系统的第一块硬盘会被命名为 **/dev/sda** 而其中的分区会命名为 **/dev/sda1** , **/dev/sda2**. 本文我们使用的石块有两个主分区的硬盘,两个分区分别为 /dev/sda1, /dev/sda2，而我们使用 /dev/sda3 来做交换分区. - -首先创建一个新分区, - -``` -$ fdisk /dev/sda -``` - -按 **' n'** 来创建新分区. 系统会询问你从哪个柱面开始, 直接按回车键使用默认值即可。然后系统询问你到哪个柱面结束, 这里我们输入交换分区的大小(比如1000MB). 这里我们输入 +1000M. - -![swap][2] - -现在我们创建了一个大小为 1000MB 的磁盘了。但是我们并没有设个分区的类型, 我们按下 **" t"** 然后回车来设置分区类型. - -现在我们要输入分区编号, 这里我们输入 **3**,然后输入磁盘分类id，交换分区的磁盘类型为 **82** (要显示所有可用的磁盘类型, 按下 **" l"** ) 然后再按下 " **w "** 保存磁盘分区表. - -![swap][4] - -再下一步使用 `mkswap` 命令来格式化交换分区 - -``` -$ mkswap /dev/sda3 -``` - -然后激活新建的交换分区 - -``` -$ swapon /dev/sda3 -``` - -然而我们的交换分区在重启后并不会自动挂载. 要做到永久挂载,我们需要添加内容道 `/etc/fstab` 文件中. 打开 `/etc/fstab` 文件并输入下面行 - -``` -$ vi /etc/fstab -``` - -``` -/dev/sda3 swap swap default 0 0 -``` - -保存并关闭文件. 现在每次重启后都能使用我们的交换分区了. - -### 第二种方法(使用 fallocate 命令) - -我推荐用这种方法因为这个是最简单，最快速的创建交换空间的方法了. Fallocate 是最被低估和使用最少的命令之一了. Fallocate 用于为文件预分配块/大小. - -使用 fallocate 创建交换空间, 我们首先在 ** '/'** 目录下创建一个名为 **swap_space** 的文件. 然后分配2GB道 swap_space 文件, - -``` -$ fallocate -l 2G /swap_space -``` - -我们运行下面命令来验证文件大小 - -``` -ls-lh /swap_space. -``` - -然后更改文件权限，让 `/swap_space` 更安全 - -``` -$ chmod 600 /swap_space** -``` - -这样只有 root 可以读写该文件了. 我们再来格式化交换分区(译者注：虽然这个swap_space应该是文件，但是我们把它当成是分区来挂载), - -``` -$ mkswap /swap_space -``` - -然后启用交换空间 - -``` -$ swapon -s -``` - -每次重启后都要重现挂载磁盘分区. 因此为了使之持久话，就像上面一样，我们编辑 `/etc/fstab` 并输入下面行 - -``` -/swap_space swap swap sw 0 0 -``` - -保存并退出文件. 现在我们的交换分区会一直被挂载了. 我们重启后可以在终端运行 **free -m** 来检查交换分区是否生效. - -我们的教程至此就结束了, 希望本文足够容易理解和学习. 如果有任何疑问欢迎提出. - - --------------------------------------------------------------------------------- - -via: http://linuxtechlab.com/create-swap-using-fdisk-fallocate/ - -作者：[Shusain][a] -译者：[lujun9972](https://github.com/lujun9972) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:http://linuxtechlab.com/author/shsuain/ -[1]:https://i1.wp.com/linuxtechlab.com/wp-content/plugins/a3-lazy-load/assets/images/lazy_placeholder.gif?resize=668%2C211 -[2]:https://i0.wp.com/linuxtechlab.com/wp-content/uploads/2017/02/fidsk.jpg?resize=668%2C211 -[3]:https://i1.wp.com/linuxtechlab.com/wp-content/plugins/a3-lazy-load/assets/images/lazy_placeholder.gif?resize=620%2C157 -[4]:https://i0.wp.com/linuxtechlab.com/wp-content/uploads/2017/02/fidsk-swap-select.jpg?resize=620%2C157 diff --git a/translated/tech/20170925 Linux Free Command Explained for Beginners (6 Examples).md b/translated/tech/20170925 Linux Free Command Explained for Beginners (6 Examples).md deleted file mode 100644 index 279bd0e75d..0000000000 --- a/translated/tech/20170925 Linux Free Command Explained for Beginners (6 Examples).md +++ /dev/null @@ -1,135 +0,0 @@ -6 个例子让初学者掌握 free 命令 -====== - -在 Linux 系统上，有时你可能想从命令行快速地了解系统的已使用和未使用的内存空间。如果你是一个 Linux 新手，有个好消息：有一条系统内置的命令可以显示这些信息：**free**。 - -在本文中，我们会讲到 free 命令的基本用法以及它所提供的一些重要的功能。文中提到的所有命令和用法都是在 Ubuntu 16.04LTS 上测试过的。 - -### Linux free 命令 - -让我们看一下 free 命令的语法： - -free [options] - -free 命令的 man 手册如是说： - -``` -free 命令显示了系统的可用和已用的物理内存及交换内存的总量，以及内核用到的缓存空间。这些信息是从 /proc/meminfo 中得到的。 -``` - -接下来我们用问答的方式了解一下 free 命令是怎么工作的。 - -### Q1. 怎么用 free 命令查看已使用和未使用的内存？ - -这很容易，您只需不加任何参数地运行 free 这条命令就可以了： - -free - -这是 free 命令在我的系统上的输出： - -[![view used and available memory using free command][1]][2] - -这些列是什么意思呢？ - -[![Free command columns][3]][4] - -total - 安装的内存的总量（等同于 /proc/meminfo 中的 MemTotal 和 SwapTotal） - -used - 已使用的内存（计算公式为：total - free - buffers - cache） - -free - 未被使用的内存（等同于 /proc/meminfo 中的 MemFree 和 SwapFree） - -shared - 通常是临时文件系统使用的内存（等同于 /proc/meminfo 中的 Shmem；在内核 2.6.32 版本上生效，参数无效则显示为 0） - -buffers - 内核缓冲区使用的内存（等同于 /proc/meminfo 中的 Buffers） - -cache - 页面缓存和 Slab 分配机制使用的内存（等同于 /proc/meminfo 中的 Cached 和 Slab） - -buff/cache - buffers 与 cache 之和 - -available - 在不计算交换空间的情况下，预计可以被新启动的应用程序所使用的内存空间。与 cache 或者 free 部分不同，这一列把页面缓存计算在内，并且不是所有的可回收 slab 内存都可以真正被回收，因为可能有被占用的部分。（等同于 /proc/meminfo 中的 MemAvailable；在内核 3.14 版本上生效，从内核 2.6.27 版本开始仿真；在其他版本上这个值与 free 这一列相同） - -### Q2. 如何更改显示的单位呢？ - -如果需要的话，你可以更改内存的显示单位。比如说，想要内存以兆为单位显示，你可以用 **-m** 这个参数： - -free -m - -[![free command display metrics change][5]][6] - -同样地，你可以用 **-b** 以字节显示、**-k** 以 KB 显示、**-m** 以 MB 显示、**-g** 以 GB 显示、**\--tera** 以 TB 显示。 - -### Q3. 怎么显示可读的结果呢？ - -free 命令提供了 **-h** 这个参数使输出转化为可读的格式。 - -free -h - -用这个参数，free 命令会自己决定用什么单位显示内存的每个数值。例如： - -[![diplsy data fromm free command in human readable form][7]][8] - -### Q4. 怎么让 free 命令以一定的时间间隔持续运行？ - -您可以用 **-s** 这个参数让 free 命令以一定的时间间隔持续地执行。您需要传递给命令行一个数字参数，做为这个时间间隔的秒数。 - -例如，使 free 命令每隔 3 秒执行一次： - -free -s 3 - -如果您需要 free 命令只执行几次，您可以用 **-c** 这个参数指定执行的次数： - -free -s 3 -c 5 - -上面这条命令可以确保 free 命令每隔 3 秒执行一次，总共执行 5 次。 - -**注**：这个功能目前在Ubuntu系统上还存在 [问题][9]，所以并未测试。 - -### Q5. 怎么使 free 基于 1000 计算内存，而不是 1024？ - -如果您指定 free 用 MB 来显示内存（用 -m 参数），但又想基于 1000 来计算结果，可以用 **\--sj** 这个参数来实现。下图展示了用与不用这个参数的结果： - -[![How to make free use power of 1000 \(not 1024\) while displaying memory figures][10]][11] - -### Q6. 如何使 free 命令显示每一列的总和？ - -如果您想要 free 命令显示每一列的总和，你可以用 **-t** 这个参数。 - -free -t - -如下图所示： - -[![How to make free display total of columns][12]][13] - -请注意 “Total” 这一行出现了。 - -### 总结 - -free 命令对于系统管理来讲是个极其有用的工具。它有很多参数可以定制化您的输出，易懂易用。我们在本文中也提到了很多有用的参数。练习完之后，请您移步至 [man 手册][14]了解更多内容。 - - --------------------------------------------------------------------------------- - -via: https://www.howtoforge.com/linux-free-command/ - -作者：[Himanshu Arora][a] -译者：[jessie-pang](https://github.com/jessie-pang) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://www.howtoforge.com -[1]:https://www.howtoforge.com/images/linux_free_command/free-command-output.png -[2]:https://www.howtoforge.com/images/linux_free_command/big/free-command-output.png -[3]:https://www.howtoforge.com/images/linux_free_command/free-output-columns.png -[4]:https://www.howtoforge.com/images/linux_free_command/big/free-output-columns.png -[5]:https://www.howtoforge.com/images/linux_free_command/free-m-option.png -[6]:https://www.howtoforge.com/images/linux_free_command/big/free-m-option.png -[7]:https://www.howtoforge.com/images/linux_free_command/free-h.png -[8]:https://www.howtoforge.com/images/linux_free_command/big/free-h.png -[9]:https://bugs.launchpad.net/ubuntu/+source/procps/+bug/1551731 -[10]:https://www.howtoforge.com/images/linux_free_command/free-si-option.png -[11]:https://www.howtoforge.com/images/linux_free_command/big/free-si-option.png -[12]:https://www.howtoforge.com/images/linux_free_command/free-t-option.png -[13]:https://www.howtoforge.com/images/linux_free_command/big/free-t-option.png -[14]:https://linux.die.net/man/1/free diff --git a/translated/tech/20171011 What is a firewall.md b/translated/tech/20171011 What is a firewall.md deleted file mode 100644 index cdbf18a5c9..0000000000 --- a/translated/tech/20171011 What is a firewall.md +++ /dev/null @@ -1,78 +0,0 @@ -什么是防火墙？ -===== -基于网络的防火墙已经在美国企业无处不在，因为它们证实了抵御日益增长的威胁的防御能力。 - -通过网络测试公司 NSS 实验室最近的一项研究发现高达 80% 的美国大型企业运行着下一代防火墙。研究公司 IDC 评估防火墙和相关的统一威胁管理市场营业额在 2015 是 76 亿美元，预计到 2020 年底将达到 127 亿美元。 - -**如果你想提升，这里是[What to consider when deploying a next generation firewall][1]** - -### 什么是防火墙？ - -防火墙充当一个监控流量的边界防御工具，要么允许它要么屏蔽它。 多年来，防火墙的功能不断增强，现在大多数防火墙不仅可以阻止已知的一组威胁，并执行高级访问控制列表策略，还可以深入检查各个包的流量和测试包，以确定它们是否安全。大多数防火墙被部署为网络硬件，用于处理流量和允许终端用户配置和管理系统的软件。越来越多的软件版防火墙部署到高度虚拟机环境中执行策略在被隔离的网络或 IaaS 公有云中。 - -随着防火墙技术的进步在过去十年中创造了新的防火墙部署选项，所以现在对于部署防火墙的最终用户来说，有一些选择。这些选择包括： - -### 有状态的防火墙 - 当首次创造防火墙时，它们是无状态的，这意味着流量通过硬件，在检查被监视的每个网络包流量的过程中，并单独屏蔽或允许它。从1990年代中后期开始，防火墙的第一个主要进展是引入状态。有状态防火墙在更全面的上下文中检查流量，同时考虑到网络连接的工作状态和特性，以提供更全面的防火墙。例如，维持这状态的防火墙允许某些流量访问某些用户，同时阻塞其他用户的同一流量。 - -### 下一代防火墙 - 多年来，防火墙增加了多种新的特性，包括深度包检查、入侵检测以及对加密流量的预防和检查。下一代防火墙（NGFWs）是指有许多先进的功能集成到防火墙的防火墙。 - -### 基于代理的防火墙 - -这些防火墙充当请求数据的最终用户和数据源之间的网关。在传递给最终用户之前，所有的流量都通过这个代理过滤。这通过掩饰信息的原始请求者的身份来保护客户端不受威胁。 - -### Web 应用防火墙 - -这些防火墙位于特定应用程序的前面，而不是在更广阔的网络的入口或则出口上。而基于代理的防火墙通常被认为是保护终端客户，WAFs 通常被认为是保护应用服务器。 - -### 防火墙硬件 - -防火墙硬件通常是一个简单的服务器，它可以充当路由器来过滤流量和运行防火墙软件。这些设备放置在企业网络的边缘，路由器和 Internet 服务提供商的连接点之间。通常企业可能在整个数据中心部署十几个物理防火墙。 用户需要根据用户基数的大小和 Internet 连接的速率来确定防火墙需要支持的吞吐量容量。 - -### 防火墙软件 - -通常，终端用户部署多个防火墙硬件端和一个中央防火墙软件系统来管理部署。 这个中心系统是配置策略和特性的地方，在那里可以进行分析，并可以对威胁作出响应。 - -### 下一代防火墙 - -多年来，防火墙增加了多种新的特性，包括深度包检查、入侵检测以及对加密流量的预防和检查。下一代防火墙（NGFWs）是指集成了这些先进功能的防火墙，这里描述的是它们中的一些。 - -### 有状态的检测 - -阻止已知不需要的流量，这是基本的防火墙功能。 - -### 抵御病毒 - -在网络流量中搜索已知病毒和漏洞，这个功能有助于防火墙接收最新威胁的更新，并不断更新以保护它们。 - -### 入侵防御系统 - -这类安全产品可以部署为一个独立的产品，但 IPS 功能正逐步融入 NGFWs。 虽然基本的防火墙技术识别和阻止某些类型的网络流量，但 IPS 使用更多的细粒度安全措施，如签名跟踪和异常检测，以防止不必要的威胁进入公司网络。 IPS 系统已经取代了以前这一技术的版本，入侵检测系统（IDS）的重点是识别威胁而不是遏制它们。 - -### 深度包检测（DPI） - -DPI 可部分或用于与 IPS 的结合，但其仍然成为一个 NGFWs 的重要特征，因为它提供细粒度分析的能力，具体到流量包和流量数据的头文件。DPI 还可以用来监测出站流量，以确保敏感信息不会离开公司网络，这种技术称为数据丢失预防（DLP）。 - -### SSL 检测 - -安全套接字层（SSL）检测是一个检测加密流量来测试威胁的方法。随着越来越多的流量进行加密，SSL 检测成为 DPI 技术，NGFWs 正在实施的一个重要组成部分。SSL 检测作为一个缓冲区，它在送到最终目的地之前解码流量以检测它。 - -### 沙盒 - -这个是被卷入 NGFWs 中的一个较新的特性，它指防火墙接收某些未知的流量或者代码，并在一个测试环境运行，以确定它是否是邪恶的能力。 - --------------------------------------------------------------------------------- - -via: https://www.networkworld.com/article/3230457/lan-wan/what-is-a-firewall-perimeter-stateful-inspection-next-generation.html - -作者：[Brandon Butler][a] -译者：[zjon](https://github.com/zjon) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:https://www.networkworld.com/author/Brandon-Butler/ -[1]:https://www.networkworld.com/article/3236448/lan-wan/what-to-consider-when-deploying-a-next-generation-firewall.html - - diff --git a/sources/tech/20171016 Fixing vim in Debian - There and back again.md b/translated/tech/20171016 Fixing vim in Debian - There and back again.md similarity index 53% rename from sources/tech/20171016 Fixing vim in Debian - There and back again.md rename to translated/tech/20171016 Fixing vim in Debian - There and back again.md index 622b9fe885..36dd92d36a 100644 --- a/sources/tech/20171016 Fixing vim in Debian - There and back again.md +++ b/translated/tech/20171016 Fixing vim in Debian - There and back again.md @@ -1,19 +1,18 @@ -translating---geekpi - -Fixing vim in Debian – There and back again +在 Debian 中修复 vim - 去而复得 ====== I was wondering for quite some time why on my server vim behaves so stupid with respect to the mouse: Jumping around, copy and paste wasn't possible the usual way. All this despite having +我一直在想，为什么我服务器上 vim 为什么在鼠标方面表现得如此愚蠢：不能像平时那样跳转、复制、粘贴。尽管在 `/etc/vim/vimrc.local` 中已经设置了 ``` set mouse= ``` -in my `/etc/vim/vimrc.local`. Finally I found out why, thanks to bug [#864074][1] and fixed it. +最后我终于知道为什么了，多谢 bug [#864074][1] 并且修复了它。 ![][2] -The whole mess comes from the fact that, when there is no `~/.vimrc`, vim loads `defaults.vim` **after** ` vimrc.local` and thus overwriting several settings put in there. +原因是，当没有 `~/.vimrc` 的时候，vim在 `vimrc.local` **之后**加载 `defaults.vim`，从而覆盖了几个设置。 -There is a comment (I didn't see, though) in `/etc/vim/vimrc` explaining this: +在 `/etc/vim/vimrc` 中有一个注释（虽然我没有看到）解释了这一点： ``` " Vim will load $VIMRUNTIME/defaults.vim if the user does not have a vimrc. " This happens after /etc/vim/vimrc(.local) are loaded, so it will override @@ -24,11 +23,11 @@ There is a comment (I didn't see, though) in `/etc/vim/vimrc` explaining this: ``` -I agree that this is a good way to setup vim on a normal installation of Vim, but the Debian package could do better. The problem is laid out clearly in the bug report: If there is no `~/.vimrc`, settings in `/etc/vim/vimrc.local` are overwritten. +我同意这是在正常安装 vim 后设置 vim 的好方法，但 Debian 包可以做得更好。在错误报告中清楚地说明了这个问题：如果没有 `~/.vimrc`，`/etc/vim/vimrc.local` 中的设置被覆盖。 -This is as counterintuitive as it can be in Debian - and I don't know any other package that does it in a similar way. +这在Debian中是违反直觉的 - 而且我也不知道其他包中是否采用类似的方法。 -Since the settings in `defaults.vim` are quite reasonable, I want to have them, but only fix a few of the items I disagree with, like the mouse. At the end what I did is the following in my `/etc/vim/vimrc.local`: +由于 `defaults.vim` 中的设置非常合理，所以我希望使用它，但只修改了一些我不同意的项目，比如鼠标。最后，我在 `/etc/vim/vimrc.local` 中做了以下操作： ``` if filereadable("/usr/share/vim/vim80/defaults.vim") source /usr/share/vim/vim80/defaults.vim @@ -42,14 +41,14 @@ set mouse= ``` -There is probably a better way to get a generic load statement that does not depend on the Vim version, but for now I am fine with that. +可能有更好的方式来获得一个不依赖于 vim 版本的通用加载语句, 但现在我对此很满意。 -------------------------------------------------------------------------------- via: https://www.preining.info/blog/2017/10/fixing-vim-in-debian/ 作者：[Norbert Preining][a] -译者：[译者ID](https://github.com/译者ID) +译者：[geekpi](https://github.com/geekpi) 校对：[校对者ID](https://github.com/校对者ID) 本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 diff --git a/translated/tech/20180101 The mysterious case of the Linux Page Table Isolation patches.md b/translated/tech/20180101 The mysterious case of the Linux Page Table Isolation patches.md deleted file mode 100644 index 2cfd429533..0000000000 --- a/translated/tech/20180101 The mysterious case of the Linux Page Table Isolation patches.md +++ /dev/null @@ -1,139 +0,0 @@ -# [关于 Linux 页面表隔离补丁的神秘情况][14] - -* * * - -_长文预警：_ 这是一个目前严格限制的、禁止披露的安全 bug，它影响到目前几乎所有实现虚拟内存的 CPU 架构，需要硬件的改变才能完全解决这个 bug。通过软件来缓解这种影响的紧急开发工作正在进行中，并且最近在 Linux 内核中已经得以实现，并且，在 11 月份，在 NT 内核中也开始了一个类似的紧急开发。在最糟糕的情况下，软件修复会导致一般工作负载出现巨大的减速（译者注：外在表现为 CPU 性能下降）。这里有一个提示，攻击会影响虚拟化环境，包括 Amazon EC2 和 Google 计算引擎，以及另外的提示是，这种精确的攻击可能涉及一个新的 Rowhammer 变种（译者注：一个由 Google 安全团队提出的 DRAM 的安全漏洞，在文章的后面部分会简单介绍）。 - -* * * - -我一般不太关心安全问题，但是，对于这个 bug 我有点好奇，而一般会去写这个主题的人似乎都很忙，要么就是知道这个主题细节的人会保持沉默。这让我在新年的第一天（元旦那天）花了几个小时深入去挖掘关于这个谜团的更多信息，并且我将这些信息片断拼凑到了一起。 - -注意，这是一件相互之间高度相关的事件，因此，它的主要描述都是猜测，除非过一段时间，它的限制禁令被取消。我所看到的，包括涉及到的供应商、许多争论和这种戏剧性场面，将在限制禁令取消的那一天出现。 - -**LWN** - -这个事件的线索出现于 12 月 20 日 LWN 上的 [内核页面表的当前状况：页面隔离][2]（致校对：就是昨天我翻译的那篇） 这篇文章。它在 10 月份被奥地利的 [TU Graz][4] 的一组研究人员第一次发表。从文章语气上明显可以看到这项工作的紧急程度，内核的核心开发者紧急加入了 [KAISER 补丁系列][3]。 - -这一系列的补丁的用途从概念上说很简单：为了阻止运行在用户空间的进程在进程页面表中，通过映射得到内核空间页面的各种攻击方式，可以很好地阻止了从非特权的用户空间代码中识别到内核虚拟地址的攻击企图。 - -这组论文描述的 KAISER，[KASLR 已死：KASLR 永存][5]（致校对：这里我觉得是[ASLR 已死：KASLR 永存]，请查看原文出处。），在它的抽象中，通过特定的引用，在内存管理硬件中去删除所有内核地址空间的信息，即便是用户代码在这个 CPU 上处于活动状态的时候。 - -这个补丁集的魅力在于它触及到了核心，内核的全部基础核心（和与用户空间的接口），显然，它应该被最优先考虑。在 Linux 中当我读到关于内存管理的变化时，通常，第一个引用发生在变化被合并的很久之前，并且，通常会进行多次的评估、拒绝、以及因各种原因爆发争论的一系列过程。 - -KAISER（就是现在的 KPTI）系列被合并还不足三个月。 - -**ASLR 概述** - -从表面上看，设计的这些补丁可以确保地址空间布局随机化仍然有效：这是一个现代操作系统的安全特性，它企图去将更多的随机位，引入到公共映射对象的地址空间中。 - -例如，在引用 /usr/bin/python 时，动态链接将对系统的 C 库、堆、线程栈、以及主要的可执行文件进行排布，去接受随机分配的地址范围： - -> $ bash -c ‘grep heap /proc/$$/maps’ -> 019de000-01acb000 rw-p 00000000 00:00 0                                  [heap] -> $ bash -c 'grep heap /proc/$$/maps’ -> 023ac000-02499000 rw-p 00000000 00:00 0                                  [heap] - -注意跨 bash 进程的开始和结束偏移量上的堆的变化。 - -这个特性的效果是，一个 buffer 管理的 bug 导致一个攻击者可以去覆写一些程序代码指向的内存地址，并且，那个地址将在程序控制流中被使用，诸如这种攻击者可以使控制流转向到一个包含他们选择的内容的 buffer 上，对于攻击者来说，使用机器代码来填充 buffer 将更困难。例如，system() C 库函数将被引用，因为，那个函数的地址在不同的运行进程上不同的。 - -这是一个简单的示例，ASLR 被设计用于去保护类似这样的许多场景，包括阻止攻击者从有可能被用来修改控制流或者实现一个攻击的程序数据的地址内容。 - -KASLR 是 “简化的” 应用到内核本身的 ASLR：在每个重新引导的系统上，属于内核的地址范围是随机的，这样就使得，虽然被攻击者转向的控制流运行在内核模式上，但是，不能猜测到为实现他们的攻击目的所需要的函数和结构的地址，比如，定位当前进程数据，将活动的 UID 从一个非特权用户提升到 root 用户，等等。 - -**坏消息：缓减这种攻击的软件运行成本过于贵重** - -老的 Linux 将内核内存映射在同一个页面表中的这个行为的主要原因是，当用户的代码触发一个系统调用、故障、或者产生中断时，用户内存也是这种行为，这样就不需要改变正在运行的进程的虚拟内存布局。 - -因为在那样，它不需要去改变虚拟内存布局，进而也就不需要去清洗掉（flush）与 CPU 性能高度依赖的缓存（致校对：意思是如果清掉这些缓存，CPU 性能就会下降），主要是通过 [转换查找缓冲器][6]（译者注：Translation Lookaside Buffer（TLB）（将虚拟地址转换为物理地址）。 - -使用已合并的页面表分割补丁后变成，内核每次开始运行时，需要将内核的缓存清掉，并且，每次用户代码恢复运行时都会这样。对于大多数工作负载，在每个系统调用中，TLB 的实际总损失将导致明显的变慢：[@grsecurity 测量的一个简单的案例][7]，在一个最新的 AMD CPU 上，Linux “du -s” 变慢了 50%。 - -**34C3** - -在今年的 CCC 上，你可以找到 TU Graz 的研究人员的另一篇，[一个纯 Javascript 的 ASLR 攻击描述][8] ，通过仔细地掌握 CPU 内存管理单元的操作时机，遍历了描述虚拟内存布局的页面表，来实现 ASLR 攻击。它通过高度精确的时间掌握和选择性回收的 CPU 缓存行的组合方式来实现这种结果，一个运行在 web 浏览器的 Javascript 程序可以找回一个 Javascript 对象的虚拟地址，使得利用浏览器内存管理 bugs 被允许进行接下来的攻击。 - -因此，从表面上看，我们有一组 KAISER 补丁，也展示了解除 ASLR 的地址的技术，并且，这个展示使用的是 Javascript，很快就可以在一个操作系统内核上进行重新部署。 - -**虚拟内存概述** - -在通常情况下，当一些机器码尝试去加载、存储、或者跳转到一个内存地址时，现代的 CPUs 必须首先去转换这个  _虚拟地址_  到一个  _物理地址_ ，通过使用一系列操作系统托管的数组（被称为页面表），来描述一个虚拟地址和安装在这台机器上的物理内存之间的映射。 - -在现代操作系统中，虚拟内存可能是仅有的一个非常重要的强大特性：它都阻止了什么呢？例如，一个濒临死亡的进程崩溃了操作系统、一个 web 浏览器 bugs 崩溃了你的桌面环境、或者，一个运行在 Amazon EC2 中的虚拟机的变化影响了同一台主机上的另一个虚拟机。 - -这种攻击的原理是，利用 CPU 上维护的大量的缓存，通过仔细地操纵这些缓存的内存，它可以去推测内存管理单元的地址，以去访问页面表的不同层级，因为一个未缓存的访问将比一个缓存的访问花费更长的时间。通过检测页面表上可访问的元素，它可能去恢复在 MMU（译者注：存储器管理单元）忙于解决的虚拟地址中的大部分比特（bits）。 - -**这种动机的证据，但是不用恐慌** - -我们找到了动机，但是到目前为止，我们并没有看到这项工作引进任何恐慌。总的来说，ASLR 并不能完全缓减这种风险，并且也是一道最后的防线：仅在这 6 个月的周期内，即便是一个没有安全意识的人也能看到一些关于解除（unmasking） ASLR 的指针的新闻，并且，实际上 ASLR 已经存在了。 - -单独的修复 ASLR 并不足于去描述这项工作高优先级背后的动机。 - -**它是硬件安全 bug 的证据** - -通过阅读这一系列补丁，可以明确许多事情。 - -第一，正如 [@grsecurity 指出][9] 的，代码中的一些注释已经被编辑（redacted），并且，描述这项工作的额外的主文档文件已经在 Linux 源代码树中看不到了。 - -测试代码已经以运行时补丁的方式构建，在系统引导时仅当内核检测到是受影响的系统时才会被应用，与对臭名昭著的 [Pentium F00F bug][10] 的缓解措施，使用完全相同的机制： - -![](http://78.media.tumblr.com/1c80c45e14c1e676b35cdd89cc9b557c/tumblr_inline_p1untxZBBD1rkm8fh_500.jpg) - -**更多的线索：Microsoft 也已经实现了页面表的分割** - -通过对 FreeBSD 源代码的一个小挖掘可以看出，目前，其它的免费操作系统没有实现页面表分割，但是，通过 [Alex Ioniscu on Twitter][11] 的启示，这项工作已经不局限于 Linux 了：从 11 月起，公开的 NT 内核也已经实现了同样的技术。 - -**猜测的结果：Rowhammer** - -在 TU Graz 上进一步挖掘对这项工作的研究，我们找到 [When rowhammer only knocks once][12]，12 月 4 日通告的一个 [新的 Rowhammer 攻击的变种][13]： - -> 在这篇论文中，我们提出了新的 Rowhammer 攻击和原始的漏洞利用，表明即便是所有防御的组合也没有效果。我们的新攻击技术，对一个位置的反复 “敲打”（hammering），打破了以前假定的触发 Rowhammer bug 的前提条件。 - -作一个快速回顾，Rowhammer 是一个对主要（全部？）种类的商品 DRAMs 的基础问题的一个类别，比如，在普通的计算机中的内存上。通过精确操作内存中的一个区域，这可能会导致内存该区域存储的相关（但是逻辑上是独立的）内容被毁坏。效果是，Rowhammer 可能被用于去反转内存中的比特（bits），使未经授权的用户代码可以访问到，比如，这个比特位描述了系统中的其它代码的访问权限。 - -我发现在 Rowhammer 上，这项工作很有意思，尤其是它反转的位接近页面表分割补丁时，但是，因为 Rowhammer 攻击要求一个目标：你必须知道你尝试去反转的比特在内存中的物理地址，并且，第一步是得到的物理地址可能是一个虚拟地址，比如，在 KASLR 中的解除（unmasking）工作。 - -**猜测的结果：它影响主要的云供应商** - -在我能看到的内核邮件列表中，除了子系统维护者的名字之外，e-mail 地址是属于 Intel、Amazon、和 Google 的雇员，这表示这两个大的云计算供应商对此特别感兴趣，这为我们提供了一个强大的线索，这项工作很大的可能是受虚拟化安全驱动的。 - -它可能会导致产生更多的猜测：虚拟机 RAM 和由这些虚拟机所使用的虚拟内存地址，最终表示为在主机上大量的相邻的数组，那些数组，尤其是在一个主机上只有两个租户的情况下，在 Xen 和 Linux 内核中是通过内存分配来确定的，这样可能会有（准确性）非常高的可预测行为。 - -**最喜欢的猜测：这是一个提升特权的攻击** - -把这些综合到一起，我并不难预测，如果我们在 2018 年使用这些存在提升特权的 bug 的发行版，或者类似的系统去驱动如此紧急的情况，并且在补丁集的抄送列表中出现如此多的感兴趣者的名字。 - -最后的一个趣闻，虽然我在阅读补丁集的时候没有找到我要的东西，但是，在一些代码中标记，paravirtual 或者 HVM Xen 是不受此影响的。 - -**Invest in popcorn, 2018 将很有趣** - -这些猜想是完全有可能的，它离实现很近，但是可以肯定的是，当这些事情被公开后，那将是一个非常令人激动的几个星期。 - --------------------------------------------------------------------------------- - -via: http://pythonsweetness.tumblr.com/post/169166980422/the-mysterious-case-of-the-linux-page-table - -作者：[python sweetness][a] -译者：[qhwdw](https://github.com/qhwdw) -校对：[校对者ID](https://github.com/校对者ID) - -本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 - -[a]:http://pythonsweetness.tumblr.com/ -[1]:http://pythonsweetness.tumblr.com/post/169217189597/quiet-in-the-peanut-gallery -[2]:http://t.umblr.com/redirect?z=https%3A%2F%2Flwn.net%2FArticles%2F741878%2F&t=ODY1YTM4MjYyYzU2NzNmM2VmYzEyMGIzODJkY2IxNDg0MDhkZDM1MSxXRG55eVpXNw%3D%3D&b=t%3AqBH2b-yWL63V8acbuG-EUQ&p=http%3A%2F%2Fpythonsweetness.tumblr.com%2Fpost%2F169166980422%2Fthe-mysterious-case-of-the-linux-page-table&m=1 -[3]:http://t.umblr.com/redirect?z=https%3A%2F%2Flwn.net%2FArticles%2F738975%2F&t=MzQxMmMyYThhNDdiMGJkZmRmZWI5NDkzZmQ3ZTM4ZDcwYzFhMjU5OSxXRG55eVpXNw%3D%3D&b=t%3AqBH2b-yWL63V8acbuG-EUQ&p=http%3A%2F%2Fpythonsweetness.tumblr.com%2Fpost%2F169166980422%2Fthe-mysterious-case-of-the-linux-page-table&m=1 -[4]:http://t.umblr.com/redirect?z=https%3A%2F%2Fwww.iaik.tugraz.at%2Fcontent%2Fresearch%2Fsesys%2F&t=NzEwZjg5YmQ1ZTNlZWIyYWE0YzgzZmZjN2ZmM2E2YjMzNDk5YTk4YixXRG55eVpXNw%3D%3D&b=t%3AqBH2b-yWL63V8acbuG-EUQ&p=http%3A%2F%2Fpythonsweetness.tumblr.com%2Fpost%2F169166980422%2Fthe-mysterious-case-of-the-linux-page-table&m=1 -[5]:http://t.umblr.com/redirect?z=https%3A%2F%2Fgruss.cc%2Ffiles%2Fkaiser.pdf&t=OTk4NGQwZTQ1NTdlNzE1ZGEyZTdlY2ExMTY1MTJhNzk2ODIzYWY1OSxXRG55eVpXNw%3D%3D&b=t%3AqBH2b-yWL63V8acbuG-EUQ&p=http%3A%2F%2Fpythonsweetness.tumblr.com%2Fpost%2F169166980422%2Fthe-mysterious-case-of-the-linux-page-table&m=1 -[6]:http://t.umblr.com/redirect?z=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FTranslation_lookaside_buffer&t=NjEyNGUzNTk2MGY3ODY3ODIxZjQ1Yjc4YWZjMGNmNmI1OWU1M2U0YyxXRG55eVpXNw%3D%3D&b=t%3AqBH2b-yWL63V8acbuG-EUQ&p=http%3A%2F%2Fpythonsweetness.tumblr.com%2Fpost%2F169166980422%2Fthe-mysterious-case-of-the-linux-page-table&m=1 -[7]:https://twitter.com/grsecurity/status/947439275460702208 -[8]:http://t.umblr.com/redirect?z=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dewe3-mUku94&t=NjczZmIzNWY3YTA2NGFiZDJmYThlMjlhMWM1YTE3NThhNzY0OGJlMSxXRG55eVpXNw%3D%3D&b=t%3AqBH2b-yWL63V8acbuG-EUQ&p=http%3A%2F%2Fpythonsweetness.tumblr.com%2Fpost%2F169166980422%2Fthe-mysterious-case-of-the-linux-page-table&m=1 -[9]:https://twitter.com/grsecurity/status/947147105684123649 -[10]:http://t.umblr.com/redirect?z=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FPentium_F00F_bug&t=Yjc4MDZhNDZjZDdiYWNkNmJkNjQ3ZDNjZmVlZmRkMGM2NDYwN2I2YSxXRG55eVpXNw%3D%3D&b=t%3AqBH2b-yWL63V8acbuG-EUQ&p=http%3A%2F%2Fpythonsweetness.tumblr.com%2Fpost%2F169166980422%2Fthe-mysterious-case-of-the-linux-page-table&m=1 -[11]:https://twitter.com/aionescu/status/930412525111296000 -[12]:http://t.umblr.com/redirect?z=https%3A%2F%2Fwww.tugraz.at%2Fen%2Ftu-graz%2Fservices%2Fnews-stories%2Fplanet-research%2Fsingleview%2Farticle%2Fwenn-rowhammer-nur-noch-einmal-klopft%2F&t=NWM1ZjZlZWU2NzFlMWIyNmI5MGZlNjJlZmM2YTlhOTIzNGY3Yjk4NyxXRG55eVpXNw%3D%3D&b=t%3AqBH2b-yWL63V8acbuG-EUQ&p=http%3A%2F%2Fpythonsweetness.tumblr.com%2Fpost%2F169166980422%2Fthe-mysterious-case-of-the-linux-page-table&m=1 -[13]:http://t.umblr.com/redirect?z=https%3A%2F%2Farxiv.org%2Fabs%2F1710.00551&t=ZjAyMDUzZWRmYjExNGNlYzRlMjE1NTliMTI2M2Y4YjkxMTFhMjI0OCxXRG55eVpXNw%3D%3D&b=t%3AqBH2b-yWL63V8acbuG-EUQ&p=http%3A%2F%2Fpythonsweetness.tumblr.com%2Fpost%2F169166980422%2Fthe-mysterious-case-of-the-linux-page-table&m=1 -[14]:http://pythonsweetness.tumblr.com/post/169166980422/the-mysterious-case-of-the-linux-page-table -[15]:http://pythonsweetness.tumblr.com/ - - diff --git a/translated/tech/20180104 How to Change Your Linux Console Fonts.md b/translated/tech/20180104 How to Change Your Linux Console Fonts.md new file mode 100644 index 0000000000..245f15924e --- /dev/null +++ b/translated/tech/20180104 How to Change Your Linux Console Fonts.md @@ -0,0 +1,88 @@ +如何更改 Linux 控制台上的字体 +====== +![](https：//www.linux.com/sites/lcom/files/styles/rendered_file/public/font-size_0.png？itok=d97vmyYa) + +我尝试尽可能的保持心灵祥和，然而总有一些事情让我意难平，比如控制台字体太小了。记住我的话，朋友，有一天你的眼睛会退化，无法再看清你编码时用的那些细小字体，到那时你就后悔莫及了。 + +幸好，Linux 死忠们，你可以更改控制台的字体。按照 Linux 一贯的尿性，不断变化的 Linux 环境使得这个问题变得不太简单明了，而 Linux 上也没有字体管理这么个东西，这使得我们很容易就被搞晕了。本文，我将会向你展示，我找到的更改字体的最简方法。 + +### Linux 控制台是个什么鬼？ + +首先让我们来澄清一下我们说的到底是个什么东西。当我提到 Linux 控制台，我指的是 TTY1-6，即你从图形环境用 `Ctrl-Alt-F1` 到 `F6` 切换到的虚拟终端。按下 `Ctrl+Alt+F7` 会切回图形环境。(不过这些热键已经不再通用，你的 Linux 发行版可能有不同的键映射。你的 TTY 的数量也可能不同，你图形环境会话也可能不在 `F7`。比如，Fedora 的默认图形会话是 `F2`，它只有一个额外的终端在 `F1`。) 我觉得能同时拥有 X 会话和终端绘画实在是太酷了。 + +Linux 控制台是内核的一部分，而且并不运行在 X 会话中。它和你在没有图形环境的无头服务器中用的控制台是一样的。我称呼在图形会话中的 X 终端为终端，而将控制台和 X 终端统称为终端模拟器。 + +但这还没完。Linux 终端从早期的 ANSI 时代开始已经经历了长久的发展，多亏了 Linux framebuffer，它现在支持 Unicode 并且对图形也有了有限的一些支持。而且出现了很多在控制台下运行的多媒体应用，这些我们在以后的文章中会提到。 + +### 控制台截屏 + +获取控制台截屏的最简单方法是让控制台跑在虚拟机内部。然后你可以在宿主系统上使用中意的截屏软件来抓取。不过借助 [fbcat][1] 和 [fbgrab][2] 你也可以直接在控制台上截屏。`fbcat` 会创建一个可移植的像素映射格式 (PPM) 图像; 这是一个高度可移植的未压缩图像格式，可以在所有的操作系统上读取，当然你也可以把它转换成任何喜欢的其他格式。`fbgrab` 则是 `fbcat` 的一个封装脚本，用来生成一个 PNG 文件。不同的人写过多个版本的 `fbgrab`。每个版本的选项都有限而且只能创建截取全屏。 + +`fbcat` 的执行需要 root 权限，而且它的输出需要重定向到文件中。你无需指定文件扩展名，只需要输入文件名就行了： +``` +$ sudo fbcat > Pictures/myfile + +``` + +在 GIMP 中裁剪后，就得到了图 1。 + +![](https：//www.linux.com/sites/lcom/files/styles/rendered_file/public/fig-1_10.png？itok=bHOxrZk9) +Figure 1：View after cropping。 + +如果能在左边空白处有一点填充就好了，如果有读者知道如何实现请在留言框中告诉我。 + +`fbgrab` 还有一些选项，你可以通过 `man fbgrab` 来查看，这些选项包括对另一个控制台进行截屏，以及延时截屏。在下面的例子中可以看到，`fbgrab` 截屏跟 `fbcat` 截屏类似，只是你无需明确进行输出重定性了： +``` +$ sudo fbgrab Pictures/myOtherfile + +``` + +### 查找字体 + +就我所知，除了查看字体存储目录 `/usr/share/consolefonts/`(Debian/etc。)，`/lib/kbd/consolefonts/` (Fedora)，`/usr/share/kbd/consolefonts` (openSUSE)，外没有其他方法可以列出已安装的字体了。 + +### 更改字体 + +可读字体不是什么新概念。我们应该尊重以前的经验！可读性是很重要的。可配置性也很重要，然而现如今却不怎么看重了。 + +在 Debian/Ubuntu/ 等系统上，可以运行 `sudo dpkg-reconfigure console-setup` 来设置控制台字体，然后在控制台运行 `setupcon` 命令来让变更生效。`setupcon` 属于 `console-setup` 软件包中的一部分。若你的 Linux 发行版中不包含该工具，可以在 [openSUSE][3] 中下载到它。 + +你也可以直接编辑 `/etc/default/console-setup` 文件。下面这个例子中设置字体为 32 点大小的 Terminus Bold 字体，这是我的最爱，并且严格限制控制台宽度为 80 列。 +``` +ACTIVE_CONSOLES="/dev/tty[1-6]" +CHARMAP="UTF-8" +CODESET="guess" +FONTFACE="TerminusBold" +FONTSIZE="16x32" +SCREEN_WIDTH="80" + +``` + +这里的 FONTFACE 和 FONTSIZE 的值来自于字体的文件名，`TerminusBold32x16.psf.gz`。是的，你需要反转 FONTSIZE 中值的顺序。计算机就是这么搞笑。然后再运行 `setupcon` 来让新配置生效。可以使用 `showconsolefont` 来查看当前所用字体的所有字符集。要查看完整的选项说明请参考 `man console-setup`。 + +### Systemd + +Systemd 与 `console-setup` 不太一样，除了字体之外，你无需安装任何东西。你只需要编辑 `/etc/vconsole.conf` 然后重启就行了。我在 Fedora 和 openSUSE 系统中安装了一些额外的大型号的 Terminus 字体包，因为默认安装的字体最大只有 16 点而我想要的是 32 点。然后将 `/etc/vconsole.conf` 的内容修改为： +``` +KEYMAP="us" +FONT="ter-v32b" + +``` + +下周我们还将学习一些更加酷的控制台小技巧，以及一些在控制台上运行的多媒体应用。 + + +-------------------------------------------------------------------------------- + +via: https://www.linux.com/learn/intro-to-linux/2018/1/how-change-your-linux-console-fonts + +作者：[Carla Schroder][a] +译者：[lujun9972](https://github.com/lujun9972) +校对：[校对者ID](https://github.com/校对者ID) + +本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出 + +[a]:https://www.linux.com/users/cschroder +[1]:http://jwilk.net/software/fbcat +[2]:https://github.com/jwilk/fbcat/blob/master/fbgrab +[3]:https://software.opensuse.org/package/console-setup