mirror of
https://github.com/LCTT/TranslateProject.git
synced 2026-08-20 03:53:30 +08:00
Merge remote-tracking branch 'LCTT/master'
This commit is contained in:
Xingyu Wang
@@ -1,22 +1,24 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: (geekpi)
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
[#]: reviewer: (wxy)
|
||||
[#]: publisher: (wxy)
|
||||
[#]: url: (https://linux.cn/article-12364-1.html)
|
||||
[#]: subject: (Style your data plots in Python with Pygal)
|
||||
[#]: via: (https://opensource.com/article/20/6/pygal-python)
|
||||
[#]: author: (Shaun Taylor-Morgan https://opensource.com/users/shaun-taylor-morgan)
|
||||
|
||||
使用 Pygal 在 Python 中设置数据图的样式
|
||||
======
|
||||
介绍一种更时尚的 Python 绘图库。
|
||||
![Python in a coffee cup.][1]
|
||||
|
||||
[Python][2] 一个可以可视化数据的库。一个更具互动性的选择是 Pygal，我认为这个库适合喜欢好看的人。它生成用户可以与之交互的漂亮的 SVG（可缩放矢量图形）文件。SVG 是交互式图形的标准格式，仅使用几行 Python 就可以带来丰富的用户体验。
|
||||
> 介绍一种更时尚的 Python 绘图库。
|
||||
|
||||
|
||||
|
||||
[Python][2] 有很多可以将数据可视化的库。其中一个互动性较强的库是 Pygal，我认为这个库适合喜欢漂亮事物的人。它可以生成用户可以与之交互的漂亮的 SVG（可缩放矢量图形）文件。SVG 是交互式图形的标准格式，仅使用几行 Python 就可以带来丰富的用户体验。
|
||||
|
||||
### 使用 Pygal 进行时尚的 Python 绘图
|
||||
|
||||
在本文中，我们要重新创建多柱状图，它代表了 1966 年至 2020 年英国大选的结果：
|
||||
在本文中，我们要重新创建多柱状图，用来表示 1966 年至 2020 年英国大选的结果：
|
||||
|
||||
![Pygal plot][3]
|
||||
|
||||
@@ -25,18 +27,14 @@
|
||||
* 运行最新版本的 Python（[Linux][4]、[Mac][5] 和 [Windows][6] 的说明）
|
||||
* 确认你运行的是与这些库兼容的 Python 版本
|
||||
|
||||
|
||||
|
||||
数据可在线获得，并可使用 pandas 导入：
|
||||
|
||||
|
||||
```
|
||||
import pandas as pd
|
||||
df = pd.read_csv('<https://anvil.works/blog/img/plotting-in-python/uk-election-results.csv>')
|
||||
df = pd.read_csv('https://anvil.works/blog/img/plotting-in-python/uk-election-results.csv')
|
||||
```
|
||||
|
||||
现在可以了。数据如下所示：
|
||||
|
||||
现在我们可以继续进行了。。数据如下所示：
|
||||
|
||||
```
|
||||
year conservative labour liberal others
|
||||
@@ -49,33 +47,29 @@ df = pd.read_csv('<https://anvil.works/blog/img/plotting-in-python/uk-election-r
|
||||
14 2019 365 202 11 72
|
||||
```
|
||||
|
||||
|
||||
|
||||
在 Pygal 中进行绘制会以一种易于阅读的方式显示。首先，我们以简化柱状图定义的方式定义样式对象。然后我们将自定义样式以及其他元数据传递给 `Bar` 对象：
|
||||
|
||||
在 Pygal 中进行绘制会以一种易于阅读的方式显示。首先，我们以一种简化柱状图定义的方式定义样式对象。然后我们将自定义样式以及其他元数据传递给 `Bar` 对象：
|
||||
|
||||
```
|
||||
import pygal
|
||||
from pygal.style import Style
|
||||
|
||||
custom_style = Style(
|
||||
colors=('#0343df', '#e50000', '#ffff14', '#929591'),
|
||||
font_family='Roboto,Helvetica,Arial,sans-serif',
|
||||
background='transparent',
|
||||
label_font_size=14,
|
||||
colors=('#0343df', '#e50000', '#ffff14', '#929591'),
|
||||
font_family='Roboto,Helvetica,Arial,sans-serif',
|
||||
background='transparent',
|
||||
label_font_size=14,
|
||||
)
|
||||
|
||||
c = pygal.Bar(
|
||||
title="UK Election Results",
|
||||
style=custom_style,
|
||||
y_title='Seats',
|
||||
width=1200,
|
||||
x_label_rotation=270,
|
||||
title="UK Election Results",
|
||||
style=custom_style,
|
||||
y_title='Seats',
|
||||
width=1200,
|
||||
x_label_rotation=270,
|
||||
)
|
||||
```
|
||||
|
||||
然后，我们将数据`添加`到 `Bar` 对象中：
|
||||
|
||||
然后，我们将数据添加到 `Bar` 对象中：
|
||||
|
||||
```
|
||||
c.add('Conservative', df['conservative'])
|
||||
@@ -88,9 +82,8 @@ c.x_labels = df['year']
|
||||
|
||||
最后，我们将图另存为 SVG 文件：
|
||||
|
||||
|
||||
```
|
||||
`c.render_to_file('pygal.svg')`
|
||||
c.render_to_file('pygal.svg')
|
||||
```
|
||||
|
||||
结果是一个交互式 SVG 图，你可以在此 gif 中看到：
|
||||
@@ -101,12 +94,7 @@ c.x_labels = df['year']
|
||||
|
||||
### 总结
|
||||
|
||||
、Python 中的某些绘图工具需要非常详细地构建每个对象，而 Pygal 从一开始就为你提供这些。如果你手边有数据并且想做一个干净、漂亮、简单的交互式图表，请尝试一下 Pygal。
|
||||
|
||||
|
||||
\---
|
||||
|
||||
_本文最初发表于[此][8]，并获得许可编辑并重新发布。_
|
||||
Python 中的某些绘图工具需要非常详细地构建每个对象，而 Pygal 从一开始就为你提供这些。如果你手边有数据并且想做一个干净、漂亮、简单的交互式图表，请尝试一下 Pygal。
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
@@ -115,7 +103,7 @@ via: https://opensource.com/article/20/6/pygal-python
|
||||
作者：[Shaun Taylor-Morgan][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[geekpi](https://github.com/geekpi)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
校对：[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
@@ -0,0 +1,107 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: (Yufei-Yan)
|
||||
[#]: reviewer: (wxy)
|
||||
[#]: publisher: (wxy)
|
||||
[#]: url: (https://linux.cn/article-12365-1.html)
|
||||
[#]: subject: (Customize your Raspberry Pi operating system for everyday use)
|
||||
[#]: via: (https://opensource.com/article/20/6/custom-raspberry-pi)
|
||||
[#]: author: (Patrick H. Mullins https://opensource.com/users/pmullins)
|
||||
|
||||
定制用于日常使用的树莓派系统
|
||||
======
|
||||
|
||||
> 安装精简版树莓派系统，让你的树莓派更加轻盈。
|
||||
|
||||
|
||||
|
||||
你有一个运行<ruby>[树莓派系统][2]<rt>Raspberry Pi OS</rt></ruby>（以前称为 Raspbian）操作系统的<ruby>树莓派<rt>Raspberry Pi</rt></ruby>，你肯定知道，它是一个非常棒的小型电脑，有一个很好的操作系统，对于初学者来说，它包括了你可能想要的一切。然而，一旦你熟悉了它，并且想用它干一些别的事情的时候，你可能不想要那个默认囊括了一切组件的操作系统。
|
||||
|
||||
在这种情况下，你两个选择：要么你可以绞尽脑汁地把所有你不需要的东西都删干净，要么你还可以用<ruby>精简版树莓派系统<rt>Raspberry Pi OS Lite</rt></ruby>来构建专门为你的需求而定制的轻量级操作系统。我的建议是，用第二种方法，既简便又节省时间。
|
||||
|
||||
### Raspberry Pi OS Lite
|
||||
|
||||
“<ruby>精简版<rt>Lite</rt></ruby>”的<ruby>树莓派系统<rt>Raspberry Pi OS</rt></ruby>其实没什么特别的，就是一个基于最新版本的 [Debian][3] 的最小化镜像。这个镜像仅包含了操作系统核心和启动到命令行的部分，而不会进入图形桌面。可以将这个作为你的定制树莓派系统的基础。这之后的所有东西都是建立在这个核心之上的。
|
||||
|
||||
前往树莓派基金会的网站上[下载][2]这个轻量级镜像。下载完成后，查看详细的[安装指南][4]，这里面介绍了在 Linux、Windows 或者 macOS 下如何烧制树莓派操作系统的 SD 卡。
|
||||
|
||||
如果你计划用树莓派作为一个极简系统来运行一些脚本和服务的话，差不多这么多就够了。如果你还想干更多事，那继续往下读。
|
||||
|
||||
### X Window
|
||||
|
||||
首先，如果偶尔需要通过图形用户界面（GUI）连接到你的树莓派，安装一个窗口系统还是不错的。
|
||||
|
||||
[X Window 系统][5]，有时候称为 X11，是 Unix 操作系统上一个常见的基本窗口系统。X11 提供了一套 GUI 桌面环境的基本框架。它可以让你通过窗口、鼠标和键盘与计算机交互。
|
||||
|
||||
#### 安装 X Window
|
||||
|
||||
下面这一行安装了能让 X11 运行的最少的包。
|
||||
|
||||
```
|
||||
sudo apt install -y --no-install-recommends xserver-xorg-core xserver-xorg xfonts-base xinit
|
||||
```
|
||||
|
||||
如果使用 `--no-install-recommends`，则只安装了主要的一些依赖（`Depends` 字段中的包）。这样可以节省很多空间，因为没有安装那些建议却不一定需要的包。
|
||||
|
||||
### 进阶：使用 Xfce 桌面环境
|
||||
|
||||
如果你愿意，可以就此停下了，然后开始使用 X Window 作为你的桌面。不过，我并不建议这么做。X Window 自带的这种最小化的窗口管理工具走的是极简主义风格，某种程度上让人感觉过时了。相反，我建议安装现代化的桌面环境，比如说像 Xfce、GNOME 或者 KDE。当用在微型计算机上时，我更倾向于 [Xfce][6] 而不是其他的，因为它就是为资源有限的系统设计的，而且你可以通过主题、图标或者其他东西对它进行定制。
|
||||
|
||||
#### 安装 Xfce
|
||||
|
||||
安装 Xfce 桌面环境相当简单。只需要：
|
||||
|
||||
```
|
||||
sudo apt install -y --no-install-recommends xfce4 desktop-base lightdm
|
||||
```
|
||||
|
||||
这就够了。你现在安装了 X Window（X11）和 Xfce 了。现在是时候来定制一下环境并且安装一些核心应用了。
|
||||
|
||||
### 核心应用
|
||||
|
||||
目前为止，你已经安装了 X Window（X11）、Xfce 桌面环境和 LightDM（一个 Xfce 自带的显示管理器）。现在，你已经有了一个可以启动并且正常使用的轻量级的完整系统。不过，在彻底完成之前，我还是喜欢装一些核心应用。
|
||||
|
||||
下面这条命令安装了一个终端程序、[Audacious][7] 音频播放器、[Ristretto][8] 图像浏览器、[Mousepad][9] 文本编辑器、[File Roller][10] 存档管理器和 [Thunar][11] 容量管理器。
|
||||
|
||||
```
|
||||
sudo apt install -y --no-install-recommends xfce4-terminal audacious ristretto
|
||||
sudo apt install -y --no-install-recommends mousepad file-roller thunar-volman
|
||||
```
|
||||
|
||||
#### 其他可选项
|
||||
|
||||
其他一些你可能想安装的东西，包括一个好的网络管理器、任务管理器、PDF 阅读器和通知工具，以及桌面背景管理器、截图工具、一些新的图标和光标主题。简单来说，如果树莓派是你的首选系统，这些都算是日常工作的一些补充。
|
||||
|
||||
```
|
||||
sudo apt install -y --no-install-recommends network-manager xfce4-taskmanager xfce4-notifyd
|
||||
sudo apt install -y --no-install-recommends xpdf gnome-icon-theme dmz-cursor-theme
|
||||
```
|
||||
|
||||
### 下一步该做什么？
|
||||
|
||||
如果一切都正常工作的话，你现在就有一个基于 Xfce 和 Debian Lite 超轻量级操作系统的树莓派了。我建议现在你去 Xfce 网站上查看其它很酷的好东西，这些你都可以安装并使用。下一步做什么完全由你决定！
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensource.com/article/20/6/custom-raspberry-pi
|
||||
|
||||
作者：[Patrick H. Mullins][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[Yufei-Yan](https://github.com/Yufei-Yan)
|
||||
校对：[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://opensource.com/users/pmullins
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/raspberrypi_cartoon.png?itok=m3TcBONJ (Cartoon graphic of Raspberry Pi board)
|
||||
[2]: https://www.raspberrypi.org/downloads/raspberry-pi-os/
|
||||
[3]: https://www.debian.org/
|
||||
[4]: https://www.raspberrypi.org/documentation/installation/installing-images/README.md
|
||||
[5]: https://en.wikipedia.org/wiki/X_Window_System
|
||||
[6]: http://xfce.org
|
||||
[7]: https://audacious-media-player.org/
|
||||
[8]: https://docs.xfce.org/apps/ristretto/start
|
||||
[9]: https://github.com/codebrainz/mousepad
|
||||
[10]: https://gitlab.gnome.org/GNOME/file-roller
|
||||
[11]: https://docs.xfce.org/xfce/thunar/thunar-volman
|
||||
[12]: https://goodies.xfce.org/
|
||||
@@ -0,0 +1,66 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
[#]: subject: (NIST aims to make frequency sharing more efficient for wireless networks)
|
||||
[#]: via: (https://www.networkworld.com/article/3561618/nist-aims-to-make-frequency-sharing-more-efficient-for-wireless-networks.html)
|
||||
[#]: author: (Patrick Nelson https://www.networkworld.com/author/Patrick-Nelson/)
|
||||
|
||||
NIST aims to make frequency sharing more efficient for wireless networks
|
||||
======
|
||||
Machine-learning formula will help different radio protocols, such as Wi-Fi and LTE, work together more efficiently in the same wireless spectrum.
|
||||
Martyn Williams/IDG
|
||||
|
||||
A machine-learning formula developed by the National Institute of Standards and Technology ([NIST][1]) has the potential to significantly improve how [5G][2] and other wireless networks select and share communications frequencies. Compared to trial-and-error methods, NIST's formula could make the process of sharing communications frequencies as much as 5,000 times more efficient, researchers claim.
|
||||
|
||||
The NIST system is based the idea that radio equipment can learn its network environments from experience rather than, as is done now, simply select frequency channels based on trial-and-error.
|
||||
|
||||
"The algorithm learns which channel provides the best outcome" under specific environmental conditions, NIST says in an [article on its website][3].
|
||||
|
||||
**READ MORE:** [How beamforming makes wireless communication faster][4]
|
||||
|
||||
"The formula could be programmed into software on transmitters in many [different] types of real-world networks," the team says.
|
||||
|
||||
Essentially, the computer-modeled algorithm is a formula that maps prior experience in environmental RF conditions. Those conditions can include the number of transmitters operating within a channel (set of adjacent frequencies), for example.
|
||||
|
||||
"… if a transmitter selects a channel that is not occupied, then the probability of a successful transmission rises, leading to a higher data rate," the article says. Likewise, when a transmitter selects a channel that doesn't have much interference on it, the signal is stronger, and you get a better data rate. The transmitter remembers which channel provides the best outcome and learns to choose that spot on the dial when it next needs a clear signal.
|
||||
|
||||
That's different from the way things generally work today. That is, a radio simply tries to find an open frequency and then communicates with like-protocol radios. In sophisticated cases, like Wi-Fi, for example, frequency hopping and [beamforming][4] are used to optimize channels.
|
||||
|
||||
Where NIST's machine-learning technique shines is in the case of shared spectrum, such as sharing Wi-Fi with License Assisted Access (LAA), the researchers explain. LAA is LTE in unlicensed spectrum, known as LTE-U, at 5 GHz. In that combination of Wi-Fi with LAA, on the same frequencies, the protocols are disparate: the radios don't communicate with each other to function in harmony, and chaos could occur the busier the band got—transmissions would bump into other transmissions. But, if all the radios were better at choosing their slot, by learning what works and what doesn't, then things would be better.
|
||||
|
||||
"This could potentially make communications in the unlicensed bands much more efficient," says Jason Coder, a NIST engineer, in the article.
|
||||
|
||||
Indeed, it "could help 5G and other wireless networks select and share communications frequencies about 5,000 times more efficiently than trial-and-error methods," NIST claims.
|
||||
|
||||
The key word here is "share," because in order to increase communications in limited spectrum, more sharing must take place—the users, such as IoT, or media streaming, are all competing for the same metaphorical real estate. Combining unlicensed and licensed bands, as is the case in LAA, will likely become more common as IoT and digital continues to expand. (Unlicensed bands are those not assigned to a specific user, like a mobile network operator; licensed bands are won in auctions and allocated.)
|
||||
|
||||
In the NIST scenario, the competing transmitters "each learn to maximize the total network data rate without communicating with each other." Therefore, multiple protocols and data types, like video or sensor data, or Wi-Fi and mobile networks, can function alongside each other.
|
||||
|
||||
NIST's formula significantly simplifies the process of assigning optimum channels to transmitters, according to the article: "The study found that an exhaustive effort [using trial and error] to identify the best solution would require about 45,600 trials, whereas the formula could select a similar solution by trying only 10 channels, just 0.02 percent of the effort."
|
||||
|
||||
The NIST researchers recently presented their research at [IEEE's 91st Vehicular Technology Conference][5], held virtually this year.
|
||||
|
||||
Join the Network World communities on [Facebook][6] and [LinkedIn][7] to comment on topics that are top of mind.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://www.networkworld.com/article/3561618/nist-aims-to-make-frequency-sharing-more-efficient-for-wireless-networks.html
|
||||
|
||||
作者：[Patrick Nelson][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://www.networkworld.com/author/Patrick-Nelson/
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://www.nist.gov/
|
||||
[2]: https://www.networkworld.com/article/3330603/5g-versus-4g-how-speed-latency-and-application-support-differ.html
|
||||
[3]: https://www.nist.gov/news-events/news/2020/05/nist-formula-may-help-5g-wireless-networks-efficiently-share-communications
|
||||
[4]: https://www.networkworld.com/article/3445039/beamforming-explained-how-it-makes-wireless-communication-faster.html
|
||||
[5]: https://events.vtsociety.org/vtc2020-spring/conference-sessions/program/
|
||||
[6]: https://www.facebook.com/NetworkWorld/
|
||||
[7]: https://www.linkedin.com/company/network-world
|
||||
@@ -0,0 +1,77 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
[#]: subject: (A minimalist Mac terminal for Linux fans)
|
||||
[#]: via: (https://opensource.com/article/20/6/iterm2-mac)
|
||||
[#]: author: (Lisa Seelye https://opensource.com/users/lisa)
|
||||
|
||||
A minimalist Mac terminal for Linux fans
|
||||
======
|
||||
Here is how I keep a terminal simple and my dotfiles secure through a
|
||||
lot of subtle complexity.
|
||||
![Coffee and laptop][1]
|
||||
|
||||
I have a confession to make: I have been a Mac user for more than 10 years now. At first, I felt a little shame, given my strong Linux background, but the Mac gives me a Unix-like shell and a great window manager. Because of that history, I have a mix of features that will run on macOS but feel familiar to Linux users. There's no reason it can't port over to Linux (and it has!).
|
||||
|
||||
### Using iTerm2 on a Mac
|
||||
|
||||
For a long time, my preferred terminal was the basic built-in Terminal.app, but I recently switched to [iTerm2][2] because it has much better customization and profile support. One of its key wins for me is that it's easy to transplant settings from Mac to Mac. For daily use, I prefer the Solarized Dark theme, but for presentations, I have a separate profile that enlarges the text and uses a plain black background with more vibrant colors.
|
||||
|
||||
The first thing I do to make iTerm2 usable is to configure the **Ctrl+Left** and **Ctrl+Right** arrows to respect the classic terminal behavior of jumping to the start and end of a word boundary. To do so, navigate to Preferences > Profiles > Your Profile > Keys and enter the following.
|
||||
|
||||
* Keyboard Shortcut: ^←
|
||||
* Action: Send Escape Sequence
|
||||
* Esc+: b
|
||||
|
||||
|
||||
|
||||
Then the other:
|
||||
|
||||
* Keyboard Shortcut: ^→
|
||||
* Action: Send Escape Sequence
|
||||
* Esc+: f
|
||||
|
||||
|
||||
|
||||
Learn more about what you can do with [iTerm2][3] and enjoy the custom experience.
|
||||
|
||||
### A simple command prompt
|
||||
|
||||
I am one of those boring terminal prompt users. I don't include Git directory or exit code, and I only use a single line. The only fancy component I use is [kubectx][4], which includes the current Kubernetes context. As an [OpenShift Dedicated][5] Site Reliability Engineer (SRE), I have to run commands with the appropriate context, and `kubectx` makes it easy to know where I am when I'm typing. So, my Bash PS1 is the boring `username@host cwd $`, save for the Kubernetes context prefix.
|
||||
|
||||
There is no doubt that I'm on the minimalist side, compared to some fancy terminals I've seen. Some people enjoy transparency, and others prefer a lot of information on their prompts—from the time to the exit code and everything else. I find it distracting in my terminals, so I enjoy those setups from afar.
|
||||
|
||||
### Beautifully complex dotfiles
|
||||
|
||||
Compared to my minimalist terminal, it's easy to see where I put my maximalist efforts: deploying my [dotfiles][6], including my `.bash_profile` and my overall Mac setup.
|
||||
|
||||
I use a [series of Makefiles][7], hosted through GitHub, to manage my Mac setup. This pulls in my [dotfile-specific deployment mechanism][8], which is also in GitHub. Why all the tooling around security you ask? IT professionals and hobbyists alike need a robust way to put secure pieces of data on new systems. Maybe you prefer your SSH config to be hidden, or maybe you're deploying credentials through a third-party system. I find it useful to keep my secure data with everything else, and I solved this problem with [Ansible Vault][9]. All my secrets are stored in Git, encrypted with Ansible Vault. Decryption is handled with Makefiles.
|
||||
|
||||
Whether I'm installing for the first time or updating existing dotfiles, I (of course) must have Ansible Vault, and to avoid having to install that everywhere, I put it in a container that I run with Docker, which I do have installed everywhere. I put the decryption passphrase into a file, `run make`, and clean up everything with `make clean`. (You can learn more by [exploring the dotfiles][8].)
|
||||
|
||||
I will say that this management scheme may be over the top, but some folks like complicated terminal prompts. So perhaps in the balance, it all evens out.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensource.com/article/20/6/iterm2-mac
|
||||
|
||||
作者：[Lisa Seelye][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://opensource.com/users/lisa
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/coffee_cafe_brew_laptop_desktop.jpg?itok=G-n1o1-o (Coffee and laptop)
|
||||
[2]: https://www.iterm2.com/
|
||||
[3]: https://www.iterm2.com/documentation.html
|
||||
[4]: https://github.com/ahmetb/kubectx
|
||||
[5]: https://www.openshift.com/products/dedicated/
|
||||
[6]: https://opensource.com/article/19/3/move-your-dotfiles-version-control
|
||||
[7]: https://github.com/lisa/mac-setup
|
||||
[8]: https://github.com/lisa/dotrc
|
||||
[9]: https://docs.ansible.com/ansible/latest/user_guide/vault.html
|
||||
336
sources/tech/20200629 Scaling a GraphQL Website.md
Normal file
336
sources/tech/20200629 Scaling a GraphQL Website.md
Normal file
@@ -0,0 +1,336 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
[#]: subject: (Scaling a GraphQL Website)
|
||||
[#]: via: (https://theartofmachinery.com/2020/06/29/scaling_a_graphql_site.html)
|
||||
[#]: author: (Simon Arneaud https://theartofmachinery.com)
|
||||
|
||||
Scaling a GraphQL Website
|
||||
======
|
||||
|
||||
For obvious reasons, I normally write abstractly about work I’ve done for other people, but I’ve been given permission to write about a website, [Vocal][1], that I did some SRE work on last year. I actually gave [a presentation at GraphQL Sydney back in February][2], but for various reasons it’s taken me this long to get it into a blog post.
|
||||
|
||||
Vocal is a GraphQL-based website that got traction and hit scaling problems that I got called in to fix. Here’s what I did. Obviously, you’ll find this post useful if you’re scaling another GraphQL website, but most of it’s representative of what you have to deal with when a site gets enough traffic to cause technical problems. If website scalability is a key interest of yours, you might want to read [my recent post about scalability][3] first.
|
||||
|
||||
### Vocal
|
||||
|
||||
![][4]
|
||||
|
||||
Vocal is a blogging platform publishing everything from diaries to movie reviews to opinion pieces to recipes to professional and amateur photography to beauty and lifestyle tips and poetry. Of course, there’s no shortage of proud pet owners with cute cat and dog pictures.
|
||||
|
||||
![][5]
|
||||
|
||||
One thing that’s a bit different about Vocal is that it lets everyday people get paid for producing works that viewers find interesting. Authors get a small amount of money per page view, and can also receive donations from other users. There are professionals using the platform to show off their work, but for most users it’s just a fun hobby that happens to make some extra pocket money as a bonus.
|
||||
|
||||
Vocal is the product of [Jerrick Media][6], a New Jersey startup. Development started in 2015 in collaboration with [Thinkmill][7], a medium-sized Sydney software development consultancy that specialises in all things JavaScript, React and GraphQL.
|
||||
|
||||
### Some spoilers for the rest of this post
|
||||
|
||||
I was told that unfortunately I can’t give hard traffic numbers for legal reasons, but publicly available information can give an idea. Alexa ranks all websites it knows of by traffic level. Here’s a plot of Alexa rank I showed in my talk, showing growth from November 2019 up to getting ranked number 5,567 in the world by February.
|
||||
|
||||
![Vocal global Alexa rank rising from#9,574 in November 2019 to #5,567 in February 2020.][8]
|
||||
|
||||
It’s normal for the curve to slow down because it requires more and more traffic to win each position. Vocal is now at around #4,900. Obviously there’s a long way to go, but that’s not shabby at all for a startup. Most startups would gladly swap their Alexa rank with Vocal.
|
||||
|
||||
Shortly after the site was upgraded, Jerrick Media ran a marketing campaign that doubled traffic. All we had to do on the technical side was watch numbers go up in the dashboards. In the past 9 months since launch, there have only been two platform issues needing staff intervention: [the once-in-five-years AWS RDS certificate rotation that landed in March][9], and an app rollout hitting a Terraform bug. I’ve been very happy with how little platform busywork is needed to keep Vocal running.
|
||||
|
||||
Here’s an overview of the technical stuff I’ll talk about in this post:
|
||||
|
||||
* Technical and historical background
|
||||
* Database migration from MongoDB to Postgres
|
||||
* Deployment infrastructure revamp
|
||||
* Making the app compatible with scaling
|
||||
* Making HTTP caching work
|
||||
* Miscellaneous performances tweaks
|
||||
|
||||
|
||||
|
||||
### Some background
|
||||
|
||||
Thinkmill built a website using [Next.js][10] (a React-based web framework), talking to a GraphQL API provided by [Keystone][11] in front of MongoDB. Keystone is a GraphQL-based headless CMS library: you define a schema in JavaScript, hook it up to some data storage, and get an automatically generated GraphQL API for data access. It’s a free and open-source software project that’s commercially backed by Thinkmill.
|
||||
|
||||
#### Vocal V2
|
||||
|
||||
The version 1 of Vocal got traction. It found a userbase that liked the product, and it grew, and eventually Jerrick Media asked Thinkmill to help develop a version 2, which was successfully launched in September last year. The Jerrick Media folk avoided the [second system effect][12] by generally basing changes on user feedback, so they were [mostly UI and feature changes that I won’t go into][13]. Instead, I’ll talk about the stuff I was brought in for: making the new site more robust and scalable.
|
||||
|
||||
For the record, I’m thankful that I got to work with Jerrick Media and Thinkmill on Vocal, and that they let me present this story, but [I’m still an independent consultant][14]. I wasn’t paid or even asked to write this post, and this is still my own personal blog.
|
||||
|
||||
### The database migration
|
||||
|
||||
Thinkmill suffered several scalability problems with using MongoDB for Vocal, and decided to upgrade Keystone to version 5 to take advantage of its new Postgres support.
|
||||
|
||||
If you’ve been in tech long enough to remember the “NoSQL” marketing from the end of the 00s, that might surprise you. The message was that relational (SQL) databases like Postgres aren’t as scalable as “webscale” NoSQL databases like MongoDB. It’s technically true, but the scalability of NoSQL databases comes from compromises in the variety of queries that can be efficiently handled. Simple, non-relational databases (like document and key-value databases) have their places, but when used as a general-purpose backend for an app, the app often outgrows the querying limitations of the database before it outgrows the theoretical scaling limit a relational database would have. Most of Vocal’s DB queries worked just fine with MongoDB, but over time more and more queries needed hacks to work at all.
|
||||
|
||||
In terms of technical requirements, Vocal is very similar to Wikipedia, one of the biggest sites in the world. Wikipedia runs on MySQL (or rather, its fork, MariaDB). Sure, some significant engineering is needed to make that work, but I don’t see relational databases being a serious threat to Vocal’s scaling in the foreseeable future.
|
||||
|
||||
At one point I checked, the managed AWS RDS Postgres instances cost less than a fifth of the old MongoDB instances, yet CPU usage of the Postgres instances was still under 10%, despite serving more traffic than the old site. That’s mostly because of a few important queries that just never were efficient under the document database architecture.
|
||||
|
||||
The migration could be a blog post of its own, but basically a Thinkmill dev built an [ETL pipeline][15] using [MoSQL][16] to do the heavy lifting. Thanks to Keystone being a FOSS project, I was also able to contribute some performance improvements to its GraphQL to SQL mapping. For that kind of stuff, I always recommend Markus Winand’s SQL blogs: [Use the Index Luke][17] and [Modern SQL][18]. His writing is friendly and accessible to non-experts, yet has most of the theory you need for writing fast and effective SQL. A good, DB-specific book on performance gives you the rest.
|
||||
|
||||
### The platform
|
||||
|
||||
#### The architecture
|
||||
|
||||
V1 was a couple of Node.js apps running on a single virtual private server (VPS) behind Cloudflare as a CDN. I’m a fan of avoiding overengineering as a high priority, so that gets a thumbs up from me. However, by the time V2 development started, it was obvious that Vocal had outgrown that simple architecture. It didn’t give Thinkmillers many options when handling big traffic spikes, and it made updates hard to deploy safely and without downtime.
|
||||
|
||||
Here’s the new architecture for V2:
|
||||
|
||||
![Architecture of Vocal V2. Requests come through a CDN to a load balancer in AWS. The load balancer distributes traffic to two apps, "Platform" and "Website". "Platform" is a Keystone app storing data in Redis and Postgres.][19]
|
||||
|
||||
Basically, the two Node.js apps have been replicated and put behind a load balancer. Yes, that’s it. In my SRE work, I often meet engineers who expect a scalable architecture to be more complicated than that, but I’ve worked on sites that are orders of magnitude bigger than Vocal but are still just replicated services behind load balancers, with DB backends. If you think about it, if the platform architecture needs to keep getting significantly more complicated as the site grows, it’s not really very scalable. Website scalability is mostly about fixing the many little implementation details that prevent scaling.
|
||||
|
||||
Vocal’s architecture might need a few additions if traffic grows enough, but the main reason it would get more complicated is new features. For example, if (for some reason) Vocal needed to handle real-time geospatial data in future, that would be a very different technical beast from blog posts, so I’d expect architectural changes for it. Most of the complexity in big site architecture is because of feature complexity.
|
||||
|
||||
If you don’t know how to make your architecture scalable, I always recommend keeping it as simple as you can. Fixing an architecture that’s too simple is easier and cheaper than fixing an architecture that’s too complex. Also, an unnecessarily complex architecture is more likely to have mistakes, and those mistakes will be harder to debug.
|
||||
|
||||
By the way, Vocal happened to be split into two apps, but that’s not important. A common scaling mistake is to prematurely split an app into smaller services in the name of scalability, but split the app in the wrong place and cause more scalability problems overall. Vocal could have scaled okay as a monolithic app, but the split is also in a good place.
|
||||
|
||||
#### The infrastructure
|
||||
|
||||
Thinkmill has a few people who have experience working with AWS, but it’s primarily a dev shop and needed something more “hands off” than the old Vocal deployment. I ended up deploying the new Vocal on [AWS Fargate][20], which is a relatively new backend to Elastic Container Service (ECS). In the old days, many people wanted ECS to be a simple “run my Docker container as a managed service” product, and were disappointed that they still had to build and manage their own server cluster. With ECS Fargate, AWS manages the cluster. It supports running Docker containers with the basic nice things like replication, health checking, rolling updates, autoscaling and simple alerting.
|
||||
|
||||
A good alternative would have been a managed Platform-as-a-Service (PaaS) like App Engine or Heroku. Thinkmill was already using them for simple projects, but often needed more flexibility with other projects. There are much bigger sites running on PaaSes, but Vocal is at a scale where a custom cloud deployment can make sense economically.
|
||||
|
||||
Another obvious alternative would have been Kubernetes. Kubernetes has a lot more features than ECS Fargate, but it’s a lot more expensive — both in resource overhead, and the staffing needed for maintenance (such as regular node upgrades). As a rule, I don’t recommend Kubernetes to any place that doesn’t have dedicated DevOps staff. Fargate has the features Vocal needs, and has let Thinkmill and Jerrick Media focus on website improvements, not infrastructure busywork.
|
||||
|
||||
Yet another option was “Serverless” function products like AWS Lambda or Google Cloud Functions. They’re great for handling services with very low or highly irregular traffic, but (as I’ll explain) ECS Fargate’s autoscaling is enough for Vocal’s backend. Another plus of these products is that they allow developers to deploy things in cloud environments without needing to learn a lot about cloud environments. The tradeoff is that the Serverless product becomes tightly coupled to the development process, and to the testing and debugging processes. Thinkmill already had enough AWS expertise in-house to manage a Fargate deployment, and any dev who knows how to make a Node.js Express Hello World app can work on Vocal without learning anything about either Serverless functions or Fargate.
|
||||
|
||||
An obvious downside of ECS Fargate is vendor lock-in. However, avoiding vendor lock-in is a tradeoff like avoiding downtime. If you’re worried about migrating, it doesn’t make sense to spend more on platform independence than you would on a migration. The total amount of Fargate-specific code in Vocal is <500 lines of [Terraform][21]. The most important thing is that the Vocal app code itself is platform agnostic. It can run on normal developer machines, and then be packaged up into a Docker container that can run practically anywhere a Docker container can, including ECS Fargate.
|
||||
|
||||
Another downside of Fargate is that it’s not trivial to set up. Like most things in AWS, it’s in a world of VPCs, subnets, IAM policies, etc. Fortunately, that kind of stuff is quite static (unlike a server cluster that requires maintenance).
|
||||
|
||||
### Making a scaling-ready app
|
||||
|
||||
There’s a bunch of stuff to get right if you want to run an app painlessly at scale. You’re doing well if you follow [the Twelve-Factor App design][22], so I won’t repeat it here.
|
||||
|
||||
There’s no point building a “scalable” system if staff can’t operate it at scale — that’s like putting a jet engine on a unicycle. An important part of making Vocal scalable was setting up stuff like CI/CD and [infrastructure as code][23]. Similarly, some deployment ideas aren’t worth it because they make production too different from the development environment (see also [point #10 of the Twelve-Factor App][24]). Every difference between production and development slows app development and can be expected to lead to a bug eventually.
|
||||
|
||||
### Caching
|
||||
|
||||
Caching is a really big topic — I once gave [a presentation on just HTTP caching][25], and that still wasn’t enough. I’ll stick to the essentials for GraphQL here.
|
||||
|
||||
First, an important warning: Whenever you have performance problems, you might wonder, “Can I make this faster by putting this value into a cache for future reuse?” **Microbenchmarks will practically _always_ tell you the answer is “yes”.** However, putting caches everywhere will tend to make your overall system **slower**, thanks to problems like cache coherency. Here’s my mental checklist for caching:
|
||||
|
||||
1. Ask if the performance problem needs to be solved with caching
|
||||
2. Really ask (non-caching performance wins tend to be more robust)
|
||||
3. Ask if the problem can be solved by improving existing caches
|
||||
4. If all else fails, maybe add a new cache
|
||||
|
||||
|
||||
|
||||
One cache system you’ll always have is the HTTP caching system, so a corollary is that it’s a good idea to use HTTP caching effectively before trying to add extra caches. I’ll focus on that in this post.
|
||||
|
||||
Another very common trap is using a hash map or something inside the app for caching. [It works great in local development but performs badly when scaled.][26] The best thing is to use an explicit caching library that supports pluggable backends like Redis or Memcached.
|
||||
|
||||
#### The basics
|
||||
|
||||
There are two types of caches in the HTTP spec: private and public. Private caches are caches that don’t share data with multiple users — in practice, the user’s browser cache. Public caches are all the rest. They include ones under your control (such as CDNs or servers like Varnish or Nginx) and ones that aren’t (proxies). Proxy caches are rarer in today’s HTTPS world, but some corporate networks have them.
|
||||
|
||||
![][27]
|
||||
|
||||
Caching lookup keys are normally based on URLs, so caching is less painful if you stick to a “same content, same URL; different content, different URL” rule. I.e., give each page a canonical URL, and avoid “clever” tricks returning varying content from one URL. Obviously, this has implications for GraphQL API endpoints (that I’ll discuss later).
|
||||
|
||||
Your servers can take custom configuration, but the primary way to configure HTTP caching is through HTTP headers you set on web responses. The most important header is `cache-control`. The following says that all caches down the line may cache the page for up to 3600 seconds (one hour):
|
||||
|
||||
```
|
||||
cache-control: max-age=3600, public
|
||||
```
|
||||
|
||||
For user-specific pages (such as user settings pages), it’s important to use `private` instead of `public` to tell public caches not to store the response and serve it to other users.
|
||||
|
||||
Another common header is `vary`. This tells caches that the response varies based on some things other than the URL. (Effectively it adds HTTP headers to the the cache key, alongside the URL.) It’s a very blunt tool, which is why I recommend using a good URL structure instead if possible, but an important use case is telling browsers that the response depends on the login cookie, so that they update pages on login/logout.
|
||||
|
||||
```
|
||||
vary: cookie
|
||||
```
|
||||
|
||||
If a page can vary based on login status, you need `cache-control: private` (and `vary: cookie`) even on the public, logged out version, to make sure responses don’t get mixed up.
|
||||
|
||||
Other useful headers include `etag` and `last-modified`, but I won’t cover them here. You might still see some old headers like `expires` and `pragma: cache`. They were made obsolete by HTTP/1.1 back in 1997, so I only use them if I want to disable caching and I’m feeling paranoid.
|
||||
|
||||
#### Clientside headers
|
||||
|
||||
Less well known is that the HTTP spec allows `cache-control` headers to be used in client requests to reduce the cache time and get a fresher response. Unfortunately `max-age` greater than 0 doesn’t seem to be widely supported by browsers, but `no-cache` can be useful if you sometimes need a fresh response after an update.
|
||||
|
||||
#### HTTP caching and GraphQL
|
||||
|
||||
As above, the normal cache key is the URL. But GraphQL APIs often use just one endpoint (let’s call it `/api/`). If you want a GraphQL query to be cachable, you need the query and its parameters to appear in the URL path, like `/api/?query={user{id}}&variables={"x":99}` (ignoring URL escaping). The trick is to configure your GraphQL client to use HTTP GET requests for queries (e.g., [set `useGETForQueries` for `apollo-link-http`][28]).
|
||||
|
||||
Mutations mustn’t be cached, so they still need to use HTTP POST requests. With POST requests, caches will only see `/api/` as the URL path, but caches will refuse to cache POST requests outright. Remember: GET for non-mutating queries, POST for mutations. There’s a case where you might want to avoid GET for a query: if the query variables contain sensitive information. URLs have a habit of appearing in log files, browser history and chat channels, so sensitive information in URLs is usually a bad idea. Things like authentication should be done as non-cachable mutations, anyway, so this is a rare case, but one worth remembering.
|
||||
|
||||
Unfortunately, there’s a problem: GraphQL queries tend to be much larger than REST API URLs. If you simply switch on GET-based queries, you’ll get some pretty big URLs, easily bigger than the ~2000 byte limit before some popular browsers and servers just won’t accept them. A solution is to send some kind of query ID, instead of sending the whole query. (I.e., something like `/api/?queryId=42&variables={"x":99}`.) Apollo GraphQL server supports two ways of doing this.
|
||||
|
||||
One way is to [extract all the GraphQL queries from the code and build a lookup table that’s shared serverside and clientside][29]. One downside is that it makes the build process more complicated. Another downside is that it couples the client project to the server project, which goes against a selling point of GraphQL. Yet another downside is that version X of your code might recognise a different set of queries from version Y of your code. This is a problem because 1) your replicated app will serve multiple versions during an update rollout, or rollback, and 2) clients might use cached JavaScript, even as you upgrade or downgrade the server.
|
||||
|
||||
Another way is what Apollo GraphQL calls [Automatic Persisted Queries (APQs)][30]. With APQs, the query ID is a hash of the query. The client optimistically makes a request to the server, referring to the query by hash. If the server doesn’t recognise the query, the client sends the full query in a POST request. The server stores that query by hash so that it can be recognised in future.
|
||||
|
||||
![][31]
|
||||
|
||||
#### HTTP caching and Keystone 5
|
||||
|
||||
As above, Vocal uses Keystone 5 for generating its GraphQL API, and Keystone 5 works with Apollo GraphQL server. How do we actually set the caching headers?
|
||||
|
||||
Apollo supports cache hints on GraphQL schemas. The neat thing is that Apollo gathers all the hints for everything that’s touched by a query, and then it automatically calculates the appropriate overall cache header values. For example, take this query:
|
||||
|
||||
```
|
||||
query userAvatarUrl {
|
||||
authenticatedUser {
|
||||
name
|
||||
avatar_url
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
If `name` has a max age of one day, and the `avatar_url` has a max age of one hour, the overall cache max age would be the minimum, one hour. `authenticatedUser` depends on the login cookie, so it needs a `private` hint, which overrides the `public` on the other fields, so the resulting header would be `cache-control: max-age=3600, private`.
|
||||
|
||||
I added [cache hint support to Keystone lists and fields][32]. Here’s a simple example of adding a cache hint to a field in the to-do list demo from the docs:
|
||||
|
||||
```
|
||||
const keystone = new Keystone({
|
||||
name: 'Keystone To-Do List',
|
||||
adapter: new MongooseAdapter(),
|
||||
});
|
||||
|
||||
keystone.createList('Todo', {
|
||||
schemaDoc: 'A list of things which need to be done',
|
||||
fields: {
|
||||
name: {
|
||||
type: Text,
|
||||
schemaDoc: 'This is the thing you need to do',
|
||||
isRequired: true,
|
||||
cacheHint: {
|
||||
scope: 'PUBLIC',
|
||||
maxAge: 3600,
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
#### One more problem: CORS
|
||||
|
||||
Cross-Origin Resource Sharing (CORS) rules create a frustrating conflict with caching in an API-based website.
|
||||
|
||||
Before getting stuck into the problem details, let me jump to the easiest solution: putting the main site and API onto one domain. If your site and API are served from one domain, you won’t have to worry about CORS rules (but you might want to consider [restricting cookies][33]). If your API is specifically for the website, this is the cleanest solution, and you can happily skip this section.
|
||||
|
||||
In Vocal V1, the Website (Next.js) and Platform (Keystone GraphQL) apps were on different domains (`vocal.media` and `api.vocal.media`). To protect users from malicious websites, modern browsers don’t just let one website interact with another. So, before allowing `vocal.media` to make requests to `api.vocal.media`, the browser would make a “pre-flight” check to `api.vocal.media`. This is an HTTP request using the `OPTIONS` method that essentially asks if the cross-origin sharing of resources is okay. After getting the okay from the pre-flight check, the browser makes the normal request that was originally intended.
|
||||
|
||||
The frustrating thing about pre-flight checks is that they are per-URL. The browser makes a new `OPTIONS` request for each URL, and the server response applies to that URL. [The server can’t say that `vocal.media` is a trusted origin for all `api.vocal.media` requests][34]. This wasn’t a serious problem when everything was a POST request to the one api endpoint, but after giving every query its own GET-able URL, every query got delayed by a pre-flight check. For extra frustration, the HTTP spec says `OPTIONS` requests can’t be cached, so you can find that all your GraphQL data is beautifully cached in a CDN right next to the user, but browsers still have to make pre-flight requests all the way to the origin server every time they use it.
|
||||
|
||||
There are a few solutions (if you can’t just use a shared domain).
|
||||
|
||||
If your API is simple enough, you might be able to exploit the [exceptions to the CORS rules][35].
|
||||
|
||||
Some cache servers can be configured to ignore the HTTP spec and cache `OPTIONS` requests anyway (e.g., Varnish-based caches and AWS CloudFront). This isn’t as efficient as avoiding the pre-flight requests completely, but it’s better than the default.
|
||||
|
||||
Another (really hacky) option is [JSONP][36]. Beware: you can create security bugs if you don’t get this right.
|
||||
|
||||
#### Making Vocal more cachable
|
||||
|
||||
After making HTTP caching work at the low level, I needed to make the app take better advantage of it.
|
||||
|
||||
A limitation of HTTP caching is that it’s all-or-nothing at the response level. Most of a response can be cachable, but if a single byte isn’t, all bets are off. As a blogging platform, most Vocal data is highly cachable, but in the old site almost no _pages_ were cachable at all because of a menu bar in the top right corner. For an anonymous user, the menu bar would show links inviting the user to log in or create an account. That bar would change to a user avatar and profile menu for signed-in users. Because the page varied based on user login status, it wasn’t possible to cache any of it in CDNs.
|
||||
|
||||
![A typical page from Vocal. Most of the page is highly cachable content, but in the old site none of it was actually cachable because of a little menu in the top right corner.][37]
|
||||
|
||||
These pages are generated by Server-Side Rendering (SSR) of React components. The fix was to take all the React components that depended on the login cookie, and force them to be [lazily rendered clientside only][38]. Now the server returns completely generic pages with placeholders for personalised components like the login menu bar. When a page loads in the user’s browser, these placeholders are filled in clientside by making calls to the GraphQL API. The generic pages can be safely cached in CDNs.
|
||||
|
||||
Not only does this trick improve cache hit ratios, it helps improve perceived page load time thanks to human psychology. Blank screens and even spinner animations make us impatient, but once the first content appears, it distracts us for several hundred milliseconds. If people click a Vocal post link from social media and the main content appears immediately from a CDN, very few will ever notice that some components aren’t fully interactive until a few hundred milliseconds later.
|
||||
|
||||
By the way, another trick for getting the first content in front of the user faster is to [stream render the SSR response as it’s generated][39], instead of waiting for the whole page to be rendered before sending it. Unfortunately, [Next.js doesn’t support that yet][40].
|
||||
|
||||
The idea of splitting responses for improved cachability also applies to GraphQL. The ability to query multiple pieces of data with one request is normally an advantage of GraphQL, but if the different parts of the response have very different cachability, it can be better overall to split them. As a simple example, Vocal’s pagination component needs to know the number of pages plus the content for the current page. Originally the component fetched both in one query, but because the total number of pages is a constant across all pages, I made it a separate query so it can be cached.
|
||||
|
||||
#### Benefits of caching
|
||||
|
||||
The obvious benefit of caching is that it reduces the load on Vocal’s backend servers. That’s good, but it’s dangerous to rely on caching for capacity, though, because you still need a backup plan for when you inevitably drop the cache one day.
|
||||
|
||||
The improved responsiveness is a better reason for caching.
|
||||
|
||||
A couple of other benefits might be less obvious. Traffic spikes tend to be highly localised. If someone with a lot of social media followers shares a link to a page, Vocal will get a big surge of traffic, but mostly to that one page and its assets. That’s why caches are good at absorbing the worst traffic spikes, making the backend traffic patterns relatively smoother and easier for autoscaling to handle.
|
||||
|
||||
Another benefit is graceful degradation. Even if the backends are in serious trouble for some reason, the most popular parts of the site will still be served from the CDN cache.
|
||||
|
||||
### Other performance tweaks
|
||||
|
||||
As I always say, the secret to scaling isn’t making things complicated. It’s making things no more complicated than needed, and then thoroughly fixing all the things that prevent scaling. Scaling Vocal involved a lot of little things that won’t fit in this post.
|
||||
|
||||
Here’s one tip: for the difficult debugging problems in distributed systems, the hardest part is usually getting the right data to see what’s going on. I can think of plenty of times that I’ve got stuck and tried to just “wing it” by guessing instead of figuring out how to find the right data. Sometimes that works, but not for the hard problems.
|
||||
|
||||
A related tip is that you can learn a lot by getting real-time data (even just log files under [`tail -F`][41]) on each component in a system, displaying it in various windows in one monitor, and just clicking around the site in another. I’m talking about things like, “Hey, why does toggling this one checkbox generate dozens of DB queries in the backend?”
|
||||
|
||||
Here’s an example of one fix. Some pages were taking more than a couple of seconds to render, but only in the deployment environment, and only with SSR. The monitoring dashboards didn’t show any CPU usage spikes, and the apps weren’t using disk, so it suggested that maybe the app was waiting on network requests, probably to a backend. In a dev environment I could watch how the app worked using [the sysstat tools][42] to record CPU/RAM/disk usage, along with Postgres statement logging and the usual app logs. [Node.js supports probes for tracing HTTP requests][43] using something like [bpftrace][44], but boring reasons meant they didn’t work in the dev environment, so instead I found the probes in the source code and made a custom Node.js build with request logging. I used [tcpdump][45] to record network data. That let me find the problem: for every API request made by Website, a new network connection was being created to Platform. (If that hadn’t worked, I guess I would have added request tracing to the apps.)
|
||||
|
||||
Network connections are fast on a local machine, but take non-negligible time on a real network. Setting up an encrypted connection (like in the production environment) takes even longer. If you’re making lots of requests to one server (like an API), it’s important to keep the connection open and reuse it. Browsers do that automatically, but Node.js doesn’t by default because it can’t know if you’re making more requests. That’s why the problem only appeared with SSR. Like many long debugging sessions, the fix was very simple: just configure SSR to [keep connections alive][46]. The rendering time of the slower pages dropped dramatically.
|
||||
|
||||
If you want to know more about this kind of stuff, I highly recommend reading [the High Performance Browser Networking book][47] (free to read online) and following up with [guides Brendan Gregg has published][48].
|
||||
|
||||
### What about your site?
|
||||
|
||||
There’s actually a lot more stuff we could have done to improve Vocal, but we didn’t do it all. That’s a big difference between doing SRE work for a startup and doing it for a big company as a permanent employee. We had goals, a budget and a launch date, and now Vocal V2 has been running for 9 months with a healthy growth rate.
|
||||
|
||||
Similarly, your site will have its own requirements, and is likely quite different from Vocal. However, I hope this post and its links give you at least some useful ideas to make something better for users.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://theartofmachinery.com/2020/06/29/scaling_a_graphql_site.html
|
||||
|
||||
作者：[Simon Arneaud][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://theartofmachinery.com
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://vocal.media
|
||||
[2]: https://www.meetup.com/en-AU/GraphQL-Sydney/events/267681845/
|
||||
[3]: https://theartofmachinery.com/2020/04/21/what_is_high_traffic.html
|
||||
[4]: https://theartofmachinery.com/images/scaling_a_graphql_site/vocal1.png
|
||||
[5]: https://theartofmachinery.com/images/scaling_a_graphql_site/vocal2.png
|
||||
[6]: https://jerrick.media
|
||||
[7]: https://www.thinkmill.com.au/
|
||||
[8]: https://theartofmachinery.com/images/scaling_a_graphql_site/alexa.png
|
||||
[9]: https://aws.amazon.com/blogs/database/amazon-rds-customers-update-your-ssl-tls-certificates-by-february-5-2020/
|
||||
[10]: https://github.com/vercel/next.js
|
||||
[11]: https://www.keystonejs.com/
|
||||
[12]: https://wiki.c2.com/?SecondSystemEffect
|
||||
[13]: https://vocal.media/resources/vocal-2-0
|
||||
[14]: https://theartofmachinery.com/about.html
|
||||
[15]: https://en.wikipedia.org/wiki/Extract,_transform,_load
|
||||
[16]: https://github.com/stripe/mosql
|
||||
[17]: https://use-the-index-luke.com/
|
||||
[18]: https://modern-sql.com/
|
||||
[19]: https://theartofmachinery.com/images/scaling_a_graphql_site/architecture.svg
|
||||
[20]: https://aws.amazon.com/fargate/
|
||||
[21]: https://www.terraform.io/docs/providers/aws/r/ecs_task_definition.html
|
||||
[22]: https://12factor.net/
|
||||
[23]: https://theartofmachinery.com/2019/02/16/talks.html
|
||||
[24]: https://12factor.net/dev-prod-parity
|
||||
[25]: https://www.meetup.com/en-AU/Port80-Sydney/events/lwcdjlyvjblb/
|
||||
[26]: https://theartofmachinery.com/2016/07/30/server_caching_architectures.html
|
||||
[27]: https://theartofmachinery.com/images/scaling_a_graphql_site/http_caches.svg
|
||||
[28]: https://www.apollographql.com/docs/link/links/http/#options
|
||||
[29]: https://www.apollographql.com/blog/persisted-graphql-queries-with-apollo-client-119fd7e6bba5
|
||||
[30]: https://www.apollographql.com/blog/improve-graphql-performance-with-automatic-persisted-queries-c31d27b8e6ea
|
||||
[31]: https://theartofmachinery.com/images/scaling_a_graphql_site/apq.png
|
||||
[32]: https://www.keystonejs.com/api/create-list/#cachehint
|
||||
[33]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies#Define_where_cookies_are_sent
|
||||
[34]: https://lists.w3.org/Archives/Public/public-webapps/2012AprJun/0236.html
|
||||
[35]: https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS#Simple_requests
|
||||
[36]: https://en.wikipedia.org/wiki/JSONP
|
||||
[37]: https://theartofmachinery.com/images/scaling_a_graphql_site/cachablepage.png
|
||||
[38]: https://nextjs.org/docs/advanced-features/dynamic-import#with-no-ssr
|
||||
[39]: https://medium.com/the-thinkmill/progressive-rendering-the-key-to-faster-web-ebfbbece41a4
|
||||
[40]: https://github.com/vercel/next.js/issues/1209
|
||||
[41]: https://linux.die.net/man/1/tail
|
||||
[42]: https://github.com/sysstat/sysstat/
|
||||
[43]: http://www.brendangregg.com/blog/2016-10-12/linux-bcc-nodejs-usdt.html
|
||||
[44]: https://theartofmachinery.com/2019/04/26/bpftrace_d_gc.html
|
||||
[45]: https://danielmiessler.com/study/tcpdump/
|
||||
[46]: https://www.npmjs.com/package/agentkeepalive
|
||||
[47]: https://hpbn.co/
|
||||
[48]: http://www.brendangregg.com/
|
||||
@@ -0,0 +1,600 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
[#]: subject: (Use intent parsers for your open source home automation project)
|
||||
[#]: via: (https://opensource.com/article/20/6/mycroft-intent-parsers)
|
||||
[#]: author: (Steve Ovens https://opensource.com/users/stratusss)
|
||||
|
||||
Use intent parsers for your open source home automation project
|
||||
======
|
||||
Use Padatious, Adapt, conversational context, and dialogs to start
|
||||
programming your first voice AI.
|
||||
![Working from home at a laptop][1]
|
||||
|
||||
In [part 1][2] and [part 2][3] of this series on the [Mycroft][4] open source voice assistant, I laid the groundwork for learning how to create a skill. In [part 3][5], I walked through creating an outline for a skill and recommended creating the skill in pure Python first to ensure the methods work as intended. That way, when things go wrong, you know it is related to how your Mycroft skill is constructed and not the code itself.
|
||||
|
||||
In this article, you will enhance the outline from part 3 by adding:
|
||||
|
||||
* Mycroft entities
|
||||
* Padatious intents
|
||||
* Adapt intents
|
||||
* Dialogs
|
||||
* Conversational contexts
|
||||
|
||||
|
||||
|
||||
The code for this project can be found in [my GitLab][6] repository.
|
||||
|
||||
Let's get to it!
|
||||
|
||||
### Improve your skill
|
||||
|
||||
As a reminder, the purpose of this project is to use Mycroft to add items to shopping lists in the [OurGroceries][7] app. However, this tutorial can be applied to a variety of home automation applications like turning on your lights, getting the morning weather report, or controlling your entertainment systems.
|
||||
|
||||
Here's what the skill's outline looks like so far:
|
||||
|
||||
|
||||
```
|
||||
from mycroft import intent_file_handler, MycroftSkill, intent_handler
|
||||
from mycroft.skills.context import adds_context, removes_context
|
||||
|
||||
class OurGroceriesSkill(MycroftSkill):
|
||||
def __init__(self):
|
||||
MycroftSkill.__init__(self)
|
||||
|
||||
# Mycroft should call this function directly when the user
|
||||
# asks to create a new item
|
||||
def create_item_on_list(self, message):
|
||||
pass
|
||||
|
||||
# Mycroft should also call this function directly
|
||||
def create_shopping_list(self, message):
|
||||
pass
|
||||
|
||||
# This is not called directly, but instead should be triggered
|
||||
# as part of context aware decisions
|
||||
def handle_dont_create_anyways_context(self):
|
||||
pass
|
||||
|
||||
# This function is also part of the context aware decision tree
|
||||
def handle_create_anyways_context(self):
|
||||
pass
|
||||
|
||||
def stop(self):
|
||||
pass
|
||||
|
||||
def create_skill():
|
||||
return OurGroceriesSkill()
|
||||
```
|
||||
|
||||
As it stands, Mycroft will load the skill successfully, but it won't do anything because all the methods have the command `pass` in them. For now, ignore the `__init__(self)` method and start working with the `create_item_on_list` method. From the comments, you can see the intent is for Mycroft to call this method directly. That means that you need to declare an _intent_. How do you do that?
|
||||
|
||||
### Working with intents
|
||||
|
||||
You may have noticed that the [outline][8] `mycroft-msk` created (in the third article) has what looks like a weird function, `@intent_file_handler('ourgroceries.intent')`, above the `handle_test` method. These are special notations in Python called decorators (if you want, head over to Real Python for a [primer on Python decorators][9]). For this tutorial, it is sufficient to know that a decorator is a way to pass your function into a prebuilt function developed by Mycroft. This saves a lot of work and boilerplate code.
|
||||
|
||||
Recall from part three of this series that this project uses two intent parsers: Padacious and Adapt, which I described in the second article.
|
||||
|
||||
#### Padatious intents
|
||||
|
||||
So how do you know which decorator to use and where to use it? Great questions! I'll start with Padatious, which is the easier of the two to understand. If you recall from the second article, [Padatious][10] skills are _trained_ based on phrases that the skill's developer decides are relevant for the skill. As Mycroft may install many skills using the Padatious intent engine, each intent is given a score by the neural network module that Mycroft employs. Mycroft then chooses the highest-scoring intent and executes its functions.
|
||||
|
||||
The phrases which Mycroft will use to train the intent are placed inside a file with a `.intent` file extension. You can have multiple `.intent` files, but you must reference each file explicitly. That means if you have `create.item.intent` and `create.category.intent`, there will be no confusion about which file your variables are populated from, as you must call them by file name. As you can see in the output from `mycroft-msk`, the decorator is intuitively named `@intent_file_handler()`. Simply use the name of the file as the argument for the decorator, such as `@intent_file_handler("create.item.intent")`.
|
||||
|
||||
Think about what phrases someone might use to add an item to a shopping list. Since the motivating factor for this skill was using Mycroft to create a grocery list, the example code uses food-related terms, but you could use generic terms. With that said, here are some phrases you might say to add an item to your grocery list:
|
||||
|
||||
* Add tomatoes to my shopping list
|
||||
* Add tomatoes to the grocery list
|
||||
* Add tomatoes to Costco list
|
||||
|
||||
|
||||
|
||||
You may choose to have some grammatically incorrect phrases as well, to account for Mycroft misunderstanding the user's voice. From the list above, what pieces of information are programmatically relevant? `tomatoes`, `shopping list`, `grocery list`, and `Costco list`. The official documentation refers to this type of object as an _entity_. You can think of an entity as a variable, if that makes more sense to you. This will become clearer later when you create an intent file. While the `mycroft-msk` command will put intents in `locale/en-us` by default, I put mine under `vocab/en-us/`. Why? Well, that's because the Adapt intent parser stores its files in `vocab`, and I prefer to keep all my intent files in the same location. My file `vocab/en-us/create.item.intent` starts with:
|
||||
|
||||
|
||||
```
|
||||
`add {Food} to my {ShoppingList}`
|
||||
```
|
||||
|
||||
This defines the _entities_ `Food` and `ShoppingList`.
|
||||
|
||||
**IMPORTANT NOTE**: Padatious entities are **not** case-sensitive, and Padatious interprets everything in lower-case. For example, `ShoppingList` will be `shoppinglist`.
|
||||
|
||||
Now that you have an intent, have Mycroft say a phrase containing your _entities_. Don't forget to add the intent decorator! Your new function will look like this:
|
||||
|
||||
|
||||
```
|
||||
@intent_file_handler("create.item.intent")
|
||||
def create_item_on_list(self, message):
|
||||
"""
|
||||
This function adds an item to the specified list
|
||||
|
||||
:param message:
|
||||
:return: Nothing
|
||||
"""
|
||||
item_to_add = message.data.get('food')
|
||||
list_name = message.data.get('shoppinglist')
|
||||
self.speak("Adding %s to %s" % (item_to_add, list_name))
|
||||
```
|
||||
|
||||
The graphic below uses three phrases:
|
||||
|
||||
* Add tomatoes to my shopping list
|
||||
* Add nails to my hardware list
|
||||
* Add buns to groceries list
|
||||
|
||||
|
||||
|
||||
Mycroft will not be able to figure out the intent behind one of these phrases. Can you guess which one and why?
|
||||
|
||||
![Mycroft processing intent][11]
|
||||
|
||||
(Steve Ovens, [CC BY-SA 4.0][12])
|
||||
|
||||
In case the video was a little too fast for you, here's the answer: Mycroft can't process the phrase `add buns to groceries list` because it is missing the keyword `my`. The intent explicitly says `add {Food} to my {ShoppingList}`. Without the word `my` as input from the user, the Padatious intent for the skill scores low and, therefore, Mycroft doesn't choose this skill to handle the request. The simplest solution is to add a new line to your intent file, like this:
|
||||
|
||||
|
||||
```
|
||||
`add {Food} to {ShoppingList}`
|
||||
```
|
||||
|
||||
Mycroft can often reload skills when it detects a change, but I prefer to restart Mycroft's skills section to make sure. I also clear the logs a lot during testing, so I run the following command to do everything in one line:
|
||||
|
||||
|
||||
```
|
||||
` ./stop-mycroft.sh skills;sudo rm -f /var/log/mycroft/skills.log; ./start-mycroft.sh skills; mycroft-cli-client`
|
||||
```
|
||||
|
||||
Testing the skill after Mycroft restarts yields the following results:
|
||||
|
||||
|
||||
```
|
||||
add buns to groceries list
|
||||
>> Adding buns to groceries list
|
||||
```
|
||||
|
||||
In case it is not clear, any response Mycroft makes in the `mycroft-cli-client` is prefixed by `>>` to indicate its response. Now that you have a basic intent, go back and review the objectives for this skill from part 3 of this series:
|
||||
|
||||
1. Login/authenticate
|
||||
2. Get a list of the current grocery lists
|
||||
3. Add item to a specific grocery list
|
||||
4. Add item to a category under a specific list
|
||||
5. Be able to add a category (since OurGroceries allows items to be placed in categories)
|
||||
|
||||
|
||||
|
||||
Ignore the first two items for now—those deal with the online portion of the project, and you need to complete the other objectives first. For the third item, you have a basic intent that, in theory, should be able to take the _entities_ Mycroft detects and turn them into variables in the Python code. For the fourth item on the list, add two new lines to your intent:
|
||||
|
||||
|
||||
```
|
||||
add {Food} to my {ShoppingList} under {Category}
|
||||
add {Food} to {ShoppingList} under {Category}
|
||||
```
|
||||
|
||||
You also need to alter your function slightly. When using the Padatious intent parser, _entities_ are returned via the `message.data.get()` function. This function will return `None` if the entity is undefined. In other words, if Mycroft cannot parse `{Category}` from an `utterance` the user makes, `message.data.get()` will return `None`. With this in mind, here is some quick test code:
|
||||
|
||||
|
||||
```
|
||||
@intent_file_handler("create.item.intent")
|
||||
def create_item_on_list(self, message):
|
||||
"""
|
||||
This function adds an item to the specified list
|
||||
|
||||
:param message:
|
||||
:return: Nothing
|
||||
"""
|
||||
item_to_add = message.data.get('food')
|
||||
list_name = message.data.get('shoppinglist')
|
||||
category_name = message.data.get('category')
|
||||
if category_name is None:
|
||||
self.speak("Adding %s to %s" % (item_to_add, list_name))
|
||||
else:
|
||||
self.speak("Adding %s to %s under the category %s" % (item_to_add, list_name, category_name))
|
||||
```
|
||||
|
||||
Here is an example that tests these code changes:
|
||||
|
||||
![Mycroft testing code changes][13]
|
||||
|
||||
(Steve Ovens, [CC BY-SA 4.0][12])
|
||||
|
||||
In the example, Mycroft responds with `>> Adding nails to my hardware list under`, yet the only time you told Mycroft to say the word `under` is when `category_name` has a value other than `None`. This is because the intent parser is interpreting the word `under` as a part of the entity `ShoppingList`. Because the utterance had the word `my` in it, the sentences that matched the utterances could have been either:
|
||||
|
||||
1. `add {Food} to my {ShoppingList}`
|
||||
or
|
||||
2. `add {Food} to my {ShoppingList} under {Category}`
|
||||
|
||||
|
||||
|
||||
Since the user did not state the `{Category}`, Mycroft selected the first statement as the most correct. This means that anything _after_ the word `my` would be cast into the entity `{ShoppingList}`. Since, therefore, `{Category}` is `None`, Mycroft speaks, "Adding nails to my hardware list under" instead of "Adding nails to my hardware list under None."
|
||||
|
||||
Padatious may seem a bit simplistic at first. For each phrase you need Mycroft to match, simply add a line to the intent file. However, with a complex intent, you could have several dozen lines attempting to cover all the different utterances you want to handle.
|
||||
|
||||
There is another option that may be worth considering. Padatious intents support _parentheses expansion_. This means you can use a form of _OR_ statements to cut down on the number for lines in an intent. Going back, the example is trying to account for three cases:
|
||||
|
||||
|
||||
```
|
||||
add {Food} to my {ShoppingList}
|
||||
add {Food} to my {ShoppingList} under {Category}
|
||||
add {Food} to the {ShoppingList}
|
||||
add {Food} to the {ShoppingList} under {Category}
|
||||
add {Food} to {ShoppingList}
|
||||
add {Food} to {ShoppingList} under {Category}
|
||||
```
|
||||
|
||||
If you want to rewrite this using an _OR_ statement to combine the `my` and `the` keywords, you could write:
|
||||
|
||||
|
||||
```
|
||||
add {Food} to ( my | the ) {ShoppingList}
|
||||
add {Food} to ( my | the ) {ShoppingList} under {Category}
|
||||
add {Food} to {ShoppingList}
|
||||
add {Food} to {ShoppingList} under {Category}
|
||||
```
|
||||
|
||||
This removes two lines from the intent. Parentheses expansion also supports making something optional. So, if you want to make `the` and `my` optional and thus allow for the phrase `add {Food} to {ShoppingList}`, it would look like:
|
||||
|
||||
|
||||
```
|
||||
add {Food} to ( | my | the ) {ShoppingList}
|
||||
add {Food} to ( | my | the ) {ShoppingList} under {Category}
|
||||
```
|
||||
|
||||
This simple change covers all three scenarios (once you restart the Mycroft skills subsystem). You can go a step further and condense this into a single line if you wish:
|
||||
|
||||
|
||||
```
|
||||
`add {Food} to ( | my | the ) {ShoppingList} ( | under {Category})`
|
||||
```
|
||||
|
||||
**Note:** For easier readability, use white spaces in your intent parentheses expansion.
|
||||
|
||||
To summarize the important points about Padatious intent parsing:
|
||||
|
||||
* You have to give several examples of phrases for Mycroft to come up with the correct match.
|
||||
* Padatious intents use entities such as `{Food}` to identify object values that can be retrieved from your Python code.
|
||||
* Entities are _always lower-case_ regardless of how you declare them in the intent file.
|
||||
* If an entity cannot be parsed from an utterance, its value is `None`.
|
||||
* The decorator for Padatious intents is `@intent_file_handler('my.intent.file.intent')`.
|
||||
|
||||
|
||||
|
||||
#### Adapt intents
|
||||
|
||||
Unlike Padatious intents, where you specify entities in the intent file, the Adapt intent parser works with a series of keywords that work in combination with regular expression (regex) files to attempt to capture an entity. You would use Adapt over Padatious when you:
|
||||
|
||||
1. Expect the utterance to be complex and the more robust parsing of regex is required
|
||||
2. Want or need Mycroft to be context-aware
|
||||
3. Need intents to be as lightweight as possible
|
||||
|
||||
|
||||
|
||||
That said, the `voc` files Adapt uses are quite flexible. They can include a single word (as shown in the [official documentation][14]), or they can include the start of a sentence you want to react to.
|
||||
|
||||
As one goal of this project is to have Mycroft create a new shopping list in the OurGroceries app, I wanted to add in some rudimentary checking so that the user is informed if a list with a similar name exists and asked if they still want to create a new list. This should cut down on list duplication and misplacement of items.
|
||||
|
||||
Mock up some code, and then you can deal with the vocab and regex files. While you could use Pytest or similar unit tests to assert specific values, for the sake of simplicity, you will create a list called "shopping list." The Python mock function will look like this:
|
||||
|
||||
|
||||
```
|
||||
def create_shopping_list(self, message):
|
||||
fake_list = ["shopping list"]
|
||||
self.new_shopping_list_name = message.data['ListName'].lower()
|
||||
for current_shopping_list in fake_list:
|
||||
try:
|
||||
if self.new_shopping_list_name in current_shopping_list:
|
||||
if self.new_shopping_list_name == current_shopping_list:
|
||||
self.speak("The shopping list %s already exists" % self.new_shopping_list_name )
|
||||
break
|
||||
else:
|
||||
self.speak("I found a similar naming list called %s" % current_shopping_list)
|
||||
# This hands off to either handle_dont_create_anyways_context or handle_create_anyways_context
|
||||
# to make a context aware decision
|
||||
self.speak("Would you like me to add your new list anyways?", expect_response=True)
|
||||
break
|
||||
else:
|
||||
self.speak("Ok creating a new list called %s" % self.new_shopping_list_name)
|
||||
except Exception as ex:
|
||||
print(ex)
|
||||
pass
|
||||
```
|
||||
|
||||
Notice I am using a `forloop` to iterate over `fake_list`. That is because, in theory, multiple lists will be returned from the OurGroceries app. Also note the `try/except` block; I have given a general pass on the exception because, right now, I don't know what kind of exceptions I may run into. As you use and debug your code, you can tighten this up a bit.
|
||||
|
||||
Another line to note is:
|
||||
|
||||
|
||||
```
|
||||
`self.speak("Would you like me to add your new list anyways?", expect_response=True) `
|
||||
```
|
||||
|
||||
This bit of code will have Mycroft prompt the user for a response and store the result. I'll talk more about this bit of code in the conversational contexts section.
|
||||
|
||||
#### Regular expressions, entities, and Adapt intents
|
||||
|
||||
Now you have some pseudo code, but you need to add the decorator for Mycroft to action your code. You need to create three files for this to work: two vocab files and one regex file. The regex file, which I will name `add.shopping.list.rx`, looks like this:
|
||||
|
||||
|
||||
```
|
||||
start a new list called (?P<ListName>.*)
|
||||
create a new list called (?P<ListName>.*)
|
||||
add a new list called (?P<ListName>.*)
|
||||
```
|
||||
|
||||
You could make this a one-liner, but for simplicity's sake, keep it as three lines. Note this strange-looking notation: `(?P<ListName>.*)`. This is the part of the code that captures and creates the entity. The entity, in this case, is called `ListName`. For checking your syntax, I recommend [Pythex][15]. It is very helpful when I am debugging my regex (I'm pretty terrible at regex).
|
||||
|
||||
**IMPORTANT NOTE:** Adapt intents are case sensitive.
|
||||
|
||||
#### Adapt and vocab files
|
||||
|
||||
Now that your regex includes the full sentences you expect, create your two vocab files. The first file is called `CreateKeyword.voc`. As you can surmise from the file name, all the words you want to associate with the `create` action should reside here. This file is very simple:
|
||||
|
||||
|
||||
```
|
||||
start a new
|
||||
create a new
|
||||
add a new
|
||||
```
|
||||
|
||||
In the documentation, you will often see only a single word per line. However, due to some Mycroft default skills using `start` and `create`, I need to add words so that Mycroft will pick my skill appropriately.
|
||||
|
||||
The second file is even easier. It's called `ListKeyword.voc` and has a single word in it:
|
||||
|
||||
|
||||
```
|
||||
`list`
|
||||
```
|
||||
|
||||
With these files defined, you can now construct your decorator:
|
||||
|
||||
|
||||
```
|
||||
`@intent_handler(IntentBuilder('CreateShoppingIntent').require('CreateKeyword').require('ListKeyword').require("ListName"))`
|
||||
```
|
||||
|
||||
The first argument in the `IntentBuilder` is `'CreateShoppingIntent'`; this is the name of the intent and is completely optional. If you want to leave this blank, you can. The `require` section is a bit confusing. When it comes to keywords, the argument for `require` is the name of the file without the file extension. In this case, one of the files is called `ListKeyword.voc`, so the argument being passed into `require` is just `'ListKeyword'`.
|
||||
|
||||
While you can name your vocab files anything you want, I highly recommend using the word `Keyword` in the file so that when you are building your `intent_handler` decorator, it is clear what you are requiring.
|
||||
|
||||
If `require` is actually an entity from a regex file, the argument for `require` is the name of the entity as you defined it in the regex. If your regex was `start a new list called (?P<NewList>.*)`, then you would write `require('NewList')`.
|
||||
|
||||
Restart the Mycroft skills subsection and try it out. You should see this in the Mycroft command-line interface:
|
||||
|
||||
|
||||
```
|
||||
add a new list called hardware
|
||||
>> Ok creating a new list called hardware
|
||||
|
||||
create a new list called hardware
|
||||
>> Ok creating a new list called hardware
|
||||
|
||||
start a new list called hardware
|
||||
>> Ok creating a new list called hardware
|
||||
```
|
||||
|
||||
#### Conversational contexts
|
||||
|
||||
Great, it works! Now add the following decorator to your function:
|
||||
|
||||
|
||||
```
|
||||
`@adds_context("CreateAnywaysContext")`
|
||||
```
|
||||
|
||||
This decorator is tied to the [conversational context][16] that Mycroft supports. Conversational contexts are essentially where you can speak normally to Mycroft and it will understand your meaning. For example, you could ask: "Who was John Quincy Adams?" After Mycroft responds, saying something like "John Quincy Adams was the sixth president of the United States," you could ask: "How old was he when he became president?" If you ask the second question first, Mycroft has no way to know who the pronoun _he_ refers to. However, in the context of this conversation, Mycroft understands that _he_ refers to John Quincy Adams.
|
||||
|
||||
Getting back to creating a conversational context, the argument for its decorator is the name of the context. This example calls the context `CreateAnywaysContext` and hence, the full decorator is `@adds_context("CreateAnywaysContext")`. This mock method is now complete. However, you now need to add two simple methods to handle the user's feedback. You can simplify the grocery list skill by requiring either a yes or a no answer. Create a `YesKeyword.voc` and a `NoKeyword.voc`, and place the words `yes` and `no` in them, respectively.
|
||||
|
||||
Now create two more methods in your Python:
|
||||
|
||||
|
||||
```
|
||||
@intent_handler(IntentBuilder('DoNotAddIntent').require("NoKeyword").require('CreateAnywaysContext').build())
|
||||
@removes_context("CreateAnywayscontext")
|
||||
def handle_dont_create_anyways_context(self):
|
||||
"""
|
||||
Does nothing but acknowledges the user does not wish to proceed
|
||||
Uses dont.add.response.dialog
|
||||
:return:
|
||||
"""
|
||||
self.speak_dialog('dont.add.response')
|
||||
|
||||
@intent_handler(IntentBuilder('AddAnywaysIntent').require("YesKeyword").require('CreateAnywaysContext').build())
|
||||
@removes_context("CreateAnywayscontext")
|
||||
def handle_create_anyways_context(self):
|
||||
"""
|
||||
If the user wants to create a similarly named list, it is handled here
|
||||
Uses do.add.response.dialog
|
||||
:return:
|
||||
"""
|
||||
self.speak_dialog('do.add.response')
|
||||
```
|
||||
|
||||
There are two things here you have not seen so far:
|
||||
|
||||
1. `@remove_context`
|
||||
2. `self.speak_dialog`
|
||||
|
||||
|
||||
|
||||
If a method that requires `CreateAnywaysContext` is called, the decorator `@remove_context` gets rid of the context so that Mycroft does not accidentally action a context more than once. While multiple contexts can be applied to a method, this project will not use them.
|
||||
|
||||
#### Dialogs
|
||||
|
||||
Dialogs are files that have several prebuilt responses that Mycroft can pick from. These dialogs are stored in `dialog/{language tag}/`, and the language tag is based on the IETF standard. Examples can be found on [Venea.net][17] in the IETF LanguageTag column.
|
||||
|
||||
Mycroft picks randomly from the list of sentences in a specified dialog file. Why would you use a dialog file instead of implementing `self.speak` in Python? The answer is simple: When you create and use a dialog file, you do not have to change the Python code to support other languages.
|
||||
|
||||
For example, if the dialog file called `dont.add.response.dialog` exists under `en-us` with the following content:
|
||||
|
||||
|
||||
```
|
||||
Ok... exiting
|
||||
Gotcha I won't add it
|
||||
Ok I'll disregard it
|
||||
Make up your mind!
|
||||
```
|
||||
|
||||
You could also create `de-de/dont.add.response.dialog` with the following content:
|
||||
|
||||
|
||||
```
|
||||
Ok... Beenden
|
||||
Erwischt Ich werde es nicht hinzufügen
|
||||
Ok, ich werde es ignorieren.
|
||||
Entscheiden Sie sich!
|
||||
```
|
||||
|
||||
In your Python code, you would use `self.speak_dialog('dont.add.response')` to randomly select one of the answers for Mycroft to use. If a user's Mycroft language is set to German, Mycroft will automatically select the correct dialog and play the dialog in German instead of English.
|
||||
|
||||
To wrap up this section, create two files under `dialog/en-us`. For `dont.add.response.dialog`, use the same content as in the above example. For `do.add.response.dialog`, use:
|
||||
|
||||
|
||||
```
|
||||
Ok adding it now
|
||||
Sure thing
|
||||
Yup yup yup
|
||||
```
|
||||
|
||||
At this point in this project, your tree should look something like this:
|
||||
|
||||
|
||||
```
|
||||
├── dialog
|
||||
│ └── en-us
|
||||
│ ├── do.add.response.dialog
|
||||
│ └── dont.add.response.dialog
|
||||
├── __init__.py
|
||||
├── regex
|
||||
│ └── en-us
|
||||
│ └── ADD.shopping.list.rx
|
||||
└── vocab
|
||||
└── en-us
|
||||
├── CREATE.item.intent
|
||||
├── CreateKeyword.voc
|
||||
└── ListKeyword.voc
|
||||
```
|
||||
|
||||
Note that I created the files by hand. If you used the `mycroft-msk create` method, you might have `locale` directories, `settingsmeta.yaml,` or other artifacts.
|
||||
|
||||
#### Wrapping up
|
||||
|
||||
We've covered a lot so far. You have implemented the Padatious intent parser to, theoretically, add a new item to a list, whether or not you put it under a category. You have also used the Adapt intent parser to add a new category. You used conversational context to prompt the user for confirmation if a similar list already exists. Finally, you learned the concept of dialogs as a way for Mycroft to provide varied confirmation responses to the user.
|
||||
|
||||
Currently, the code looks like:
|
||||
|
||||
|
||||
```
|
||||
from mycroft import intent_file_handler, MycroftSkill, intent_handler
|
||||
from mycroft.skills.context import adds_context, removes_context
|
||||
from adapt.intent import IntentBuilder
|
||||
|
||||
class OurGroceriesSkill(MycroftSkill):
|
||||
def __init__(self):
|
||||
MycroftSkill.__init__(self)
|
||||
|
||||
# Mycroft should call this function directly when the user
|
||||
# asks to create a new item
|
||||
@intent_file_handler("create.item.intent")
|
||||
def create_item_on_list(self, message):
|
||||
"""
|
||||
This function adds an item to the specified list
|
||||
|
||||
:param message:
|
||||
:return: Nothing
|
||||
"""
|
||||
item_to_add = message.data.get('food')
|
||||
list_name = message.data.get('shoppinglist')
|
||||
category_name = message.data.get('category')
|
||||
if category_name is None:
|
||||
self.speak("Adding %s to %s" % (item_to_add, list_name))
|
||||
else:
|
||||
self.speak("Adding %s to %s under the category %s" % (item_to_add, list_name, category_name))
|
||||
|
||||
# Mycroft should also call this function directly
|
||||
@intent_handler(IntentBuilder('CreateShoppingIntent').require('CreateKeyword').require('ListKeyword').require("ListName"))
|
||||
def create_shopping_list(self, message):
|
||||
fake_list = ["shopping list"]
|
||||
self.new_shopping_list_name = message.data['ListName'].lower()
|
||||
for current_shopping_list in fake_list:
|
||||
try:
|
||||
if self.new_shopping_list_name in current_shopping_list:
|
||||
if self.new_shopping_list_name == current_shopping_list:
|
||||
self.speak("The shopping list %s already exists" % self.new_shopping_list_name )
|
||||
break
|
||||
else:
|
||||
self.speak("I found a similar naming list called %s" % current_shopping_list)
|
||||
# This hands off to either handle_dont_create_anyways_context or handle_create_anyways_context
|
||||
# to make a context aware decision
|
||||
self.speak("Would you like me to add your new list anyways?", expect_response=True)
|
||||
break
|
||||
else:
|
||||
self.speak("Ok creating a new list called %s" % self.new_shopping_list_name)
|
||||
except AttributeError:
|
||||
pass
|
||||
# This is not called directly, but instead should be triggered
|
||||
# as part of context aware decisions
|
||||
@intent_handler(IntentBuilder('DoNotAddIntent').require("NoKeyword").require('CreateAnywaysContext').build())
|
||||
@removes_context("CreateAnywayscontext")
|
||||
def handle_dont_create_anyways_context(self):
|
||||
"""
|
||||
Does nothing but acknowledges the user does not wish to proceed
|
||||
Uses dont.add.response.dialog
|
||||
:return:
|
||||
"""
|
||||
self.speak_dialog('dont.add.response')
|
||||
|
||||
# This function is also part of the context aware decision tree
|
||||
@intent_handler(IntentBuilder('AddAnywaysIntent').require("YesKeyword").require('CreateAnywaysContext').build())
|
||||
@removes_context("CreateAnywayscontext")
|
||||
def handle_create_anyways_context(self):
|
||||
"""
|
||||
If the user wants to create a similarly named list, it is handled here
|
||||
Uses do.add.response.dialog
|
||||
:return:
|
||||
"""
|
||||
self.speak_dialog('do.add.response')
|
||||
|
||||
def stop(self):
|
||||
pass
|
||||
|
||||
def create_skill():
|
||||
return OurGroceriesSkill()
|
||||
```
|
||||
|
||||
|
||||
|
||||
In the next article, I will go into logging, getting settings from the web UI, and continuing to fill out the skill into something more useful.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensource.com/article/20/6/mycroft-intent-parsers
|
||||
|
||||
作者：[Steve Ovens][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://opensource.com/users/stratusss
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/wfh_work_home_laptop_work.png?itok=VFwToeMy (Working from home at a laptop)
|
||||
[2]: https://opensource.com/article/20/6/open-source-voice-assistant
|
||||
[3]: https://opensource.com/article/20/6/mycroft
|
||||
[4]: https://mycroft.ai/
|
||||
[5]: https://opensource.com/article/20/6/outline-mycroft-voice-assistant-skill
|
||||
[6]: https://gitlab.com/stratus-ss/mycroft-ourgroceries-skill
|
||||
[7]: https://www.ourgroceries.com/overview
|
||||
[8]: https://opensource.com/article/20/6/outline-mycroft-voice-assistant-skill#decorator
|
||||
[9]: https://realpython.com/primer-on-python-decorators/
|
||||
[10]: https://mycroft-ai.gitbook.io/docs/skill-development/user-interaction/intents/padatious-intents
|
||||
[11]: https://opensource.com/sites/default/files/uploads/mycroft-padatious-first-intent.gif (Mycroft processing intent)
|
||||
[12]: https://creativecommons.org/licenses/by-sa/4.0/
|
||||
[13]: https://opensource.com/sites/default/files/uploads/mycroft-padatious-category1.gif (Mycroft testing code changes)
|
||||
[14]: https://mycroft-ai.gitbook.io/docs/skill-development/user-interaction/intents/adapt-intents
|
||||
[15]: https://pythex.org/
|
||||
[16]: https://mycroft-ai.gitbook.io/docs/skill-development/user-interaction/conversational-context
|
||||
[17]: https://www.venea.net/web/culture_code
|
||||
147
sources/tech/20200629 Using Bash traps in your scripts.md
Normal file
147
sources/tech/20200629 Using Bash traps in your scripts.md
Normal file
@@ -0,0 +1,147 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
[#]: subject: (Using Bash traps in your scripts)
|
||||
[#]: via: (https://opensource.com/article/20/6/bash-trap)
|
||||
[#]: author: (Seth Kenlon https://opensource.com/users/seth)
|
||||
|
||||
Using Bash traps in your scripts
|
||||
======
|
||||
Traps help your scripts end cleanly, whether they run successfully or
|
||||
not.
|
||||
![Hands programming][1]
|
||||
|
||||
It's easy to detect when a shell script starts, but it's not always easy to know when it stops. A script might end normally, just as its author intends it to end, but it could also fail due to an unexpected fatal error. Sometimes it's beneficial to preserve the remnants of whatever was in progress when a script failed, and other times it's inconvenient. Either way, detecting the end of a script and reacting to it in some pre-calculated manner is why the [Bash][2] `trap` directive exists.
|
||||
|
||||
### Responding to failure
|
||||
|
||||
Here's an example of how one failure in a script can lead to future failures. Say you have written a program that creates a temporary directory in `/tmp` so that it can unarchive and process files before bundling them back together in a different format:
|
||||
|
||||
|
||||
```
|
||||
#!/usr/bin/env bash
|
||||
CWD=`pwd`
|
||||
TMP=${TMP:-/tmp/tmpdir}
|
||||
|
||||
## create tmp dir
|
||||
mkdir $TMP
|
||||
|
||||
## extract files to tmp
|
||||
tar xf "${1}" --directory $TMP
|
||||
|
||||
## move to tmpdir and run commands
|
||||
pushd $TMP
|
||||
for IMG in *.jpg; do
|
||||
mogrify -verbose -flip -flop $IMG
|
||||
done
|
||||
tar --create --file "${1%.*}".tar *.jpg
|
||||
|
||||
## move back to origin
|
||||
popd
|
||||
|
||||
## bundle with bzip2
|
||||
bzip2 --compress $TMP/"${1%.*}".tar \
|
||||
--stdout > "${1%.*}".tbz
|
||||
|
||||
## clean up
|
||||
/usr/bin/rm -r /tmp/tmpdir
|
||||
```
|
||||
|
||||
Most of the time, the script works as expected. However, if you accidentally run it on an archive filled with PNG files instead of the expected JPEG files, it fails halfway through. One failure leads to another, and eventually, the script exits without reaching its final directive to remove the temporary directory. As long as you manually remove the directory, you can recover quickly, but if you aren't around to do that, then the next time the script runs, it has to deal with an existing temporary directory full of unpredictable leftover files.
|
||||
|
||||
One way to combat this is to reverse and double-up on the logic by adding a precautionary removal to the start of the script. While valid, that relies on brute force instead of structure. A more elegant solution is `trap`.
|
||||
|
||||
### Catching signals with trap
|
||||
|
||||
The `trap` keyword catches _signals_ that may happen during execution. You've used one of these signals if you've ever used the `kill` or `killall` commands, which call `SIGTERM` by default. There are many other signals that shells respond to, and you can see most of them with `trap -l` (as in "list"):
|
||||
|
||||
|
||||
```
|
||||
$ trap --list
|
||||
1) SIGHUP 2) SIGINT 3) SIGQUIT 4) SIGILL 5) SIGTRAP
|
||||
6) SIGABRT 7) SIGBUS 8) SIGFPE 9) SIGKILL 10) SIGUSR1
|
||||
11) SIGSEGV 12) SIGUSR2 13) SIGPIPE 14) SIGALRM 15) SIGTERM
|
||||
16) SIGSTKFLT 17) SIGCHLD 18) SIGCONT 19) SIGSTOP 20) SIGTSTP
|
||||
21) SIGTTIN 22) SIGTTOU 23) SIGURG 24) SIGXCPU 25) SIGXFSZ
|
||||
26) SIGVTALRM 27) SIGPROF 28) SIGWINCH 29) SIGIO 30) SIGPWR
|
||||
31) SIGSYS 34) SIGRTMIN 35) SIGRTMIN+1 36) SIGRTMIN+2 37) SIGRTMIN+3
|
||||
38) SIGRTMIN+4 39) SIGRTMIN+5 40) SIGRTMIN+6 41) SIGRTMIN+7 42) SIGRTMIN+8
|
||||
43) SIGRTMIN+9 44) SIGRTMIN+10 45) SIGRTMIN+11 46) SIGRTMIN+12 47) SIGRTMIN+13
|
||||
48) SIGRTMIN+14 49) SIGRTMIN+15 50) SIGRTMAX-14 51) SIGRTMAX-13 52) SIGRTMAX-12
|
||||
53) SIGRTMAX-11 54) SIGRTMAX-10 55) SIGRTMAX-9 56) SIGRTMAX-8 57) SIGRTMAX-7
|
||||
58) SIGRTMAX-6 59) SIGRTMAX-5 60) SIGRTMAX-4 61) SIGRTMAX-3 62) SIGRTMAX-2
|
||||
63) SIGRTMAX-1 64) SIGRTMAX
|
||||
```
|
||||
|
||||
Any of these signals may be anticipated with `trap`. In addition to these, `trap` recognizes:
|
||||
|
||||
* `EXIT`: Occurs when a process exits
|
||||
* `ERR`: Occurs when a process exits with a non-zero status
|
||||
* `DEBUG`: A Boolean representing debug mode
|
||||
|
||||
|
||||
|
||||
To set a trap in Bash, use `trap` followed by a list of commands you want to be executed, followed by a list of signals to trigger it.
|
||||
|
||||
For instance, this trap detects a `SIGINT`, the signal sent when a user presses **Ctrl+C** while a process is running:
|
||||
|
||||
|
||||
```
|
||||
`trap "{ echo 'Terminated with Ctrl+C'; }" SIGINT`
|
||||
```
|
||||
|
||||
The example script with temporary directory problems can be fixed with a trap detecting `SIGINT`, errors, and successful exits:
|
||||
|
||||
|
||||
```
|
||||
#!/usr/bin/env bash
|
||||
CWD=`pwd`
|
||||
TMP=${TMP:-/tmp/tmpdir}
|
||||
|
||||
trap \
|
||||
"{ /usr/bin/rm -r $TMP ; exit 255; }" \
|
||||
SIGINT SIGTERM ERR EXIT
|
||||
|
||||
## create tmp dir
|
||||
mkdir $TMP
|
||||
tar xf "${1}" --directory $TMP
|
||||
|
||||
## move to tmp and run commands
|
||||
pushd $TMP
|
||||
for IMG in *.jpg; do
|
||||
mogrify -verbose -flip -flop $IMG
|
||||
done
|
||||
tar --create --file "${1%.*}".tar *.jpgh
|
||||
|
||||
## move back to origin
|
||||
popd
|
||||
|
||||
## zip tar
|
||||
bzip2 --compress $TMP/"${1%.*}".tar \
|
||||
--stdout > "${1%.*}".tbz
|
||||
```
|
||||
|
||||
For complex actions, you can simplify `trap` statements with [Bash functions][3].
|
||||
|
||||
### Traps in Bash
|
||||
|
||||
Traps are useful to ensure that your scripts end cleanly, whether they run successfully or not. It's never safe to rely completely on automated garbage collection, so this is a good habit to get into in general. Try using them in your scripts, and see what they can do!
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensource.com/article/20/6/bash-trap
|
||||
|
||||
作者：[Seth Kenlon][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://opensource.com/users/seth
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/programming-code-keyboard-laptop.png?itok=pGfEfu2S (Hands programming)
|
||||
[2]: https://opensource.com/resources/what-bash
|
||||
[3]: https://opensource.com/article/20/6/how-write-functions-bash
|
||||
@@ -1,110 +0,0 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: (Yufei-Yan)
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
[#]: subject: (Customize your Raspberry Pi operating system for everyday use)
|
||||
[#]: via: (https://opensource.com/article/20/6/custom-raspberry-pi)
|
||||
[#]: author: (Patrick H. Mullins https://opensource.com/users/pmullins)
|
||||
|
||||
定制你的树莓派（Raspberry Pi）用于日常使用
|
||||
======
|
||||
|
||||
安装 Raspberry Pi OS Lite 可以更加轻量化你的树莓派（Raspberry Pi）
|
||||
![Cartoon graphic of Raspberry Pi board][1]
|
||||
|
||||
如果你有一个运行 [Raspberry Pi OS][2]（以前称为 Raspbian）操作系统的树莓派（Raspberry Pi）的话，你懂的，他就是一个运行了可以为初学者提供一切可能的伟大操作系统的小电脑，这个电脑简直太棒了。然而，一旦你开始熟悉他并且想开始用他干一些别的事情的时候，你可能不再想要那个默认包含了所有东西的操作系统。
|
||||
|
||||
当这种情况发生的时候，你两个选择：要么你可以尽可能的把所有你不需要的东西都删干净，要么你还可以用 Raspberry Pi OS Lite 来构建专门为你的需求而定制的轻量级操作系统。我的建议是，用第二种方法，既简便又节省时间。
|
||||
|
||||
### Raspberry Pi OS Lite
|
||||
|
||||
Raspberry Pi OS 的“Lite”版本其实没什么特别的，就是一个基于最新 [Debian][3] 版本的最小化镜像。这个镜像仅包含了操作系统核心和启动到命令行的部分，而不会进入图形桌面。考虑用这个作为你的定制 Raspberry Pi OS 的基础吧。这之后的所有东西都是建立在这个核心之上的。
|
||||
|
||||
到树莓派基金会（Raspberry Pi Foundation）的网站上[下载][2]这个轻量级镜像。下载完成后，查看详细的[安装指南][4]，这里面介绍了在 Linux、Windows 或者 macOS 下如何烧制一个树莓派（Raspberry Pi）操作系统的 SD 卡。
|
||||
|
||||
如果你计划用树莓派作为一个最小化操作系统运行一些脚本和服务的话，差不多这么多就够了。如果你还想干更多事，那继续往下读。
|
||||
|
||||
### X Windows
|
||||
|
||||
首先，为了偶尔需要通过用户界面（GUI）连接到你的树莓派（Raspberry Pi），安装一个可视化的系统还是不错的。
|
||||
|
||||
[X Window 系统][5]，有时候称为 X11，是 Unix 操作系统上一个很常见且基础的可视化系统。X11提供了一套 GUI 桌面环境下的基础框架。它可以让你通过窗口、鼠标和键盘与计算机交互。
|
||||
|
||||
#### 安装 X Windows
|
||||
|
||||
下面这一行安装了能让 X11 运行的最少的包。
|
||||
|
||||
```
|
||||
`sudo apt install -y --no-install-recommends xserver-xorg-core xserver-xorg xfonts-base xinit`
|
||||
```
|
||||
|
||||
通过 `--no-install-recommends`，只安装了主要的一些依赖（Depends 字段中的包）。这样可以节省很多空间，因为没有安装那些建议却不需要的包。
|
||||
|
||||
### 进阶：使用 Xfce 桌面环境
|
||||
|
||||
如果你喜欢的话，可以就此停下了，然后开始使用 X Windows 作为你的桌面。然而， 我并不建议这么做。X Windows自带的这种最小化的窗口管理工具某种程度上让人感觉过时了。相反，我建议安装现代化的桌面环境，比如说像 Xfce、GNOME 或者 KDE。当我们说的是微型计算机的时候，我更倾向于 [Xfce][6] 而不是其他的，因为他就是为资源有限的系统设计的，而且你可以通过主题、图标或者其他东西对他进行定制。
|
||||
|
||||
#### Install Xfce
|
||||
#### 安装 Xfce
|
||||
|
||||
安装 Xfce 桌面环境相当简单。只需要：
|
||||
|
||||
```
|
||||
`sudo apt install -y --no-install-recommends xfce4 desktop-base lightdm`
|
||||
```
|
||||
|
||||
这就够了。你现在安装了 X Windows （X11）和 Xfce 了。现在是时候来定制一下环境并且安装一些核心的应用了。
|
||||
|
||||
### 核心应用
|
||||
|
||||
目前为止，你已经安装了 X Windows（X11），Xfce 桌面环境，和 LightDM（一个 Xfce 自带的显示管理器）。现在，你已经有了一个可以启动并且正常使用的轻量、完整的系统。然而，在彻底完成之前，我还是喜欢装一些核心应用。
|
||||
|
||||
The following command installs a terminal app, the [Audacious][7] audio player, the [Ristretto][8] image viewer, the [Mousepad][9] text editor, the [File Roller][10] archive manager, and the [Thunar][11] volume manager:
|
||||
|
||||
下面这条命令安装了一个终端程序，[Audacious][7] 音乐播放器，[Ristretto][8] 图像浏览器，[Mousepad][9] 文本编辑器，[File Roller][10] 文档管理器和 [Thunar][11] 容量管理器。
|
||||
|
||||
```
|
||||
sudo apt install -y --no-install-recommends xfce4-terminal audacious ristretto
|
||||
sudo apt install -y --no-install-recommends mousepad file-roller thunar-volman
|
||||
```
|
||||
|
||||
#### 其他可选项
|
||||
|
||||
其他一些你可能想安装的东西，包括一个好的网络管理器，任务管理器，PDF 阅读器和通知工具，再加上桌面背景管理器，截图工具，一些新的图标和光标主题。简单来说，如果树莓派（Raspberry Pi）是你的首选系统，这些都算是日常工作的一些补充。
|
||||
|
||||
|
||||
```
|
||||
sudo apt install -y --no-install-recommends network-manager xfce4-taskmanager xfce4-notifyd
|
||||
sudo apt install -y --no-install-recommends xpdf gnome-icon-theme dmz-cursor-theme
|
||||
```
|
||||
|
||||
### 下一步该做什么？
|
||||
|
||||
如果一切都正常工作的话，你现在就有一个基于 Xfce 和 Debian Lite 超轻量级操作系统的树莓派了（Raspberry Pi）。我建议现在上 Xfce 网站上查看其他所有不错的工具，这些东西你都可以安装并使用。下一步做什么完全由你决定！
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensource.com/article/20/6/custom-raspberry-pi
|
||||
|
||||
作者：[Patrick H. Mullins][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[Yufei-Yan](https://github.com/Yufei-Yan)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://opensource.com/users/pmullins
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/raspberrypi_cartoon.png?itok=m3TcBONJ (Cartoon graphic of Raspberry Pi board)
|
||||
[2]: https://www.raspberrypi.org/downloads/raspberry-pi-os/
|
||||
[3]: https://www.debian.org/
|
||||
[4]: https://www.raspberrypi.org/documentation/installation/installing-images/README.md
|
||||
[5]: https://en.wikipedia.org/wiki/X_Window_System
|
||||
[6]: http://xfce.org
|
||||
[7]: https://audacious-media-player.org/
|
||||
[8]: https://docs.xfce.org/apps/ristretto/start
|
||||
[9]: https://github.com/codebrainz/mousepad
|
||||
[10]: https://gitlab.gnome.org/GNOME/file-roller
|
||||
[11]: https://docs.xfce.org/xfce/thunar/thunar-volman
|
||||
[12]: https://goodies.xfce.org/
|
||||
Reference in New Issue
Block a user