mirror of
https://github.com/LCTT/TranslateProject.git
synced 2026-08-23 04:03:29 +08:00
Merge remote-tracking branch 'LCTT/master'
This commit is contained in:
Xingyu Wang
@@ -1,34 +1,34 @@
|
||||
[#]: collector: "lujun9972"
|
||||
[#]: translator: "way-ww"
|
||||
[#]: reviewer: " "
|
||||
[#]: publisher: " "
|
||||
[#]: url: " "
|
||||
[#]: reviewer: "wxy"
|
||||
[#]: publisher: "wxy"
|
||||
[#]: url: "https://linux.cn/article-11491-1.html"
|
||||
[#]: subject: "How to Run the Top Command in Batch Mode"
|
||||
[#]: via: "https://www.2daygeek.com/linux-run-execute-top-command-in-batch-mode/"
|
||||
[#]: author: "Magesh Maruthamuthu https://www.2daygeek.com/author/magesh/"
|
||||
|
||||
如何在批处理模式下运行 Top 命令
|
||||
如何在批处理模式下运行 top 命令
|
||||
======
|
||||
|
||||
**[Top 命令][1]** 是每个人都在使用的用于 **[监控 Linux 系统性能][2]** 的最好的命令。
|
||||
|
||||
|
||||
除了很少的几个操作， 你可能已经知道 top 命令的绝大部分操作， 如果我没错的话， 批处理模式就是其中之一。
|
||||
[top 命令][1] 是每个人都在使用的用于 [监控 Linux 系统性能][2] 的最好的命令。你可能已经知道 `top` 命令的绝大部分操作，除了很少的几个操作，如果我没错的话，批处理模式就是其中之一。
|
||||
|
||||
大部分的脚本编写者和开发人员都知道这个， 因为这个操作主要就是用来编写脚本。
|
||||
大部分的脚本编写者和开发人员都知道这个，因为这个操作主要就是用来编写脚本。
|
||||
|
||||
如果你不了解这个， 不用担心，我们将在这里介绍它。
|
||||
如果你不了解这个，不用担心，我们将在这里介绍它。
|
||||
|
||||
### 什么是 Top 命令的批处理模式
|
||||
### 什么是 top 命令的批处理模式
|
||||
|
||||
批处理模式允许你将 top 命令的输出发送至其他程序或者文件中。
|
||||
批处理模式允许你将 `top` 命令的输出发送至其他程序或者文件中。
|
||||
|
||||
在这个模式中， top 命令将不会接收输入并且持续运行直到迭代次数达到你用 “-n” 选项指定的次数为止。
|
||||
在这个模式中，`top` 命令将不会接收输入并且持续运行，直到迭代次数达到你用 `-n` 选项指定的次数为止。
|
||||
|
||||
如果你想解决 Linux 服务器上的任何性能问题， 你需要正确的 **[理解 top 命令的输出][3]** 。
|
||||
如果你想解决 Linux 服务器上的任何性能问题，你需要正确的 [理解 top 命令的输出][3]。
|
||||
|
||||
### 1) 如何在批处理模式下运行 top 命令
|
||||
|
||||
默认地， top 命令按照 CPU 的使用率来排序输出结果， 所以当你在批处理模式中运行以下命令时， 它会执行同样的操作并打印前 35 行。
|
||||
默认地，`top` 命令按照 CPU 的使用率来排序输出结果，所以当你在批处理模式中运行以下命令时，它会执行同样的操作并打印前 35 行：
|
||||
|
||||
```
|
||||
# top -bc | head -35
|
||||
@@ -72,7 +72,7 @@ PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
|
||||
|
||||
### 2) 如何在批处理模式下运行 top 命令并按内存使用率排序结果
|
||||
|
||||
在批处理模式中运行以下命令按内存使用率对结果进行排序
|
||||
在批处理模式中运行以下命令按内存使用率对结果进行排序：
|
||||
|
||||
```
|
||||
# top -bc -o +%MEM | head -n 20
|
||||
@@ -99,19 +99,17 @@ KiB Swap: 1048572 total, 514640 free, 533932 used. 2475984 avail Mem
|
||||
8632 nobody 20 0 256844 25744 2216 S 0.0 0.7 0:00.03 /usr/sbin/httpd -k start
|
||||
```
|
||||
|
||||
**上面命令的详细信息:**
|
||||
|
||||
* **-b :** 批处理模式选项
|
||||
* **-c :** 打印运行中的进程的绝对路径
|
||||
* **-o :** 指定进行排序的字段
|
||||
* **head :** 输出文件的第一部分
|
||||
* **-n :** 打印前 n 行
|
||||
|
||||
上面命令的详细信息：
|
||||
|
||||
* `-b`：批处理模式选项
|
||||
* `-c`：打印运行中的进程的绝对路径
|
||||
* `-o`：指定进行排序的字段
|
||||
* `head`：输出文件的第一部分
|
||||
* `-n`：打印前 n 行
|
||||
|
||||
### 3) 如何在批处理模式下运行 top 命令并按照指定的用户进程对结果进行排序
|
||||
|
||||
如果你想要按照指定用户进程对结果进行排序请运行以下命令
|
||||
如果你想要按照指定用户进程对结果进行排序请运行以下命令：
|
||||
|
||||
```
|
||||
# top -bc -u mysql | head -n 10
|
||||
@@ -128,13 +126,11 @@ KiB Swap: 1048572 total, 514640 free, 533932 used. 2649412 avail Mem
|
||||
|
||||
### 4) 如何在批处理模式下运行 top 命令并按照处理时间进行排序
|
||||
|
||||
在批处理模式中使用以下 top 命令按照处理时间对结果进行排序。 这展示了任务从启动以来已使用的总 CPU 时间
|
||||
|
||||
但是如果你想要检查一个进程在 Linux 上运行了多长时间请看接下来的文章。
|
||||
|
||||
* **[检查 Linux 中进程运行时间的五种方法][4]**
|
||||
在批处理模式中使用以下 `top` 命令按照处理时间对结果进行排序。这展示了任务从启动以来已使用的总 CPU 时间。
|
||||
|
||||
但是如果你想要检查一个进程在 Linux 上运行了多长时间请看接下来的文章：
|
||||
|
||||
* [检查 Linux 中进程运行时间的五种方法][4]
|
||||
|
||||
```
|
||||
# top -bc -o TIME+ | head -n 20
|
||||
@@ -163,7 +159,7 @@ KiB Swap: 1048572 total, 514640 free, 533932 used. 2440332 avail Mem
|
||||
|
||||
### 5) 如何在批处理模式下运行 top 命令并将结果保存到文件中
|
||||
|
||||
如果出于解决问题的目的， 你想要和别人分享 top 命令的输出， 请使用以下命令重定向输出到文件中
|
||||
如果出于解决问题的目的，你想要和别人分享 `top` 命令的输出，请使用以下命令重定向输出到文件中：
|
||||
|
||||
```
|
||||
# top -bc | head -35 > top-report.txt
|
||||
@@ -209,9 +205,9 @@ KiB Swap: 1048572 total, 514640 free, 533932 used. 2659084 avail Mem
|
||||
|
||||
### 如何按照指定字段对结果进行排序
|
||||
|
||||
在 top 命令的最新版本中， 按下 **“f”** 键进入字段管理界面。
|
||||
在 `top` 命令的最新版本中， 按下 `f` 键进入字段管理界面。
|
||||
|
||||
要使用新字段进行排序， 请使用 **“up/down”** 箭头选择正确的选项， 然后再按下 **“s”** 键进行排序。 最后按 **“q”** 键退出此窗口。
|
||||
要使用新字段进行排序， 请使用 `up`/`down` 箭头选择正确的选项，然后再按下 `s` 键进行排序。最后按 `q` 键退出此窗口。
|
||||
|
||||
```
|
||||
Fields Management for window 1:Def, whose current sort field is %CPU
|
||||
@@ -269,9 +265,9 @@ Fields Management for window 1:Def, whose current sort field is %CPU
|
||||
nsUSER = USER namespace Inode
|
||||
```
|
||||
|
||||
对 top 命令的旧版本， 请按 **“shift+f”** 或 **“shift+o”** 键进入字段管理界面进行排序。
|
||||
对 `top` 命令的旧版本，请按 `shift+f` 或 `shift+o` 键进入字段管理界面进行排序。
|
||||
|
||||
要使用新字段进行排序， 请选择相应的排序字段字母， 然后按下 **“Enter”** 排序。
|
||||
要使用新字段进行排序，请选择相应的排序字段字母， 然后按下回车键排序。
|
||||
|
||||
```
|
||||
Current Sort Field: N for window 1:Def
|
||||
@@ -323,7 +319,7 @@ via: https://www.2daygeek.com/linux-run-execute-top-command-in-batch-mode/
|
||||
作者：[Magesh Maruthamuthu][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[way-ww](https://github.com/way-ww)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
校对：[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
@@ -1,24 +1,26 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: (lnrCoder)
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
[#]: reviewer: (wxy)
|
||||
[#]: publisher: (wxy)
|
||||
[#]: url: (https://linux.cn/article-11492-1.html)
|
||||
[#]: subject: (DevSecOps pipelines and tools: What you need to know)
|
||||
[#]: via: (https://opensource.com/article/19/10/devsecops-pipeline-and-tools)
|
||||
[#]: author: (Sagar Nangare https://opensource.com/users/sagarnangare)
|
||||
|
||||
你需要知道的 DevSecOps 流程及工具
|
||||
======
|
||||
DevSecOps 对 DevOps 进行了改进，以确保安全性仍然是该过程的一个重要部分。
|
||||
![An intersection of pipes.][1]
|
||||
|
||||
到目前为止，DevOps 在 IT 世界中已广为人知，但其并非完美无缺。试想一下，你已经在一个项目的现代应用程序交付中实施了所有 DevOps 工程实践。你已经到达开发流程的末尾，但是渗透测试团队（内部或外部）检测到安全漏洞并提出了报告。 现在，你必须重新启动所有流程，并要求开发人员修复该漏洞。
|
||||
> DevSecOps 对 DevOps 进行了改进，以确保安全性仍然是该过程的一个重要部分。
|
||||
|
||||
在基于 DevOps 的软件开发生命周期（SDLC）系统中，这并不繁琐，但它确实会浪费时间并影响交付进度。如果安全性从 SDLC 初期就已经集成，那么你可能已经跟踪到了该故障，并在开发流程中就消除了它。但是，如上述情形那样，将安全性推到开发流程的最后将导致更长的开发生命周期。
|
||||
|
||||
|
||||
到目前为止，DevOps 在 IT 世界中已广为人知，但其并非完美无缺。试想一下，你在一个项目的现代应用程序交付中实施了所有 DevOps 工程实践。你已经到达开发流程的末尾，但是渗透测试团队（内部或外部）检测到安全漏洞并提出了报告。现在，你必须重新启动所有流程，并要求开发人员修复该漏洞。
|
||||
|
||||
在基于 DevOps 的软件开发生命周期（SDLC）系统中，这并不繁琐，但它确实会浪费时间并影响交付进度。如果从 SDLC 初期就已经集成了安全性，那么你可能已经跟踪到了该故障，并在开发流程中就消除了它。但是，如上述情形那样，将安全性推到开发流程的最后将导致更长的开发生命周期。
|
||||
|
||||
这就是引入 DevSecOps 的原因，它以自动化的方式巩固了整个软件交付周期。
|
||||
|
||||
在现代 DevOps 方法中，组织广泛使用容器托管应用程序，我们看到 [Kubernetes][2] 和 [Istio][3] 使用的较多。但是，这些工具都有其自身的漏洞。例如，云原生计算基金会(CNCF)最近完成了一项 [kubernetes 安全审计][4]，发现了几个问题。DevOps 开发流程中使用的所有工具在流程运行时都需要进行安全检查，DevSecOps 会推动管理员监视工具的存储库以获取升级和补丁。
|
||||
在现代 DevOps 方法中，组织广泛使用容器托管应用程序，我们看到 [Kubernetes][2] 和 [Istio][3] 使用的较多。但是，这些工具都有其自身的漏洞。例如，云原生计算基金会（CNCF）最近完成了一项 [kubernetes 安全审计][4]，发现了几个问题。DevOps 开发流程中使用的所有工具在流程运行时都需要进行安全检查，DevSecOps 会推动管理员去监视工具的存储库以获取升级和补丁。
|
||||
|
||||
### 什么是 DevSecOps?
|
||||
|
||||
@@ -26,22 +28,22 @@ DevSecOps 对 DevOps 进行了改进，以确保安全性仍然是该过程的
|
||||
|
||||
要使用 [DevSecOps][5]，你需要：
|
||||
|
||||
* 从SDLC开始就引入安全性概念，以最大程度地减少软件代码中的漏洞。
|
||||
* 确保每个人（包括开发人员和IT运营团队）共同承担在其任务中遵循安全实践的责任。
|
||||
* 在DevOps工作流程开始时集成安全控件，工具和流程。这些将在软件交付的每个阶段启用自动安全检查。
|
||||
* 从 SDLC 开始就引入安全性概念，以最大程度地减少软件代码中的漏洞。
|
||||
* 确保每个人（包括开发人员和 IT 运营团队）共同承担在其任务中遵循安全实践的责任。
|
||||
* 在 DevOps 工作流程开始时集成安全控件、工具和流程。这些将在软件交付的每个阶段启用自动安全检查。
|
||||
|
||||
DevOps 一直致力于在开发和发布过程中包括安全性以及质量保证（QA），数据库管理和其他所有方面。然而，DevSecOps 是该过程的一个演进，以确保安全永远不会被遗忘，成为该过程的一个重要部分。
|
||||
DevOps 一直致力于在开发和发布过程中包括安全性以及质量保证（QA）、数据库管理和其他所有方面。然而，DevSecOps 是该过程的一个演进，以确保安全永远不会被遗忘，成为该过程的一个重要部分。
|
||||
|
||||
### 了解 DevSecOps 流程
|
||||
|
||||
典型的 DevOps 流程有不同的阶段；典型的 SDLC 流程包括计划，编码，构建，测试，发布和部署等阶段。在 DevSecOps 中，每个阶段都会应用特定的安全检查。
|
||||
典型的 DevOps 流程有不同的阶段；典型的 SDLC 流程包括计划、编码、构建、测试、发布和部署等阶段。在 DevSecOps 中，每个阶段都会应用特定的安全检查。
|
||||
|
||||
* **计划:** 执行安全性分析并创建测试计划，以确定在何处，如何以及何时进行测试的方案。
|
||||
* **编码:** 部署整理工具和 Git 控件以保护密码和 API 密钥。
|
||||
* **构建:** 在构建执行代码时，请结合使用静态应用程序安全测试（SAST）工具来跟踪代码中的缺陷，然后再部署到生产环境中。 这些工具针对特定的编程语言。
|
||||
* **测试:** 在运行时使用动态应用程序安全测试（DAST）工具来测试您的应用程序。 这些工具可以检测与用户身份验证，授权，SQL 注入以及与 API 相关的端点相关的错误。
|
||||
* **发布:** 在发布应用程序之前，请使用安全分析工具来进行全面的渗透测试和漏洞扫描。
|
||||
* **部署:** 在运行时完成上述测试后，将安全的版本发送到生产中以进行最终部署。
|
||||
* **计划**：执行安全性分析并创建测试计划，以确定在何处、如何以及何时进行测试的方案。
|
||||
* **编码**：部署整理工具和 Git 控件以保护密码和 API 密钥。
|
||||
* **构建**：在构建执行代码时，请结合使用静态应用程序安全测试（SAST）工具来跟踪代码中的缺陷，然后再部署到生产环境中。这些工具针对特定的编程语言。
|
||||
* **测试**：在运行时使用动态应用程序安全测试（DAST）工具来测试您的应用程序。 这些工具可以检测与用户身份验证，授权，SQL 注入以及与 API 相关的端点相关的错误。
|
||||
* **发布**：在发布应用程序之前，请使用安全分析工具来进行全面的渗透测试和漏洞扫描。
|
||||
* **部署**：在运行时完成上述测试后，将安全的版本发送到生产中以进行最终部署。
|
||||
|
||||
### DevSecOps 工具
|
||||
|
||||
@@ -56,7 +58,7 @@ via: https://opensource.com/article/19/10/devsecops-pipeline-and-tools
|
||||
作者：[Sagar Nangare][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[lnrCoder](https://github.com/lnrCoder)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
校对：[wxy](https://github.com/wxy)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
[#]: subject: (VMware on AWS gets an on-premises option)
|
||||
[#]: via: (https://www.networkworld.com/article/3446796/vmware-on-aws-gets-an-on-premises-option.html)
|
||||
[#]: author: (Michael Cooney https://www.networkworld.com/author/Michael-Cooney/)
|
||||
|
||||
VMware on AWS gets an on-premises option
|
||||
======
|
||||
Amazon Relational Database Service on VMware automates database provisioning for customers running VMware vSphere 6.5 or later, and it supports Microsoft SQL Server, PostgreSQL, and MySQL.
|
||||
Getty Images
|
||||
|
||||
VMware has taken another step to integrate its virtual kingdom with Amazon Web Services' world with an [on-premise service][1] that will let customers automate database provisioning and management.
|
||||
|
||||
The package, [Amazon Relational Database Service][2] (RDS) on VMware is available now for customers running VMware vSphere 6.5 or later and supports Microsoft SQL Server, PostgreSQL, and MySQL. Other DBs will be supported in the future, the companies said.
|
||||
|
||||
****[**** Read also: [How to plan a software-defined data-center network][3] ****|**** [Get regularly scheduled insights by signing up for Network World newsletters.][4]**]**
|
||||
|
||||
The RDS lets customers run native RDS Database instances on a vSphere platform and manage those instances from the AWS Management Console in the cloud. It automates database provisioning, operating-system and database patching, backups, point-in-time restore and compute scaling, as well as database-instance health management, VMware said.
|
||||
|
||||
[][5]
|
||||
|
||||
BrandPost Sponsored by HPE
|
||||
|
||||
[HPE Synergy For Dummies][5]
|
||||
|
||||
Here’s how IT can provide an anytime, anywhere, any workload infrastructure.
|
||||
|
||||
With the service customers such as software developers and database administrators get native access to the Amazon Relational Database Service using their familiar AWS Management Console, CLI, and RDS APIs, Chris Wolf, vice president and CTO, global field and industry at VMware wrote in a [blog][6] about the service. “Operations teams can quickly stand up an RDS instance anywhere they run vSphere, and manage it using all of their existing tools and processes.”
|
||||
|
||||
Wolf said the service should greatly simplify managing databases linked to its flagship vSphere system.
|
||||
|
||||
Advertisement
|
||||
|
||||
Managing databases on vSphere or natively has always been a tedious exercise that steals the focus of highly skilled database administrators, Wolf stated. “VMware customers will now be able to expand the benefits of automation and standardization of their database workloads inside of vSphere and focus more of their time and energy on improving their applications for their customers.”
|
||||
|
||||
The RDS is just the part of the enterprise data center/cloud integration work VMware and AWS have been up to in the past year.
|
||||
|
||||
In August [VMware said it added VMware HCX][7] capabilities to enable push-button migration and interconnectivity between VMware Cloud on AWS Software-Defined Data Centers running in different AWS Regions. It has also added new Elastic vSAN support to bolster storage scaling.
|
||||
|
||||
Once applications are migrated to the cloud, customers can extend their capabilities through the integration of native AWS services. In the future, through technology such as Bitfusion and partnerships with other vendors such as NVIDIA, customers will be able to enrich existing applications and power new enterprise applications.
|
||||
|
||||
VMware and NVIDIA also announced their intent to deliver accelerated GPU services for VMware Cloud on AWS. These services will let customers migrate VMware vSphere-based applications and containers to the cloud, unchanged, where they can be modernized to take advantage of high-performance computing, machine learning, data analytics and video-processing applications, VMware said.
|
||||
|
||||
And last November [AWS tied in VMware][8] to its on-premises Outposts development, which comes in two versions. The first, VMware Cloud on AWS Outposts, lets customers use the same VMware control plane and APIs they currently deploy. The other is an AWS-native variant that lets customers use the same APIs and control plane they use to run in the AWS cloud, but on premises, according to AWS.
|
||||
|
||||
Outposts can be upgraded with the latest hardware and next-generation instances to run all native AWS and VMware applications, [AWS stated][9]. A second version, VMware Cloud on AWS Outposts, lets customers use a VMware control plane and APIs to run the hybrid environment.
|
||||
|
||||
The idea with Outposts is that customers can use the same programming interface, same APIs, same console and CLI they use on the AWS cloud for on-premises applications, develop and maintain a single code base, and use the same deployment tools in the AWS cloud and on premises, AWS wrote.
|
||||
|
||||
VMware isn’t the only vendor cozying up to AWS. Cisco has done a variety of integration work with the cloud service provider as well. In [April Cisco released Cloud ACI for AWS][10] to let users configure inter-site connectivity, define policies and monitor the health of network infrastructure across hybrid environments, Cisco said. The AWS service utilizes the Cisco Cloud APIC [Application Policy Infrastructure Controller] to provide connectivity, policy translation and enhanced visibility of workloads in the public cloud, Cisco said.
|
||||
|
||||
“This solution brings a suite of capabilities to extend your on-premises data center into true multi-cloud architectures, helping to drive policy and operational consistency, independent of where your applications or data reside. [It] uses the native AWS constructs for policy translation and gives end-to-end visibility into the customer's multi-cloud workloads and connectivity,” Cisco said.
|
||||
|
||||
Join the Network World communities on [Facebook][11] and [LinkedIn][12] to comment on topics that are top of mind.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://www.networkworld.com/article/3446796/vmware-on-aws-gets-an-on-premises-option.html
|
||||
|
||||
作者：[Michael Cooney][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://www.networkworld.com/author/Michael-Cooney/
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://aws.amazon.com/blogs/aws/now-available-amazon-relational-database-service-rds-on-vmware/
|
||||
[2]: https://blogs.vmware.com/vsphere/2019/10/how-amazon-rds-on-vmware-works.html
|
||||
[3]: https://www.networkworld.com/article/3284352/data-center/how-to-plan-a-software-defined-data-center-network.html
|
||||
[4]: https://www.networkworld.com/newsletters/signup.html
|
||||
[5]: https://www.networkworld.com/article/3399618/hpe-synergy-for-dummies.html?utm_source=IDG&utm_medium=promotions&utm_campaign=HPE19718&utm_content=sidebar (HPE Synergy For Dummies)
|
||||
[6]: https://cloud.vmware.com/community/2019/10/16/announcing-general-availability-amazon-rds-vmware/
|
||||
[7]: https://www.networkworld.com/article/3434397/vmware-fortifies-its-hybrid-cloud-portfolio-with-management-automation-aws-and-dell-offerings.html
|
||||
[8]: https://www.networkworld.com/article/3324043/aws-does-hybrid-cloud-with-on-prem-hardware-vmware-help.html
|
||||
[9]: https://aws.amazon.com/outposts/
|
||||
[10]: https://www.networkworld.com/article/3388679/cisco-taps-into-aws-for-data-center-cloud-applications.html
|
||||
[11]: https://www.facebook.com/NetworkWorld/
|
||||
[12]: https://www.linkedin.com/company/network-world
|
||||
@@ -0,0 +1,80 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
[#]: subject: (“Making software liquid: a DevOps company founder’s journey from OSS community to billion-dollar darling”)
|
||||
[#]: via: (https://opensourceforu.com/2019/10/making-software-liquid-a-devops-company-founders-journey-from-oss-community-to-billion-dollar-darling/)
|
||||
[#]: author: (Editor Team https://opensourceforu.com/author/editor/)
|
||||
|
||||
“Making software liquid: a DevOps company founder’s journey from OSS community to billion-dollar darling”
|
||||
======
|
||||
|
||||
![][1]
|
||||
|
||||
_JFrog claims to make software development easier and faster, and enable firms to reduce their development costs. To understand the basis of this promise, **Rahul Chopra, editorial director, EFY Group**, spoke to **Fred Simon, co-founder and chief architect, JFrog** and here’s what he discovered…_
|
||||
|
||||
**Q. How would you explain JFrog’s solutions to a senior business decision maker?**
|
||||
|
||||
**A.** It’s a fair question, as we have been – and continue to be – a developer-driven company that makes tools and solutions for developers. Originally, we were a team of engineers working on Java and had the task of solving some package management pain during the J2EE (now Java EE) transformation. So, it was all hyper-technical and hard to explain to non-engineers.
|
||||
|
||||
Today, it’s a bit easier. As the world moves towards cloud-native applications as the default and every company is now a software company, the benefits of software management and delivery are now mission-critical. We see that as this industry maturity has taken place, software conversations are now management-level conversations. So, it’s now a very simple proposition: You are getting business demands for faster, smarter, more secure software. And you must “release software fast, or you die,” as we like to say. Competition is fierce, so if you can provide value to the end-user faster and smarter without downtime – what we call “Liquid Software,” you have a competitive edge. JFrog helps you achieve these goals faster as a DevOps organization.
|
||||
|
||||
**Q. How does the explanation change, when explaining it to a senior techie like a CTO?**
|
||||
|
||||
**A.** At this level, it is even simpler. You have historically released software once a quarter or even once a year or more. You know that this demand has changed with cloud, microservices and agility movements. We give you the ability to get a new version rapidly, control where the version was validated and how it ends up in runtime as quickly as possible. We’ve been doing this for more than 10 years at scale. When we started, our customers managed a gigabyte, then later a terabyte and today we have customers with petabytes of binary software creating dozens or hundreds of builds a day.
|
||||
|
||||
**Q. You mentioned the word ‘control’. But, a lot of developers do not like control. So, how would you explain JFrog’s promise to them, what would your pitch be?**
|
||||
|
||||
**A.** The word “control” to a developer’s ear can sound like something being imposed on them. It’s “someone else’s” control. But the developer roots of JFrog demand that we provide speed and agility to developers, giving them as much control over their environments as possible without sacrificing their speed.
|
||||
|
||||
**Q. According to you, the drivers within the company are the developers, who then take it to DevOps, then to CTO and the CEO signs the cheque? Is that how it works?**
|
||||
|
||||
**A.** Yes. JFrog to date has only had an inside sales force with no outbound sales. The first time we ever talked to a company was because the developer said that they needed a professional version of JFrog tools. Then we started the discussion with the managers and so on up the chain. Developers – as some like our friends at RedMonk have said – are still the kingmakers.
|
||||
|
||||
**Q. Can you explain the term ‘Liquid Software’ that’s been mentioned quite a few times on your website?**
|
||||
|
||||
**A.** The concept of Liquid Software is to enable continuous, secure, seamless updates of every piece of software that is running without any downtime. This is very different than the traditional build, package, distribute once a year model. The old way doesn’t scale. The new world of Liquid Software makes the update process nearly seamless from code to end device.
|
||||
|
||||
**Q. Has the shift to “everything-as-a-service” become the main driver for this concept of Liquid Software?**
|
||||
|
||||
**A.** Yes. People are not making software as a service, they are delivering services. They are using our Liquid Software pipeline delivery process for every kind of delivery. It’s important to note that “as a service” isn’t just for cloud, but is in fact the standard for every type of software delivery.
|
||||
|
||||
**Q. How’s JFrog connected with open source and how does it shift to an enterprise paid version? What is the licensing model at JFrog?**
|
||||
|
||||
**A.** We have an open-source version licensed under AGPL. This open source version allows you to do many Java-related works, and is sometimes where developers start to “kick the tires.”.There is also an edition specifically for C/C++ developer utilizing the Conan framework. Since most development shops do more than one type of development, our commercial versions – starting with a Pro subscription – universally support all package types. From there, there are other plans available that include HA, Security and Compliance tools, Distribution and more. We have also recently added JFrog Pipelines for full automation of your pipelines across the organization. So, you can choose what makes the most sense for them, and JFrog can grow alongside your needs as you mature your DevOps and DevSecOps infrastructure.
|
||||
|
||||
**Q. Do you have a different set of solutions for developers depending on whether they are developing for the web, mobile, IoT?**
|
||||
|
||||
**A.** No, we are universal. You don’t need to re-install different things for different technology. So, if you are a Ruby developer or a Python developer or if you have Docker, Debian, Microsoft or NuGet, then you get everything in one single tool. Pipelines are so unique to each organization that we need to support all of it.
|
||||
|
||||
**Q. Are there any specific solutions or capabilities that you have developed for IoT?**
|
||||
|
||||
**A.** Yes. Quite early on we worked with customers on an IoT offering. We provided an IoT-specific solution, which is an extension of Debian for IoT, and we also have Conan and Yocto. Controlling and increasing the speed of delivery is something that is in the early stages of the IoT environment. So we are helping in this integration and providing tools that are enabling different technologies on your JFrog platform that are tailored to an IoT environment.
|
||||
|
||||
**Q. Overall, how important is India, both as development and tech-support centre for JFrog globally as well as a market for JFrog?**
|
||||
|
||||
**A.** JFrog opened its first office in India more than three years ago, with a development office working on JFrog Insight and JFrog Mission Control (which provide pipeline tooling and performance visibility). We purchased an organization called Shippable at the beginning of this year for their technology and their R&D team, who then created JFrog Pipelines product. They are also located in India, so India has been and is increasingly important from both an R&D and support perspective. A lot of our senior support force is in India, so we need really good developers working at JFrog to handle the high-tech support volume. We are already at 60 employees in Bangalore and have recently appointed a General Manager. As you know, JFrog is now a company of more than 500 people. We are also growing our marketing and sales teams in India that will help drive the DevOps revolution for Indian customers.
|
||||
|
||||
**Q. Are these more of a global account that have shops in India or are these Indian companies?**
|
||||
|
||||
**A.** Both. We started with the global companies with R&D in India. Today, we have companies throughout India that are directly buying from us.
|
||||
|
||||
**Q. A little bit about your personal journey, when and how did you connect with the open-source world?**
|
||||
|
||||
**A.** I will date myself and say that in 1992, I used to play with Mosaic while I was in University, and I created a web server based on open source web stacks. Gloriously geeky stuff, but it put me in the OSS community right from the beginning. When I was a kid, I used to share code and OSS was the way I learned how to code in the first place. It’s clear to me that OSS is the future for creating innovative software, and I – and JFrog – continue to support and contribute to development communities globally. I look forward to seeing OSS and OSS communities drive the innovations of the future.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensourceforu.com/2019/10/making-software-liquid-a-devops-company-founders-journey-from-oss-community-to-billion-dollar-darling/
|
||||
|
||||
作者：[Editor Team][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://opensourceforu.com/author/editor/
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://i2.wp.com/opensourceforu.com/wp-content/uploads/2019/10/Copy-of-IMG_0219a-_39_-new.jpg?resize=350%2C472&ssl=1
|
||||
@@ -0,0 +1,104 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
[#]: subject: (Gartner: 10 infrastructure trends you need to know)
|
||||
[#]: via: (https://www.networkworld.com/article/3447397/gartner-10-infrastructure-trends-you-need-to-know.html)
|
||||
[#]: author: (Michael Cooney https://www.networkworld.com/author/Michael-Cooney/)
|
||||
|
||||
Gartner: 10 infrastructure trends you need to know
|
||||
======
|
||||
Gartner names the most important factors affecting infrastructure and operations
|
||||
[Daniel Páscoa][1] [(CC0)][2]
|
||||
|
||||
ORLANDO – Corporate network infrastructure is only going to get more involved over the next two to three years as automation, network challenges and hybrid cloud become more integral to the enterprise.
|
||||
|
||||
Those were some of the main infrastructure trend themes espoused by Gartner vice president and distinguished analyst [David Cappuccio][3] at the research firm’s IT Symposium/XPO here this week.
|
||||
|
||||
[[Get regularly scheduled insights by signing up for Network World newsletters.]][4]
|
||||
|
||||
Cappuccio noted that Gartner’s look at the top infrastructure and operational trends reflect offshoots of technologies – such as cloud computing, automation and networking advances the company’s [analysts have talked][5] about many times before.
|
||||
|
||||
[][6]
|
||||
|
||||
BrandPost Sponsored by HPE
|
||||
|
||||
[Take the Intelligent Route with Consumption-Based Storage][6]
|
||||
|
||||
Combine the agility and economics of HPE storage with HPE GreenLake and run your IT department with efficiency.
|
||||
|
||||
Gartners “Top Ten Trends Impacting Infrastructure and Operations” list is:
|
||||
|
||||
### Automation-strategy rethink
|
||||
|
||||
[Automation][7] has been going on at some level for years, Cappuccio said, but the level of complexity as it is developed and deployed further is what’s becoming confusing. The amounts and types of automation need to be managed and require a shift to a team development approach led by an automation architect that can be standardized across business units. Cappuccio said. What would help? Gartner says by 2025, more than 90 percent of enterprises will have an automation architect, up from less than 20 percent today.
|
||||
|
||||
Advertisement
|
||||
|
||||
### Hybrid IT Impacts Disaster Recovery Confidence
|
||||
|
||||
Hybrid IT which includes a mix of data center, SAAS, PAAS, branch offices, [edge computing][8] and security services makes it hard to promise enterprise resources will be available or backed-up, Cappuccio said. Overly-simplistic IT disaster recovery plans may only deliver partial success. By 2021, the root cause of 90 percent of cloud-based availability issues will be the failure to fully use cloud service provider native redundancy capabilities, he said. Enterprises need to leverage their automation investments and other IT tools to refocus how systems are recovered.
|
||||
|
||||
### Scaling DevOps agility demands platform rethinking
|
||||
|
||||
IT’s role in many companies has almost become that of a product manager for all its different DevOps teams. IT needs to build consistency across the enterprise because they don’t want islands of DeVOps teams across the company. By 2023, 90 percent of enterprises will fail to scale DevOps initiatives if shared self-service platform approaches are not adopted, Gartner stated.
|
||||
|
||||
### Infrastructure - and your data - are everywhere
|
||||
|
||||
By 2022, more than 50 percent of enterprise-generated data will be created and processed outside the [data cente][9]r or cloud, up from less than 10 percent in 2019. Infrastructure is everywhere, Cappuccio said and every time data is moved it creates challenges. How does IT manage data-everywhere scenarios? Cappuccio advocated mandating data-driven infrastructure impact-assessment at early stages of design, investing in infrastructure tools to manage data wherever it resides, and modernizing existing backup architectures to be able to protect data wherever it resides.
|
||||
|
||||
### Overwhelming Impact of IoT
|
||||
|
||||
The issue here is that most [IoT][10] implementations are not driven by IT, and they typically involve different protocols and vendors that don’t usually deal with an IT organization. In the end, who controls and manages IoT becomes an issue and it creates security and operational risks. Cappuccio said companies need to engage with business leaders to shape IoT strategies and establish a center of excellence for IoT.
|
||||
|
||||
### Distributed cloud
|
||||
|
||||
The methods of putting cloud services or cloud-like services on-premises but letting a vendor manage that cloud are increasing. Google has Athos and AWS will soon roll out OutPosts, for example, so this environment is going to change a lot in the next two years, Cappuccio said. This is a nascent market so customers should beware. Enterprises should also be prepared to set boundaries and determine who is responsible for software upgrades, patching and performance.
|
||||
|
||||
### Immersive Experience
|
||||
|
||||
Humans used to learn about and adapt to technology. Today, technology learns and adapts to humans, Cappuccio said. “We have created a world where customers have a serious expectation of perfection. We have designed applications where perfection is the norm.” Such systems are great for mindshare, marketshare and corporate reputation, but as soon as there’s one glitch that’s all out the window.
|
||||
|
||||
### Democratization of IT
|
||||
|
||||
Application development is no longer the realm of specialists. There has been the rollout of simpler development tools like [low code][11] or [no code][12] packages and a focus on bringing new applications to market quickly. That may bring a quicker time-to-market for the business but could be riskier for IT, Cappuccio said. IT leaders perhaps can’t control such rapid development, but it needs to understand what’s happening.
|
||||
|
||||
### What's next for networking?
|
||||
|
||||
There are tons of emerging trends around networking such as mesh, secure-access service edge, network automation, network-on-demand service, network automation, and firewalls as a service. “After decades of focusing on network performance and availability, future network innovation will target operational simplicity, automation, reliability and flexible business models,” Cappuccio said. Enterprises need to automate “everywhere” and balance what technologies are safe vs. what is agile, he said.
|
||||
|
||||
### Hybrid digital-infrastructure management
|
||||
|
||||
The general idea here is that CIOs face the challenge of selecting the right mixture of cloud and traditional IT for the organization. The mix of many different elements such as edge, [hybrid cloud][13], workflow and management creates complex infrastructures. Gartner recommends a focus on workflow visualization – utilizing an in integrated toolset and developing a center of excellence to work on the issues, Cappuccio said.
|
||||
|
||||
Join the Network World communities on [Facebook][14] and [LinkedIn][15] to comment on topics that are top of mind.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://www.networkworld.com/article/3447397/gartner-10-infrastructure-trends-you-need-to-know.html
|
||||
|
||||
作者：[Michael Cooney][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://www.networkworld.com/author/Michael-Cooney/
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://unsplash.com/photos/tjiPN3e45WE
|
||||
[2]: https://creativecommons.org/publicdomain/zero/1.0/
|
||||
[3]: https://www.linkedin.com/in/davecappuccio/
|
||||
[4]: https://www.networkworld.com/newsletters/signup.html
|
||||
[5]: https://www.networkworld.com/article/2160904/gartner--10-critical-it-trends-for-the-next-five-years.html
|
||||
[6]: https://www.networkworld.com/article/3440100/take-the-intelligent-route-with-consumption-based-storage.html?utm_source=IDG&utm_medium=promotions&utm_campaign=HPE20773&utm_content=sidebar ( Take the Intelligent Route with Consumption-Based Storage)
|
||||
[7]: https://www.networkworld.com/article/3223189/how-network-automation-can-speed-deployments-and-improve-security.html
|
||||
[8]: https://www.networkworld.com/article/3224893/what-is-edge-computing-and-how-it-s-changing-the-network.html
|
||||
[9]: https://www.networkworld.com/article/3223692/what-is-a-data-centerhow-its-changed-and-what-you-need-to-know.html
|
||||
[10]: https://www.networkworld.com/article/3207535/what-is-iot-how-the-internet-of-things-works.html
|
||||
[11]: https://www.mendix.com/low-code-guide/
|
||||
[12]: https://kissflow.com/no-code/
|
||||
[13]: https://www.networkworld.com/article/3268448/what-is-hybrid-cloud-really-and-whats-the-best-strategy.html
|
||||
[14]: https://www.facebook.com/NetworkWorld/
|
||||
[15]: https://www.linkedin.com/company/network-world
|
||||
@@ -0,0 +1,85 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
[#]: subject: (Disney’s Streaming Service is Having Troubles with Linux)
|
||||
[#]: via: (https://itsfoss.com/disney-plus-linux/)
|
||||
[#]: author: (Ankush Das https://itsfoss.com/author/ankush/)
|
||||
|
||||
Disney’s Streaming Service is Having Troubles with Linux
|
||||
======
|
||||
|
||||
You might be already using Amazon Prime Video (comes free with [Amazon Prime membership][1]) or [Netflix on your Linux system][2]. Google Chrome supports these streaming services out of the box. You can also [watch Netflix on Firefox in Linux][3] but you have to explicitly enable DRM content.
|
||||
|
||||
However we just learned that Disney’s upcoming streaming service, Disney+ does not work in the same way.
|
||||
|
||||
A user, Hans de Goede, on [LiveJournal][4] revealed this from his experience with Disney+ in the testing period. In fact, the upcoming streaming service Disney+ does not support Linux at all, at least for now.
|
||||
|
||||
### The trouble with Disney+ and DRM
|
||||
|
||||
![][5]
|
||||
|
||||
As Hans explains in his [post][4], he subscribed to the streaming service in the testing period because of the availability of Disney+ in Netherlands.
|
||||
|
||||
Hans tested it on Fedora with mainstream browsers like Firefox and Chrome. However, every time, an error was encountered – “**Error Code 83**“.
|
||||
|
||||
So, he reached out to Disney support to solve the issue – but interestingly they weren’t even properly aware of the issue as it took them a week to give him a response.
|
||||
|
||||
Here’s how he puts his experience:
|
||||
|
||||
> So I mailed the Disney helpdesk about this, explaining how Linux works fine with Netflix, AmazonPrime video and even the web-app from my local cable provider. They promised to get back to me in 24 hours, the eventually got back to me in about a week. They wrote: “We are familiar with Error 83. This often happens if you want to play Disney + via the web browser or certain devices. Our IT department working hard to solve this. In the meantime, I want to advise you to watch Disney + via the app on a phone or tablet. If this error code still occurs in a few days, you can check the help center …” this was on September 23th.
|
||||
|
||||
They just blatantly advised him to use his phone/tablet to access the streaming service instead. That’s genius!
|
||||
|
||||
### Disney should reconsider their DRM implementation
|
||||
|
||||
What is DRM?
|
||||
|
||||
Digital Rights Management ([DRM][6]) technologies attempt to control what you can and can’t do with the media and hardware you’ve purchased.
|
||||
|
||||
Even though they want to make sure that their content remains protected from pirates (which won’t make a difference either), it creates a problem with the support for multiple platforms.
|
||||
|
||||
How on earth do you expect more people to subscribe to your streaming service when you do not even support platforms like Linux? So many media center devices run on Linux. This will be a big setback if Disney continues like this.
|
||||
|
||||
To shed some light on the issue, a user on [tweakers.net][7] found out that it is a [Widevine][8] error. Here, it generally means that your device is incompatible with the security level of DRM implemented.
|
||||
|
||||
It turns out that it isn’t just limited to Linux – but a lot of users are encountering the same error on other platforms as well.
|
||||
|
||||
In addition to the wave of issues, the Widevine error also points to a fact that Disney+ may not even work on Chromebooks, some Android smartphones, and Linux desktops in general.
|
||||
|
||||
Seriously, Disney?
|
||||
|
||||
### Go easy, Disney!
|
||||
|
||||
A common DRM (low-level security) implementation with Disney+ should make it accessible on every platform including Linux systems.
|
||||
|
||||
Disney+ might want to re-think about the DRM implementation if they want to compete with other streaming platforms like Netflix and Amazon Prime Video.
|
||||
|
||||
Personally, I would prefer to stay with Netflix if Disney does not care about supporting multiple platforms.
|
||||
|
||||
It is not actually about supporting “Linux” but conveniently making the streaming service available for more platforms which could justify its subscription fee.
|
||||
|
||||
What do you think about this? Let me know your thoughts in the comments below.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://itsfoss.com/disney-plus-linux/
|
||||
|
||||
作者：[Ankush Das][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://itsfoss.com/author/ankush/
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://www.amazon.com/tryprimefree?tag=chmod7mediate-20
|
||||
[2]: https://itsfoss.com/watch-netflix-in-ubuntu-linux/
|
||||
[3]: https://itsfoss.com/netflix-firefox-linux/
|
||||
[4]: https://hansdegoede.livejournal.com/22338.html
|
||||
[5]: https://i0.wp.com/itsfoss.com/wp-content/uploads/2019/10/disney-plus-linux.jpg?resize=800%2C450&ssl=1
|
||||
[6]: https://www.eff.org/issues/drm
|
||||
[7]: https://tweakers.net/nieuws/157224/disney+-start-met-gratis-proefperiode-van-twee-maanden-in-nederland.html?showReaction=13428408#r_13428408
|
||||
[8]: https://www.widevine.com/
|
||||
@@ -1,81 +0,0 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: (geekpi)
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
[#]: subject: (Linux sudo flaw can lead to unauthorized privileges)
|
||||
[#]: via: (https://www.networkworld.com/article/3446036/linux-sudo-flaw-can-lead-to-unauthorized-privileges.html)
|
||||
[#]: author: (Sandra Henry-Stocker https://www.networkworld.com/author/Sandra-Henry_Stocker/)
|
||||
|
||||
Linux sudo flaw can lead to unauthorized privileges
|
||||
======
|
||||
Exploiting a newly discovered sudo flaw in Linux can enable certain users with to run commands as root despite restrictions against it.
|
||||
Thinkstock
|
||||
|
||||
A newly discovered and serious flaw in the [**sudo**][1] command can, if exploited, enable users to run commands as root in spite of the fact that the syntax of the **/etc/sudoers** file specifically disallows them from doing so.
|
||||
|
||||
Updating **sudo** to version 1.8.28 should address the problem, and Linux admins are encouraged to do so as soon as possible.
|
||||
|
||||
[[Get regularly scheduled insights by signing up for Network World newsletters.]][2]
|
||||
|
||||
How the flaw might be exploited depends on specific privileges granted in the **/etc/sudoers** file. A rule that allows a user to edit files as any user except root, for example, would actually allow that user to edit files as root as well. In this case, the flaw could lead to very serious problems.
|
||||
|
||||
[][3]
|
||||
|
||||
BrandPost Sponsored by HPE
|
||||
|
||||
[Take the Intelligent Route with Consumption-Based Storage][3]
|
||||
|
||||
Combine the agility and economics of HPE storage with HPE GreenLake and run your IT department with efficiency.
|
||||
|
||||
For a user to exploit the flaw, **a user** needs to be assigned privileges in the **/etc/sudoers **file that allow that user to run commands as some other users, and the flaw is limited to the command privileges that are assigned in this way.
|
||||
|
||||
This problem affects versions prior to 1.8.28. To check your sudo version, use this command:
|
||||
|
||||
```
|
||||
$ sudo -V
|
||||
Sudo version 1.8.27 <===
|
||||
Sudoers policy plugin version 1.8.27
|
||||
Sudoers file grammar version 46
|
||||
Sudoers I/O plugin version 1.8.27
|
||||
```
|
||||
|
||||
The vulnerability has been assigned [CVE-2019-14287][4] in the **Common Vulnerabilities and Exposures** database. The risk is that any user who has been given the ability to run even a single command as an arbitrary user may be able to escape the restrictions and run that command as root – even if the specified privilege is written to disallow running the command as root.
|
||||
|
||||
The lines below are meant to give the user "jdoe" the ability to edit files with **vi** as any user except root (**!root** means "not root") and nemo the right to run the **id** command as any user except root:
|
||||
|
||||
```
|
||||
# affected entries on host "dragonfly"
|
||||
jdoe dragonfly = (ALL, !root) /usr/bin/vi
|
||||
nemo dragonfly = (ALL, !root) /usr/bin/id
|
||||
```
|
||||
|
||||
However, given the flaw, either of these users would be able to circumvent the restriction and edit files or run the **id** command as root as well.
|
||||
|
||||
The flaw can be exploited by an attacker to run commands as root by specifying the user ID "-1" or "4294967295."
|
||||
|
||||
The response of "1" demonstrates that the command is being run as root (showing root's user ID).
|
||||
|
||||
Joe Vennix from Apple Information Security both found and analyzed the problem.
|
||||
|
||||
Join the Network World communities on [Facebook][5] and [LinkedIn][6] to comment on topics that are top of mind.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://www.networkworld.com/article/3446036/linux-sudo-flaw-can-lead-to-unauthorized-privileges.html
|
||||
|
||||
作者：[Sandra Henry-Stocker][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://www.networkworld.com/author/Sandra-Henry_Stocker/
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://www.networkworld.com/article/3236499/some-tricks-for-using-sudo.html
|
||||
[2]: https://www.networkworld.com/newsletters/signup.html
|
||||
[3]: https://www.networkworld.com/article/3440100/take-the-intelligent-route-with-consumption-based-storage.html?utm_source=IDG&utm_medium=promotions&utm_campaign=HPE20773&utm_content=sidebar ( Take the Intelligent Route with Consumption-Based Storage)
|
||||
[4]: http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-14287
|
||||
[5]: https://www.facebook.com/NetworkWorld/
|
||||
[6]: https://www.linkedin.com/company/network-world
|
||||
@@ -1,5 +1,5 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: translator: (geekpi)
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
|
||||
@@ -0,0 +1,108 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
[#]: subject: (How collaboration fueled a development breakthrough at Greenpeace)
|
||||
[#]: via: (https://opensource.com/open-organization/19/10/collaboration-breakthrough-greenpeace)
|
||||
[#]: author: (Laura Hilliger https://opensource.com/users/laurahilliger)
|
||||
|
||||
How collaboration fueled a development breakthrough at Greenpeace
|
||||
======
|
||||
We're building an innovative platform to connect environmental
|
||||
advocates—but system complexity threatened to slow us down. Opening up
|
||||
was the answer.
|
||||
![The Open Organization at Greenpeace][1]
|
||||
|
||||
Activists really don't like feeling stuck.
|
||||
|
||||
We thrive on forward momentum and the energy it creates. When that movement grinds to a halt, even for a moment, our ability to catalyze passion in others stalls too.
|
||||
|
||||
And my colleagues and I at Greenpeace International were feeling stuck.
|
||||
|
||||
We'd managed to launch a prototype of Planet 4, [Greenpeace's new, open engagement platform][2] for activists and communities. It's live in more than 38 countries (with many more sites). More than 1.75 million people are using it. We've topped more than 3.1 million pageviews.
|
||||
|
||||
To get here, we [spent more than 650 hours in meetings, drank 1,478 litres of coffee, and fixed more than 300 bugs][3]. But it fell short of our vision; it _still_ wasn't [the minimum lovable product][4] we wanted and we didn't know how to move it forward.
|
||||
|
||||
We were stuck.
|
||||
|
||||
Planet 4's complexity was daunting. We didn't always have the right people to address the numerous challenges the project raised. We didn't know if we'd ever realize our vision. Yet a commitment to openness had gotten us here, and I knew a commitment to openness would get us through this, too.
|
||||
|
||||
As [the story of Planet 4][5] continues, I'll explain how it did.
|
||||
|
||||
### An opportunity
|
||||
|
||||
By 2016, my work helping Greenpeace International become a more open organization—[which I described in the first part of this series][6]—was beginning to bear fruit. We were holding regular [community calls][7]. We were releasing project updates frequently and publicly. We were networking with global stakeholders across the organization to define what Planet 4 needed to be. We were [architecting the project with participation in mind][8].
|
||||
|
||||
Becoming open is an organic process. There's no standard "game plan" for implementing process and practices in an organization. Success depends on the people, the tools, the project, the very fabric of the culture you're working inside.
|
||||
|
||||
Inside Greenpeace, we were beginning to see that success.
|
||||
|
||||
A commitment to openness had gotten us here, and I knew a commitment to openness would get us through this, too.
|
||||
|
||||
For some, this open way of working was inspiring and engaging. For others it was terrifying. Some thought asking for everyone's input was ridiculous. Some thought only "experts" should be part of the conversations, a viewpoint that doesn't mesh well with [the principle of inclusivity][9]. I appreciate expertise—don't get me wrong—but the problem with only asking for "expert" opinions is that you exclude people who might have more interest, passion, and knowledge than someone with a formal title.
|
||||
|
||||
Planet 4 was a vision—not just of a new and open engagement platform, but of an organization that could make _use_ of this platform. And it raised problems on both those fronts:
|
||||
|
||||
* **Data and systems integration:** As a network of 28 independent offices all over the world, Greenpeace has a complex technical landscape. While Greenpeace International provides system _recommendations_ and _support_, individual National and Regional Offices are free to make their own systems choices, even if they aren't the supported ones. This is a good thing; different tools better address different needs for different offices. But it's challenging, too, because the absence of standardization means a lack of expertise in all those systems.
|
||||
* **Organizational culture and work styles:** Planet 4 devoured many of Greenpeace's internal strategies and visions, then spit them out into a way that promised to move toward the type of organization we wanted to be. It was challenging the organizational status quo.
|
||||
|
||||
|
||||
|
||||
Our team was too small, our work too big, and the landscape of working in a global non-profit too complex. The team was struggling, and we needed help.
|
||||
|
||||
Then, in 2018, I saw an opportunity.
|
||||
|
||||
As an [Open Organization Ambassador][10], I'd been to Red Hat Summit to speak on a panel about open organizational principles. There I noticed a session exploring what [Red Hat had done to help UNICEF][11], another global non-profit, with its digital transformation efforts. Surely, I thought, Red Hat and Greenpeace could work together, too.
|
||||
|
||||
So I did something that shouldn't seem so revolutionary or audacious: I found the Red Hatter responsible for the company's collaboration with UNICEF, Alexandra Machado, and I _said hello_. I wasn't just introducing myself; I was approaching Alexandra on behalf of a global community of open-minded advocates.
|
||||
|
||||
And it worked.
|
||||
|
||||
### Accelerating
|
||||
|
||||
Together, Alexandra and I spent more than a year coordinating a collaboration that could help Greenpeace move forward. Earlier this year, we started to succeed.
|
||||
|
||||
Planet 4 was a vision—not just of a new and open engagement platform, but of an organization that could make use of this platform. And it raised problems on both those fronts.
|
||||
|
||||
In late May, members of the Planet 4 project and a team from Red Hat's App Dev Center of Excellence met in Amsterdam. The goal: Accelerate us.
|
||||
|
||||
We'd spend an entire week together in a design sprint aimed at helping us chart a speedy path toward making our vision for the Planet 4 engagement platform a reality, beginning with navigating its technical complexity. And in the process, we'd lean heavily on the open way of working we'd learned to embrace.
|
||||
|
||||
At the sprint, our teams got to know each other. We dumped everything on the table. In a radically open and honest way, the Greenpeace team helped the Red Hat team from Waterford understand the technical and cultural hurdles we faced. We explained our organization and our tech stack, our vision and our dreams. Red Hatters noticed our passion and worked alongside us to explore possible technologies that could make our vision a reality.
|
||||
|
||||
Through a series of exercises—including a particularly helpful session of [event storming][12]—we confirmed that our dream was not only the right one to have but also fully realizable. We talked through the dynamics of the systems we are addressing, and, in the end, the Red Hat team helped us envision a prototype for integrated systems that the Greenpeace team could take forward. We've already begun user testing.
|
||||
|
||||
_Listen to Patrick Carney of Red Hat Open Innovation Labs explain event storming._
|
||||
|
||||
On top of that, our new allies wrote a technical report that laid out the complexities we could _see_ but not _address_—and in a way that spurred internal conversations forward. We found ourselves, a few weeks after the event, moving forward at speed.
|
||||
|
||||
Finally, we were unstuck.
|
||||
|
||||
In the final chapter of Planet 4's story, I'll explain what the experience taught us about the power of openness.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensource.com/open-organization/19/10/collaboration-breakthrough-greenpeace
|
||||
|
||||
作者：[Laura Hilliger][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://opensource.com/users/laurahilliger
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/images/open-org/open-org-greenpeace-article-2-blog-thumbnail-520x292.png?itok=YNEKRAxS (The Open Organization at Greenpeace)
|
||||
[2]: http://greenpeace.org/international
|
||||
[3]: https://medium.com/planet4/p4-in-2018-3bec1cc12be8
|
||||
[4]: https://medium.com/planet4/past-the-prototype-d3e0a4d3a171
|
||||
[5]: https://opensource.com/tags/open-organization-greenpeace
|
||||
[6]: https://opensource.com/open-organization/19/10/open-platform-greenpeace-1
|
||||
[7]: https://opensource.com/open-organization/16/1/community-calls-will-increase-participation-your-open-organization
|
||||
[8]: https://opensource.com/open-organization/16/8/best-results-design-participation
|
||||
[9]: https://opensource.com/open-organization/resources/open-org-definition
|
||||
[10]: https://opensource.com/open-organization/resources/meet-ambassadors
|
||||
[11]: https://www.redhat.com/en/proof-of-concept-series
|
||||
[12]: https://openpracticelibrary.com/practice/event-storming/
|
||||
@@ -1,5 +1,5 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: translator: (lnrCoder)
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
@@ -180,7 +180,7 @@ via: https://www.2daygeek.com/find-get-size-of-directory-folder-linux-disk-usage
|
||||
|
||||
作者：[Magesh Maruthamuthu][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
译者：[lnrCoder](https://github.com/lnrCoder)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
@@ -0,0 +1,227 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
[#]: subject: (How to Go About Linux Boot Time Optimisation)
|
||||
[#]: via: (https://opensourceforu.com/2019/10/how-to-go-about-linux-boot-time-optimisation/)
|
||||
[#]: author: (B Thangaraju https://opensourceforu.com/author/b-thangaraju/)
|
||||
|
||||
How to Go About Linux Boot Time Optimisation
|
||||
======
|
||||
|
||||
[![][1]][2]
|
||||
|
||||
_Booting an embedded device or a piece of telecommunication equipment quickly is crucial for time-critical applications and also plays a very major role in improving the user experience. This article gives some important tips on how to enhance the boot-up time of any device._
|
||||
|
||||
Fast booting or fast rebooting plays a crucial role in various situations. It is critical for an embedded system to boot up fast in order to maintain the high availability and better performance of all the services. Imagine a telecommunications device running a Linux operating system that does not have fast booting enabled. All the systems, services and the users dependent on that particular embedded device might be affected. It is really important that devices maintain high availability in their services, for which fast booting and rebooting play a crucial role.
|
||||
|
||||
A small failure or shutdown of a telecom device, even for a few seconds, can play havoc with countless users working on the Internet. Thus, it is really important for a lot of time-dependent devices and telecommunication devices to incorporate fast booting in their devices to help them get back to work quicker. Let us understand the Linux boot-up procedure from Figure 1.
|
||||
|
||||
![Figure 1: Boot-up procedure][3]
|
||||
|
||||
![Figure 2: Boot chart][4]
|
||||
|
||||
**Monitoring tools and the boot-up procedure**
|
||||
A user should take note of a number of factors before making changes to a machine. These include the current booting speed of the machine and also the services, processes or applications that are taking up resources and increasing the boot-up time.
|
||||
|
||||
**Boot chart:** To monitor the boot-up speed and the various services that start while booting up, the user can install the boot chart using the following command:
|
||||
|
||||
```
|
||||
sudo apt-get install pybootchartgui.
|
||||
```
|
||||
|
||||
Each time you boot up, the boot chart saves a _.png_ (portable network graphics) file in the log, which enables the user to view the _png_ files to get an understanding about the system’s boot-up process and services. Use the following command for this purpose:
|
||||
|
||||
```
|
||||
cd /var/log/bootchart
|
||||
```
|
||||
|
||||
The user might need an application to view the _.png_ files. Feh is an X11 image viewer that targets console users. It doesn’t have a fancy GUI, unlike most other image viewers, but it simply displays pictures. Feh can be used to view the _.png_ files. You can install it using the following command:
|
||||
|
||||
```
|
||||
sudo apt-get install feh
|
||||
```
|
||||
|
||||
You can view the _png_ files using _feh xxxx.png_.
|
||||
Figure 2 shows the boot chart when a boot chart _png_ file is viewed.
|
||||
However, a boot chart is not necessary for Ubuntu versions later than 15.10. To get very brief information regarding boot up speed, use the following command:
|
||||
|
||||
```
|
||||
systemd-analyze
|
||||
```
|
||||
|
||||
![Figure 3: Output of systemd-analyze][5]
|
||||
|
||||
Figure 3 shows the output of the command _systemd-analyze_.
|
||||
The command _systemd-analyze_ blame is used to print a list of all running units based on the time they took to initialise. This information is very helpful and can be used to optimise boot-up times. systemd-analyze blame doesn’t display results for services with _Type=simple_, because systemd considers such services to be started immediately; hence, no measurement of the initialisation delays can be done.
|
||||
|
||||
![Figure 4: Output of systemd-analyze blame][6]
|
||||
|
||||
Figure 4 shows the output of _systemd-analyze_ blame.
|
||||
The following command prints a tree of the time-critical chain of units:
|
||||
|
||||
```
|
||||
command systemd-analyze critical-chain
|
||||
```
|
||||
|
||||
Figure 5 shows the output of the command _systemd-analyze critical-chain_.
|
||||
|
||||
![Figure 5: Output of systemd-analyze critical-chain][7]
|
||||
|
||||
**Steps to reduce the boot-up time**
|
||||
Shown below are the various steps that can be taken to reduce boot-up time.
|
||||
|
||||
**BUM (Boot-Up-Manager):** BUM is a run level configuration editor that allows the configuration of _init_ services when the system boots up or reboots. It displays a list of every service that can be started at boot. The user can toggle individual services on and off. BUM has a very clean GUI and is very easy to use.
|
||||
|
||||
BUM can be installed in Ubuntu 14.04 using the following command:
|
||||
|
||||
```
|
||||
sudo apt-get install bum
|
||||
```
|
||||
|
||||
To install it in versions later than 15.10, download the packages from the link _<http://apt.ubuntu.com/p/bum> 13_.
|
||||
|
||||
Start with basic things and disable services related to the scanner and printer. You can also disable Bluetooth and all other unwanted devices and services if you are not using any of them. I strongly recommend that you study the basics about the services before disabling them, as it might affect the machine or operating system. Figure 6 shows the GUI of BUM.
|
||||
|
||||
![Figure 6: BUM][8]
|
||||
|
||||
**Editing the rc file:** To edit the rc file, you need to go to the rc directory. This can be done using the following command:
|
||||
|
||||
```
|
||||
cd /etc/init.d.
|
||||
```
|
||||
|
||||
However, root privileges are needed to access _init.d_, which basically contains start/stop scripts that are used to control (start, stop, reload, restart) the daemon while the system is running or during boot.
|
||||
|
||||
The _rc_ file in _init.d_ is called a run control script. During booting, init executes the _rc_ script and plays its role. To improve the booting speed, we make changes to the _rc_ file. Open the _rc_ file (once you are in the _init.d_ directory) using any file editor.
|
||||
|
||||
For example, by entering _vim rc_, you can change the value of _CONCURRENCY=none_ to _CONCURRENCY=shell_. The latter allows certain startup scripts to be executed simultaneously, rather than serially.
|
||||
|
||||
In the latest versions of the kernel, the value should be changed to _CONCURRENCY=makefile_.
|
||||
Figures 7 and 8 show the comparison of boot-up times before and after editing the rc file. The improvement in the boot-up speed can be noticed. The time to boot before editing the rc file was 50.98 seconds, whereas the time to boot after making the changes to the rc file is 23.85 seconds.
|
||||
However, the above-mentioned changes don’t work on operating systems later than the Ubuntu version 15.10, since the operating systems with the latest kernel use the systemd file and not the _init.d_ file any more.
|
||||
|
||||
![Figure 7: Boot speed before making changes to the rc file][9]
|
||||
|
||||
![Figure 8: Boot speed after making changes to the rc file][10]
|
||||
|
||||
**E4rat:** E4rat stands for e4 ‘reduced access time’ (ext4 file system only). It is a project developed by Andreas Rid and Gundolf Kiefer. E4rat is an application that helps in achieving a fast boot with the help of defragmentation. It also accelerates application startups. E4rat eliminates both seek times and rotational delays using physical file reallocation. This leads to a high disk transfer rate.
|
||||
E4rat is available as a .deb package and you can download it from its official website _<http://e4rat.sourceforge.net/>_.
|
||||
|
||||
Ubuntu’s default ureadahead package conflicts with e4rat. So a few packages have to be installed using the following command:
|
||||
|
||||
```
|
||||
sudo dpkg purge ureadahead ubuntu-minimal
|
||||
```
|
||||
|
||||
Now install the dependencies for e4rat using the following command:
|
||||
|
||||
```
|
||||
sudo apt-get install libblkid1 e2fslibs
|
||||
```
|
||||
|
||||
Open the downloaded _.deb_ file and install it. Boot data is now needed to be gathered properly to work with e4rat.
|
||||
|
||||
Follow the steps given below to get e4rat running properly and to increase the boot-up speed.
|
||||
|
||||
* Access the Grub menu while booting. This can be done by holding the shift button when the system is booting.
|
||||
* Choose the option (kernel version) that is normally used to boot and press ‘e’.
|
||||
* Look for the line starting with _linux /boot/vmlinuz_ and add the following code at the end of the line (hit space after the last letter of the sentence):
|
||||
|
||||
|
||||
|
||||
```
|
||||
- init=/sbin/e4rat-collect or try - quiet splash vt.handsoff =7 init=/sbin/e4rat-collect
|
||||
```
|
||||
|
||||
* Now press _Ctrl+x_ to continue booting. This lets e4rat collect data after booting. Work on the machine, open and close applications for the next two minutes.
|
||||
* Access the log file by going to the e4rat folder and using the following command:
|
||||
|
||||
|
||||
|
||||
```
|
||||
cd /var/log/e4rat
|
||||
```
|
||||
|
||||
* If you do not find any log file, repeat the above mentioned process. Once the log file is there, access the Grub menu again and press ‘e’ as your option.
|
||||
* Enter ‘single’ at the end of the same line that you have edited before. This will help you access the command line. If a different menu appears asking for anything, choose Resume normal boot. If you don’t get to the command prompt for some reason, hit Ctrl+Alt+F1.
|
||||
* Enter your details once you see the login prompt.
|
||||
* Now enter the following command:
|
||||
|
||||
|
||||
|
||||
```
|
||||
sudo e4rat-realloc /var/lib/e4rat/startup.log
|
||||
```
|
||||
|
||||
This process takes a while, depending on the machine’s disk speed.
|
||||
|
||||
* Now restart your machine using the following command:
|
||||
|
||||
|
||||
|
||||
```
|
||||
sudo shutdown -r now
|
||||
```
|
||||
|
||||
* Now, we need to configure Grub to run e4rat at every boot.
|
||||
* Access the grub file using any editor. For example, _gksu gedit /etc/default/grub._
|
||||
* Look for a line starting with _GRUB CMDLINE LINUX DEFAULT=_, and add the following line in between the quotes and before whatever options there are:
|
||||
|
||||
|
||||
|
||||
```
|
||||
init=/sbin/e4rat-preload 18
|
||||
```
|
||||
|
||||
* It should look like this:
|
||||
|
||||
|
||||
|
||||
```
|
||||
GRUB CMDLINE LINUX DEFAULT = init=/sbin/e4rat- preload quiet splash
|
||||
```
|
||||
|
||||
* Save and close the Grub menu and update Grub using _sudo update-grub_.
|
||||
* Reboot the system and you will find noticeable changes in boot speed.
|
||||
|
||||
|
||||
|
||||
Figures 9 and 10 show the differences between the boot-up time before and after installing e4rat. The improvement in the boot-up speed can be noticed. The time taken to boot before using e4rat was 22.32 seconds, whereas the time taken to boot after using e4rat is 9.065 seconds
|
||||
|
||||
![Figure 9: Boot speed before using e4rat][11]
|
||||
|
||||
![Figure 10: Boot speed after using e4rat][12]
|
||||
|
||||
**A few simple tweaks**
|
||||
A good boot-up speed can also be achieved using very small tweaks, two of which are listed below.
|
||||
**SSD:** Using solid-state devices rather than normal hard disks or other storage devices will surely improve your booting speed. SSDs also help in achieving great speeds in transferring files and running applications.
|
||||
|
||||
**Disabling GUI:** The graphical user interface, desktop graphics and window animations take up a lot of resources. Disabling the GUI is another good way to achieve great boot-up speed.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensourceforu.com/2019/10/how-to-go-about-linux-boot-time-optimisation/
|
||||
|
||||
作者：[B Thangaraju][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://opensourceforu.com/author/b-thangaraju/
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://i0.wp.com/opensourceforu.com/wp-content/uploads/2019/10/Screenshot-from-2019-10-07-13-16-32.png?resize=696%2C496&ssl=1 (Screenshot from 2019-10-07 13-16-32)
|
||||
[2]: https://i0.wp.com/opensourceforu.com/wp-content/uploads/2019/10/Screenshot-from-2019-10-07-13-16-32.png?fit=700%2C499&ssl=1
|
||||
[3]: https://i2.wp.com/opensourceforu.com/wp-content/uploads/2019/10/fig-1.png?resize=350%2C302&ssl=1
|
||||
[4]: https://i0.wp.com/opensourceforu.com/wp-content/uploads/2019/10/fig-2.png?resize=350%2C412&ssl=1
|
||||
[5]: https://i1.wp.com/opensourceforu.com/wp-content/uploads/2019/10/fig-3.png?resize=350%2C69&ssl=1
|
||||
[6]: https://i0.wp.com/opensourceforu.com/wp-content/uploads/2019/10/fig-4.png?resize=350%2C535&ssl=1
|
||||
[7]: https://i2.wp.com/opensourceforu.com/wp-content/uploads/2019/10/fig-5.png?resize=350%2C206&ssl=1
|
||||
[8]: https://i0.wp.com/opensourceforu.com/wp-content/uploads/2019/10/fig-6.png?resize=350%2C449&ssl=1
|
||||
[9]: https://i2.wp.com/opensourceforu.com/wp-content/uploads/2019/10/fig-7.png?resize=350%2C85&ssl=1
|
||||
[10]: https://i1.wp.com/opensourceforu.com/wp-content/uploads/2019/10/fig-8.png?resize=350%2C72&ssl=1
|
||||
[11]: https://i2.wp.com/opensourceforu.com/wp-content/uploads/2019/10/fig-9.png?resize=350%2C61&ssl=1
|
||||
[12]: https://i0.wp.com/opensourceforu.com/wp-content/uploads/2019/10/fig-10.png?resize=350%2C61&ssl=1
|
||||
@@ -0,0 +1,498 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
[#]: subject: (How to program with Bash: Logical operators and shell expansions)
|
||||
[#]: via: (https://opensource.com/article/19/10/programming-bash-part-2)
|
||||
[#]: author: (David Both https://opensource.com/users/dboth)
|
||||
|
||||
How to program with Bash: Logical operators and shell expansions
|
||||
======
|
||||
Learn about logical operators and shell expansions, in the second
|
||||
article in this three-part series on programming with Bash.
|
||||
![Women in computing and open source v5][1]
|
||||
|
||||
Bash is a powerful programming language, one perfectly designed for use on the command line and in shell scripts. This three-part series (which is based on my [three-volume Linux self-study course][2]) explores using Bash as a programming language on the command-line interface (CLI).
|
||||
|
||||
The [first article][3] explored some simple command-line programming with Bash, including using variables and control operators. This second article looks into the types of file, string, numeric, and miscellaneous logical operators that provide execution-flow control logic and different types of shell expansions in Bash. The third and final article in the series will explore the **for**, **while**, and **until** loops that enable repetitive operations.
|
||||
|
||||
Logical operators are the basis for making decisions in a program and executing different sets of instructions based on those decisions. This is sometimes called flow control.
|
||||
|
||||
### Logical operators
|
||||
|
||||
Bash has a large set of logical operators that can be used in conditional expressions. The most basic form of the **if** control structure tests for a condition and then executes a list of program statements if the condition is true. There are three types of operators: file, numeric, and non-numeric operators. Each operator returns true (0) if the condition is met and false (1) if the condition is not met.
|
||||
|
||||
The functional syntax of these comparison operators is one or two arguments with an operator that are placed within square braces, followed by a list of program statements that are executed if the condition is true, and an optional list of program statements if the condition is false:
|
||||
|
||||
|
||||
```
|
||||
if [ arg1 operator arg2 ] ; then list
|
||||
or
|
||||
if [ arg1 operator arg2 ] ; then list ; else list ; fi
|
||||
```
|
||||
|
||||
The spaces in the comparison are required as shown. The single square braces, **[** and **]**, are the traditional Bash symbols that are equivalent to the **test** command:
|
||||
|
||||
|
||||
```
|
||||
`if test arg1 operator arg2 ; then list`
|
||||
```
|
||||
|
||||
There is also a more recent syntax that offers a few advantages and that some sysadmins prefer. This format is a bit less compatible with different versions of Bash and other shells, such as ksh (the Korn shell). It looks like:
|
||||
|
||||
|
||||
```
|
||||
`if [[ arg1 operator arg2 ]] ; then list`
|
||||
```
|
||||
|
||||
#### File operators
|
||||
|
||||
File operators are a powerful set of logical operators within Bash. Figure 1 lists more than 20 different operators that Bash can perform on files. I use them quite frequently in my scripts.
|
||||
|
||||
Operator | Description
|
||||
---|---
|
||||
-a filename | True if the file exists; it can be empty or have some content but, so long as it exists, this will be true
|
||||
-b filename | True if the file exists and is a block special file such as a hard drive like **/dev/sda** or **/dev/sda1**
|
||||
-c filename | True if the file exists and is a character special file such as a TTY device like **/dev/TTY1**
|
||||
-d filename | True if the file exists and is a directory
|
||||
-e filename | True if the file exists; this is the same as **-a** above
|
||||
-f filename | True if the file exists and is a regular file, as opposed to a directory, a device special file, or a link, among others
|
||||
-g filename | True if the file exists and is **set-group-id**, **SETGID**
|
||||
-h filename | True if the file exists and is a symbolic link
|
||||
-k filename | True if the file exists and its "sticky'" bit is set
|
||||
-p filename | True if the file exists and is a named pipe (FIFO)
|
||||
-r filename | True if the file exists and is readable, i.e., has its read bit set
|
||||
-s filename | True if the file exists and has a size greater than zero; a file that exists but that has a size of zero will return false
|
||||
-t fd | True if the file descriptor **fd** is open and refers to a terminal
|
||||
-u filename | True if the file exists and its **set-user-id** bit is set
|
||||
-w filename | True if the file exists and is writable
|
||||
-x filename | True if the file exists and is executable
|
||||
-G filename | True if the file exists and is owned by the effective group ID
|
||||
-L filename | True if the file exists and is a symbolic link
|
||||
-N filename | True if the file exists and has been modified since it was last read
|
||||
-O filename | True if the file exists and is owned by the effective user ID
|
||||
-S filename | True if the file exists and is a socket
|
||||
file1 -ef file2 | True if file1 and file2 refer to the same device and iNode numbers
|
||||
file1 -nt file2 | True if file1 is newer (according to modification date) than file2, or if file1 exists and file2 does not
|
||||
file1 -ot file2 | True if file1 is older than file2, or if file2 exists and file1 does not
|
||||
|
||||
_**Fig. 1: The Bash file operators**_
|
||||
|
||||
As an example, start by testing for the existence of a file:
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ File="TestFile1" ; if [ -e $File ] ; then echo "The file $File exists." ; else echo "The file $File does not exist." ; fi
|
||||
The file TestFile1 does not exist.
|
||||
[student@studentvm1 testdir]$
|
||||
```
|
||||
|
||||
Next, create a file for testing named **TestFile1**. For now, it does not need to contain any data:
|
||||
|
||||
|
||||
```
|
||||
`[student@studentvm1 testdir]$ touch TestFile1`
|
||||
```
|
||||
|
||||
It is easy to change the value of the **$File** variable rather than a text string for the file name in multiple locations in this short CLI program:
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ File="TestFile1" ; if [ -e $File ] ; then echo "The file $File exists." ; else echo "The file $File does not exist." ; fi
|
||||
The file TestFile1 exists.
|
||||
[student@studentvm1 testdir]$
|
||||
```
|
||||
|
||||
Now, run a test to determine whether a file exists and has a non-zero length, which means it contains data. You want to test for three conditions: 1. the file does not exist; 2. the file exists and is empty; and 3. the file exists and contains data. Therefore, you need a more complex set of tests—use the **elif** stanza in the **if-elif-else** construct to test for all of the conditions:
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ File="TestFile1" ; if [ -s $File ] ; then echo "$File exists and contains data." ; fi
|
||||
[student@studentvm1 testdir]$
|
||||
```
|
||||
|
||||
In this case, the file exists but does not contain any data. Add some data and try again:
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ File="TestFile1" ; echo "This is file $File" > $File ; if [ -s $File ] ; then echo "$File exists and contains data." ; fi
|
||||
TestFile1 exists and contains data.
|
||||
[student@studentvm1 testdir]$
|
||||
```
|
||||
|
||||
That works, but it is only truly accurate for one specific condition out of the three possible ones. Add an **else** stanza so you can be somewhat more accurate, and delete the file so you can fully test this new code:
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ File="TestFile1" ; rm $File ; if [ -s $File ] ; then echo "$File exists and contains data." ; else echo "$File does not exist or is empty." ; fi
|
||||
TestFile1 does not exist or is empty.
|
||||
```
|
||||
|
||||
Now create an empty file to test:
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ File="TestFile1" ; touch $File ; if [ -s $File ] ; then echo "$File exists and contains data." ; else echo "$File does not exist or is empty." ; fi
|
||||
TestFile1 does not exist or is empty.
|
||||
```
|
||||
|
||||
Add some content to the file and test again:
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ File="TestFile1" ; echo "This is file $File" > $File ; if [ -s $File ] ; then echo "$File exists and contains data." ; else echo "$File does not exist or is empty." ; fi
|
||||
TestFile1 exists and contains data.
|
||||
```
|
||||
|
||||
Now, add the **elif** stanza to discriminate between a file that does not exist and one that is empty:
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ File="TestFile1" ; touch $File ; if [ -s $File ] ; then echo "$File exists and contains data." ; elif [ -e $File ] ; then echo "$File exists and is empty." ; else echo "$File does not exist." ; fi
|
||||
TestFile1 exists and is empty.
|
||||
[student@studentvm1 testdir]$ File="TestFile1" ; echo "This is $File" > $File ; if [ -s $File ] ; then echo "$File exists and contains data." ; elif [ -e $File ] ; then echo "$File exists and is empty." ; else echo "$File does not exist." ; fi
|
||||
TestFile1 exists and contains data.
|
||||
[student@studentvm1 testdir]$
|
||||
```
|
||||
|
||||
Now you have a Bash CLI program that can test for these three different conditions… but the possibilities are endless.
|
||||
|
||||
It is easier to see the logic structure of the more complex compound commands if you arrange the program statements more like you would in a script that you can save in a file. Figure 2 shows how this would look. The indents of the program statements in each stanza of the **if-elif-else** structure help to clarify the logic.
|
||||
|
||||
|
||||
```
|
||||
File="TestFile1"
|
||||
echo "This is $File" > $File
|
||||
if [ -s $File ]
|
||||
then
|
||||
echo "$File exists and contains data."
|
||||
elif [ -e $File ]
|
||||
then
|
||||
echo "$File exists and is empty."
|
||||
else
|
||||
echo "$File does not exist."
|
||||
fi
|
||||
```
|
||||
|
||||
_**Fig. 2: The command line program rewritten as it would appear in a script**_
|
||||
|
||||
Logic this complex is too lengthy for most CLI programs. Although any Linux or Bash built-in commands may be used in CLI programs, as the CLI programs get longer and more complex, it makes more sense to create a script that is stored in a file and can be executed at any time, now or in the future.
|
||||
|
||||
#### String comparison operators
|
||||
|
||||
String comparison operators enable the comparison of alphanumeric strings of characters. There are only a few of these operators, which are listed in Figure 3.
|
||||
|
||||
Operator | Description
|
||||
---|---
|
||||
-z string | True if the length of string is zero
|
||||
-n string | True if the length of string is non-zero
|
||||
string1 == string2
|
||||
or
|
||||
string1 = string2 | True if the strings are equal; a single **=** should be used with the test command for POSIX conformance. When used with the **[[** command, this performs pattern matching as described above (compound commands).
|
||||
string1 != string2 | True if the strings are not equal
|
||||
string1 < string2 | True if string1 sorts before string2 lexicographically (refers to locale-specific sorting sequences for all alphanumeric and special characters)
|
||||
string1 > string2 | True if string1 sorts after string2 lexicographically
|
||||
|
||||
_**Fig. 3: Bash string logical operators**_
|
||||
|
||||
First, look at string length. The quotes around **$MyVar** in the comparison must be there for the comparison to work. (You should still be working in **~/testdir**.)
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ MyVar="" ; if [ -z "" ] ; then echo "MyVar is zero length." ; else echo "MyVar contains data" ; fi
|
||||
MyVar is zero length.
|
||||
[student@studentvm1 testdir]$ MyVar="Random text" ; if [ -z "" ] ; then echo "MyVar is zero length." ; else echo "MyVar contains data" ; fi
|
||||
MyVar is zero length.
|
||||
```
|
||||
|
||||
You could also do it this way:
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ MyVar="Random text" ; if [ -n "$MyVar" ] ; then echo "MyVar contains data." ; else echo "MyVar is zero length" ; fi
|
||||
MyVar contains data.
|
||||
[student@studentvm1 testdir]$ MyVar="" ; if [ -n "$MyVar" ] ; then echo "MyVar contains data." ; else echo "MyVar is zero length" ; fi
|
||||
MyVar is zero length
|
||||
```
|
||||
|
||||
Sometimes you may need to know a string's exact length. This is not a comparison, but it is related. Unfortunately, there is no simple way to determine the length of a string. There are a couple of ways to do it, but I think using the **expr** (evaluate expression) command is easiest. Read the man page for **expr** for more about what it can do. Note that quotes are required around the string or variable you're testing.
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ MyVar="" ; expr length "$MyVar"
|
||||
0
|
||||
[student@studentvm1 testdir]$ MyVar="How long is this?" ; expr length "$MyVar"
|
||||
17
|
||||
[student@studentvm1 testdir]$ expr length "We can also find the length of a literal string as well as a variable."
|
||||
70
|
||||
```
|
||||
|
||||
Regarding comparison operators, I use a lot of testing in my scripts to determine whether two strings are equal (i.e., identical). I use the non-POSIX version of this comparison operator:
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ Var1="Hello World" ; Var2="Hello World" ; if [ "$Var1" == "$Var2" ] ; then echo "Var1 matches Var2" ; else echo "Var1 and Var2 do not match." ; fi
|
||||
Var1 matches Var2
|
||||
[student@studentvm1 testdir]$ Var1="Hello World" ; Var2="Hello world" ; if [ "$Var1" == "$Var2" ] ; then echo "Var1 matches Var2" ; else echo "Var1 and Var2 do not match." ; fi
|
||||
Var1 and Var2 do not match.
|
||||
```
|
||||
|
||||
Experiment some more on your own to try out these operators.
|
||||
|
||||
#### Numeric comparison operators
|
||||
|
||||
Numeric operators make comparisons between two numeric arguments. Like the other operator classes, most are easy to understand.
|
||||
|
||||
Operator | Description
|
||||
---|---
|
||||
arg1 -eq arg2 | True if arg1 equals arg2
|
||||
arg1 -ne arg2 | True if arg1 is not equal to arg2
|
||||
arg1 -lt arg2 | True if arg1 is less than arg2
|
||||
arg1 -le arg2 | True if arg1 is less than or equal to arg2
|
||||
arg1 -gt arg2 | True if arg1 is greater than arg2
|
||||
arg1 -ge arg2 | True if arg1 is greater than or equal to arg2
|
||||
|
||||
_**Fig. 4: Bash numeric comparison logical operators**_
|
||||
|
||||
Here are some simple examples. The first instance sets the variable **$X** to 1, then tests to see if **$X** is equal to 1. In the second instance, **X** is set to 0, so the comparison is not true.
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ X=1 ; if [ $X -eq 1 ] ; then echo "X equals 1" ; else echo "X does not equal 1" ; fi
|
||||
X equals 1
|
||||
[student@studentvm1 testdir]$ X=0 ; if [ $X -eq 1 ] ; then echo "X equals 1" ; else echo "X does not equal 1" ; fi
|
||||
X does not equal 1
|
||||
[student@studentvm1 testdir]$
|
||||
```
|
||||
|
||||
Try some more experiments on your own.
|
||||
|
||||
#### Miscellaneous operators
|
||||
|
||||
These miscellaneous operators show whether a shell option is set or a shell variable has a value, but it does not discover the value of the variable, just whether it has one.
|
||||
|
||||
Operator | Description
|
||||
---|---
|
||||
-o optname | True if the shell option optname is enabled (see the list of options under the description of the **-o** option to the Bash set builtin in the Bash man page)
|
||||
-v varname | True if the shell variable varname is set (has been assigned a value)
|
||||
-R varname | True if the shell variable varname is set and is a name reference
|
||||
|
||||
_**Fig. 5: Miscellaneous Bash logical operators**_
|
||||
|
||||
Experiment on your own to try out these operators.
|
||||
|
||||
### Expansions
|
||||
|
||||
Bash supports a number of types of expansions and substitutions that can be quite useful. According to the Bash man page, Bash has seven forms of expansions. This article looks at five of them: tilde expansion, arithmetic expansion, pathname expansion, brace expansion, and command substitution.
|
||||
|
||||
#### Brace expansion
|
||||
|
||||
Brace expansion is a method for generating arbitrary strings. (This tool is used below to create a large number of files for experiments with special pattern characters.) Brace expansion can be used to generate lists of arbitrary strings and insert them into a specific location within an enclosing static string or at either end of a static string. This may be hard to visualize, so it's best to just do it.
|
||||
|
||||
First, here's what a brace expansion does:
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ echo {string1,string2,string3}
|
||||
string1 string2 string3
|
||||
```
|
||||
|
||||
Well, that is not very helpful, is it? But look what happens when you use it just a bit differently:
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ echo "Hello "{David,Jen,Rikki,Jason}.
|
||||
Hello David. Hello Jen. Hello Rikki. Hello Jason.
|
||||
```
|
||||
|
||||
That looks like something useful—it could save a good deal of typing. Now try this:
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ echo b{ed,olt,ar}s
|
||||
beds bolts bars
|
||||
```
|
||||
|
||||
I could go on, but you get the idea.
|
||||
|
||||
#### Tilde expansion
|
||||
|
||||
Arguably, the most common expansion is the tilde (**~**) expansion. When you use this in a command like **cd ~/Documents**, the Bash shell expands it as a shortcut to the user's full home directory.
|
||||
|
||||
Use these Bash programs to observe the effects of the tilde expansion:
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ echo ~
|
||||
/home/student
|
||||
[student@studentvm1 testdir]$ echo ~/Documents
|
||||
/home/student/Documents
|
||||
[student@studentvm1 testdir]$ Var1=~/Documents ; echo $Var1 ; cd $Var1
|
||||
/home/student/Documents
|
||||
[student@studentvm1 Documents]$
|
||||
```
|
||||
|
||||
#### Pathname expansion
|
||||
|
||||
Pathname expansion is a fancy term expanding file-globbing patterns, using the characters **?** and *****, into the full names of directories that match the pattern. File globbing refers to special pattern characters that enable significant flexibility in matching file names, directories, and other strings when performing various actions. These special pattern characters allow matching single, multiple, or specific characters in a string.
|
||||
|
||||
* **?** — Matches only one of any character in the specified location within the string
|
||||
* ***** — Matches zero or more of any character in the specified location within the string
|
||||
|
||||
|
||||
|
||||
This expansion is applied to matching directory names. To see how this works, ensure that **testdir** is the present working directory (PWD) and start with a plain listing (the contents of my home directory will be different from yours):
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ ls
|
||||
chapter6 cpuHog.dos dmesg1.txt Documents Music softlink1 testdir6 Videos
|
||||
chapter7 cpuHog.Linux dmesg2.txt Downloads Pictures Templates testdir
|
||||
testdir cpuHog.mac dmesg3.txt file005 Public testdir tmp
|
||||
cpuHog Desktop dmesg.txt link3 random.txt testdir1 umask.test
|
||||
[student@studentvm1 testdir]$
|
||||
```
|
||||
|
||||
Now list the directories that start with **Do**, **testdir/Documents**, and **testdir/Downloads**:
|
||||
|
||||
|
||||
```
|
||||
Documents:
|
||||
Directory01 file07 file15 test02 test10 test20 testfile13 TextFiles
|
||||
Directory02 file08 file16 test03 test11 testfile01 testfile14
|
||||
file01 file09 file17 test04 test12 testfile04 testfile15
|
||||
file02 file10 file18 test05 test13 testfile05 testfile16
|
||||
file03 file11 file19 test06 test14 testfile09 testfile17
|
||||
file04 file12 file20 test07 test15 testfile10 testfile18
|
||||
file05 file13 Student1.txt test08 test16 testfile11 testfile19
|
||||
file06 file14 test01 test09 test18 testfile12 testfile20
|
||||
|
||||
Downloads:
|
||||
[student@studentvm1 testdir]$
|
||||
```
|
||||
|
||||
Well, that did not do what you wanted. It listed the contents of the directories that begin with **Do**. To list only the directories and not their contents, use the **-d** option.
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ ls -d Do*
|
||||
Documents Downloads
|
||||
[student@studentvm1 testdir]$
|
||||
```
|
||||
|
||||
In both cases, the Bash shell expands the **Do*** pattern into the names of the two directories that match the pattern. But what if there are also files that match the pattern?
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ touch Downtown ; ls -d Do*
|
||||
Documents Downloads Downtown
|
||||
[student@studentvm1 testdir]$
|
||||
```
|
||||
|
||||
This shows the file, too. So any files that match the pattern are also expanded to their full names.
|
||||
|
||||
#### Command substitution
|
||||
|
||||
Command substitution is a form of expansion that allows the STDOUT data stream of one command to be used as the argument of another command; for example, as a list of items to be processed in a loop. The Bash man page says: "Command substitution allows the output of a command to replace the command name." I find that to be accurate if a bit obtuse.
|
||||
|
||||
There are two forms of this substitution, **`command`** and **$(command)**. In the older form using back tics (**`**), using a backslash (**\**) in the command retains its literal meaning. However, when it's used in the newer parenthetical form, the backslash takes on its meaning as a special character. Note also that the parenthetical form uses only single parentheses to open and close the command statement.
|
||||
|
||||
I frequently use this capability in command-line programs and scripts where the results of one command can be used as an argument for another command.
|
||||
|
||||
Start with a very simple example that uses both forms of this expansion (again, ensure that **testdir** is the PWD):
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ echo "Todays date is `date`"
|
||||
Todays date is Sun Apr 7 14:42:46 EDT 2019
|
||||
[student@studentvm1 testdir]$ echo "Todays date is $(date)"
|
||||
Todays date is Sun Apr 7 14:42:59 EDT 2019
|
||||
[student@studentvm1 testdir]$
|
||||
```
|
||||
|
||||
The **-w** option to the **seq** utility adds leading zeros to the numbers generated so that they are all the same width, i.e., the same number of digits regardless of the value. This makes it easier to sort them in numeric sequence.
|
||||
|
||||
The **seq** utility is used to generate a sequence of numbers:
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ seq 5
|
||||
1
|
||||
2
|
||||
3
|
||||
4
|
||||
5
|
||||
[student@studentvm1 testdir]$ echo `seq 5`
|
||||
1 2 3 4 5
|
||||
[student@studentvm1 testdir]$
|
||||
```
|
||||
|
||||
Now you can do something a bit more useful, like creating a large number of empty files for testing:
|
||||
|
||||
|
||||
```
|
||||
`[student@studentvm1 testdir]$ for I in $(seq -w 5000) ; do touch file-$I ; done`
|
||||
```
|
||||
|
||||
In this usage, the statement **seq -w 5000** generates a list of numbers from one to 5,000. By using command substitution as part of the **for** statement, the list of numbers is used by the **for** statement to generate the numerical part of the file names.
|
||||
|
||||
#### Arithmetic expansion
|
||||
|
||||
Bash can perform integer math, but it is rather cumbersome (as you will soon see). The syntax for arithmetic expansion is **$((arithmetic-expression))**, using double parentheses to open and close the expression.
|
||||
|
||||
Arithmetic expansion works like command substitution in a shell program or script; the value calculated from the expression replaces the expression for further evaluation by the shell.
|
||||
|
||||
Once again, start with something simple:
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ echo $((1+1))
|
||||
2
|
||||
[student@studentvm1 testdir]$ Var1=5 ; Var2=7 ; Var3=$((Var1*Var2)) ; echo "Var 3 = $Var3"
|
||||
Var 3 = 35
|
||||
```
|
||||
|
||||
The following division results in zero because the result would be a decimal value of less than one:
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ Var1=5 ; Var2=7 ; Var3=$((Var1/Var2)) ; echo "Var 3 = $Var3"
|
||||
Var 3 = 0
|
||||
```
|
||||
|
||||
Here is a simple calculation I often do in a script or CLI program that tells me how much total virtual memory I have in a Linux host. The **free** command does not provide that data:
|
||||
|
||||
|
||||
```
|
||||
[student@studentvm1 testdir]$ RAM=`free | grep ^Mem | awk '{print $2}'` ; Swap=`free | grep ^Swap | awk '{print $2}'` ; echo "RAM = $RAM and Swap = $Swap" ; echo "Total Virtual memory is $((RAM+Swap))" ;
|
||||
RAM = 4037080 and Swap = 6291452
|
||||
Total Virtual memory is 10328532
|
||||
```
|
||||
|
||||
I used the **`** character to delimit the sections of code used for command substitution.
|
||||
|
||||
I use Bash arithmetic expansion mostly for checking system resource amounts in a script and then choose a program execution path based on the result.
|
||||
|
||||
### Summary
|
||||
|
||||
This article, the second in this series on Bash as a programming language, explored the Bash file, string, numeric, and miscellaneous logical operators that provide execution-flow control logic and the different types of shell expansions.
|
||||
|
||||
The third article in this series will explore the use of loops for performing various types of iterative operations.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensource.com/article/19/10/programming-bash-part-2
|
||||
|
||||
作者：[David Both][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://opensource.com/users/dboth
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/OSDC_women_computing_5.png?itok=YHpNs_ss (Women in computing and open source v5)
|
||||
[2]: http://www.both.org/?page_id=1183
|
||||
[3]: https://opensource.com/article/19/10/programming-bash-part-1
|
||||
389
sources/tech/20191022 Initializing arrays in Java.md
Normal file
389
sources/tech/20191022 Initializing arrays in Java.md
Normal file
@@ -0,0 +1,389 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
[#]: subject: (Initializing arrays in Java)
|
||||
[#]: via: (https://opensource.com/article/19/10/initializing-arrays-java)
|
||||
[#]: author: (Chris Hermansen https://opensource.com/users/clhermansen)
|
||||
|
||||
Initializing arrays in Java
|
||||
======
|
||||
Arrays are a helpful data type for managing collections elements best
|
||||
modeled in contiguous memory locations. Here's how to use them
|
||||
effectively.
|
||||
![Coffee beans and a cup of coffee][1]
|
||||
|
||||
People who have experience programming in languages like C or FORTRAN are familiar with the concept of arrays. They’re basically a contiguous block of memory where each location is a certain type: integers, floating-point numbers, or what-have-you.
|
||||
|
||||
The situation in Java is similar, but with a few extra wrinkles.
|
||||
|
||||
### An example array
|
||||
|
||||
Let’s make an array of 10 integers in Java:
|
||||
|
||||
|
||||
```
|
||||
int[] ia = new int[10];
|
||||
```
|
||||
|
||||
What’s going on in the above piece of code? From left to right:
|
||||
|
||||
1. The **int[]** to the extreme left declares the _type_ of the variable as an array (denoted by the **[]**) of **int**.
|
||||
|
||||
2. To the right is the _name_ of the variable, which in this case is **ia**.
|
||||
|
||||
3. Next, the **=** tells us that the variable defined on the left side is set to what’s to the right side.
|
||||
|
||||
4. To the right of the **=** we see the word **new**, which in Java indicates that an object is being _initialized_, meaning that storage is allocated and its constructor is called ([see here for more information][2]).
|
||||
|
||||
5. Next, we see **int[10]**, which tells us that the specific object being initialized is an array of 10 integers.
|
||||
|
||||
|
||||
|
||||
|
||||
Since Java is strongly-typed, the type of the variable **ia** must be compatible with the type of the expression on the right-hand side of the **=**.
|
||||
|
||||
### Initializing the example array
|
||||
|
||||
Let’s put this simple array in a piece of code and try it out. Save the following in a file called **Test1.java**, use **javac** to compile it, and use **java** to run it (in the terminal of course):
|
||||
|
||||
|
||||
```
|
||||
import java.lang.*;
|
||||
|
||||
public class Test1 {
|
||||
|
||||
public static void main([String][3][] args) {
|
||||
int[] ia = new int[10]; // See note 1 below
|
||||
[System][4].out.println("ia is " + ia.getClass()); // See note 2 below
|
||||
for (int i = 0; i < ia.length; i++) // See note 3 below
|
||||
[System][4].out.println("ia[" + i + "] = " + ia[i]); // See note 4 below
|
||||
}
|
||||
|
||||
}
|
||||
```
|
||||
|
||||
Let’s work through the most important bits.
|
||||
|
||||
1. Our declaration and initialization of the array of 10 integers, **ia**, is easy to spot.
|
||||
2. In the line just following, we see the expression **ia.getClass()**. That’s right, **ia** is an _object_ belonging to a _class_, and this code will let us know which class that is.
|
||||
3. In the next line following that, we see the start of the loop **for (int i = 0; i < ia.length; i++)**, which defines a loop index variable **i** that runs through a sequence from zero to one less than **ia.length**, which is an expression that tells us how many elements are defined in the array **ia**.
|
||||
4. Next, the body of the loop prints out the values of each element of **ia**.
|
||||
|
||||
|
||||
|
||||
When this program is compiled and run, it produces the following results:
|
||||
|
||||
|
||||
```
|
||||
me@mydesktop:~/Java$ javac Test1.java
|
||||
me@mydesktop:~/Java$ java Test1
|
||||
ia is class [I
|
||||
ia[0] = 0
|
||||
ia[1] = 0
|
||||
ia[2] = 0
|
||||
ia[3] = 0
|
||||
ia[4] = 0
|
||||
ia[5] = 0
|
||||
ia[6] = 0
|
||||
ia[7] = 0
|
||||
ia[8] = 0
|
||||
ia[9] = 0
|
||||
me@mydesktop:~/Java$
|
||||
```
|
||||
|
||||
The string representation of the output of **ia.getClass()** is **[I**, which is shorthand for "array of integer." Similar to the C programming language, Java arrays begin with element zero and extend up to element **<array size> – 1**. We can see above that each of the elements of **ia** are set to zero (by the array constructor, it seems).
|
||||
|
||||
So, is that it? We declare the type, use the appropriate initializer, and we’re done?
|
||||
|
||||
Well, no. There are many other ways to initialize an array in Java.
|
||||
|
||||
### Why do I want to initialize an array, anyway?
|
||||
|
||||
The answer to this question, like that of all good questions, is "it depends." In this case, the answer depends on what we expect to do with the array once it is initialized.
|
||||
|
||||
In some cases, arrays emerge naturally as a type of accumulator. For example, suppose we are writing code for counting the number of calls received and made by a set of telephone extensions in a small office. There are eight extensions, numbered one through eight, plus the operator’s extension, numbered zero. So we might declare two arrays:
|
||||
|
||||
|
||||
```
|
||||
int[] callsMade;
|
||||
int[] callsReceived;
|
||||
```
|
||||
|
||||
Then, whenever we start a new period of accumulating call statistics, we initialize each array as:
|
||||
|
||||
|
||||
```
|
||||
callsMade = new int[9];
|
||||
callsReceived = new int[9];
|
||||
```
|
||||
|
||||
At the end of each period of accumulating call statistics, we can print out the stats. In very rough terms, we might see:
|
||||
|
||||
|
||||
```
|
||||
import java.lang.*;
|
||||
import java.io.*;
|
||||
|
||||
public class Test2 {
|
||||
|
||||
public static void main([String][3][] args) {
|
||||
|
||||
int[] callsMade;
|
||||
int[] callsReceived;
|
||||
|
||||
// initialize call counters
|
||||
|
||||
callsMade = new int[9];
|
||||
callsReceived = new int[9];
|
||||
|
||||
// process calls...
|
||||
// an extension makes a call: callsMade[ext]++
|
||||
// an extension receives a call: callsReceived[ext]++
|
||||
|
||||
// summarize call statistics
|
||||
|
||||
[System][4].out.printf("%3s%25s%25s\n","ext"," calls made",
|
||||
"calls received");
|
||||
for (int ext = 0; ext < callsMade.length; ext++)
|
||||
[System][4].out.printf("%3d%25d%25d\n",ext,
|
||||
callsMade[ext],callsReceived[ext]);
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
```
|
||||
|
||||
Which would produce output something like this:
|
||||
|
||||
|
||||
```
|
||||
me@mydesktop:~/Java$ javac Test2.java
|
||||
me@mydesktop:~/Java$ java Test2
|
||||
ext calls made calls received
|
||||
0 0 0
|
||||
1 0 0
|
||||
2 0 0
|
||||
3 0 0
|
||||
4 0 0
|
||||
5 0 0
|
||||
6 0 0
|
||||
7 0 0
|
||||
8 0 0
|
||||
me@mydesktop:~/Java$
|
||||
```
|
||||
|
||||
Not a very busy day in the call center.
|
||||
|
||||
In the above example of an accumulator, we see that the starting value of zero as set by the array initializer is satisfactory for our needs. But in other cases, this starting value may not be the right choice.
|
||||
|
||||
For example, in some kinds of geometric computations, we might need to initialize a two-dimensional array to the identity matrix (all zeros except for the ones along the main diagonal). We might choose to do this as:
|
||||
|
||||
|
||||
```
|
||||
double[][] m = new double[3][3];
|
||||
for (int d = 0; d < 3; d++)
|
||||
m[d][d] = 1.0;
|
||||
```
|
||||
|
||||
In this case, we rely on the array initializer **new double[3][3]** to set the array to zeros, and then use a loop to set the diagonal elements to ones. In this simple case, we might use a shortcut that Java provides:
|
||||
|
||||
|
||||
```
|
||||
double[][] m = {
|
||||
{1.0, 0.0, 0.0},
|
||||
{0.0, 1.0, 0.0},
|
||||
{0.0, 0.0, 1.0}};
|
||||
```
|
||||
|
||||
This type of visual structure is particularly appropriate in this sort of application, where it can be a useful double-check to see the actual layout of the array. But in the case where the number of rows and columns is only determined at run time, we might instead see something like this:
|
||||
|
||||
|
||||
```
|
||||
int nrc;
|
||||
// some code determines the number of rows & columns = nrc
|
||||
double[][] m = new double[nrc][nrc];
|
||||
for (int d = 0; d < nrc; d++)
|
||||
m[d][d] = 1.0;
|
||||
```
|
||||
|
||||
It’s worth mentioning that a two-dimensional array in Java is actually an array of arrays, and there’s nothing stopping the intrepid programmer from having each one of those second-level arrays be a different length. That is, something like this is completely legitimate:
|
||||
|
||||
|
||||
```
|
||||
int [][] differentLengthRows = {
|
||||
{ 1, 2, 3, 4, 5},
|
||||
{ 6, 7, 8, 9},
|
||||
{10,11,12},
|
||||
{13,14},
|
||||
{15}};
|
||||
```
|
||||
|
||||
There are various linear algebra applications that involve irregularly-shaped matrices, where this type of structure could be applied (for more information see [this Wikipedia article][5] as a starting point). Beyond that, now that we understand that a two-dimensional array is actually an array of arrays, it shouldn’t be too much of a surprise that:
|
||||
|
||||
|
||||
```
|
||||
differentLengthRows.length
|
||||
```
|
||||
|
||||
tells us the number of rows in the two-dimensional array **differentLengthRows**, and:
|
||||
|
||||
|
||||
```
|
||||
differentLengthRows[i].length
|
||||
```
|
||||
|
||||
tells us the number of columns in row **i** of **differentLengthRows**.
|
||||
|
||||
### Taking the array further
|
||||
|
||||
Considering this idea of array size that is determined at run time, we see that arrays still require us to know that size before instantiating them. But what if we don’t know the size until we’ve processed all of the data? Does that mean we have to process it once to figure out the size of the array, and then process it again? That could be hard to do, especially if we only get one chance to consume the data.
|
||||
|
||||
The [Java Collections Framework][6] solves this problem in a nice way. One of the things provided there is the class **ArrayList**, which is like an array but dynamically extensible. To demonstrate the workings of **ArrayList**, let’s create one and initialize it to the first 20 [Fibonacci numbers][7]:
|
||||
|
||||
|
||||
```
|
||||
import java.lang.*;
|
||||
import java.util.*;
|
||||
|
||||
public class Test3 {
|
||||
|
||||
public static void main([String][3][] args) {
|
||||
|
||||
ArrayList<Integer> fibos = new ArrayList<Integer>();
|
||||
|
||||
fibos.add(0);
|
||||
fibos.add(1);
|
||||
for (int i = 2; i < 20; i++)
|
||||
fibos.add(fibos.get(i-1) + fibos.get(i-2));
|
||||
|
||||
for (int i = 0; i < fibos.size(); i++)
|
||||
[System][4].out.println("fibonacci " + i +
|
||||
" = " + fibos.get(i));
|
||||
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Above, we see:
|
||||
|
||||
* The declaration and instantiation of an **ArrayList** that is used to store **Integer**s.
|
||||
* The use of **add()** to append to the **ArrayList** instance.
|
||||
* The use of **get()** to retrieve an element by index number.
|
||||
* The use of **size()** to determine how many elements are already in the **ArrayList** instance.
|
||||
|
||||
|
||||
|
||||
Not shown is the **put()** method, which places a value at a given index number.
|
||||
|
||||
The output of this program is:
|
||||
|
||||
|
||||
```
|
||||
fibonacci 0 = 0
|
||||
fibonacci 1 = 1
|
||||
fibonacci 2 = 1
|
||||
fibonacci 3 = 2
|
||||
fibonacci 4 = 3
|
||||
fibonacci 5 = 5
|
||||
fibonacci 6 = 8
|
||||
fibonacci 7 = 13
|
||||
fibonacci 8 = 21
|
||||
fibonacci 9 = 34
|
||||
fibonacci 10 = 55
|
||||
fibonacci 11 = 89
|
||||
fibonacci 12 = 144
|
||||
fibonacci 13 = 233
|
||||
fibonacci 14 = 377
|
||||
fibonacci 15 = 610
|
||||
fibonacci 16 = 987
|
||||
fibonacci 17 = 1597
|
||||
fibonacci 18 = 2584
|
||||
fibonacci 19 = 4181
|
||||
```
|
||||
|
||||
**ArrayList** instances can also be initialized by other techniques. For example, an array can be supplied to the **ArrayList** constructor, or the **List.of()** and **Arrays.asList()** methods can be used when the initial elements are known at compile time. I don’t find myself using these options all that often since my primary use case for an **ArrayList** is when I only want to read the data once.
|
||||
|
||||
Moreover, an **ArrayList** instance can be converted to an array using its **toArray()** method, for those who prefer to work with an array once the data is loaded; or, returning to the current topic, once the **ArrayList** instance is initialized.
|
||||
|
||||
The Java Collections Framework provides another kind of array-like data structure called a **Map**. What I mean by "array-like" is that a **Map** defines a collection of objects whose values can be set or retrieved by a key, but unlike an array (or an **ArrayList**), this key need not be an integer; it could be a **String** or any other complex object.
|
||||
|
||||
For example, we can create a **Map** whose keys are **String**s and whose values are **Integer**s as follows:
|
||||
|
||||
|
||||
```
|
||||
Map<[String][3],Integer> stoi = new Map<[String][3],Integer>();
|
||||
```
|
||||
|
||||
Then we can initialize this **Map** as follows:
|
||||
|
||||
|
||||
```
|
||||
stoi.set("one",1);
|
||||
stoi.set("two",2);
|
||||
stoi.set("three",3);
|
||||
```
|
||||
|
||||
And so on. Later, when we want to know the numeric value of **"three"**, we can retrieve it as:
|
||||
|
||||
|
||||
```
|
||||
stoi.get("three");
|
||||
```
|
||||
|
||||
In my world, a **Map** is useful for converting strings occurring in third-party datasets into coherent code values in my datasets. As a part of a [data transformation pipeline][8], I will often build a small standalone program to clean the data before processing it; for this, I will almost always use one or more **Map**s.
|
||||
|
||||
Worth mentioning is that it’s quite possible, and sometimes reasonable, to have **ArrayLists** of **ArrayLists** and **Map**s of **Map**s. For example, let’s assume we’re looking at trees, and we’re interested in accumulating the count of the number of trees by tree species and age range. Assuming that the age range definition is a set of string values ("young," "mid," "mature," and "old") and that the species are string values like "Douglas fir," "western red cedar," and so forth, then we might define a **Map** of **Map**s as:
|
||||
|
||||
|
||||
```
|
||||
Map<[String][3],Map<[String][3],Integer>> counter =
|
||||
new Map<[String][3],Map<[String][3],Integer>>();
|
||||
```
|
||||
|
||||
One thing to watch out for here is that the above only creates storage for the _rows_ of **Map**s. So, our accumulation code might look like:
|
||||
|
||||
|
||||
```
|
||||
// assume at this point we have figured out the species
|
||||
// and age range
|
||||
if (!counter.containsKey(species))
|
||||
counter.put(species,new Map<[String][3],Integer>());
|
||||
if (!counter.get(species).containsKey(ageRange))
|
||||
counter.get(species).put(ageRange,0);
|
||||
```
|
||||
|
||||
At which point, we can start accumulating as:
|
||||
|
||||
|
||||
```
|
||||
counter.get(species).put(ageRange,
|
||||
counter.get(species).get(ageRange) + 1);
|
||||
```
|
||||
|
||||
Finally, it’s worth mentioning that the (new in Java 8) Streams facility can also be used to initialize arrays, **ArrayList** instances, and **Map** instances. A nice discussion of this feature can be found [here][9] and [here][10].
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensource.com/article/19/10/initializing-arrays-java
|
||||
|
||||
作者：[Chris Hermansen][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://opensource.com/users/clhermansen
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/java-coffee-mug.jpg?itok=Bj6rQo8r (Coffee beans and a cup of coffee)
|
||||
[2]: https://opensource.com/article/19/8/what-object-java
|
||||
[3]: http://www.google.com/search?hl=en&q=allinurl%3Adocs.oracle.com+javase+docs+api+string
|
||||
[4]: http://www.google.com/search?hl=en&q=allinurl%3Adocs.oracle.com+javase+docs+api+system
|
||||
[5]: https://en.wikipedia.org/wiki/Irregular_matrix
|
||||
[6]: https://en.wikipedia.org/wiki/Java_collections_framework
|
||||
[7]: https://en.wikipedia.org/wiki/Fibonacci_number
|
||||
[8]: https://towardsdatascience.com/data-science-for-startups-data-pipelines-786f6746a59a
|
||||
[9]: https://stackoverflow.com/questions/36885371/lambda-expression-to-initialize-array
|
||||
[10]: https://stackoverflow.com/questions/32868665/how-to-initialize-a-map-using-a-lambda
|
||||
@@ -0,0 +1,258 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: ( )
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
[#]: subject: (NGT: A library for high-speed approximate nearest neighbor search)
|
||||
[#]: via: (https://opensource.com/article/19/10/ngt-open-source-library)
|
||||
[#]: author: (Masajiro Iwasaki https://opensource.com/users/masajiro-iwasaki)
|
||||
|
||||
NGT: A library for high-speed approximate nearest neighbor search
|
||||
======
|
||||
NGT is a high-performing, open source deep learning library for
|
||||
large-scale and high-dimensional vectors.
|
||||
![Houses in a row][1]
|
||||
|
||||
Approximate nearest neighbor ([ANN][2]) search is used in deep learning to make a best guess at the point in a given set that is most similar to another point. This article explains the differences between ANN search and traditional search methods and introduces [NGT][3], a top-performing open source ANN library developed by [Yahoo! Japan Research][4].
|
||||
|
||||
### Nearest neighbor search for high-dimensional data
|
||||
|
||||
Different search methods are used for different data types. For example, full-text search is for text data, content-based image retrieval is for images, and relational databases are for data relationships. Deep learning models can easily generate vectors from various kinds of data so that the vector space has embedded relationships among source data. This means that if two source data are similar, the two vectors from the data will be located near each other in the vector space. Therefore, all you have to do is search the vectors instead of the source data.
|
||||
|
||||
Moreover, the vectors not only represent the text and image characteristics of the source data, but they also represent products, human beings, organizations, and so forth. Therefore, you can search for similar documents and images as well as products with similar attributes, human beings with similar skills, clothing with similar features, and so on. For example, [Yahoo! Japan][5] provides a similarity-based fashion-item search using NGT.
|
||||
|
||||
![Nearest neighbour search][6]
|
||||
|
||||
Since the number of dimensions in deep learning models tends to increase, ANN search methods are indispensable when searching for more than several million high-dimensional vectors. ANN search methods allow you to search for neighbors to the specified query vector in high-dimensional space.
|
||||
|
||||
There are many nearest-neighbor search methods to choose from. [ANN Benchmarks][7] evaluates the best-known ANN search methods, including Faiss (Facebook), Flann, and Hnswlib. According to this benchmark, NGT achieves top-level performance.
|
||||
|
||||
### NGT algorithms
|
||||
|
||||
The NGT index combines a graph and a tree. This result is a very good search performance, with the graph's vertices representing searchable objects. Neighboring vertices are connected by edges.
|
||||
|
||||
This animation shows how a graph is constructed.
|
||||
|
||||
![NGT graph construction][8]
|
||||
|
||||
In the search procedure, neighboring vertices to the specified query can be found descending the graph. Densely connected vertices enable users to explore the graph effectively.
|
||||
|
||||
![NGT graph][9]
|
||||
|
||||
NGT provides a command-line tool, along with C, C++, and Python APIs. This article focuses on the command-line tool and the Python API.
|
||||
|
||||
### Using NGT with the command-line tool
|
||||
|
||||
#### Linux installation
|
||||
|
||||
Download the [latest version of NGT][10] as a ZIP file and install it on Linux with:
|
||||
|
||||
|
||||
```
|
||||
unzip NGT-x.x.x.zip
|
||||
cd NGT-x.x.x
|
||||
mkdir build
|
||||
cd build
|
||||
cmake ..
|
||||
make
|
||||
make install
|
||||
```
|
||||
|
||||
Since NGT libraries are installed in **/usr/local/lib(64)** by default, add the directory to the search path:
|
||||
|
||||
|
||||
```
|
||||
export PATH="$PATH:/opt/local/bin"
|
||||
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib"
|
||||
```
|
||||
|
||||
#### Sample dataset generation
|
||||
|
||||
Before you can search for a large-scale dataset, you must generate an NGT dataset. As an example, [download the][11] [fastText][11] [dataset][11] from the [fastText website][12], then convert it to the NGT registration format with:
|
||||
|
||||
|
||||
```
|
||||
curl -O <https://dl.fbaipublicfiles.com/fasttext/vectors-english/wiki-news-300d-1M-subword.vec.zip>
|
||||
unzip wiki-news-300d-1M-subword.vec.zip
|
||||
tail -n +2 wiki-news-300d-1M-subword.vec | cut -d " " -f 2- > objects.ssv
|
||||
```
|
||||
|
||||
**Objects.ssv** is a registration file that has 1 million objects. One object in the file is extracted as a query:
|
||||
|
||||
|
||||
```
|
||||
`head -10000 objects.ssv | tail -1 > query.ssv`
|
||||
```
|
||||
|
||||
#### Index construction
|
||||
|
||||
An **ngt_index** can be constructed using the following command:
|
||||
|
||||
|
||||
```
|
||||
`ngt create -d 300 -D c index objects.ssv`
|
||||
```
|
||||
|
||||
_-d_ specifies the number of dimensions of the vector. _-D c_ means using cosine similarity.
|
||||
|
||||
#### Approximate nearest neighbor search
|
||||
|
||||
The **ngt_index** can be searched for with the queries using:
|
||||
|
||||
|
||||
```
|
||||
`ngt search -n 10 index query.ssv`
|
||||
```
|
||||
|
||||
**-n** specifies the number of resulting objects.
|
||||
|
||||
The search results are:
|
||||
|
||||
|
||||
```
|
||||
Query No.1
|
||||
Rank ID Distance
|
||||
1 10000 0
|
||||
2 21516 0.184495
|
||||
3 201860 0.240375
|
||||
4 71865 0.241284
|
||||
5 339589 0.267265
|
||||
6 485158 0.280977
|
||||
7 7961 0.283865
|
||||
8 924513 0.286571
|
||||
9 28870 0.286654
|
||||
10 395274 0.290466
|
||||
Query Time= 0.000972628 (sec), 0.972628 (msec)
|
||||
Average Query Time= 0.000972628 (sec), 0.972628 (msec), (0.000972628/1)
|
||||
```
|
||||
|
||||
Please see the [NGT command-line README][13] for more information.
|
||||
|
||||
### Using NGT from Python
|
||||
|
||||
Although NGT has C and C++ APIs, the [ngtpy][14] Python binding for NGT is the simplest option for programming.
|
||||
|
||||
#### Installing ngtpy
|
||||
|
||||
Install the Python binding (ngtpy) through PyPI with:
|
||||
|
||||
|
||||
```
|
||||
`pip3 install ngt`
|
||||
```
|
||||
|
||||
#### Sample dataset generation
|
||||
|
||||
Generate data files for Python sample programs from the sample data set you downloaded by using this code:
|
||||
|
||||
|
||||
```
|
||||
dataset_path = 'wiki-news-300d-1M-subword.vec'
|
||||
with open(dataset_path, 'r') as fi, open('objects.tsv', 'w') as fov,
|
||||
open('words.tsv', 'w') as fow:
|
||||
n, dim = map(int, fi.readline().split())
|
||||
fov.write('{0}¥t{1}¥n'.format(n, dim))
|
||||
for line in fi:
|
||||
tokens = line.rstrip().split(' ')
|
||||
fow.write(tokens[0] + '¥n')
|
||||
fov.write('{0}¥n'.format('¥t'.join(tokens[1:])))
|
||||
```
|
||||
|
||||
#### Index construction
|
||||
|
||||
Construct the NGT index with:
|
||||
|
||||
|
||||
```
|
||||
import ngtpy
|
||||
|
||||
index_path = 'index'
|
||||
with open('objects.tsv', 'r') as fin:
|
||||
n, dim = map(int, fin.readline().split())
|
||||
ngtpy.create(index_path, dim, distance_type='Cosine') # create an index
|
||||
index = ngtpy.Index(index_path) # open the index
|
||||
print('inserting objects...')
|
||||
for line in fin:
|
||||
object = list(map(float, line.rstrip().split('¥t')))
|
||||
index.insert(object) # insert objects
|
||||
print('building objects...')
|
||||
index.build_index()
|
||||
print('saving the index...')
|
||||
index.save()
|
||||
```
|
||||
|
||||
#### Approximate nearest neighbor search
|
||||
|
||||
Here is an example ANN search program:
|
||||
|
||||
|
||||
```
|
||||
import ngtpy
|
||||
|
||||
print('loading words...')
|
||||
with open('words.tsv', 'r') as fin:
|
||||
words = list(map(lambda x: x.rstrip('¥n'), fin.readlines()))
|
||||
|
||||
index = ngtpy.Index('index', zero_based_numbering = False) # open index
|
||||
query_id = 10000
|
||||
query_object = index.get_object(query_id) # get the object for a query
|
||||
|
||||
result = index.search(query_object) # aproximate nearest neighbor search
|
||||
print('Query={}'.format(words[query_id - 1]))
|
||||
print('Rank¥tID¥tDistance¥tWord')
|
||||
for rank, object in enumerate(result):
|
||||
print('{}¥t{}¥t{:.6f}¥t{}'.format(rank + 1, object[0], object[1], words[object[0] - 1]))
|
||||
```
|
||||
|
||||
And here are the search results, which are the same as the NGT command-line option's results:
|
||||
|
||||
|
||||
```
|
||||
loading words...
|
||||
Query=Horse
|
||||
Rank ID Distance Word
|
||||
1 10000 0.000000 Horse
|
||||
2 21516 0.184495 Horses
|
||||
3 201860 0.240375 Horseback
|
||||
4 71865 0.241284 Horseman
|
||||
5 339589 0.267265 Prancing
|
||||
6 485158 0.280977 Horsefly
|
||||
7 7961 0.283865 Dog
|
||||
8 924513 0.286571 Horsing
|
||||
9 28870 0.286654 Pony
|
||||
10 395274 0.290466 Blood-Horse
|
||||
```
|
||||
|
||||
For more information, please see [ngtpy README][14].
|
||||
|
||||
Approximate nearest neighbor (ANN) principles are important features for analyzing data. Learning how to use it in your own projects, or to make sense of data that you're analyzing, is a powerful way to make correlations and interpret information. With NGT, you can use ANN in whatever way you require, or build upon it to add custom features.
|
||||
|
||||
Introduction to Apache Hadoop, an open source software framework for storage and large scale...
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://opensource.com/article/19/10/ngt-open-source-library
|
||||
|
||||
作者：[Masajiro Iwasaki][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[译者ID](https://github.com/译者ID)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://opensource.com/users/masajiro-iwasaki
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://opensource.com/sites/default/files/styles/image-full-size/public/lead-images/house_home_colors_live_building.jpg?itok=HLpsIfIL (Houses in a row)
|
||||
[2]: https://en.wikipedia.org/wiki/Nearest_neighbor_search#Approximate_nearest_neighbor
|
||||
[3]: https://github.com/yahoojapan/NGT
|
||||
[4]: https://research-lab.yahoo.co.jp/en/
|
||||
[5]: https://www.yahoo.co.jp/
|
||||
[6]: https://opensource.com/sites/default/files/browser-visual-search_new.jpg (Nearest neighbour search)
|
||||
[7]: https://github.com/erikbern/ann-benchmarks
|
||||
[8]: https://opensource.com/sites/default/files/uploads/ngt_movie2.gif (NGT graph construction)
|
||||
[9]: https://opensource.com/sites/default/files/uploads/ngt_movie1.gif (NGT graph)
|
||||
[10]: https://github.com/yahoojapan/NGT/releases/latest
|
||||
[11]: https://dl.fbaipublicfiles.com/fasttext/vectors-english/wiki-news-300d-1M-subword.vec.zip
|
||||
[12]: https://fasttext.cc/
|
||||
[13]: https://github.com/yahoojapan/NGT/blob/master/bin/ngt/README.md
|
||||
[14]: https://github.com/yahoojapan/NGT/blob/master/python/README-ngtpy.md
|
||||
@@ -0,0 +1,69 @@
|
||||
[#]: collector: (lujun9972)
|
||||
[#]: translator: (geekpi)
|
||||
[#]: reviewer: ( )
|
||||
[#]: publisher: ( )
|
||||
[#]: url: ( )
|
||||
[#]: subject: (Linux sudo flaw can lead to unauthorized privileges)
|
||||
[#]: via: (https://www.networkworld.com/article/3446036/linux-sudo-flaw-can-lead-to-unauthorized-privileges.html)
|
||||
[#]: author: (Sandra Henry-Stocker https://www.networkworld.com/author/Sandra-Henry_Stocker/)
|
||||
|
||||
Linux sudo 漏洞可能导致未经授权的特权
|
||||
======
|
||||
在 Linux 中利用新发现的 sudo 漏洞可以使某些用户以 root 身份运行命令，尽管对此还有所限制。
|
||||
|
||||
[**sudo**][1] 命令中最近发现了一个严重漏洞，如果被利用，那么即使在 **/etc/sudoers** 文件中明确禁止了该用户，它们也可以以 root 身份运行命令。。
|
||||
|
||||
将 **sudo** 更新到版本 1.8.28 应该可以解决该问题，因此鼓励 Linux 管理员尽快这样做。
|
||||
|
||||
如何利用此漏洞取决于 **/etc/sudoers** 中授予的特定权限。例如，一条规则允许用户以除 root 用户之外的任何用户身份来编辑文件，这实际上将允许该用户也以 root 用户身份来编辑文件。在这种情况下，该漏洞可能会导致非常严重的问题。
|
||||
|
||||
要让用户能够利用此漏洞，需要在 **/etc/sudoers ** 中为**用户**分配权限，以使该用户可以像其他用户一样运行命令，并且该漏洞仅限于以这种方式分配的命令特权。
|
||||
|
||||
此问题影响 1.8.28 之前的版本。要检查你的 sudo 版本，请使用以下命令：
|
||||
|
||||
```
|
||||
$ sudo -V
|
||||
Sudo version 1.8.27 <===
|
||||
Sudoers policy plugin version 1.8.27
|
||||
Sudoers file grammar version 46
|
||||
Sudoers I/O plugin version 1.8.27
|
||||
```
|
||||
|
||||
该漏洞已在“常见漏洞和披露”数据库中分配了编号 [CVE-2019-14287][4]。它的风险是，任何被指定能以任意用户运行单个命令的用户，即使被明确禁止以 root 身份运行，它都能逃脱限制。
|
||||
|
||||
下面这些行让 “jdoe” 能够以除了 root 用户之外的其他身份使用 **vi**编辑文件（**!root**表示“非 root”），同时 nemo有权运行以除了 root 身份以外的任何用户使用 **id** 命令：
|
||||
|
||||
```
|
||||
# affected entries on host "dragonfly"
|
||||
jdoe dragonfly = (ALL, !root) /usr/bin/vi
|
||||
nemo dragonfly = (ALL, !root) /usr/bin/id
|
||||
```
|
||||
|
||||
但是，由于存在漏洞，这些用户中的任何一个都将能够绕过限制并编辑文件，或者也可以 root 用户身份运行 **id** 命令。
|
||||
|
||||
|
||||
攻击者可以通过指定用户 ID 为 “-1” 或 “4294967295” 来以 root 身份运行命令。
|
||||
|
||||
响应 “1” 表明该命令以 root 身份运行（显示 root 的用户 ID）。
|
||||
|
||||
苹果信息安全团队的 Joe Vennix 找到并分析该问题。
|
||||
|
||||
在 [Facebook][5] 和 [LinkedIn][6] 加入 Network World 社区，评论热门主题。
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
via: https://www.networkworld.com/article/3446036/linux-sudo-flaw-can-lead-to-unauthorized-privileges.html
|
||||
|
||||
作者：[Sandra Henry-Stocker][a]
|
||||
选题：[lujun9972][b]
|
||||
译者：[geekpi](https://github.com/geekpi)
|
||||
校对：[校对者ID](https://github.com/校对者ID)
|
||||
|
||||
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
|
||||
|
||||
[a]: https://www.networkworld.com/author/Sandra-Henry_Stocker/
|
||||
[b]: https://github.com/lujun9972
|
||||
[1]: https://www.networkworld.com/article/3236499/some-tricks-for-using-sudo.html
|
||||
[4]: http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-14287
|
||||
[5]: https://www.facebook.com/NetworkWorld/
|
||||
[6]: https://www.linkedin.com/company/network-world
|
||||
Reference in New Issue
Block a user