FreeBSD Quarterly Status Report - First Quarter 2019
4 June, 2019 by trasz@freebsd.org | freebsd
FreeBSD Project Quarterly Status Report - 1st Quarter 2019
As spring leads into summer, we reflect back on what the FreeBSD
project has accomplished in the first quarter of 2019. Events included
FOSDEM and AsiaBSDCon, the FreeBSD Journal is now free to everyone,
ASLR is available in -CURRENT and KPTI can be controlled per-process.
The run up to 11.3-RELEASE has begun, and a team is applying syzkaller
guided fuzzing to the kernel, plus so much more. Catch up on many new
and ongoing efforts throughout the project, and find where you can
pitch in.
__________________________________________________________________
FreeBSD Team Reports
* Continuous Integration
* FreeBSD Core Team
* FreeBSD Foundation
* FreeBSD Release Engineering Team
* Ports Collection
Projects
* AXP803 PMIC driver update
* Broadcom ARM64 SoC support
* C Runtime changes
* Capsicum
* CFT - Package Base
* ENA FreeBSD Driver Update
* FreeBSD boot security improvements
* FUSE
* Kernel ZLIB Update
* LLVM's lld as the FreeBSD system linker
* mlx5 Drivers Update
* PCI Express Resets
* Security-Related changes
Architectures
* FreeBSD/RISC-V Update
Ports
* FreeBSD GNOME status report
* FreeBSD KDE status report
Third-Party Projects
* FreeBSD Wiki Apple Intel Mac mini update
* Fuzzing FreeBSD with syzkaller
* sysctlmibinfo API 1.0
* sysctlview 1.0
* University of Waterloo Co-operative Education Students
__________________________________________________________________
FreeBSD Team Reports
Entries from the various official and semi-official teams, as found in
the Administration Page.
Continuous Integration
Links
FreeBSD Jenkins Instance
URL: https://ci.FreeBSD.org
FreeBSD CI artifact archive
URL: https://artifact.ci.FreeBSD.org/
FreeBSD Jenkins wiki
URL: https://wiki.freebsd.org/Jenkins
freebsd-testing Mailing List
URL: https://lists.FreeBSD.org/mailman/listinfo/freebsd-testing
freebsd-ci Repository
URL: https://github.com/freebsd/freebsd-ci
Tickets related to freebsd-testing@
URL: https://preview.tinyurl.com/y9maauwg
Hosted CI wiki
URL: https://wiki.freebsd.org/HostedCI
FreeBSD CI weekly report
URL: https://hackfoldr.org/freebsd-ci-report/
Contact: Jenkins Admin <jenkins-admin@FreeBSD.org>
Contact: Li-Wen Hsu <lwhsu@FreeBSD.org>
The FreeBSD CI team maintains continuous integration system and related
tasks for the FreeBSD project. The CI system regularly checks the
changes committed to the project's Subversion repository can be
successfully built, and performs various tests and analysis of the
results. The results from build jobs are archived in an artifact
server, for the further testing and debugging needs. The CI team
members examine the failing builds and unstable tests, and work with
the experts in that area to fix the code or adjust test infrastructure.
Starting from this quarter, we started to publish CI weekly report at
freebsd-testing@ mailing list. The archive is available at
https://hackfoldr.org/freebsd-ci-report/
We also worked on extending test executing environment to improve the
code coverage, temporarily disabling flakey test cases, and opening
tickets to work with domain experts. The details are of these efforts
are available in the weekly CI reports.
We published the draft FCP for CI policy and are ready to accept
comments.
Please see freebsd-testing@ related tickets for more information.
Work in progress:
* Fixing the failing test cases and builds
* Adding drm ports building test against -CURRENT
* Implementing automatic tests on bare metal hardware
* Implementing the embedded testbed
* Planning for running ztest and network stack tests
* Help more 3rd software get CI on FreeBSD through a hosted CI
solution
__________________________________________________________________
FreeBSD Core Team
Contact: FreeBSD Core Team <core@FreeBSD.org>
The FreeBSD Core Team is the governing body of FreeBSD.
Core initiated a Release Engineering Charter Modernization working
group. The purpose of the working group is to present (to Core) a
modernized version of the Release Engineering Charter and a first
version of a new Release Engineering Team Operations Plan. The group
hopes to complete its goals and dissolve by 2019-06-30.
The Core Team invites all members of the FreeBSD community to complete
the 2019 FreeBSD Community Survey.
https://www.research.net/r/freebsd2019
The purpose of the survey is to collect quantitative data from the
public in order to help guide the project's priorities and efforts. It
will remain open for 17 days and close at midnight May 13 UTC (Monday
5pm PDT). (Editor's note: Survey has finished)
Core voted to approve source commit bits for Johannes Lundberg
(johalun@) and Mitchell Horne (mhorne@) and associate membership for
Philip Jocks. Core also voted to revoke Michael Dexter's documentation
bit.
After a long lapse of not closing idle source commit bits, core has
taken in the commit bit for these developers. We thank each for
contributing to the project as a source committer.
* Alfred Perlstein (alfred@)
* Eric Badger (badger@)
* Daniel Eischen (deischen@)
* Ermal Luçi (eri@)
* Tony Finch (fanf@)
* Justin T. Gibbs (gibbs@)
* Imre Vadász (ivadasz@)
* Julio Merino (jmmv@)
* John W. De Boskey (jwd@)
* Kai Wang (kaiw@)
* Luigi Rizzo (luigi@)
* Neel Natu (neel@)
* Craig Rodrigues (rodrigc@)
* Stanislav Sedov (stas@)
* Thomas Quinot (thomas@)
* Andrew Thompson (thompsa@)
* Pyun YongHyeon (yongari@)
* Zbigniew Bodek (zbb@)
__________________________________________________________________
FreeBSD Foundation
Contact: Deb Goodkin <deb@FreeBSDFoundation.org>
The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated
to supporting and promoting the FreeBSD Project and community
worldwide. Funding comes from individual and corporate donations and is
used to fund and manage software development projects, conferences and
developer summits, and provide travel grants to FreeBSD contributors.
The Foundation purchases and supports hardware to improve and maintain
FreeBSD infrastructure and provides resources to improve security,
quality assurance, and release engineering efforts; publishes marketing
material to promote, educate, and advocate for the FreeBSD Project;
facilitates collaboration between commercial vendors and FreeBSD
developers; and finally, represents the FreeBSD Project in executing
contracts, license agreements, and other legal arrangements that
require a recognized legal entity.
Here are some highlights of what we did to help FreeBSD last quarter:
We kicked off the year with an all-day board meeting in Berkeley, where
FreeBSD began, to put together high-level plans for 2019. This included
prioritizing technologies and features we should support, long-term
planning for the next 2-5 years, and philosophical discussions on our
purpose and goals.
Partnerships and Commercial User Support
We began the year by meeting with a few commercial users, to help them
navigate working with the Project, and understanding how they are using
FreeBSD. We're also in the process of setting up meetings for Q2 and
throughout the rest of 2019. Because we're a 501(c)(3) non-profit, we
don't directly support commercial users. However, these meetings allow
us to focus on facilitating collaboration with the community.
Fundraising Efforts
Our work is 100% funded by your donations. We kicked off the year with
many individual and corporate donations, including donations and
commitments from NetApp, Netflix, Intel, Tarsnap, Beckhoff Automation,
E-Card, VMware, and Stormshield. We are working hard to get more
commercial users to give back to help us continue our work supporting
FreeBSD. Please consider making a donation to help us continue and
increase our support for FreeBSD at: www.FreeBSDfoundation.org/donate/.
We also have the Partnership Program, to provide more benefits for our
larger commercial donors. Find out more information at
https://www.FreeBSDfoundation.org/FreeBSD-foundation-partnership-program/
and share with your companies!
OS Improvements
The Foundation improves the FreeBSD operating system by employing our
technical staff to maintain and improve critical kernel subsystems, add
features and functionality, and fix problems. This also includes
funding separate project grants like the arm64 port, porting the
blacklistd access control daemon, and the integration of VIMAGE
support, to make sure that FreeBSD remains a viable solution for
research, education, computing, products and more.
Over the quarter there were 241 commits from nine Foundation-sponsored
staff members and grant recipients.
We kicked off or continued the following projects last quarter:
* FUSE file system kernel support (update and bug fixes)
* Linuxulator testing and diagnostics improvements
* SDIO and WiFi infrastructure improvements
* x86-64 scalability and performance improvements
* OpenZFS Online RAID-Z Expansion
Having software developers on staff has allowed us to jump in and work
directly on projects to improve FreeBSD like:
* amd64 and i386 pmap improvements and bugfixes
* address userland threading library issues
* improve i386 support to keep the platform viable
* improve FreeBSD on RISC-V
* application of the Capsicum sandboxing framework
* build system improvements and bug fixes
* respond to reports of security issues
* implement vulnerability mitigations
* tool chain updates and improvements
* adding kernel code coverage support for the Syzkaller
coverage-guided system call fuzzer
* improved Syzkaller support for FreeBSD
* improve the usability of freebsd-update
* improve network stack stability and address race conditions
* ensure FreeBSD provides userland interfaces required by
contemporary applications
* implement support for machine-dependent optimized subroutines
* update and correct documentation and manpages
* DTrace bug fixes
* update the FreeBSD Valgrind port and try to upstream the changes
Continuous Integration and Quality Assurance
The Foundation provides a full-time staff member who is working on
improving our automated testing, continuous integration, and overall
quality assurance efforts.
During the first quarter of 2019, Foundation staff continued improving
the project's CI infrastructure, working with contributors to fix
failing build and test cases, and working with other teams in the
project for their testing needs. In this quarter, we started publishing
the CI weekly report on the freebsd-testing@ mailing list.
See the FreeBSD CI section of this report for more information.
Release Engineering
The Foundation provides a full-time staff member to oversee the release
engineering efforts. This has provided timely and reliable releases
over the last five years.
During the first quarter of 2019, the FreeBSD Release Engineering team
continued providing weekly development snapshots for 13-CURRENT,
12-STABLE, and 11-STABLE.
In addition, the Release Engineering team published the schedule for
the upcoming 11.3-RELEASE cycle, the fourth release from the stable/11
branch, which builds on the stability and reliability of 11.2-RELEASE.
The upcoming 11.3-RELEASE schedule can be found at:
https://www.freebsd.org/releases/11.3R/schedule.html
FreeBSD 11.3 is currently targeted for final release in early July
2019.
Please see the FreeBSD Release Engineering Team section of this
quarterly status report for additional details surrounding the above
mentioned work.
Supporting FreeBSD Infrastructure
The Foundation provides hardware and support to improve FreeBSD
infrastructure. Last quarter, we continued supporting FreeBSD hardware
located around the world.
FreeBSD Advocacy and Education
A large part of our efforts are dedicated to advocating for the
Project. This includes promoting work being done by others with
FreeBSD; producing advocacy literature to teach people about FreeBSD
and help make the path to starting using FreeBSD or contributing to the
Project easier; and attending and getting other FreeBSD contributors to
volunteer to run FreeBSD events, staff FreeBSD tables, and give FreeBSD
presentations.
The FreeBSD Foundation sponsors many conferences, events, and summits
around the globe. These events can be BSD-related, open source, or
technology events geared towards underrepresented groups. We support
the FreeBSD-focused events to help provide a venue for sharing
knowledge, to work together on projects, and to facilitate
collaboration between developers and commercial users. This all helps
provide a healthy ecosystem. We support the non-FreeBSD events to
promote and raise awareness of FreeBSD, to increase the use of FreeBSD
in different applications, and to recruit more contributors to the
Project.
Check out some of the advocacy and education work we did last quarter:
* Attended FOSDEM 2019 where we: staffed the FreeBSD Stand, sponsored
the co-located FreeBSD Developer Summit, and gave the 25 Years of
FreeBSD presentation in the BSD Dev room.
* Sponsored and presented at SANOG33 in Thimphu, Bhutan
* Represented FreeBSD at APRICOT 2019 in Yuseong-gu, Daejeon South
Korea
* Sponsored the USENIX FAST conference in Boston, MA as an Industry
Partner
* Ran our first ever FreeBSD track at SCALE 17x, which included an
all-day Getting Started with FreeBSD workshop. We were thrilled
with the turnout of almost 30 participants and received a lot of
positive feedback. Thanks to Roller Angel who taught the class with
the help of Deb Goodkin and Gordon Tetlow. We also promoted FreeBSD
at the FreeBSD table in the Expo Hall.
* Sponsored, presented, and exhibited at FOSSASIA in Singapore
* Sponsored AsiaBSDCon 2019
* Committed to sponsoring Rootconf, BSDCan, and EuroBSDcon
* Created registration systems for the Aberdeen Hackathon and the
upcoming 2019 Vienna FreeBSD Security Hackathon
* Provided FreeBSD advocacy material
* Provided 3 travel grants to FreeBSD contributors to attend many of
the above events.
We continued producing FreeBSD advocacy material to help people promote
FreeBSD around the world.
Read more about our conference adventures in the conference recaps and
trip reports in our monthly newsletters.
We help educate the world about FreeBSD by publishing the
professionally produced FreeBSD Journal. We're excited to announce that
with the release of the January/February 2019 issue, the FreeBSD
Journal is now a free publication. Find out more and access the latest
issues at www.FreeBSDfoundation.org/journal/.
You can find out more about events we attended and upcoming events at
www.FreeBSDfoundation.org/news-and-events/.
We also engaged with a new website developer to help us improve our
website to make it easier for community members to find information
more easily and to make the site more efficient.
Legal/FreeBSD IP
The Foundation owns the FreeBSD trademarks, and it is our
responsibility to protect them. We also provide legal support for the
core team to investigate questions that arise.
Go to www.FreeBSDfoundation.org to find out how we support FreeBSD and
how we can help you!
__________________________________________________________________
FreeBSD Release Engineering Team
Links
FreeBSD 11.3-RELEASE schedule
URL: https://www.freebsd.org/releases/11.3R/schedule.html
FreeBSD development snapshots
URL: https://download.freebsd.org/ftp/snapshots/ISO-IMAGES/
Contact: FreeBSD Release Engineering Team <re@FreeBSD.org>
The FreeBSD Release Engineering Team is responsible for setting and
publishing release schedules for official project releases of FreeBSD,
announcing code freezes and maintaining the respective branches, among
other things.
During the first quarter of 2019, the FreeBSD Release Engineering team
published the initial schedule for the upcoming the 11.3-RELEASE.
FreeBSD 11.3-RELEASE will be the fourth release from the stable/11
branch, building on the stability and reliability of 11.2-RELEASE.
FreeBSD 11.3-RELEASE is currently targed for release in early July,
2019.
Additionally throughout the quarter, several development snapshots
builds were released for the head, stable/12, and stable/11 branches.
Much of this work was sponsored by the FreeBSD Foundation.
__________________________________________________________________
Ports Collection
Links
About FreeBSD Ports
URL: https://www.FreeBSD.org/ports/
Contributing to Ports
URL: https://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributing/ports-contributing.html
FreeBSD Ports Monitoring
URL: http://portsmon.freebsd.org/index.html
Ports Management Team
URL: https://www.freebsd.org/portmgr/index.html
Contact: René Ladan <portmgr-secretary@FreeBSD.org>
Contact: FreeBSD Ports Management Team <portmgr@FreeBSD.org>
As always, below is a summary of what happened in the Ports Tree during
the last quarter.
During 2019q1, the number of ports dropped slightly to just over
32,500. At the end of the quarter, we had 2092 open port PRs. The last
quarter saw 8205 commits from 167 committers. So more PRs were closed
and more commits were made than in 2018q4.
During the last quarter, we welcomed Kai Knoblich (kai@) and said
goodbye to Matthew Rezny (rezny@).
On the infrastructure side, two new USES were introduced (azurepy and
sdl) and USES=gecko was removed. The default versions of Lazarus and
LLVM were bumped to 2.0.0 and 8.0 respectively. Some big port
frameworks that were end-of-life were removed: PHP 5.6, Postgresql 9.3,
Qt4, WebKit-Gtk and XPI. Firefox was updated to 66.0.2, Firefox-ESR to
60.6.1, and Chromium was updated to 72.0.3626.121.
During the last quarter, antoine@ ran 30 exp-runs for package updates,
moving from GNU ld to LLVM ld, and switching clang to DWARF4.
__________________________________________________________________
Projects
Projects that span multiple categories, from the kernel and userspace
to the Ports Collection or external projects.
AXP803 PMIC driver update
Contact: Ganbold Tsagaankhuu <ganbold@FreeBSD.org>
The AXP803 is a highly integrated PMIC that targets Li-battery (Li-ion
or Li-polymer) applications. It provides flexible power management
solution for processors such as the Allwinner A64 SoC. This SoC is used
by Pinebook.
The following updates were performed on the AXP803 driver:
* Enabled necessary bits when activating interrupts. This allows
reading some events from the interrupt status registers. These
events are reported to devd via system "PMU" and subsystem
"Battery", "AC" and "USB" such as plugged/unplugged, battery
absent, charged and charging.
* Added sensors support for AXP803/AXP813. Sensor values such as
battery charging, charge state, voltage, charging current,
discharging current, battery capacity can be obtained via sysctl.
* Added sysctl for setting battery charging current. The charging
current can be set using steps from 0 to 13. These steps correspond
to 200mA to 2800mA, with a granularity of 200mA/step.
__________________________________________________________________
Broadcom ARM64 SoC support
Contact: Michal Stanek <mst@semihalf.com>
Contact: Marcin Wojtas <mw@semihalf.com>
The Semihalf team continued working on FreeBSD support for the Broadcom
BCM5871X SoC series
BCM5871X are quad-core 64-bit ARMv8 Cortex-A57 communication processors
targeted for networking applications such as 10G routers, gateways,
control plane processing and NAS.
Completed since the last update:
* iProc PCIe root complex (internal and external buses)
* OTP (One Time Programmable memory) driver
In progress:
* BNXT Ethernet support
* Crypto engine acceleration for IPsec offloading.
Todo:
* Upstreaming of work. This work is expected to be submitted/merged
to HEAD in the second half of 2019.
This project was sponsored by Juniper Networks, Inc.
__________________________________________________________________
C Runtime changes
Contact: Konstantin Belousov <kib@freebsd.org>
Several changes where made to the C runtime which generally improves
the environment provided to an application.
Fix for libraries with initial exec TLS mode
Some libraries, most prominent of which is NVidia-provided and thus
binary-only libGL.so.1, use so called initial exec mode for TLS
variables access. This is the fastest mode of TLS access, but its
drawback is that it only reliably work when the main binary is linked
against the library, i.e. dlopen-ing the library to load it at runtime
is not guaranteed to work.
This mode works by placing the TLS variables for objects in one area
allocated during the executable initialization, which somewhat explains
the name of the mode. An obvious consequence is that if such library is
loaded later, there is no space in the TLS area for an application to
put its TLS variables.
The FreeBSD dynamic linker is aware of misbehaviour of the app
builders, and provides some amount of slack in the TLS area to give
space for such libraries. But it appeared that the initial content of
the TLS segment from libraries was not distributed among the threads'
TLS areas, still breaking libraries which use initial exec mode for
TLS.
Another issue that somewhat mitigates mis-use of the mode is the
DF_STATIC_TLS flag in the dynamic section. This flag allows the linker
to check for the space earlier and avoid loading dependencies if there
is no total required space. This linker flag was implemented by the BFD
ld linker, but not by the LLVM lld linker.
The FreeBSD dynamic linker was fixed to properly distribute TLS
initialization data to all threads' initial segments, which required
reasonably extensive per-architecture changes to libc and libthr.
Simultaneously, LLD was improved to mark libraries using initial exec
TLS mode with the appropriate flag.
These measures should make FreeBSD more resilent to improperly linked
libraries. The most interesting fix is to users of the nvidia libgl
library, because it cannot be fixed by relinking.
Use rtld malloc in libthr
The FreeBSD implementation of mutexes in libthr allocates some memory
to keep the mutex data needed for mutex initialization. In contrast,
the malloc implementation used by FreeBSD, jemalloc(3), requires
working pthread mutexes for operation.
This creates a chicken-and-egg problem during executable startup, and
requires jemalloc to provide fragile hacks to make it possible to
initialize mutexes. This has been a constant source of mismatches on
imports of new versions of jemalloc.
The FreeBSD rtld implementation already contained a very light-weight
malloc implementation, suitable for limited use in pre-C-runtime
environments. This seemed to be the ideal fit for an allocator for the
pthread private mutexes memory. By using this allocator, a method to
address the cyclic dependencies between jemalloc and libthr could
finally be implemented.
The entry points in the rtld malloc.c were renamed to avoid a clash
with the libc exported symbols, and now the file is linked statically
into libthr, providing an allocator for private mutexes and pthread key
storage. The later was already switched to direct use of mmap(2) for
similar reasons. Now less memory is wasted when key storage requires
less than a page.
Destructors order bug
Alexander Kabaev (kan@) noted that C++ destructors for the static
objects from the linked shared libraries are executed before C++
destructors of the static objects from the main binary. This was
verified both for clang++ and g++, but amusingly not for
__attribute__(((destructor))).
The bug was introduced when init functions and init arrays for main
binary startup are called from the rtld instead of csu (C startup code
linked to the binary, typically from crt1.o). The cause is due to the
somewhat complicated way of how destructors are called both by
fini/fini arrays and rtld-registered atexit(3) handler.
Solution is to register rtld atexit(3) handler before main binary init
functions are called, using new internal ABI __libc_atexit() function.
It is amusing that the bug was not noticed for so many years.
This project was sponsored by The FreeBSD Foundation.
__________________________________________________________________
Capsicum
Links
Capsicum Wiki Page
URL: https://wiki.FreeBSD.org/Capsicum
Contact: Enji Cooper <ngie@freebsd.org>
Contact: Mark Johnston <markj@FreeBSD.org>
Contact: Ed Maste <emaste@FreeBSD.org>
Contact: Mariusz Zaborski <oshogbo@FreeBSD.org>
Contact: Bora Özarslan <borako.ozarslan@gmail.com>
Three themes for Capsicum work were:
* Importing Google's Capsicum test suite into FreeBSD
* Porting and sandboxing openrsync for FreeBSD
* Applying capsicum to additional base system utilities
The Googletest-based Capsicum test cases are now integrated into
FreeBSD. After some discussion with David Drysdale - the main
maintainer and developer for the Capsicum port on Linux - we decided
that from now the FreeBSD will be upstream for Capsicum test cases.
The next major step was sandboxing openrsync. In the course of that
work we extended our fileargs service with two new functionalities. We
modified the fileargs service to allow limiting the operations which
can be performed, and can now delegate lstat to the Casper service.
Furthermore, openrsync highly depends on the fts API. We spend some
time in optimizing fts and making it sandbox friendly by introducing
fts_openat function and removing the need to change the working
directory to traverse the paths. The changes to the fts API are now in
the tests phase.
Moreover, we improved bootstrapping for non-FreeBSD machines. Thanks to
this work we can now build tools needed to bootstrap FreeBSD which use
Casper services. In the base system strings is now sandboxed as a
result.
We also sandboxed rtsol, rtsold, and savecore.
We host biweekly Capsicum calls. The notes from the meetings are
published in FreeBSD's Capsium meeting repository on GitHub. If you
would like to join the call do not hesitate to send us an email.
__________________________________________________________________
CFT - Package Base
Links
Package Base CFT - FAQ
URL: https://trueos.github.io/pkgbase-docs/
Contact: Kris Moore <kmoore@FreeBSD.org>
The TrueOS project has been working on a Package Base implementation,
and is pleased to issue its first CFT to the FreeBSD community.
The TrueOS packaging work has been in development for close to 6
months, and differs from the original FreeBSD package base effort, in
that it is an "out of tree" implementation. It allows any version of
FreeBSD to be packaged, and only requires a patch to poudriere, as well
as some minor ports enhancements, the first which is currently in
review. For more information on the current status, please refer to the
FAQ page.
Additionally there will be a working-group at BSDCan 2019, and we
encourage porters to attend and join the discussion.
This project was sponsored by iXsystems Inc.
__________________________________________________________________
ENA FreeBSD Driver Update
Links
ENA README
URL: https://github.com/amzn/amzn-drivers/blob/master/kernel/fbsd/ena/README
Contact: Michal Krawczyk <mk@semihalf.com>
Contact: Marcin Wojtas <mw@semihalf.com>
ENA (Elastic Network Adapter) is the smart NIC available in the
virtualized environment of Amazon Web Services (AWS). The ENA driver
supports multiple transmit and receive queues and can handle up to 100
Gb/s of network traffic, depending on the instance type on which it is
used.
ENAv2 has been under development for FreeBSD, similar to Linux and
DPDK. Since the last update internal review and improvements of the
patches were done, followed by validation on various AWS instances.
To do:
* Upstream of the ENAv2 patches
Recently, AWS released the A1 instances which are arm64 instances. The
FreeBSD kernel was fixed, so the ENA can be used on those instances
with no issues. There were changes required in resource activation in
the ENA driver r345371 and the addition of a missing bus release method
to the nexus module for aarch64 r345373. With these changes, the ENA
driver can run on A1 instances without any known issues.
This project was sponsored by Amazon.com Inc.
__________________________________________________________________
FreeBSD boot security improvements
Links
Veriexec manifest verification in kernel
URL: https://svnweb.freebsd.org/changeset/base/345830
TPM as entropy source
URL: https://svnweb.freebsd.org/changeset/base/345438
UEFI support in libsecureboot
URL: https://svnweb.freebsd.org/changeset/base/344840
Contact: Michal Stanek <mst@semihalf.com>
Contact: Marcin Wojtas <mw@semihalf.com>
Contact: Kornel Duleba <mindal@semihalf.com>
FreeBSD gained TPM 2.0 (Trusted Platform Module) support at the end of
2018. A kernel configuration option, TPM_HARVEST, was also added to use
the TPM RNG as system entropy source. When used this way, the TPM can
be harvested every ten seconds for entropy which is mixed into the OS
entropy pool. The kernel option is currently disabled by default in
amd64 GENERIC kernel configuration.
UEFI Secure Boot support, developed by Semihalf, has been merged with
sjg's Veriexec support, resulting in a unified library named
libsecureboot. This library is used for verification of kernel and
modules by the loader. The library uses BearSSL as the cryptographic
backend. The library supports loading trusted and blacklisted
certificates from UEFI (DB/DBx databases) and can use them as trust
anchors for the verification.
The library is also used by Veriexec to verify and parse the
authentication database (called 'manifest') in the kernel. Previously
the manifest was verified and parsed by a userspace application, then
sent to the kernel via /dev/veriexec, which was a significant
limitation and a security weakness.
To do:
* Backport to stable branches.
Special thanks to sjg and Juniper for fruitful cooperation around
Veriexec and the libsecureboot development.
This project was sponsored by Stormshield.
__________________________________________________________________
FUSE
Contact: Alan Somers <asomers@FreeBSD.org>
FUSE (File system in USErspace) allows a userspace program to implement
a file system. It is widely used to support out-of-tree file systems
like NTFS, as well as for exotic pseudo file systems like sshfs.
FreeBSD's fuse driver was added as a GSoC project in 2012. Since that
time, it has been largely neglected. The FUSE software is buggy and
out-of-date. Our implementation is about 11 years behind.
The FreeBSD Foundation has agreed to fund a project to improve the
state of the FreeBSD FUSE driver. So far I've written a test suite for
the fusefs(5) module, fixed 1 previously reported bug, discovered and
fixed 6 new bugs, fixed all of fusefs's Coverity CIDs, made some minor
performance enhancements and done some general cleanup. During the next
quarter I plan to continue fixing bugs, and I'll also raise the
driver's API level as high as I can before the quarter runs out. We're
currently at 7.8; the highest defined level is 7.28.
This project was sponsored by The FreeBSD Foundation.
__________________________________________________________________
Kernel ZLIB Update
Links
Review D19706
URL: https://reviews.freebsd.org/D19706
Contact: Yoshihiro Ota <ota@j.email.ne.jp>
The FreeBSD system still uses an ancient (over 20 year-old) version of
zlib (version 1.0.4). The FreeBSD kernel zlib implementation has
special enhancements only used by netgraph. There is a separate version
of code derived from unzip 5.12 used to inflate gzip files in the
kernel which could be replaced with a more modern zlib. More detailed
information is written in sys/modules/zlib/README in the review.
In order to use the latest zlib, version 1.2.11, work has been done to
revisit all existing zlib uses in the system. Most of the code works
with the newer version of zlib as is. The unzip code will need some
conversion work to use the newer zlib. A few callers will be made
simplier by using some newer APIs available in the updated zlib. There
are some zombie programs that have been broken and I would like to
delete.
This will clean up zombie programs and duplicated zlib code. This will
also make future zlib version updates easier.
These changes touch some very sensitive areas of the system, such as
kernel loading, or are architecture specific like armv6/armv7, and also
touch some legacy code like kgzip+kgzldr on i386. Testers and active
users of these legacy zlib code are welcomed.
* armv elf_trampoline Arm up to v5 can boot from gzipped kernel. This
code is modified to use newer API for simplicity. Please verify
gzipped kernel still boots with new code (Current code has fall
back to legacy zlib in case of failure). Please also elaborate how
to link such kernel, too. I'm still trying to figure that out.
* netgraph compression/decompression Please help testing and/or teach
how to test. Netgraph compiles in the FreeBSD zlib version inside.
* gzipped a.out Does anyone use gzipped a.out executables, still? If
so, does someone have an easy and safe program to run? Is a.out
format i386 only?
* zfs boot Can we boot from gzipped file system today?
* CTF Checking how I can test.
__________________________________________________________________
LLVM's lld as the FreeBSD system linker
Links
LLD on the FreeBSD Wiki
URL: https://wiki.freebsd.org/LLD
lld exp-run
URL: https://bugs.freebsd.org/214864
Contact: Ed Maste <emaste@freebsd.org>
In FreeBSD-HEAD and 12.0 the default FreeBSD system linker (i.e.,
/usr/bin/ld) is LLVM's lld, on amd64, arm64, and armv7. For i386 in
12.0 lld is used as the bootstrap linker (i.e., to build the kernel and
base system) but it is not enabled as the system linker because of
multiple issues building FreeBSD ports with it enabled.
The primary issue affecting i386 with lld is that many ports build
position-dependent code (i.e., non-PIC) for use in shared libraries.
This either comes from omitting the -fPIC compiler flag, or using
hand-written position-dependent assembly. Compared with other CPU
architectures i386 position-independent code is rather inefficient,
which may be responsible for port authors making an explicit decision
to avoid PIC.
By default lld does not allow position-dependent code in shared objects
(in particular, it does not permit relocations against read-only
segments - typically containing the`.text` section).
Over the last quarter many commits were made to the ports tree to fix
the build when the system linker is lld - either building PIC code, or
adding the -znotext linker flag to permit relocations against read-only
segments, or just switching the port to link with GNU ld if it is
incompatible with lld in some other way.
At this point there are only a few dozen open bug reports for issues
linking ports with lld as the system linker, and I expect FreeBSD 12.1
to use lld as the system linker on i386 as well.
Tasks:
* Fix freepascal/Lazarus ports with lld
* Triage and address remaining port failures
* Holistic review of lld workarounds in the ports tree, to identify
changes that are no longer needed, should be addressed in lld, or
should be sent upstream
This project was sponsored by The FreeBSD Foundation.
__________________________________________________________________
mlx5 Drivers Update
Links
Mellanox OFED for FreeBSD Documentation
URL: http://www.mellanox.com/page/products_dyn?product_family3&mtag=freebsd_driver
Contact: Slava Shwartsman, Hans Petter Selasky, Konstantin Belousov <freebsd-drivers@mellanox.com>
The mlx5 driver provides support for PCI Express adapters based on
ConnectX-4(LX), ConnectX-5(EX) and ConnectX-6(DX). The mlx5en driver
provides support for Ethernet and the mlx5ib driver provides support
for InfiniBand and RDMA over Converged Ethernet, RoCE.
Following updates done in mlx5 drivers:
* Added support for ConnectX-6 and ConnectX-6dx devices, which
support of up to 200Gb/s interface speeds!
* Added TLS hardware offload support for ConnectX-6dx devices. TLS Tx
crypto offload is a new feature for network devices. It enables the
kernel TLS socket to skip encryption and authentication operations
on the transmit side of the data path, delegating those to the NIC.
In turn, the network adapter encrypts packets that belong to an
offloaded TLS socket on the fly. The Mellanox network adapter does
not modify any packet headers. It expects to receive fully framed
TCP packets with TLS records as payload. The NIC replaces plaintext
with ciphertext and fills the authentication tag. The adapter does
not hold any state beyond the context needed to encrypt the next
expected packet, i.e. expected TCP sequence number and crypto
state.
* Add support for Dynamic Receive Queue Interrupt Moderation. Dynamic
Interrupt Moderation (DIM) refers to any action made by hardware
and/or software on run time to control interrupt rate on the
system. The moderation action itself should not interfere with the
system's operation and should not require any human interaction. In
networking, dynamic interrupt moderation is used for controlling
the rate of interrupts generated by the hardware for multiple
traffic scenarios.
* Enhanced support for self-healing mechanism: In a rare occasion
when Mellanox network adapters fail, due to a firmware bug for
example, the driver will sense the catastrophic error. As a result
of this failure detection, the device driver can trigger a firmware
reset for the device so it can recover - without the need to reboot
the entire host.
* Added support for in-driver firmware updating using mlx5tool.
This project was sponsored by Mellanox Technologies.
__________________________________________________________________
PCI Express Resets
Contact: Konstantin Belousov <konstantinb@mellanox.com>
Sometimes the need to reset a device attached to the system presents
itself. Preferrably this device reset can be accomplished without
causing the whole machine to reboot. It is easy to do with USB devices
if the physical access is available -- you can just re-plug the device.
For in-chassis devices, built-in, or on add-on cards, it is not
possible to reset the device with physical action, unless the device is
hot-plugged. Nonetheless, for typical modern PCIe devices, and most
built-in PCI-emulation devices, the reset can be initiated using
software actions.
If device is a real plugged-in PCIe device, then reset can be initiated
by disabling and then re-training PCIe-link by the upstream port
controls. For most PCI devices, which support the PCI power management
specification, the proven way to accomplish the reset is to put the
device into state D3 (off) and then return to the previous power state.
FreeBSD was missing a way to conveniently request user- or
driver-initiated reset of devices. While it was possible to manually
fiddle with registers using pciconf, this is impractical for users, and
requires a lot of boilerplate code from drivers.
A new BUS_RESET_CHILD() method was added to the newbus bus interface,
and implementations added for PCIe bridges and PCI devices. The
libdevctl(3) library call and devctl(8) command provide convenient
userspace accessors for applications and administrators.
During the reset, the device driver must stop its operations with the
device. One way to achieve this is to detach drivers before reset, and
re-attach after the device afterwards. This is mostly fine for network
interfaces, but other devices require more coordination to handle
properly. For example, an NVMe disk device being detached it means that
all mounted volumes abruptly disapper from VFS view. Due to this, the
BUS_RESET_CHILD() method allows the caller to select either
detach/re-attach or suspend/resume driver actions around the reset.
Mellanox uses the infrastructure to perform reset of the mlx(5) card
after firmware reset without server reboot. It is believed that 'devctl
reset' will be more widely useful.
This project was sponsored by Mellanox Technologies.
__________________________________________________________________
Security-Related changes
Contact: Konstantin Belousov <kib@freebsd.org>
ASLR
The ASLR (Address Space Layout Randomization) patch from review D5603
was committed into svn. While debate continues about the current and
forward-looking value ASLR provides, having an implementation in the
FreeBSD source tree makes it easily available to those who wish to use
it. This also moves the conversation past the relative merits to more
comprehensive security controls.
KPTI per-process control
The KPTI (Kernel Page Table Isolation) implementation was structured so
that most selections of page isolation mode were local to the current
address space. In other words, the global control variable pti was
almost unused in the code paths, instead the user/kernel %cr3 values
were directly loaded into registers or compared to see if the user page
table was trimmed. Some missed bits of code were provided by Isilon,
and then bugs were fixed and last places of direct use of pti were
removed.
Now when the system starts in the pti-enabled mode, proccontrol(1) can
be used by root to selectively disable KPTI mode for children of a
process. The motivation is that if you trust the program that you run,
you can get the speed of non-pti syscalls back, but still run your
normal user session in PTI mode. E.g., firefox would be properly
isolated.
Feature-control bits
Every FreeBSD executable now contains a bit mask intended for
enabling/disabling security-related features which makes sense for the
binary. This mask is part of the executable segments loaded on image
activation, and thus is part of any reasonable way to authenticate the
binary content.
For instance, the ASLR compatibility is de-facto the property of the
image and not of the process executing the image. The first (zero) bit
in the mask controls ASLR opt-out. Other OSes (e.g. Solaris) used an
OS-specific dynamic flag, which has the same runtime properties but
leaves less bits to consume in the feature-control mask.
The feature-control mask is read both by kernel and by rtld during
image activation. It is expected that more features will be added to
FreeBSD and the mask can be used for enabling/disabling those
features..
It is expected that a tool to manipulate the mask will be provided
shortly, see review D19290.
This project was sponsored by The FreeBSD Foundation.
__________________________________________________________________
Architectures
Updating platform-specific features and bringing in support for new
hardware platforms.
FreeBSD/RISC-V Update
Contact: Ruslan Bukin <br@FreeBSD.org>
Contact: Mitchell Horne <mhorne@FreeBSD.org>
Contact: Mark Johnston <markj@FreeBSD.org>
Work has continued on RISC-V port in the past quarter.
Support for transparent superpage promotion was added to the RISC-V
port, meaning that applications will now automatically use large page
mappings when possible. Per-CPU pmap activation tracking was added,
reducing the overhead of various pmap operations. This noticeably
improves the responsiveness of FreeBSD when running in a multi-CPU
virtual machine.
A RISC-V implementation of minidumps was completed. Support for
debugging RISC-V kernel dumps will land in devel/gdb after the next GDB
release.
It is now possible to compile the in-tree LLVM's RISC-V target by
setting WITH_LLVM_TARGET_RISCV=YES in /etc/src.conf. The use of LLVM to
compile the RISC-V port is currently experimental and further
investigation is ongoing.
Work is ongoing to bring up FreeBSD on SiFive's HiFive Unleashed
development board now that one has been obtained by a FreeBSD
developer. We also expect to work on support for a new version of the
SBI specification.
This project was sponsored by The FreeBSD Foundation, DARPA, AFRL.
__________________________________________________________________
Ports
Changes affecting the Ports Collection, whether sweeping changes that
touch most of the tree, or individual ports themselves.
FreeBSD GNOME status report
Links
GNOME FreeBSD
URL: https://freebsd.org/gnome/
GNOME development Repo
URL: https://github.com/freebsd/freebsd-ports-gnome
Contact: Koop Mast <kwm@FreeBSD.org>
Contact: Eric Turgeon <ericbsd@FreeBSD.org>
Ports activity in this quarter were:
* The x11-toolkits/gtk30 port updated to 3.24.5 and later to 3.24.7.
* The www/webkit2-gtk3 port was updated to 2.24.0.
* And the old insecure webkit-gtk2 and webkit-gtk3 where finally
removed.
Work in progress, the branches are available in the GNOME development
repo, see the link above.
* Eric Turgeon is working on MATE 1.22 in the mate-1.22 branch. And
is almost complete.
* Charlie Li (IRC: vishwin) is working on a long overdue update of
the cinnamon desktop. This update is almost complete. The only real
blocker is that the screensaver can't be unlocked after it
activates. The work is in the cinnamon branch.
* Koop Mast works on GNOME 3.32. The desktop is usable apart from gdm
which is currently non-functional. Due to lack of free time the
work is going slowly. This work is available in the gnome-3.32
branch.
People who are willing to contribute can find us on #freebsd-gnome on
freenode.
__________________________________________________________________
FreeBSD KDE status report
Links
KDE FreeBSD
URL: https://freebsd.kde.org/
Contact: Adriaan de Groot <adridg@FreeBSD.org>
Contact: Tobias C. Berner <tcberner@FreeBSD.org>
The two biggest accomplishements this quarter were:
* Qt4 and all its consumers have been removed from the ports tree.
* www/qt5-webengine has been updated from the ancient 5.9.4 to 5.12.x
by kai@
Further we have kept the KDE Frameworks, Plasma and Applications ports
up to date with upstreams releases, which thanks to upstreams'
FreeBSD-CI uses less and less patches.
All the kde@ maintained ports (including cmake) have been kept up to
date with their releases.
The plans for the next quarter are in no particular order
* Cleanup PyQt ports and pyqt.mk
* Improve qt.mk components
* Update sddm to 0.18.x
* Implement user management functionality in system settings (write
non-logind backend)
People who are willing to contribute can find us on #kde-freebsd on
freenode, and the kde@FreeBSD.org mailing list. Further we accept
pull-requests and contributions on
github.com/freebsd/freebsd-ports-kde.
__________________________________________________________________
Third-Party Projects
Many projects build upon FreeBSD or incorporate components of FreeBSD
into their project. As these projects may be of interest to the broader
FreeBSD community, we sometimes include brief updates submitted by
these projects in our quarterly report. The FreeBSD project makes no
representation as to the accuracy or veracity of any claims in these
submissions.
FreeBSD Wiki Apple Intel Mac mini update
Links
FreeBSD Wiki
URL: https://wiki.freebsd.org/IntelMacMini
Contact: Trevor Roydhouse <fbsdwiki@gmx.net>
The FreeBSD Wiki page for the Apple Intel Mac minis has been
comprehensively updated over the last quarter to drag it from 2009 into
2019.
There are now detailed instructions for installing FreeBSD as the only
operating system on models from 2007 through 2014 and itemised model
specific information detailing FreeBSD support.
If anyone is interested, help is needed to provide more specific
information for the macmini 1,1 and 6,1 through 8,1 models and to test
patches for the asmc(4) driver for temperature sensor feedback and for
setting fan speed. If you would like to help and have access to these
Mac minis, please contact me.
Future tasks:
* Create and test more patches for asmc(4) to cover all Intel Mac
minis
* Provide more information for 2006, 2012, 2014 and 2018 Mac minis
* Instructions for dual boot (macOS/FreeBSD) installations
__________________________________________________________________
Fuzzing FreeBSD with syzkaller
Links
syzkaller
URL: https://github.com/google/syzkaller
Contact: Mark Johnston <markj@FreeBSD.org>
Contact: Andrew Turner <andrew@FreeBSD.org>
Contact: Michael Tuexen <tuexen@FreeBSD.org>
Contact: Ed Maste <emaste@FreeBSD.org>
Syzkaller is a coverage-guided system call fuzzer. It was originally
developed for Linux. It programmatically creates programs consisting of
sequences of random system calls and executes them in a VM (virtual
machine). Using feedback from a kernel code coverage facility called
kcov, syskaller mutates the generated test programs in an attempt to
expand the executed coverage of code paths within the kernel. Sometimes
exercising a seldom or infrequently used code path will crash the
kernel. When syzkaller manages to crash the running kernel in the VM,
it attempts to generate a minimal test case which reproduces the crash,
simplifying debugging. Syzkaller is very effective at finding kernel
bugs and has uncovered hundreds of issues in Linux. Over the past
couple of years, syzkaller's author, Dmitry Vyukov, has added support
for other operating systems, including FreeBSD.
Recently, a number of FreeBSD developers have been using syzkaller to
find and fix bugs in the FreeBSD kernel. If interested, one can search
the commit logs for "syzkaller" to find examples. Syzkaller can be run
on a FreeBSD or Linux host to fuzz FreeBSD running in QEMU instances.
It can also fuzz FreeBSD instances running on GCE (Google Compute
Engine). Additionally, Google maintains a dedicated cluster of GCE
hosts to continuously fuzz the latest builds of several different OS
kernels. A FreeBSD target was recently added. Subscribe to the
syzkaller-freebsd-bugs Google Group to receive notifications for newly
discovered bugs.
Work is ongoing to improve syzkaller's coverage of FreeBSD's system
calls. In particular, syzkaller needs to be taught about all of the
target kernel's entry points and argument types in order to be useful.
Many of the standard POSIX system calls are already covered, but most
FreeBSD-specific system calls are not. Similarly, many ioctl(2)
definitions are missing.
Some in-progress work aims to add support for bhyve as a VM backend for
syzkaller, making it easier to fuzz FreeBSD VMs hosted on FreeBSD.
Currently that can be done using QEMU, but QEMU on FreeBSD lacks
support for hardware acceleration. See the PR for the implementation.
Finally, a number of bugs identified by syzkaller have yet to be fixed.
If you are interested in helping out with any of the above, please mail
the contacts listed above.
This project was sponsored by The FreeBSD Foundation.
__________________________________________________________________
sysctlmibinfo API 1.0
Links
gitlab.com/alfix/sysctlmibinfo
URL: https://gitlab.com/alfix/sysctlmibinfo
Contact: Alfonso Sabato Siciliano <alfonso.siciliano@email.com>
Port: devel/libsysctlmibinfo
The sysctl() system call can get or set the value of a 'property' of
the system. A 'property' has others info (description, type, label,
etc.), they are necessary to build an utility like /sbin/sysctl,
example:
% sysctl -d kern.ostype
kern.ostype: Operating system type
% sysctl -t kern.ostype
kern.ostype: string
Primarily sysctlmibinfo wraps the undocumented kernel interface and
provides an easy C API: sysctlmif_name(), sysctlmif_description(),
sysctlmif_info(), sysctlmif_label(), sysctlmif_nextnode() and
sysctlmif_nextleaf(), to retrieve the info of a 'property'.
Moreover sysctlmibinfo provides a high level API: defines a struct
sysctlmif_object and has some function: sysctlmif_filterlist(),
sysctlmif_grouplist() and sysctlmif_tree(), to build lists and trees of
objects.
You can use this library to quickly build a custom sysctl utility. For
example, the core of deskutils/sysctlview (a graphical explorer for the
sysctl MIB Tree) is just a call to sysctlmif_tree() and a visit to the
resulting tree to show its sysctlmif_object nodes.
Note, actually a 'property' is an OID of the sysctl MIB, it is
implemented by a struct sysctl_oid defined in sys/sysctl.h.
__________________________________________________________________
sysctlview 1.0
Links
gitlab.com/alfix/sysctlview
URL: https://www.gitlab.com/alfix/sysctlview
Contact: Alfonso Sabato Siciliano <alfonso.siciliano@email.com>
Port: deskutils/sysctlview
The FreeBSD's kernel maintains a Management Information Base where the
objects are properties to tuning the system using the sysctl() syscall
and the /sbin/sysctl utility. The sysctlview utility is a "graphical
sysctl MIB explorer", it depends on gtkmm (to build a GUI) and
sysctlmibinfo (to retrieve the info from the kernel).
The version 1.0 provides two "TreeView":
* "Main" to show 'name', 'description', 'type', 'format' and 'value'
* "Flags" to show 'name' and a column for each 'flag' defined in
sys/sysctl.h
The rows are "clickable" to display others info (e.g., 'label').
Currently sysctlview can show numeric and string values, the support
for some opaque value will be added in the future.
__________________________________________________________________
University of Waterloo Co-operative Education Students
Contact: Ed Maste <emaste@freebsd.org>
For the January-April 2019 term the FreeBSD Foundation has again
brought on two co-operative education (co-op) students from the
University of Waterloo.
Gerald Aryeetey is a 2nd year Computer Engineering student. Gerald
started looking at a FreeBSD tool chain issue - our static library
archiver (ar) did not read or write archives in the 64-bit format.
Gerald submitted a libarchive change to support 64-bit archives
followed by change to FreeBSD's ar to add 64-bit support.
Gerald later looked at a number of freebsd-update issues in FreeBSD's
bugzilla database, and submitted many fixes. Around a dozen have been
committed to FreeBSD, and more are in review.
Gerald also worked on the FreeBSD Foundation's hardware continuous
integration effort. The prototype installation is building FreeBSD on a
commit-by-commit basis and testing on a BeagleBone Black and a Pine64
LTS. The prototype will be converted to a permanent, public
installation in the near future, after which additional test devices
will be added.
For his final project Gerald intends to write a device driver for the
Microchip LAN743x PCIe NIC.
Bora Özarslan is a 3rd year student in Computing and Financial
Management. Bora's initial focus was also on tool chain issues in
FreeBSD, starting with improvements or bug fixes in FreeBSD's readelf
(from the ELF Tool Chain project).
Bora developed a tool to modify feature control bits in ELF binaries -
for example, allowing binaries incompatible with ASLR to request to
opt-out. As part of his readelf work Bora also added support to report
the status of the feature control bits.
Bora continued investigating security topics, looking at applying
Capsicum sandboxing to Kristaps' BSD licensed rsync implementation,
openrsync. This work required first implementing fileargs_lstat support
in cap_fileargs (which as now been committed) as well as changes to the
fts directory hierarchy routines (which have not yet been committed to
FreeBSD).
For the rest of the work term Bora will investigate and test unmodified
Linux Docker containers on FreeBSD, to evaluate the state of
Linuxulator support.
This project was sponsored by The FreeBSD Foundation.
__________________________________________________________________