FreeBSD Quarterly Status Report - First Quarter 2017
16 May, 2017 by bjk@FreeBSD.org | freebsd
FreeBSD Project Quarterly Status Report - 1st Quarter 2017
While a few of these projects indicate they are a "plan B" or an
"attempt III", many are still hewing to their original plans, and all
have produced impressive results. Please enjoy this vibrant collection
of reports, covering the first quarter of 2017.
--Benjamin Kaduk
__________________________________________________________________
The deadline for submissions covering the period from April to June
2017 is July 7, 2017.
__________________________________________________________________
FreeBSD Team Reports
* The FreeBSD Core Team
* The FreeBSD Foundation
* The FreeBSD Ports Collection
* The FreeBSD Release Engineering Team
Projects
* Ceph on FreeBSD
* OpenBSM
* Porting Software to CloudABI: Sandboxed Bitcoin!
* Support for eMMC Flash and Faster SD Card Modes
* TrustedBSD
Kernel
* FreeBSD on Hyper-V and Azure
* Intel 10G and 40G Network Driver Updates
* Linuxulator
* MMC Stack Using the CAM Framework
* pNFS Server Plan B
Architectures
* 64-bit PowerPC Book-E Support
* FreeBSD on Marvell Armada38x
* FreeBSD/s390x Attempt III
Ports
* MySQL
* Rust
Documentation
* The FreeBSD Dutch Documentation Project
__________________________________________________________________
FreeBSD Team Reports
The FreeBSD Core Team
Contact: FreeBSD Core Team <core@FreeBSD.org>
Core's primary function is to ensure the long-term viability of the
FreeBSD project. A very large part of that is to ensure that the
interactions between developers remain cordial, and consequently that
the project appears welcoming to newcomers.
Normally, most of Core's activities around this are done in private --
a quiet word in the right ear, some discrete peacemaking, occasional
reading of the riot act. Most of the time, this is all that is
necessary.
Unfortunately, this quarter we had an instance where such private
measures failed to achieve the desired result, and we ended up ejecting
a developer. This developer is an extremely talented programmer and has
made significant contributions to the Ports Collection. Despite this,
portmgr found him to be sufficiently disruptive and abrasive that in
their judgement, the project was better off overall to sever his
connection to itself, and core backed them up in that. We are sorry
that events came to this sad conclusion, but we remain convinced that
this was a necessary step to safeguard the character of our community.
In a more positive light, Core has been working on a proposal to
recognise notable contributors to the FreeBSD project who are not (or
perhaps not yet) suitable to be put forward as new committers. In
addition to the usual routes of recognising people that write numbers
of good bug reports or that supply patches or that volunteer to
maintain ports, this will also allow recognition of people who
contribute by such things as organising FreeBSD events or who promote
FreeBSD through social media. A formal announcement of Core's proposal
is imminent.
During January, the core secretary held an exercise to contact all
source committers who had been inactive for more than 18 months and
persuade them to hand in their commit bits if they were not planning to
resume working on FreeBSD in the near future. This is meant to be a
routine function -- the "grim reaper" -- that aims to keep the list of
people with the ability to commit pretty much in synchrony with the
list of people that are actively committing. The regular process had
fallen out of activity several years ago, and we needed to clear the
decks before restarting. Ultimately, this resulted in some 20
developers-emeritus handing in their commit bits.
No new commit bits were awarded during this quarter.
Core is also taking soundings on producing a 10.4-RELEASE. This is not
in the current plan, but a number of developers and important FreeBSD
users would be keen to see it happen, given some of the work that has
gone into the stable/10 branch since 10.3-RELEASE. On the other hand,
this would represent an additional support burden for the Security
Team, including maintaining versions of software that have been
declared obsolete upstream, in particular OpenSSL. As an even-numbered
release, 10.4-RELEASE would have a "normal" rather than an "extended"
lifetime which means it should not result in extending the support
lifetime of the stable/10 branch.
In other news, Core arranged for the old and largely inactive
marketing@FreeBSD.org mailing list to be wound up, and for any
remaining activities to be transferred to the FreeBSD Foundation.
Core also asked clusteradm to turn off Internet-wide access to the
finger server on freefall.freebsd.org. Many developers have included
details such as phone numbers into the GECOS field of their FreeBSD
password database entries, and these would be revealed by the finger
server -- details which are nowadays generally felt inadvisable to
expose publicly. finger is still available internally within
freefall.freebsd.org. Core recommends that GECOS data is limited to
just your full name, and we have updated the standard "new committer"
e-mail template to reflect that.
Core is looking for new volunteers to help out with several of the
teams that manage various aspects of the project. In particular,
Postmaster and the Security Team are in need of new blood. Recruiting
for a new member of the Security Team is well under way, but anyone
interested in joining any of the teams is encouraged to make themselves
known either to Core or directly to the teams concerned.
__________________________________________________________________
The FreeBSD Foundation
Links
FreeBSD Foundation Website
URL: https://www.FreeBSDFoundation.org/
Quarterly Newsletter
URL: https://www.FreeBSDfoundation.org/wp-content/uploads/2017/03/FreeBSD-Foundation-Q1-2017-Update.pdf
2017 Storage Summit
URL: https://wiki.FreeBSD.org/201702StorageSummit
Contact: Deb Goodkin <deb@FreeBSDFoundation.org>
The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated
to supporting and promoting the FreeBSD Project and community
worldwide. Funding comes from individual and corporate donations and is
used to fund and manage software development projects, conferences and
developer summits, and provide travel grants to FreeBSD contributors.
The Foundation purchases and supports hardware to improve and maintain
FreeBSD infrastructure; publishes marketing material to promote,
educate, and advocate for the FreeBSD Project; facilitates
collaboration between commercial vendors and FreeBSD developers; and
finally, represents the FreeBSD Project in executing contracts, license
agreements, and other legal arrangements that require a recognized
legal entity.
Our work is 100% funded by your donations. We kicked off the new year
with some large contributions from Intel and NetApp, to help us raise
over $400,000 last quarter! We engaged in discussions with new and old
commercial users to help facilitate collaboration, explain how the
Project works, and to ask for financial contributions to help us keep
FreeBSD the innovative, secure, and reliable operating system they
depend on. Please consider making a donation today!
https://www.FreeBSDfoundation.org/donate/.
The Foundation improves the FreeBSD operating system by employing our
technical staff to maintain and improve critical kernel subsystems, add
features and functionality, and fix problems. Our contributions also
include funding separate project grants like the arm64 port, blacklistd
access control daemon, and integration of VIMAGE support, to make sure
FreeBSD remains a viable solution for research, education, computing,
products and more.
This quarter's project development highlights include:
* 168 commits sponsored by the FreeBSD Foundation in the src tree
(base system) development branch, across three staff members and
four grant recipients/other developers.
* Multiple funded grants, including the cfumass project, now
committed to FreeBSD-HEAD, and improvements to the blacklistd
daemon and FreeBSD/arm64 port.
* Staff contributions including improvements to toolchain and build
tool components, run time libraries, arm64, mips64 and 32- and
64-bit x86 architectures, release image build tooling, packaged
base, and VM subsystem bug fixes.
* Significant progress on the 64-bit inode project, which is nearly
ready for commit.
FreeBSD Advocacy and Education
A large part of our efforts are dedicated to advocating for the
Project. This includes promoting work being done by others with
FreeBSD; producing advocacy literature to teach people about FreeBSD
and help make the path to starting to use FreeBSD or contribute to the
Project easier; and attending and getting other FreeBSD contributors to
volunteer to run FreeBSD events, staff FreeBSD tables, and give FreeBSD
presentations.
Some of the highlights of our advocacy and education work over the last
quarter:
* Promoted FreeBSD at: FOSDEM, SCALE, AsiaBSDcon, and FOSSASIA
* Promoted BSDCan, SCALE, USENIX LISA, vBSDcon and EuroBSDcon Calls
for Participation
* Promoted Google Summer of Code participation on social media and
created a flyer for people to post at their universities
* Published a New Faces of FreeBSD Story: Joseph Kong
* Set up a Marketing Partnership with the USENIX Association and SNIA
* Published and Promoted the Jan/Feb 2017 issue of the FreeBSD
Journal: https://www.FreeBSDfoundation.org/journal/
* Published monthly Development Projects Updates on our blog
* Secured a FreeBSD table at OSCON and promoted available discounts
Conferences and Events
The FreeBSD Foundation sponsors many conferences, events, and summits
around the globe. These events can be BSD-related, open source, or
technology events geared towards underrepresented groups.
We support the FreeBSD-focused events to help provide a venue for
sharing knowledge, to work together on projects, and to facilitate
collaboration between developers and commercial users; this all helps
provide a healthy ecosystem. We support the non-FreeBSD events to
promote and raise awareness about FreeBSD, to increase the use of
FreeBSD in different applications, and to recruit more contributors to
the Project.
We also sponsored and/or attended the following events last quarter:
* FOSDEM FreeBSD developer summit (sponsor)
* AsiaBSDCon -- Tokyo, Japan (sponsor)
* Organized and ran the FreeBSD Storage Summit in Santa Clara, CA
* Board member Philip Paeps gave a FreeBSD presentation at FOSSASIA
* Attended FOSSASIA, FOSDEM, and SCALE
Release Engineering
The Foundation provides a full-time staff member to lead the release
engineering efforts. This has provided timely and reliable releases
over the last few years. Some highlights from last quarter include:
* Continued the production of weekly development snapshots for the
12-CURRENT, 11-STABLE, and 10-STABLE branches.
* Published the initial FreeBSD 11.1-RELEASE schedule to the Project
website.
Legal/FreeBSD IP
The Foundation owns the FreeBSD trademarks, and it is our
responsibility to protect them. We continued to review requests and
grant permission to use the trademarks.
Many more details about how we supported FreeBSD last quarter can be
found in our Q1 newsletter!
__________________________________________________________________
The FreeBSD Ports Collection
Links
About FreeBSD Ports
URL: https://www.FreeBSD.org/ports/
Contributing to Ports
URL: https://www.FreeBSD.org/doc/en_US.ISO8859-1/articles/contributing/ports-contributing.html
FreeBSD Ports Monitoring
URL: http://portsmon.FreeBSD.org/index.html
Ports Management Team
URL: https://www.FreeBSD.org/portmgr/index.html
FreeBSD portmgr on Twitter (@FreeBSD_portmgr)
URL: https://twitter.com/FreeBSD_portmgr/
FreeBSD Ports Management Team on Facebook
URL: https://www.facebook.com/portmgr
FreeBSD Ports Management Team on Google+
URL: https://plus.google.com/communities/108335846196454338383
Contact: René Ladan <portmgr-secretary@FreeBSD.org>
Contact: FreeBSD Ports Management Team <portmgr@FreeBSD.org>
The number of ports is currently just 500 short of 30,000. The current
number of PRs is close to 2,400, of which 620 are unassigned. The last
quarter saw 6656 commits from 167 comitters. Both the number of ports
and the number of unassigned PRs have increased in the last quarter.
In the last quarter, we welcomed 7 new committers: Eugene Grosbein
(eugen), Johannes Dieterich (jmd), Larry Rosenman (ler), Mahdi Mokhtari
(mmohki), Matthew Rezny (rezny), Tobias Kortkamp (tobik), and Vladimir
Kondratyev (wulf). dumbbell@ was already a src committer and got an
extension for the Ports Tree. We also welcomed back krion@ and miwi@.
We took 6 bits in for safe-keeping: itetcu@, leeym@, mva@, olivierd@,
pgollucci@, and sanpei@.
There were no changes to the membership of portmgr.
antoine@ worked on USES=samba to prepare for the removal of the
long-outdated Samba 3.6 ports and replace them with modern versions.
The new default versions are: FreePascal 3.0.2, Ruby 2.3, and Samba
4.4. A new variable USE_LOCALE was created to add the LANG and LC_ALL
environment variables to all builds. Out-of-tree patches can now be
added with the new EXTRA_PATCH_TREE variable. The error messages for
invalid OPTIONS_SINGLE options were improved.
Some of the major port updates last quarter were: pkg 1.10.1, linux
c6_64, Firefox 52.0.2, Chromium 57.0.2987.110, GCC 4.9.4, Gnome 3.18.0,
Xorg 1.18.4, Qt 4.8.7 and 5.7.1, and PHP 7.1.
antoine@ ran 31 exp-runs to test version updates and under-the-hood
changes.
Open tasks:
1. The number of unassigned and open PRs is still growing, so if you
have some spare time, please close some of those.
__________________________________________________________________
The FreeBSD Release Engineering Team
Links
FreeBSD 11.1-RELEASE Schedule
URL: https://www.freebsd.org/releases/11.1R/schedule.html
FreeBSD development Snapshots
URL: http://ftp.freebsd.org/pub/FreeBSD/snapshots/ISO-IMAGES/
Contact: FreeBSD Release Engineering Team <re@FreeBSD.org>
The FreeBSD Release Engineering Team is responsible for setting and
publishing release schedules for official project releases of FreeBSD,
announcing code freezes, and maintaining the respective branches, among
other things.
The FreeBSD Release Engineering Team continued producing weekly
development snapshots for the 12-CURRENT, 11-STABLE, and 10-STABLE
branches.
In addition, the FreeBSD 11.1-RELEASE schedule was added to the Project
website. Please note, however, the schedule on the website is still
subject to change.
This project was sponsored by The FreeBSD Foundation.
__________________________________________________________________
Projects
Ceph on FreeBSD
Links
Ceph Main Site
URL: http://ceph.com
Main Repository
URL: https://github.com/ceph/ceph
My FreeBSD Fork
URL: https://github.com/wjwithagen/ceph
Contact: Willem Jan Withagen <wjw@digiware.nl>
Ceph is a distributed object store and file system designed to provide
excellent performance, reliability and scalability.
* Object Storage
Ceph provides seamless access to objects using native language
bindings or radosgw, a REST interface that is compatible with
applications written for S3 and Swift.
* Block Storage
Ceph's RADOS Block Device (RBD) provides access to block device
images that are striped and replicated across the entire storage
cluster.
* File System
Ceph provides a POSIX-compliant network file system that aims for
high performance, large data storage, and maximum compatibility
with legacy applications.
I started looking into Ceph because the HAST solution with CARP and
ggate did not really do what I was looking for. But I aim to run a Ceph
storage cluster of storage nodes that are running ZFS. User stations
would be running bhyve on RBD disks that are stored in Ceph.
Compiling for FreeBSD will now build most of the tools available in
Ceph.
Notable progress since the last report:
* The most important change is that a port has been submitted:
net/ceph-devel. However, it does not yet contain ceph-fuse.
* Regular updates to the ceph-devel port are expected, with the next
one coming in April.
* ceph-fuse works, allowing one to mount a CephFS filesystem on a
FreeBSD system and perform normal operations.
* ceph-disk prepare and activate work for FileStore on ZFS, allowing
for easy creation of OSDs.
* RBD is actually buildable and can be used to manage RADOS BLOCK
DEVICEs.
* Most of the awkward dependencies on Linux-isms are deleted -- only
/bin/bash is here to stay.
To get things running on a FreeBSD system, run pkg install
net/ceph-devel or clone https://github.com/wjwithagen/ceph and build
manually by running ./do_freebsd.sh in the checkout root.
Parts not (yet) included:
* KRBD: Kernel Rados Block Devices are implemented in the Linux
kernel, but not yet in the FreeBSD kernel. It is possible that
ggated could be used as a template, since it does similar things,
just between two disks. It also has a userspace counterpart, which
could ease development.
* BlueStore: FreeBSD and Linux have different AIO APIs, and that
incompatibility needs to resolved somehow. Additionally, there is
discussion in FreeBSD about aio_cancel not working for all
devicetypes.
* CephFS as native filesystem: though ceph-fuse works, it can be
advantageous to have an in-kernel implementation for heavy
workloads. Cython tries to access an internal field in struct
dirent, which does not compile.
Open tasks:
1. Run integration tests to see if the FreeBSD daemons will work with
a Linux Ceph platform.
2. Compile and test the userspace RBD (Rados Block Device). This
currently works but testing has been limitted.
3. Investigate and see if an in-kernel RBD device could be developed
akin to ggate.
4. Investigate the keystore, which can be embedded in the kernel on
Linux and currently prevents building Cephfs and some other parts.
The first question is whether it is really required, or only KRBD
requires it.
5. Scheduler information is not used at the moment, because the
schedulers work rather differently between Linux and FreeBSD. But
at a certain point in time, this will need some attention (in
src/common/Thread.cc).
6. Improve the FreeBSD init scripts in the Ceph stack, both for
testing purposes and for running Ceph on production machines. Work
on ceph-disk and ceph-deploy to make it more FreeBSD- and
ZFS-compatible.
7. Build a test cluster and start running some of the teuthology
integration tests on it. Teuthology wants to build its own libvirt
and that does not quite work with all the packages FreeBSD already
has in place. There are many details to work out here.
8. Design a vitual disk implementation that can be used with bhyve and
attached to an RBD image.
__________________________________________________________________
OpenBSM
Links
OpenBSM: Open Source Basic Security Module (BSM) Audit Implementation
URL: http://www.openbsm.org
OpenBSM on GitHub
URL: https://github.com/openbsm/openbsm
FreeBSD Audit Handbook Chapter
URL: https://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/audit.html
DTrace Audit Provider
URL: https://reviews.FreeBSD.org/D10149
DARPA CADETS project
URL: https://www.cl.cam.ac.uk/research/security/cadets/
TODO List on GitHub
URL: https://github.com/openbsm/openbsm/blob/master/TODO
Contact: Christian Brueffer <brueffer@FreeBSD.org>
Contact: Robert Watson <rwatson@FreeBSD.org>
Contact: TrustedBSD audit mailing list
<trustedbsd-audit@TrustedBSD.org>
OpenBSM is a BSD-licensed implementation of Sun's Basic Security Module
(BSM) API and file format. It is the userspace side of the CAPP Audit
implementations in FreeBSD and Mac OS X. Additionally, the audit trail
processing tools are expected to work on Linux.
During this quarter, experimental support for UUIDs in BSM trails was
added to OpenBSM. A DTrace audit provider using this functionality has
been developed as part of the DARPA CADETS project and is in review
(https://reviews.FreeBSD.org/D10149). In the OpenBSM GitHub repository,
support for Coverity static analysis was added via TravisCI.
Additionally, the OpenBSM 1.2-alpha5 release has been merged into the
FreeBSD HEAD branch.
This project was sponsored by DARPA/AFRL (in part).
Open tasks:
1. Test the latest release on different versions of FreeBSD, Mac OS X
and Linux. Testing on the latest versions of Mac OS X would be
particularly appreciated.
2. Fix problems that have been reported via GitHub and the FreeBSD bug
tracker.
3. Implement the features mentioned in the TODO list on GitHub.
__________________________________________________________________
Porting Software to CloudABI: Sandboxed Bitcoin!
Links
How to Use CloudABI on FreeBSD
URL: https://nuxi.nl/cloudabi/freebsd/
LevelDB for CloudABI
URL: https://nuxi.nl/blog/2017/02/18/porting-leveldb-to-cloudabi.html
Memcached for CloudABI
URL: https://nuxi.nl/blog/2017/03/15/sandboxed-memcached.html
Bitcoin for CloudABI
URL: https://laanwj.github.io/2017/03/02/porting-bitcoin-core-to-cloudabi.html
Contact: Ed Schouten <ed@FreeBSD.org>
CloudABI is a framework that allows you to develop strongly sandboxed
applications a lot more easily. It is a programming environment that
exclusively uses FreeBSD's Capsicum facilities. Any features
incompatible with Capsicum have been removed entirely, which means that
it is easier to determine how code needs to be adjusted to behave
correctly while sandboxed. In essence, you only need to patch up the
code until it builds.
Last year we have managed to port a lot of exciting libraries over to
CloudABI. Highlights include sandboxing aware versions of Boost and
LevelDB. Now that these libraries are readily available, we are at the
point where we can shift our focus towards porting full applications.
In late February one of the lead developers of the Bitcoin reference
implementation got in touch, as he is very interested in creating a
copy of Bitcoin that is better protected against security bugs. You do
not want a security bug in the networking/consensus code to allow an
attacker to steal coins from your local wallet!
As I think that this is a use case that demonstrates the strength of
CloudABI well, I've made addressing any issues reported by the Bitcoin
developers a top priority. Once the Bitcoin port is complete, we want
to provide binary packages of it as well.
This project was sponsored by Nuxi, the Netherlands.
Open tasks:
1. Though getting Bitcoin to work is pretty awesome, don't let that
distract us from porting other pieces of software over as well! Are
you the maintainer of a piece of software that could benefit from
sandboxing? Be sure to try building it using the CloudABI
toolchain!
2. One of the pieces of software that got ported over to CloudABI some
time ago is Memcached. Are you a user of Memcached? If so, feel
free to give the sandboxed version of Memcached for CloudABI a try!
3. So far, CloudABI can be used to run software written in C, C++ and
Python. Would you like to see any other programming language work
on CloudABI as well? Be sure to help out!
__________________________________________________________________
Support for eMMC Flash and Faster SD Card Modes
Contact: Marius Strobl <marius@FreeBSD.org>
In r315430, support for eMMC partitions has been added to mmc(4) and
mmcsd(4) in FreeBSD 12. Besides the user data area, i.e., the default
partition, eMMC v4.41 and later devices can additionally provide up to:
* 1 enhanced user data area partition
* 2 boot partitions
* 1 RPMB (Replay Protected Memory Block) partition
* 4 general purpose partitions (optionally with an enhanced or
extended attribute)
Apart from simply subdividing eMMC flash devices or having UEFI code in
the boot partition, as is done on some Intel NUCs, another use case for
partition support is the activation of pseudo-SLC mode, which
manufacturers of eMMC chips typically associate with the enhanced user
data area and/or the "enhanced" attribute of general purpose
partitions.
In order to be able to partition eMMC devices, r315430 also added a
Linux-compatible ioctl(2) interface to mmcsd(4). This allows the use of
the GNU mmc-utils (found in ports as sysutils/mmc-utils) on FreeBSD.
Besides partitioning eMMC devices, the mmc tool can also be used to
query for lifetime estimates and pre-EOL information of eMMC flash, as
well as to query some basic information from SD cards.
CAVEAT EMPTOR: Partitioning eMMC devices is a one-time operation.
Additionally, in order to make eMMC flash devices more usable, support
for DDR (Dual Data Rate) bus speed mode at a maximum of 52 MHz (DDR52)
has been added to mmc(4) and sdhci(4) in r315598, which will appear in
FreeBSD 12. Compared to high speed mode (the previous maximum) at 52
MHz, DDR52 mode increases the performance of the tested eMMC chips from
~45 MB/s to ~80 MB/s.
So far, support for DDR52 mode has been enabled for the eMMC
controllers found in the Intel Apollo Lake, Bay Trail and Braswell
chipsets. Note, however, that the eMMC and SDHCI controllers of the
Apollo Lake variant occasionally lock up due to a silicon bug (which is
independent of running in DDR52 mode). The only viable workaround for
that problem appears to be the implementation of support for ADMA2 mode
in sdhci(4) (currently, sdhci(4) supports only the encumbered SDMA mode
or no DMA at all).
However, r315598 also brought in infrastructure and a fair amount of
code for using even faster transfer modes with eMMC devices and SD
cards respectively, i.e., up to HS400ES with eMMC and the UHS-I modes
up to SDR104 with SD cards.
The intent is to merge these changes back to FreeBSD 10 and 11.
Open tasks:
1. Add support for eMMC HS200, HS400, and HS400ES transfer modes.
2. Add support for SD card UHS-I transfer modes (SDR12 to SDR104).
3. Make mmcsd(4) more robust and correctly follow the relevant
specifications for existing features, e.g., calculate and handle
erase timeouts, do a SEND_STATUS when CMD6 is invoked with an R1B
response to the extent not already fixed as part of r315430, get
the remainder of the existing code to properly check and handle
return codes, etc.
__________________________________________________________________
TrustedBSD
Links
TrustedBSD Website
URL: http://www.trustedbsd.org
TrustedBSD on GitHub
URL: https://github.com/trustedbsd
Contact: Christian Brueffer <brueffer@FreeBSD.org>
Contact: Robert Watson <rwatson@FreeBSD.org>
Contact: TrustedBSD announce mailing list
<trustedbsd-announce@TrustedBSD.org>
The TrustedBSD Project is an open-source community developing advanced
security features for the open-source FreeBSD operating system. Started
in April 2000, the project developed support for extended attributes,
access control lists (ACLs), UFS2, OpenPAM, security event auditing,
OpenBSM, a flexible kernel access control framework, mandatory access
control, and the GEOM storage layer. The results of this work may be
found not just in FreeBSD, but also NetBSD, OpenBSD, Linux, and Apple's
Mac OS X and iOS operating systems. Today, the project continues to
maintain and enhance these mature features in FreeBSD.
During this quarter, the TrustedBSD project transitioned from the
FreeBSD Perforce server to GitHub. This was made possible by Alexis
Sarghel, who owned the user "trustedbsd" on GitHub and graciously
transferred this account to the TrustedBSD project. To date, the
repositories hosting the TrustedBSD website and the SEBSD repository
have been moved.
__________________________________________________________________
Kernel
FreeBSD on Hyper-V and Azure
Links
FreeBSD Virtual Machines on Microsoft Hyper-V
URL: https://wiki.FreeBSD.org/HyperV
Supported Linux and FreeBSD Virtual Machines for Hyper-V on Windows
URL: https://technet.microsoft.com/en-us/library/dn531030.aspx
Contact: Sepherosa Ziehau <sepherosa@gmail.com>
Contact: Hongjiang Zhang <honzhan@microsoft.com>
Contact: Dexuan Cui <decui@microsoft.com>
Contact: Kylie Liang <kyliel@microsoft.com>
SR-IOV support for NICs is implemented. So far, we have only tested
with the Mellanox ConnectX-3 VF card, which works despite some issues
(Bug 216493: https://bugs.FreeBSD.org/bugzilla/show_bug.cgi?id!6493).
Updates for UEFI VMs (i.e., Hyper-V Generation 2 VMs):
1. After the loader issue (Bug 211746) is fixed, UEFI VMs can now boot
with Secure Boot disabled;
2. A synthetic keyboard driver has been added. Currently it is only in
HEAD, but MFCs to stable/10 and stable/11 are planned to occur
soon;
3. A SCSI DVD detection issue (Bug 218248) was fixed. Without the fix,
the VM would fail to boot.
This project was sponsored by Microsoft.
__________________________________________________________________
Intel 10G and 40G Network Driver Updates
Links
Commit adding X553 ix/ixv Support for iflib
URL: https://reviews.FreeBSD.org/D9851
Commit converting ixgbe to iflib
URL: https://reviews.FreeBSD.org/D5213
Contact: Jeb Cramer <jeb.j.cramer@intel.com>
Contact: Eric Joyner <eric.joyner@intel.com>
Contact: Krzysztof Galazka <krzysztof.galazka@intel.com>
This driver update is for the Intel ix/ixv and ixl/ixlv network
drivers, and includes support for several new hardware releases.
ix/ixv:
* Added support for X553 SoC devices based on the Denverton platform
ixl/ixlv:
* Added support for X722 10G SoC devices based on the Lewisburg
chipset
* Added an interface for the upcoming iWarp driver for X722 devices
* Added support for XXV710 25G devices
Open tasks:
1. ix/ixv iflib support is currently under review in Phabricator. It
will be refactored to include D5213.
2. Initial work for ixl/ixlv iflib support is in progress.
__________________________________________________________________
Linuxulator
Contact: Dimitry Chagin <dchagin@FreeBSD.org>
Contact: Edward Tomasz Napierała <trasz@FreeBSD.org>
Contact: Mahdi Mokhtari <mmokhi@FreeBSD.org>
In this quarter, we are pleased to announce two (of many) works
achieved in the Linuxulator.
We added a new placeholder marker UNIMPLEMENTED to accompany the
previously existing DUMMY, for distinguishing syscalls that the Linux
kernel itself does not implement from those that we currently do not
implement. Now our linux_dummy.c is clearer for newcomers to follow,
and they will quickly know which areas they can start working on.
Support for two new syscalls, preadv and pwritev, was added to the
Linuxulator.
Open tasks:
1. We plan to implement the execveat syscall for the native FreeBSD
syscall table and then port/wrap it for use in the Linuxulator.
__________________________________________________________________
MMC Stack Using the CAM Framework
Links
Project Information
URL: https://bakulin.de/freebsd/mmccam.html
Source Code
URL: https://github.com/kibab/FreeBSD/tree/mmccam-collapsed-commits
Contact: Ilya Bakulin <kibab@FreeBSD.org>
The goal of this project is to reimplement the existing MMC/SD stack
using the CAM framework. This will permit utilizing the well-tested CAM
locking model and debugging features. It will also be possible to
process interrupts generated by the inserted card, which is a
prerequisite for implementing the SDIO interface. SDIO support is
necessary for communicating with the WiFi/BT modules found on many
development boards, such as the Raspberry Pi 3.
Another feature that the new stack will have is support for sending SD
commands from userland applications using cam(3). This will allow for
building device drivers in userland and make debugging much easier.
The new stack is able to attach to an SD card and bring it to an
operational state so that it is possible to read and write to the card.
The stack has been tested to work on the Beaglebone Black and Wandboard
Quad platforms.
Currently the code is being prepared for inclusion in the FreeBSD
source tree. cam(3) is being extended to support SDIO-specific
functions (reading registers, managing interrupts, etc.).
Open tasks:
1. Integrate the code into FreeBSD HEAD to facilitate testing.
2. Begin writing a driver for Broadcom-based WLAN chips (found on the
Raspberry Pi 3 and Wandboard).
3. Begin writing a driver for Marvell-based WLAN chips (found on the
GlobalScale Dreamplug and some Chromebooks).
__________________________________________________________________
pNFS Server Plan B
Contact: Rick Macklem <rmacklem@FreeBSD.org>
Parallel NFS (pNFS) is an extension to the NFSv4 protocol that allows
for file accesses within a single logical mount to be performed against
multiple file servers, with the potential for data access to occur in
parallel. The pNFS "layout" in use specifies how the division occurs,
with metadata operations occurring against the main server, and bulk
data operations (read/write/setattr/etc.) occurring via a
layout-specific scheme between the client and data servers.
My first attempt at a pNFS server using GlusterFS was a dud. It worked,
but performance was so poor that it was not usable. This attempt that I
call "Plan B", only uses FreeBSD, with one FreeBSD server handling the
metadata operations and multiple FreeBSD servers configured to serve
data. An NFSv4.1 client that supports the pNFS File Layout will be able
to read and write to the data servers directly, spreading out the RPC
load and allowing growth beyond that of what a single FreeBSD NFS
server could achieve.
There is no support for the Flex Files Layout or mirroring at this
time. I hope to use the Flex Files Layout to add mirroring support over
the next year or so. Striping is also not supported, but I have no
plans for implementing it at the moment.
Plan B is working quite well now and should be available for testing by
the end of April. I will announce how to do this on the
freebsd-fs@FreeBSD.org mailing list when it is available.
Open tasks:
1. Testing by others will be needed, once it is available.
__________________________________________________________________
Architectures
64-bit PowerPC Book-E Support
Contact: Justin Hibbits <jhibbits@FreeBSD.org>
The Book-E platform target now supports 64-bit mode ("powerpc64"). It
includes a 63-bit address space split, but the page table directory
list uses holes to expand to the full address space, leaving gaps in
the address space where page mappings are repeated. This may change in
the future.
As with the AIM powerpc64 port, Book-E supports running powerpc
(32-bit) binaries as well, and has even been tested with a 32-bit init
and 64-bit shell.
Several of the SoC drivers are supported, however, the dTSEC ethernet
controller is not yet supported. Work is ongoing to support it.
A QORIQ64 config is included, targeting the P5 and T* series SoCs from
Freescale.
Thanks to Juniper Networks for providing patches against an older
internally maintained FreeBSD version, which enabled this porting
effort, and for providing historical context for quirks of the pmap
changes.
Open tasks:
1. Port the dTSEC driver to 64-bit. There are assumptions in the
reference driver of operating in a 32-bit environment. It may be
easier to port the Linux driver instead, which would also give ARM
support for this ethernet controller.
2. Take advantage of pointer alignment to squeeze more bits out of the
page tables; it should be possible to squeeze at least 3 more bits
out, one at each level.
__________________________________________________________________
FreeBSD on Marvell Armada38x
Contact: Marcin Wojtas <mw@semihalf.com>
Contact: Zbigniew Bodek <zbb@semihalf.com>
Final testing and productionization of support for the Marvell
Armada38x platform is under way. The rebase and cleanup is going well,
with patches functioning on top of HEAD and ready for upstreaming.
Specific tasks completed include:
* Platform benchmarking and low-level optimizations (internal bus,
L1/L2 cache prefetch) -- already submitted)
* Enable the PL310 L2 cache controller -- currently under review
* NETA tests, optimizations and PHY-handling rework
* e6000sw PHY handling rework and fixes
* Fix and enable performance counter support
* Fix timers and extract watchdog support to a separate driver
* Crypto driver fixes -- merged
* AHCI controller support -- merged
* Thermal driver -- merged
* Merge additional support for new boards (SolidRun ClearFog and
DB-88F6285-AP)
This project was sponsored by Stormshield, and Semihalf.
Open tasks:
1. Submit the remaining fixes and drivers.
__________________________________________________________________
FreeBSD/s390x Attempt III
Contact: Bjoern A. Zeeb <bz@FreeBSD.org>
A long time ago, in the FreeBSD 5 times, there was an initial port of
FreeBSD to s390 (32bit) and s390x (64bit) which booted past init on
good days in an emulator.
As an attempt to revive the s390x/systemz efforts I started to get
FreeBSD s390x to build with clang/llvm 3.9. At this time, it is
possible to build world and a GENERIC kernel skeleton (not doing
anything yet) using external binutils.
The primary idea of this initial work was to allow for incremental
addition of the necessary architecture-specific code. Having the build
framework in place will allow third-party developers to simply type
make, as they are willing to contribute to the port without having to
know FreeBSD build specifics. After some cleanup and further updates to
a more recent HEAD I am planning to push the current work to a public
repo to facilitate collaboration.
Open tasks:
1. Write a wiki page with per-architecture specific tasks that need to
be done, based on the current work and the experience from arm64
and riscv.
2. Implement both the userspace and kernel per-architecture gaps.
3. Figure out a way to get access to IBM's zPDT or better emulators to
ease implementation, testing, and debugging.
__________________________________________________________________
Ports
MySQL
Links
MySQL80 Overview
URL: https://www.mysql.com/why-mysql/presentations/mysql-80-overview/
MySQL80 InnoDB New Features
URL: https://www.mysql.com/why-mysql/presentations/innodb-whats-new-mysql-80/
Contact: Mahdi Mokhtari <mmokhi@FreeBSD.org>
Contact: Mark Felder <feld@FreeBSD.org>
This quarter a new -dev version of MySQL landed in the Ports
Collection, MySQL 8.0. It introduces many new features, though we had
to (re)-patch parts of it which were merged by MySQL from MySQL5.7.
We also updated MySQL 5.6 to its latest version and closed many PRs
related to it, mostly relating to using FreeBSD-provided ports for
libraries instead of the bundled copies. And of course there were
plenty of security updates.
We can also report that the problem of having to specify
${mysql_optfile}, which some people encountered while using MySQL, is
now considered to be solved in all MySQL versions: 5.6, 5.7, and 8.0.
Now the init script will search all default locations, for backwards
compatibility with the variety of locations used for configuration
files, before it gives up and reports an error.
Open tasks:
1. Test the new version and report problems.
__________________________________________________________________
Rust
Links
Wiki Portal
URL: https://wiki.FreeBSD.org/Rust
Guide to Bootstraping Rust on FreeBSD
URL: https://gist.github.com/dumbbell/b587da50ef014078da9e732a4331ebad
Bug Report to Track Progress on Bootstrapping
URL: https://bugs.FreeBSD.org/bugzilla/show_bug.cgi?id!6143
Contact: Jean-Sébastien Pédron <dumbbell@FreeBSD.org>
Contact: Thomas Zander <riggs@FreeBSD.org>
In the Ports Collection, Rust was updated to 1.16.0 and Cargo to
0.17.0, the latest versions at the time of this writing.
lang/rust-nightly was also updated to a snapshot from February and it
is now enabled on i386. It is lagging a bit behind upstream, but Rustup
works nicely on FreeBSD if you need to try any versions/channels of
Rust.
Work has started to bootstrap Rust on non-x86 architectures. Patches to
add FreeBSD/aarch64 support were submitted and accepted upstream.
FreeBSD/sparc64 is in progress. The lang/rust-nightly port is also
being adapted to compile natively on FreeBSD/aarch64. This work is
critical, in particular because Firefox will shortly require Rust. If
you want to help, please refer to the guide linked above.
The compiler, rustc, is crashing sometimes when there is a compilation
error. Therefore, there is a bit of work to do to improve stability.
There is some code duplication between lang/rust* and devel/cargo.
Those Makefiles deserve a bit of cleanup. It might be useful to create
a USES=rust Makefile helper.
Open tasks:
1. Bootstrap Rust on more platforms.
2. Investigate compiler crashes.
3. Create a USES=rust Makefile helper and simplify the Rust and Cargo
ports.
4. Investigate how to speed up lang/rust* compilation time.
__________________________________________________________________
Documentation
The FreeBSD Dutch Documentation Project
Links
The Dutch Translation Project
URL: https://www.FreeBSD.org/docproj/translations.html#dutch
Contact: René Ladan <rene@FreeBSD.org>
Contact: Remko Lodder <remko@FreeBSD.org>
Work has started on an initial translation of the FreeBSD Handbook to
the Dutch language via the "po" system. While we have an (outdated)
version of the Handbook available via the older XML files, we are now
trying to get back into shape with the PO file.
Rene started working on two articles already and did some translation
of strings for the FDP-Primer, while Remko has started working on the
Handbook. If you think you can assist with either, please send Rene and
Remko an email so that we can start coordinating work.
In addition, since we have a translation set already from the XML
files, it would be interesting to see whether we can merge them easily
into the PO structure. If you have ideas on that, contact us a.s.a.p.
This project was sponsored by Snow B.V. (in part).
Open tasks:
1. Identify a way to merge the current XML translations into the
nl_NL.po files.
2. Merge the translations into the .po files.
3. Update the remaining open items into the po files.
4. Remove the old/outdated translation files from the main repo and
use the po and book.xml files to generate the Dutch handbook and
other files.
5. Identify whether we can also translate the htdocs pages via the PO
system.
__________________________________________________________________