Gentoo Smolt

With Smolt we collect information about hard- and software to better prioritize development efforts.

Smolt consists of a three main components:

• GUI client (based on Qt 4.x)
• Command line client
• Server “smoon” (based on TurboGears 1.x)

The client is used to submit data to the server. While the GUI client is run manually, the command line client is either run manually or through a cron job.

The server takes care of

• Handling submission
• Generation of reports
• Display of reports

These classes of data are being collected unless disabled:

• Hardware focus
  • CPU information
    • BogoMIPS
      e.g. 5999.85
    • Vendor
      e.g. GenuineIntel
    • Model
      e.g. Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz
    • Stepping
      e.g. 10
    • CPU Family
      e.g. 6
    • Model Number
      e.g. 23
    • Number of CPUs
      e.g. 2
    • Speed
      e.g. 3003
  • Memory capacities
    • System Memory in MB
    • System Swap in MB
  • Vendor
    e.g. LENOVO
  • System
    e.g. Latitude D630
  • Form factor
    one of desktop, laptop, server, handheld, unknown
  • Device information
    • TODO
• Software focus (possible hardware implications)
  • Platform
    e.g. i686
  • Operating system
    e.g. Gentoo Base System release 1.12.11.1
  • Default run level
    e.g. 3
  • Language
    e.g. en_US.UTF-8
  • Kernel
    e.g. 2.6.27-gentoo-r8
  • SELinux information
    • SELinux Enabled
      either 1 or 0
    • SELinux Enforce
      e.g. Enforcing
    • SELinux Policy
      e.g. targeted
  • File system information
    • TODO

• Software focus (possible hardware implications)
  • Architecture-related
    • Architecture (from ARCH and CHOST variables)
    • Stability (from ACCEPT_KEYWORDS variable)
    • System profile (from /etc/make.profile)
  • Features (from FEATURES variable)
  • Compile flags
    • CFLAGS
    • CXXFLAGS
    • LDFLAGS (yes, that's link flags …)
    • MAKEOPTS
  • Global use flags from
    • “conf” i.e. /etc/make.conf
    • “globals” i.e. /etc/make.globals
    • “defaults” i.e. /etc/make.profile stack
  • Mirrors
    • Distfiles mirrors (from GENTOO_MIRRORS variable)
    • rsync mirror (from SYNC variable)
  • User-masked packages (from /etc/package.mask)
  • Repositories from
    • PORTDIR variable
    • PORTDIR_OVERLAY variable
  • Installed packages from /var/db/pkg
    • Category and package name
    • Version and revision
    • Slot
    • Keyword status
      i.e. what kind of package.keywords entry would be needed to re-install the package now. Can be one of:
      • ”” means the package is no less stable than the system itself
      • ”~arch” means an unstable package on a stable system
      • ”**” catches case KEYWORDS=”” mainly
    • Masked flag
    • Unmasked flag
    • In-world-file flag from /var/lib/portage/world
    • Source repository name
      e.g. gentoo or sunrise
    • Enabled flags (subset of ISUE from the related ebuild)

First of all Smolt is free open source software. One of the benefits that brings is that you (or someones else) can freely study what Smolt does. That possibility alone is a reason why many people trust software libre better than proprietary software.

Smolt goes a long way to protect your privacy:

1. Smolt submits only the data you allow it to: Each class of data can be excluded from submission.
2. Prior to submission you get the chance to view what data would be submitted in detail.
3. Auto-filtering filters out potentially private information for you automaticaly

This allows people with private setups to safely submit their non-private data to the Smolt server.

Also, feel free to contact me with your privacy questions and concerns.

Here's a list of easy steps to a privacy-safe Smolt submission:

1. Consider how private your setup is and what risks you face
   • Is it a machine at work?
     Does your setup contain private employer or customer specific bits, e.g. ebuilds for unreleased or internal products?
2. Disable directly affected classes of data in client.cfg
   e.g. if you use private mirrors set mirrors = False in section [gentoo]
3. Disable indirectly affected class of data
   e.g. if you considered your architecture to be private you would exclude compile and use flags from the submission, too. Admittedly a bit constructed but it should make the point clear.
4. Run the Smolt client and review what data would be submitted in detail. Do further adjustments to the config if needed.

Where possible Smolt is auto-filtering potentially private information. Such places are:

• Mirrors
  • Package tree mirror
    is filtered unless hostname matches (*.)gentoo.org
  • Distfiles mirrors
    are filtered unless found in mirrors3.xml
• Repositories
  are filtered unless in layman-global.txt or on the whitelist (e.g. “gentoo” or “funtoo”)
• Global use flags
  are filtered unless in one of these sets:
  • registered global use flags i.e. /usr/portage/profiles/use.desc
  • registered local use flags i.e. /usr/portage/profiles/use.local.desc
  • arch auto-use-flags i.e. /usr/portage/profiles/arch.list
  • registered global use_expand flags i.e. /usr/portage/profiles/desc/*.desc
• Installed packages
  • Packages itself
    are filtered unless
    • there exists an ebuild for that package (without regard to version) in a non-private overlay at the moment or
    • they were installed from a non-private overlay in the past (even if either the overlay went away or the ebuild was removed from it) .. or both
  • Repository origin
    is reset unless the repository is considered non-private. (see Repositories above)
  • Use flags
    are filtered according to the same rules as global use flags (see above)

To further increase transparency the Smolt client is logging the last 9 submissions for you in rotating fashion in ~/.smolt/.

Three types of logs are kept:

• ~/.smolt/submission-distro.html.[1-9] (distro-specific data sent, HTML)
• ~/.smolt/submission.json.[1-9] (all data sent, JSON)
• ~/.smolt/submission.rst.[1-9] (all data sent, reStructuredText)

The last submission is in files submission*.1, the second last in files submission*.2.

On the client side the user can configure which classes of data he wants to keep to himself and which classes he is willing to share. The default config is read from /etc/smolt/client.cfg and can be overridden by ~/.smolt/client.cfg. Both files are ConfigParser files, a superset of regular .ini files.

• any
  • arch
  • cpu
  • devices
  • distro
  • file_systems
  • form_factor
  • kernel
  • language
  • model
  • ram_size
  • run_level
  • selinux
  • swap_size
  • vendor
• gentoo
  • arch_related
  • compile_flags
  • features
  • global_use_flags
  • installed_packages
  • installed_packages_use_flags
  • mirrors_sync
  • mirrors_distfiles
  • package_mask
  • repositories
  • system_profile

[any]
arch = True
cpu = True
devices = True
distro = True
file_systems = True
form_factor = True
kernel = True
language = True
model = True
ram_size = True
run_level = True
selinux = True
swap_size = True
vendor = True
  
[gentoo]
arch_related = True
compile_flags = True
features = True
global_use_flags = True
installed_packages = True
installed_packages_use_flags = True
mirrors_sync = True
mirrors_distfiles = True
package_mask = True
repositories = True
system_profile = True

The server instance can be configured though a file dev.cfg. For now the only important keys are:

• global
  • sqlalchemy.dburi
    (Database connection URI)
  • server.socket_port
    (Port to listen on)

[global]
sqlalchemy.dburi='mysql://smoon:smoon@localhost/smoon'
server.socket_port=45678