Autospec

autospec is a tool to assist in the automated creation and maintainence of RPM packaging. It will continuously run updated builds based on new information discovered from build failures, until it has a complete and valid .spec file. The tool makes use of mock to achieve this.

autospec is available under the terms of the GPL, version 3.0

Copyright (C) 2015 Intel Corporation

autospec is configured by means of a simple INI-style configuration file. The default location of this file is assumed to be common/autospec.conf, relative to the directory in which autospec is executed.

Example autospec.conf file:

[autospec]
git = [email protected]/%(NAME)s
license_fetch = http://yourhost/hash.php
license_show = http://yourhost/showone.php?hash=%(HASH)s
upstream = http://yourhost/tarballs/%(HASH)s/%(NAME)s

git

The upstream git repository URL base

license_fetch

Optional URL to use for scanning license files

license_show

Optional URL to interact with online license checker

upstream

Base URL for stored upstream tarballs

Usage: python3 autospec.py [options] URL

In order to run correctly, autospec requires the following components:

• python3
• correctly configured mock

If autospec is not configured to use a license server, then you will need a common/licenses file - which should be an up to date list of licenses to facilitate automatic license detection during the scan of a tarball. For correctness, license names should be in the SPDX identifier format. Each line in the file constitutes a license definition, for example:

750b9d9cc986bfc80b47c9672c48ca615cac0c87 | BSD-3-Clause
175e59be229a5bedc6be93e958a970385bb04a62 | Apache-2.0
794a893e510ca5c15c9c97a609ce47b0df74fc1a | BSD-2-Clause

It is possible to influence precisely how autospec will behave in order to gain fine control over the build itself. These files may be used to alter the default behaviour of the configure routine, to blacklist build dependencies from being automatically added, and such.

These files are expected to live in same directory that the resulting .spec will live.

release

This file contains the current release number that will be used in the .spec. This is also bumped and generated on existing and new packages, respectively. This results in less manual work via automatic management.

$package.license

In certain cases, the package license may not be automatically discovered. In this instance, autospec will exit with an error. Update this file to contain the valid SPDX identifier for any license(s) for the package, replacing $package in the filename with the actual package name.

buildreq_add

Each line in the file provides the name of a package to add as a build dependency to the .spec.

pkgconfig_add

Each line in the file is assumed to be a pkgconfig() build dependency. Add the pkg-config names here, as autospec will automatically transform the names into their pkgconfig($name) style when generating the .spec.

buildreq_ban

Each line in the file is a build dependency that under no circumstance should be automatically added to the build dependencies. This is useful to block automatic configuration routines adding undesired functionality, or to omit any automatically discovered dependencies during tarball scanning.

pkgconfig_ban

Each line in this file is a pkgconfig() build dependency that should not be added automatically to the build, much the same as buildreq_ban. As with pkgconfig_add, these names are automatically transformed by autospec into their correct pkgconfig($name)) style.

configure

This file contains configuration flags to pass to the %configure macro for autotools based tarballs. As an example, adding --disable-static to ./configure for an autootools based tarball would result in %configure --disable-static being emitted in the .spec.

cmake_args

This file contains arguments that should be passed to the %cmake macro for CMake based tarballs. As an example, adding -DUSE_LIB64=ON to ./cmake_args would result in %cmake -DUSE_LIB64=ON being emitted in the .spec.

broken_parallel_build

If this file exists, then parallelisation will be disabled in the build. This usually means that %{?_smp_mflags} will not be passed to make

make_args

The contents of this file are appended to the make invocation. This may be useful for passing arguments to make, i.e. make TOOLDIR=/usr

make_install_args

Much like make_args, this will pass arguments to the make install macro in the .spec

install_macro

The contents of this file be used instead of the automatically detected install routine, i.e. use this if %make_install is insufficient.

subdir

Not all packages have their Makefile's available in the root of the tarball. An example of this may be cross-platform projects that split Makefile's into the unix subdirectory. Set the name in this file and the .spec will emit the correct pushd and popd lines to utilise these directories for each step in the build.

build_pattern

In certain situations, the automatically detected build pattern may not work for the given package. This one line file allows you to override the build pattern that autospec will use. The supported build_pattern types are:

• configure: Traditional %configure autotools route
• configure_ac: Like configure, but performs ``%reconfigure to regenerate ./configure
• autogen: Similar to configure_ac but uses the existing ./autogen.sh instead of %reconfigure
• distutils: Only build the Pythonic package with Python 2
• distutils23: Build the Pythonic package using both Python 2 and Python 3

series

This file contains a list of patches to apply during the build, using the %patch macro. As such it is affected by -p1 style modifiers.

excludes

This file is used to generate %exclude lines in the .spec. This is useful for omitting files from being included in the resulting package. Each line in the file should be a full path name.

keepstatic

If this file exists, then %define keepstatic 1 is emitted in the .spec. As a result, any static archive (.a) files will not be removed by rpmbuild.

extras

Each line in the file should be a full path within the resulting package, that you wish to be placed into an automatic -extras subpackage. This allows one to keep the main package slim and split out optional functionality or files.

setuid

Each line in this file should contain the full path to a binary in the resulting build that should have the setuid attribute set with the %attr macro.

attrs

Each line in this file should be a full %attr macro line that will be included in the .spec to have fine-grained control over the permissions and ownership of files in the package.

By default, autospec will attempt to detect potential test suites that can be run in the %check portion of the .spec.

skip_test_suite

If this file exists, autospec will not emit any %check functionality.

unit_tests_must_pass

This file is automatically created upon successful completion of a package build. This allows one to identify regressions in test failures when updating or altering a package. autospec will fail a package that does not pass it's test suite if this file exists.

make_check_command

Override or set the command to use in the %check portion of the .spec. This may be useful when a package uses a custom test suite, or requires additional work/parameters, to work correctly.

allow_test_failures

The existence of this file will allow test failures, and will still emit the %check code in a way that allows the build to continue.

asneeded

If this file exists, the .spec will disable the LD_AS_NEEDED variable. Supporting binutils (such as found in Clear Linux Project for Intel Architecture) will then revert to their normal behaviour, instead of enforcing -Wl,-as-needed in the most correct sense.

optimize_size

If this file exists, the CFLAGS/LDFLAGS will be extended to build the package optimised for size, and not for speed. Use this when size is more critical than performance.

funroll-loops

If this file exists, the CFLAGS/LDFLAGS will be extended to build the package optimised for speed. In short this where speed is of paramount importance, and will use -03 by default.

insecure_build

If this file exists, the CFLAGS/LDFLAGS will be replaced, using the smallest -02 based generic flags possible. This is useful for operating systems employing heavy optimisations or full RELRO by default.

autospec will attempt to use a number of patterns to determine the name and version of the package by examining the URL. For most tarballs this is simple, if they are of the format $name-$version.tar.$compression.

For websites such as bitbucket or GitHub, using get$ and v$.tar.* style links, the project name itself is used from the URL and the version is determined by stripping down the tag.

CPAN Perl packages, R packages, and rubygems.org rubygems are automatically prefixed with their language name: perl-, R- and rubygem- respectively.

When these automated detections are not desirable, it is possible to override these with the --name flag when invoking autospec

autospec can optionally talk to a license server instead of checking local hashsum files, which enables greater coverage for license detection. The URL set in license_fetch is expected to be a simple script that talks HTTP.

This URL should accept POST requests with the following keys:

hash

Contains the SHA-1 hash of the potential license file being checked.

package

The name of the package being examined

text

The contents of the potential license file

Implementations return a plain text response with the SPDX identifier of the license, if known. An empty response is assumed to mean that this license is unknown, in which case autospec will emit the license_show URL. The implementation should show the now-stored license file via a web page, and enable a human to make a decision on the license. This is then stored internally, allowing future requests to automatically know the license type when this hash is encounted again.