-rw-r--r-- 6528 saferewrite-20250228/README-analysis raw
Running the ./analyze script (see README) produces various
build/*/*/*/analysis/* files. Here's a guide to what those mean.

(There are also build/compiler/*/analysis/* files, which you might look
at if you expect a compiler to be used but it isn't being used.)

The first two stars in build/*/*/*/analysis/* are organized the same way
as src/*/*. For example, src/cmp_64xint16/frodo2 is a (wrong) rewrite of
src/cmp_64xint16/ref, and build/cmp_64xint16/frodo2 contains analysis of
that rewrite. The third star is the compiler name, with spaces and
equality signs converted to underscores. There are many possibilities
for the rest of the filename, the analysis/* part. Most possibilities
are listed below.

analysis/equals-*: empty file indicating that this implementation
produces, for all possible inputs, the same result as the implementation
specified by the last star. This comes from an SMT solver applied to
unrolled code for both implementations. The choice of implementation to
test against is currently the following: sort implementation-compiler
pairs (that were successfully unrolled) to put "ref" first, then break
ties by shorter implementation name, then break ties by implementation
name lexicographically, then break ties by compiler position in the list
of compilers; test each non-"ref" implementation against the "ref"
implementation for the same compiler if that's earlier in the list, or
else against the first entry in the list. The first entry in the list is
intentionally tested against itself as a check of the test mechanism.

analysis/unrolled: the unrolled code for this implementation.

analysis/passed-valgrind: empty file indicating that valgrind did not
find data flow from inputs to conditional branches or array indices.

analysis/unsafe-*differentfrom-*: example of inputs and outputs for
which this implementation produces a different result from the
implementation specified by the last star. There are (currently) two
flavors of these files: analysis/unsafe-randomtest-*-differentfrom-* is
found by testing a few random inputs; analysis/unsafe-differentfrom-* is
found by SMT solving applied to unrolled code. (For example,
cmp_64xint16/libc4 compares 127 bytes instead of 128; random tests won't
detect that, but SMT solving easily does. As another example,
int32_positive_mask/shift2optbug with gcc -O3 without -fwrapv produces a
bug caught by SMT solving.)

analysis/unsafe-valgrinderror: empty file indicating that valgrind found
data flow from inputs to conditional branches or array indices.

analysis/unsafe-unrollsplit-*: empty file indicating that angr found
data flow from inputs to something splitting angr's universes; the * is
the number of different angr universes. (Code that splits into too many
universes will instead trigger warning-unrollerror.)

analysis/warning-compilefailed-*: nonzero exit code from compiler; the *
is the name of the file being (unsuccessfully) compiled. (For example,
int32_sort2/compilebug is an implementation that, because of a typo in
an argument declaration, doesn't compile.)

analysis/warning-compileoutput-*: output from compiler; the * is the
name of the file being (successfully or unsuccessfully) compiled.

analysis/warning-linkfailed-*: nonzero exit code from linker; the * is
the name of the file being (unsuccessfully) linked, minus an "analysis-"
prefix. (For example, int32_sort2/linkbug is an implementation that,
because of a typo in a function name, doesn't link.)

analysis/warning-linkoutput-*: output from linker; the * is the name of
the file being (successfully or unsuccessfully) linked, minus an
"analysis-" prefix.

analysis/warning-cargofailed-*: similar to warning-linkfailed-* but for
Rust.

analysis/warning-cargooutput-*: similar to warning-linkoutput-* but for
Rust.

analysis/warning-executefailed: error message in trying to run
analysis-execute: e.g., missing libraries, or missing interpreter for
cross-compiled code.

analysis/warning-valgrinderror: empty file indicating that something
went wrong with running valgrind.

analysis/warning-unrollerror: error message saying that saferewrite is
giving up on running angr: e.g., a "saferewrite limiting split" message
indicating that code has split into more universes than saferewrite will
allow, or a "State errored" message that can probably be worked around
with more effort in saferewrite.

analysis/warning-unrollexception: similar to warning-unrollerror, but a
more severe type of failure that saferewrite isn't expecting.

analysis/warning-unrollmem: list of "mem" or "reg" variables, beyond the
specified inputs, for which angr found data flow into the specified
outputs. This can be a memory bug in the code being tested (e.g.,
cmp_64xint16/libc5 reads beyond the end of the buffer; as further
examples, apibug_inpu/api and apibug_outputt/api define too few inputs
and too many outputs respectively for code that would otherwise
correctly copy an input array to an output array), but it can also come
from compilers using partially uninitialized variables in ways that are
actually safe (something that usually happens with, e.g., registers on
mips64).

analysis/warning-unrollmismatch: example of inputs and outputs where
execution of the compiled code produces a different result from
saferewrite's simulation of the unrolled code. This could be a bug in
saferewrite, but a memory bug in the code being tested (e.g.,
cmp_64xint16/libc5 again) or an API bug in the test (e.g.,
apibug_inpu/api or apibug_outputt/api) can produce
warning-unrollmismatch along with warning-unrollmem.

analysis/warning-unusedinputs: list of specified inputs for which angr
found no data flow into the specified outputs. This is typically a bug
in the code being tested. (For example, recall that cmp_64xint16/libc4
fails to compare the last byte of input; this triggers
warning-unusedinputs. As further examples, apibug_inputt/api and
apibug_outpu/api define too many inputs and too few outputs respectively
for code that would otherwise correctly copy an input array to an output
array; these both trigger warning-unusedinputs.)

analysis/warning-div: empty file indicating that the unrolled code uses
integer divisions. Maybe this will be extended to floating-point later.

analysis/warning-mul: empty file indicating that the unrolled code uses
integer multiplications. (For example, int32_sort2/mul is an
implementation using multiplications.) Maybe this will be extended to
floating-point later.

analysis/seconds: lists the time taken by various analysis stages.