TBFILT(1)

NAME
tbfilt - filter for and analysis of test bench log files

SYNOPSIS
tbfilt --tee OFILE [OPTION]...
tbfilt [OPTION]... [FILES]...

DESCRIPTION

  Principle of Operation
tbfilt is the central tool to analyze the log files created by the test benches. It scans the test bench output for messages which indicate a test failure and based on this marks a test as PASSed or FAILed. It can be used in two modes:

    -   as a filter during test bench execution, typically in a setup like
                tbw <test_bench> 2>&1 | tbfilt --tee=<log_file>
        tbfilt reads the output from the test bench via stdin and a pipe, filters out the messages indicating a failure and shows them on stdout, and saves the full test bench output to the file given in the --tee option. In this mode tbfilt works similar to a
                tee ... | egrep ...
        pipeline with a very involved egrep selection expression. The exit status of tbfilt is 1 in case the test is considered as FAILed.

    -   as log file analysis tool. In this case, the test bench log files are either specified explicitly as arguments or determined via the --find or --all options. If the --summary option is specified a one-line summary for each test log file is displayed. The format of this summary is configurable via the --format, --wide, and --compact options and via the TBFILT_FORMAT environment variable. The exit status of tbfilt is 1 in case any of the tests is considered as FAILed.

  Filter Criteria
A line that contains any of the following strings is considered an indication of a FAILed test:
    -E:
    -F:
    ERROR
    FAIL
    :(assertion warning):
    :(assertion error):
    :(assertion failure):
As an exception to the general rules above the following assertion messages are accepted: Assertion warnings from IEEE libraries at startup (t=0ns) are ignored. They are hard to avoid in complex models and generally don't indicate a real issue.
    -   assertion warnings from IEEE libraries at startup (t=0ns). They are hard to avoid in complex models and in general, don't indicate a real issue. The best is to suppress them in ghdl(1) with the option '--ieee-asserts=disable-at-0'. That's usually done via the TBW_GHDL_OPTS environment variable used by tbw(1).
    -   assertion failure with the text 'Simulation Finished'. It is used to end simulations in ghdl(1) in some test benches.
tbfilt also expects a line in one of the formats
          xxx ns: DONE               -- tb'swithout clock
        xxx.x ns    xxx: DONE        -- single clock tb's
        xxx.x ns    xxx: DONE-xxx    -- multiclock tb's  (max taken)
and considers a test FAILed if it is missing. In addition, lines containing
    -W:
    PASS
will be displayed. If the --pcom option is specified also all lines starting with 'C'. Finally, tbfilt checks for a line of the format
        real xmx.xxxs   user xmx.xxxs   sys xmx.xxxs
and extracts the test bench execution times from this. It can be generated by a bash(1) 'time' command when
        export TIMEFORMAT=$'real %3lR   user %3lU   sys %3lS'
is set. The wrapper scripts brun_tbw(1) or tbrun_tbwrri(1) are in general used to set this up correctly.

OPTIONS

  Filter Options

--tee=ofile
    All log file input is written to ofile. Typically used to save the test bench output in a file when tbfilt is used in filter mode in a pipeline and reads from stdin.

--pcom
    Enables that lines starting with "C" are also displayed.

  File Selection Options

--find=patt
    When given the input files are determined with a find(1) command. The selection pattern patt is used with a find -regex in egrep mode. This is functionally similar to a
            find -regextype egrep -regex 'patt' | sort | tbfilt ....
    pipeline. When no '*' wildcard is found in patt it is assumed to be a mode specification and the pattern is prefixed by
            .*/tb_.*_
    and suffixed by
            .*\.log
    to select all log files of a given mode (e.g. 'bsim').

--all
    When given uses as input files all test bench files which conform the naming convention. Is equivalent to the option
            --find '.*/tb_.*_[bfsorept]sim(_.*)?\.log'

  Summary Options

--summary
    Selects summary mode. Only a single summary line per input file is written. The format is configurable via the --format, --wide, and --compact options and via the TBFILT_FORMAT environment variable. The precedence is (in increasing priority):
    -   build default ('%ec %pf %nf')
    -   TBFILT_FORMAT option
    -   --wide option
    -   --compact option
    -   --format option

--wide
    Selects a wide format for summary outputs, designed to give the most pertinent information. Uses a format of "%fd %fs %tr %tc %sc %ec %pf %nf".

--compact
    Selects a compact format for summary outputs, designed to give the key info on an 80 character wide line. Uses a format of "%fa %tg %sg %ec %pf %ns".

--nohead
    Suppresses the headline of summary outputs. Useful if the summary output is piped into sort or other tools.

--format=fmt
    Defined the format of the summary lines. The format specification fmt string is a sequence of conversion specifications of the form '%xx', which will be replaced by the respective values and other characters which are simply copied (usually a blank as delimiter). The supported conversion specifications are:
    %fd modification date of input file (as yyyy-mm-dd)
    %ft modification time of input file (as hh:mm:ss)
    %fs modification time of input file short format (as hh:mm)
    %fa age of input file in seconds, minutes, hours or days
    %tr real (wall clock) time of test bench run
    %tu user time of test bench run
    %ts system time of test bench run
    %tc total cpu (user+system) time of test bench run
    %tg show '%tc c' if cpu time is significant, otherwise '%tr r'
    %st simulation time in ns
    %ss simulation time short format (in usec, msec, or sec)
    %sc main system clock cycles till DONE
    %sg use %sc, if available, otherwise %ss
    %sp cpu time per simulation clock cycle (in usec or msec)
    %sm estimate of system clock rate (in MHz)
    %ec error count
    %pf PASS or FAIL, derived from error count
    %nf full file name (with path)
    %ns short file name (without path)

EXIT STATUS
In case of an error, the exit codes are:
    1   run time errors, like a bad option, file not found, etc
    2   at least one test FAILed, test bench has FAILed

ENVIRONMENT

TBFILT_FORMAT
    Defines the default summary format and overwrites the build-in default of "%ec %pf %nf".

TBW_GHDL_OPTS
    Used by tbw(1) to define additional options for ghld-based simulations. Usually set to "--unbuffered --ieee-asserts=disable-at-0", see tbw(1) documentation.

EXAMPLES

  As Filter
Usually used together with tbw(1) in pipelines like
        tbw <test_bench> 2>&1 | tbfilt --tee=<log_file>
Since tbfilt expects also the output of a bash(1) 'time' command in the input stream the setup of the pipeline is more involved. In general the wrapper scripts tbrun_tbw(1) or tbrun_tbwrri(1) are used.

  As Analysis Tool
To generate a compact overview of all test bench outputs use
        cd $RETROBASE
        tbfilt -all -summary -compact
To generate a report indicating all FAILed test use
        cd $RETROBASE
        tbfilt -all -summary -nohead | grep FAIL

SEE ALSO
tbw(1), tbrun_tbw(1), tbrun_tbwrri(1)

AUTHOR
Walter F.J. Mueller <W.F.J.Mueller@gsi.de>