mirror of
https://github.com/wfjm/w11.git
synced 2026-01-12 00:43:01 +00:00
66f3f5d085
- the CI/CD workflow did not defined the TBW_GHDL_OPTS environment variable, as consequence VHDL IEEE package warnings at t=0ms were not suppressed and polluted the log files - that problem was already in .travis.yml and was carried over to ci.yml - now fixed, and log files are not cluttered anymore with essentially unavoidable but harmless warnings at t=0ms.
319 lines
9.7 KiB
Groff
319 lines
9.7 KiB
Groff
.\" -*- nroff -*-
|
|
.\" $Id: tbfilt.1 1234 2022-05-03 18:28:48Z mueller $
|
|
.\" SPDX-License-Identifier: GPL-3.0-or-later
|
|
.\" Copyright 2016-2022 by Walter F.J. Mueller <W.F.J.Mueller@gsi.de>
|
|
.\"
|
|
.\" ------------------------------------------------------------------
|
|
.
|
|
.TH TBFILT 1 2022-05-02 "Retro Project" "Retro Project Manual"
|
|
.\" ------------------------------------------------------------------
|
|
.SH NAME
|
|
tbfilt \- filter for and analysis of test bench log files
|
|
.\" ------------------------------------------------------------------
|
|
.SH SYNOPSIS
|
|
.
|
|
.SY tbfilt
|
|
.B \-\-tee
|
|
.I OFILE
|
|
.RI [ OPTION ]...
|
|
.
|
|
.SY tbfilt
|
|
.RI [ OPTION ]...
|
|
.RI [ FILES ]...
|
|
.YS
|
|
.
|
|
.\" ------------------------------------------------------------------
|
|
.SH DESCRIPTION
|
|
.SS Principle of Operation
|
|
\fBtbfilt\fP is the central tool to analyze the log files created by the test
|
|
benches. It scans the test bench output for messages which indicate a test
|
|
failure and based on this marks a test as \fBPASS\fPed or \fBFAIL\fPed.
|
|
It can be used in two modes:
|
|
|
|
.RS 2
|
|
.IP "-" 2
|
|
as a filter during test bench execution, typically in a setup like
|
|
.EX
|
|
tbw <test_bench> 2>&1 | tbfilt --tee=<log_file>
|
|
.EE
|
|
tbfilt reads the output from the test bench via stdin and a pipe, filters
|
|
out the messages indicating a failure and shows them on stdout, and saves
|
|
the full test bench output to the file given in the \fB\-\-tee\fP option.
|
|
In this mode tbfilt works similar to a
|
|
.EX
|
|
tee ... | egrep ...
|
|
.EE
|
|
pipeline with a very involved egrep selection expression.
|
|
The exit status of tbfilt is 1 in case the test is considered as \fBFAIL\fPed.
|
|
.
|
|
.IP "-" 2
|
|
as log file analysis tool. In this case, the test bench log files are either
|
|
specified explicitly as arguments or determined via the \fB\-\-find\fP or
|
|
\fB\-\-all\fP options.
|
|
If the \fB\-\-summary\fP option is specified a one-line summary for each
|
|
test log file is displayed. The format of this summary is configurable via
|
|
the \fB\-\-format\fP, \fB\-\-wide\fP, and \fB\-\-compact\fP options and via
|
|
the \fBTBFILT_FORMAT\fP environment variable.
|
|
The exit status of tbfilt is 1 in case any of the tests is considered as
|
|
\fBFAIL\fPed.
|
|
.
|
|
.RE
|
|
.PP
|
|
.
|
|
.SS Filter Criteria
|
|
A line that contains any of the following strings is considered as an
|
|
indication of a \fBFAIL\fPed test:
|
|
.RS 2
|
|
.PD 0
|
|
.IP "\fB-E:\fR"
|
|
.IP "\fB-F:\fR"
|
|
.IP "\fBERROR\fR"
|
|
.IP "\fBFAIL\fR"
|
|
.IP "\fB:(assertion warning):\fR"
|
|
.IP "\fB:(assertion error):\fR"
|
|
.IP "\fB:(assertion failure):\fR"
|
|
.PD
|
|
.RE
|
|
|
|
As an exception to the general rules above the following assertion messages
|
|
are accepted:
|
|
Assertion warnings from IEEE libraries at startup (t=0ns) are ignored. They are
|
|
hard to avoid in complex models and generally don't indicate a real issue.
|
|
.RS 2
|
|
.PD 0
|
|
.IP "-" 2
|
|
assertion warnings from IEEE libraries at startup (t=0ns). They are hard to
|
|
avoid in complex models and in general don't indicate a real issue. The best
|
|
is to suppress them in \fBghdl\fP(1) with the
|
|
option '--ieee-asserts=disable-at-0'.
|
|
That's usually done via the \fBTBW_GHDL_OPTS\fP environment variable used
|
|
by \fBtbw\fP(1).
|
|
.IP "-" 2
|
|
assertion failure with the text 'Simulation Finished'. It is used to end
|
|
simulations in \fBghdl\fP(1) in some test benches.
|
|
.PD
|
|
.RE
|
|
|
|
tbfilt also expects a line in one of the formats
|
|
.EX
|
|
xxx ns: DONE -- tb'swithout clock
|
|
xxx.x ns xxx: DONE -- single clock tb's
|
|
xxx.x ns xxx: DONE-xxx -- multiclock tb's (max taken)
|
|
.EE
|
|
and considers a test \fBFAIL\fPed if it is missing.
|
|
|
|
In addition, lines containing
|
|
.RS 4
|
|
.PD 0
|
|
.IP "\fB-W:\fR"
|
|
.IP "\fBPASS\fR"
|
|
.PD
|
|
.RE
|
|
will be displayed. If the \fB\-\-pcom\fP option is specified also all lines
|
|
starting with 'C'.
|
|
|
|
Finally, tbfilt checks for a line of the format
|
|
.EX
|
|
real xmx.xxxs user xmx.xxxs sys xmx.xxxs
|
|
.EE
|
|
and extracts the test bench execution times from this. It can be generated
|
|
by a \fBbash\fP(1) 'time' command when
|
|
.EX
|
|
export TIMEFORMAT=$'real %3lR user %3lU sys %3lS'
|
|
.EE
|
|
is set. The wrapper scripts \fB\tbrun_tbw\fP(1) or \fBtbrun_tbwrri\fP(1)
|
|
are in general used to set this up correctly.
|
|
.
|
|
.\" ------------------------------------------------------------------
|
|
.SH OPTIONS
|
|
.
|
|
.\" --------------------------------------------------------
|
|
.SS Filter Options
|
|
.
|
|
.\" -- --tee -------------------------------------
|
|
.IP "\fB\-\-tee=\fIofile\fR"
|
|
All log file input is written to \fIofile\fP. Typically used to save the
|
|
test bench output in a file when tbfilt is used in filter mode in a pipeline
|
|
and reads from stdin.
|
|
.
|
|
.\" -- --pcom ------------------------------------
|
|
.IP \fB\-\-pcom\fP
|
|
Enables that lines starting with "C" are also displayed.
|
|
.
|
|
.\" --------------------------------------------------------
|
|
.SS File Selection Options
|
|
.
|
|
.\" -- --find ------------------------------------
|
|
.IP "\fB\-\-find=\fIpatt\fR"
|
|
When given the input files are determined with a \fBfind\fP(1) command.
|
|
The selection pattern \fIpatt\fR is used with a find -regex in egrep mode.
|
|
This is functionally similar to a
|
|
.EX
|
|
find -regextype egrep -regex '\fIpatt\fR' | sort | tbfilt ....
|
|
.EE
|
|
pipeline.
|
|
|
|
When no '*' wildcard is found in \fIpatt\fR it is assumed to be a mode
|
|
specification and the pattern is prefixed by
|
|
.EX
|
|
.*/tb_.*_
|
|
.EE
|
|
and suffixed by
|
|
.EX
|
|
.*\\.log
|
|
.EE
|
|
to select all log files of a given mode (e.g. 'bsim').
|
|
.
|
|
.\" -- --all -------------------------------------
|
|
.IP \fB\-\-all\fP
|
|
When given uses as input files all test bench files which conform the
|
|
naming convention. Is equivalent to the option
|
|
.EX
|
|
--find '.*/tb_.*_[bfsorept]sim(_.*)?\\.log'
|
|
.EE
|
|
.
|
|
.\" --------------------------------------------------------
|
|
.SS Summary Options
|
|
.
|
|
.\" -- --summary----------------------------------
|
|
.IP \fB\-\-summary\fP
|
|
Selects summary mode. Only a single summary line per input file is written.
|
|
The format is configurable via the \fB\-\-format\fP, \fB\-\-wide\fP, and
|
|
\fB\-\-compact\fP options and via the \fBTBFILT_FORMAT\fP environment variable.
|
|
The precedence is (in increasing priority):
|
|
.RS
|
|
.PD 0
|
|
.IP " -" 4
|
|
build default ('%ec %pf %nf')
|
|
.IP " -"
|
|
\fBTBFILT_FORMAT\fP option
|
|
.IP " -"
|
|
\fB\-\-wide\fP option
|
|
.IP " -"
|
|
\fB\-\-compact\fP option
|
|
.IP " -"
|
|
\fB\-\-format\fP option
|
|
.PD
|
|
.RE
|
|
.
|
|
.\" -- --wide ------------------------------------
|
|
.IP \fB\-\-wide\fP
|
|
Selects a wide format for summary outputs, designed to give the most pertinent
|
|
information. Uses a format of "%fd %fs %tr %tc %sc %ec %pf %nf".
|
|
.
|
|
.\" -- --compact ---------------------------------
|
|
.IP \fB\-\-compact\fP
|
|
Selects a compact format for summary outputs, designed to give the key info
|
|
on an 80 character wide line. Uses a format of "%fa %tg %sg %ec %pf %ns".
|
|
.
|
|
.\" -- --nohead ----------------------------------
|
|
.IP \fB\-\-nohead\fP
|
|
Suppresses the headline of summary outputs. Useful of summary output is
|
|
piped into sort or other tools.
|
|
.
|
|
.\" -- --format ----------------------------------
|
|
.IP "\fB\-\-format=\fIfmt\fR"
|
|
Defined the format of the summary lines.
|
|
The format specification \fIfmt\fR string is a sequence of conversion
|
|
specifications of the form '%xx', which will be replaces by the respective
|
|
values and other characters which are simply copied (usually a blank as
|
|
delimiter).
|
|
The supported conversion specifications are:
|
|
.RS
|
|
.PD 0
|
|
.IP \fB%fd\fP 5
|
|
modification date of input file (as yyyy-mm-dd)
|
|
.IP \fB%ft\fP
|
|
modification time of input file (as hh:mm:ss)
|
|
.IP \fB%fs\fP
|
|
modification time of input file short format (as hh:mm)
|
|
.IP \fB%fa\fP
|
|
age of input file in seconds, minutes, hours or days
|
|
.IP \fB%tr\fP
|
|
real (wall clock) time of test bench run
|
|
.IP \fB%tu\fP
|
|
user time of test bench run
|
|
.IP \fB%ts\fP
|
|
system time of test bench run
|
|
.IP \fB%tc\fP
|
|
total cpu (user+system) time of test bench run
|
|
.IP \fB%tg\fP
|
|
show '%tc c' if cpu time is significant, otherwise '%tr r'
|
|
.IP \fB%st\fP
|
|
simulation time in ns
|
|
.IP \fB%ss\fP
|
|
simulation time short format (in usec, msec, or sec)
|
|
.IP \fB%sc\fP
|
|
main system clock cycles till DONE
|
|
.IP \fB%sg\fP
|
|
use %sc, if available, otherwise %ss
|
|
.IP \fB%sp\fP
|
|
cpu time per simulation clock cycle (in usec or msec)
|
|
.IP \fB%sm\fP
|
|
estimate of system clock rate (in MHz)
|
|
.IP \fB%ec\fP
|
|
error count
|
|
.IP \fB%pf\fP
|
|
PASS or FAIL, derived from error count
|
|
.IP \fB%nf\fP
|
|
full file name (with path)
|
|
.IP \fB%ns\fP
|
|
short file name (without path)
|
|
.PD
|
|
.RE
|
|
.
|
|
.\" ------------------------------------------------------------------
|
|
.SH EXIT STATUS
|
|
In case of an error the exit codes are:
|
|
.RS
|
|
.PD 0
|
|
.IP \fB1\fP 3
|
|
run time errors, like bad option, file not found, etc
|
|
.IP \fB2\fP 3
|
|
at least one test FAILed, test bench has FAILed
|
|
.PD
|
|
.RE
|
|
.
|
|
.\" ------------------------------------------------------------------
|
|
.SH ENVIRONMENT
|
|
.IP "\fBTBFILT_FORMAT\fR" 4
|
|
Defines the default summary format and overwrites the build-in default of
|
|
"%ec %pf %nf".
|
|
.IP "\fBTBW_GHDL_OPTS\fR"
|
|
Used by \fBtbw\fP(1) to define additional options for GHDL based simulations.
|
|
Usually set to "--unbuffered --ieee-asserts=disable-at-0", see \fBtbw\fP(1)
|
|
documentation.
|
|
.
|
|
.\" ------------------------------------------------------------------
|
|
.SH EXAMPLES
|
|
.SS As Filter
|
|
Usually used together with \fBtbw\fP(1) in pipelines like
|
|
.EX
|
|
tbw <test_bench> 2>&1 | tbfilt --tee=<log_file>
|
|
.EE
|
|
Since tbfilt expects also the output of a \fBbash\fP(1) 'time' command
|
|
in the input stream the setup of the pipeline is more involved.
|
|
In general the wrapper scripts \fBtbrun_tbw\fP(1) or \fBtbrun_tbwrri\fP(1)
|
|
are used.
|
|
.
|
|
.SS As Analysis Tool
|
|
To generate a compact overview of all test bench outputs use
|
|
.EX
|
|
cd $RETROBASE
|
|
tbfilt -all -summary -compact
|
|
.EE
|
|
To generate a report indicating all \fBFAIL\fPed test use
|
|
.EX
|
|
cd $RETROBASE
|
|
tbfilt -all -summary -nohead | grep FAIL
|
|
.EE
|
|
.
|
|
.\" ------------------------------------------------------------------
|
|
.SH "SEE ALSO"
|
|
.BR tbw (1),
|
|
.BR tbrun_tbw (1),
|
|
.BR tbrun_tbwrri (1)
|
|
.\" ------------------------------------------------------------------
|
|
.SH AUTHOR
|
|
Walter F.J. Mueller <W.F.J.Mueller@gsi.de>
|