This patch adds support in gcc+gcov for modified condition/decision
coverage (MC/DC) with the -fcondition-coverage flag. MC/DC is a type of
test/code coverage and it is particularly important for safety-critical
applicaitons in industries like aviation and automotive. Notably, MC/DC
is required or recommended by:
* DO-178C for the most critical software (Level A) in avionics.
* IEC 61508 for SIL 4.
* ISO 26262-6 for ASIL D.
From the SQLite webpage:
Two methods of measuring test coverage were described above:
"statement" and "branch" coverage. There are many other test
coverage metrics besides these two. Another popular metric is
"Modified Condition/Decision Coverage" or MC/DC. Wikipedia defines
MC/DC as follows:
* Each decision tries every possible outcome.
* Each condition in a decision takes on every possible outcome.
* Each entry and exit point is invoked.
* Each condition in a decision is shown to independently affect
the outcome of the decision.
In the C programming language where && and || are "short-circuit"
operators, MC/DC and branch coverage are very nearly the same thing.
The primary difference is in boolean vector tests. One can test for
any of several bits in bit-vector and still obtain 100% branch test
coverage even though the second element of MC/DC - the requirement
that each condition in a decision take on every possible outcome -
might not be satisfied.
https://sqlite.org/testing.html#mcdc
MC/DC comes in different flavors, the most important being unique cause
MC/DC and masking MC/DC. This patch implements masking MC/DC, which is
works well with short circuiting semantics, and according to John
Chilenski's "An Investigation of Three Forms of the Modified Condition
Decision Coverage (MCDC) Criterion" (2001) is as good as unique cause at
catching bugs.
Whalen, Heimdahl, and De Silva "Efficient Test Coverage Measurement for
MC/DC" describes an algorithm for finding the masking table from an AST
walk, but my algorithm figures this out by analyzing the control flow
graph. The CFG is considered a reduced ordered binary decision diagram
and an input vector a path through the BDD, which is recorded. Specific
edges will mask ("null out") the contribution from earlier path
segments, which can be determined by finding short circuit endpoints.
Masking is most easily understood as circuiting of terms in the
reverse-ordered Boolean function, and the masked conditions do not
affect the decision like short-circuited conditions do not affect the
decision.
A tag/discriminator mapping from gcond->uid is created during
gimplification and made available through the function struct. The
values are unimportant as long as basic conditions constructed from a
single Boolean expression are given the same identifier. This happens in
the breaking down of ANDIF/ORIF trees, so the coverage generally works
well for frontends that create such trees.
Like Whalen et al this implementation records coverage in fixed-size
bitsets which gcov knows how to interpret. Recording conditions only
requires a few bitwise operations per condition and is very fast, but
comes with a limit on the number of terms in a single boolean
expression; the number of bits in a gcov_unsigned_type (which is usually
typedef'd to uint64_t). For most practical purposes this is acceptable,
and by default a warning will be issued if gcc cannot instrument the
expression. This is a practical limitation in the implementation, and
not a limitation of the algorithm, so support for more conditions can be
supported by introducing arbitrary-sized bitsets.
In action it looks pretty similar to the branch coverage. The -g short
opt carries no significance, but was chosen because it was an available
option with the upper-case free too.
gcov --conditions:
3: 17:void fn (int a, int b, int c, int d) {
3: 18: if ((a && (b || c)) && d)
conditions covered 3/8
condition 0 not covered (true false)
condition 1 not covered (true)
condition 2 not covered (true)
condition 3 not covered (true)
1: 19: x = 1;
-: 20: else
2: 21: x = 2;
3: 22:}
gcov --conditions --json-format:
"conditions": [
{
"not_covered_false": [
0
],
"count": 8,
"covered": 3,
"not_covered_true": [
0,
1,
2,
3
]
}
],
Expressions with constants may be heavily rewritten before it reaches
the gimplification, so constructs like int x = a ? 0 : 1 becomes
_x = (_a == 0). From source you would expect coverage, but it gets
neither branch nor condition coverage. The same applies to expressions
like int x = 1 || a which are simply replaced by a constant.
The test suite contains a lot of small programs and functions. Some of
these were designed by hand to test for specific behaviours and graph
shapes, and some are previously-failed test cases in other programs
adapted into the test suite.
gcc/ChangeLog:
* builtins.cc (expand_builtin_fork_or_exec): Check
condition_coverage_flag.
* collect2.cc (main): Add -fno-condition-coverage to OBSTACK.
* common.opt: Add new options -fcondition-coverage and
-Wcoverage-too-many-conditions.
* doc/gcov.texi: Add --conditions documentation.
* doc/invoke.texi: Add -fcondition-coverage documentation.
* function.cc (free_after_compilation): Free cond_uids.
* function.h (struct function): Add cond_uids.
* gcc.cc: Link gcov on -fcondition-coverage.
* gcov-counter.def (GCOV_COUNTER_CONDS): New.
* gcov-dump.cc (tag_conditions): New.
* gcov-io.h (GCOV_TAG_CONDS): New.
(GCOV_TAG_CONDS_LENGTH): New.
(GCOV_TAG_CONDS_NUM): New.
* gcov.cc (class condition_info): New.
(condition_info::condition_info): New.
(condition_info::popcount): New.
(struct coverage_info): New.
(add_condition_counts): New.
(output_conditions): New.
(print_usage): Add -g, --conditions.
(process_args): Likewise.
(output_intermediate_json_line): Output conditions.
(read_graph_file): Read condition counters.
(read_count_file): Likewise.
(file_summary): Print conditions.
(accumulate_line_info): Accumulate conditions.
(output_line_details): Print conditions.
* gimplify.cc (next_cond_uid): New.
(reset_cond_uid): New.
(shortcut_cond_r): Set condition discriminator.
(tag_shortcut_cond): New.
(gimple_associate_condition_with_expr): New.
(shortcut_cond_expr): Set condition discriminator.
(gimplify_cond_expr): Likewise.
(gimplify_function_tree): Call reset_cond_uid.
* ipa-inline.cc (can_early_inline_edge_p): Check
condition_coverage_flag.
* ipa-split.cc (pass_split_functions::gate): Likewise.
* passes.cc (finish_optimization_passes): Likewise.
* profile.cc (struct condcov): New declaration.
(cov_length): Likewise.
(cov_blocks): Likewise.
(cov_masks): Likewise.
(cov_maps): Likewise.
(cov_free): Likewise.
(instrument_decisions): New.
(read_thunk_profile): Control output to file.
(branch_prob): Call find_conditions, instrument_decisions.
(init_branch_prob): Add total_num_conds.
(end_branch_prob): Likewise.
* tree-core.h (struct tree_exp): Add condition_uid.
* tree-profile.cc (struct conds_ctx): New.
(CONDITIONS_MAX_TERMS): New.
(EDGE_CONDITION): New.
(topological_cmp): New.
(index_of): New.
(single_p): New.
(single_edge): New.
(contract_edge_up): New.
(struct outcomes): New.
(conditional_succs): New.
(condition_index): New.
(condition_uid): New.
(masking_vectors): New.
(emit_assign): New.
(emit_bitwise_op): New.
(make_top_index_visit): New.
(make_top_index): New.
(paths_between): New.
(struct condcov): New.
(cov_length): New.
(cov_blocks): New.
(cov_masks): New.
(cov_maps): New.
(cov_free): New.
(find_conditions): New.
(struct counters): New.
(find_counters): New.
(resolve_counter): New.
(resolve_counters): New.
(instrument_decisions): New.
(tree_profiling): Check condition_coverage_flag.
(pass_ipa_tree_profile::gate): Likewise.
* tree.h (SET_EXPR_UID): New.
(EXPR_COND_UID): New.
libgcc/ChangeLog:
* libgcov-merge.c (__gcov_merge_ior): New.
gcc/testsuite/ChangeLog:
* lib/gcov.exp: Add condition coverage test function.
* g++.dg/gcov/gcov-18.C: New test.
* gcc.misc-tests/gcov-19.c: New test.
* gcc.misc-tests/gcov-20.c: New test.
* gcc.misc-tests/gcov-21.c: New test.
* gcc.misc-tests/gcov-22.c: New test.
* gcc.misc-tests/gcov-23.c: New test.
gcc/ChangeLog:
* coverage.c (get_coverage_counts): Skip sanity check for TOP N counters
as they have variable number of counters.
* gcov-dump.c (main): Add new option -r.
(print_usage): Likewise.
(tag_counters): All new raw format.
* gcov-io.h (struct gcov_kvp): New.
(GCOV_TOPN_VALUES): Remove.
(GCOV_TOPN_VALUES_COUNTERS): Likewise.
(GCOV_TOPN_MEM_COUNTERS): New.
(GCOV_TOPN_DISK_COUNTERS): Likewise.
(GCOV_TOPN_MAXIMUM_TRACKED_VALUES): Likewise.
* ipa-profile.c (ipa_profile_generate_summary): Use
GCOV_TOPN_MAXIMUM_TRACKED_VALUES.
(ipa_profile_write_edge_summary): Likewise.
(ipa_profile_read_edge_summary): Likewise.
(ipa_profile): Remove usage of GCOV_TOPN_VALUES.
* profile.c (sort_hist_values): Sort variable number
of counters.
(compute_value_histograms): Special case for TOP N counters
that have dynamic number of key-value pairs.
* value-prof.c (dump_histogram_value): Dump variable number
of key-value pairs.
(stream_in_histogram_value): Stream in variable number
of key-value pairs for TOP N counter.
(get_nth_most_common_value): Deal with variable number
of key-value pairs.
(dump_ic_profile): Use GCOV_TOPN_MAXIMUM_TRACKED_VALUES
for loop iteration.
(gimple_find_values_to_profile): Set GCOV_TOPN_MEM_COUNTERS
to n_counters.
* doc/gcov-dump.texi: Document new -r option.
libgcc/ChangeLog:
* libgcov-driver.c (prune_topn_counter): Remove.
(prune_counters): Likewise.
(merge_one_data): Special case TOP N counters
as they have variable length.
(write_top_counters): New.
(write_one_data): Special case TOP N.
(dump_one_gcov): Do not prune TOP N counters.
* libgcov-merge.c (merge_topn_values_set): Remove.
(__gcov_merge_topn): Use gcov_topn_add_value.
* libgcov-profiler.c (__gcov_topn_values_profiler_body):
Likewise here.
* libgcov.h (gcov_counter_add): New.
(gcov_counter_set_if_null): Likewise.
(gcov_topn_add_value): New.
PR ipa/92924
* common.opt: Add -fprofile-reproducibility.
* doc/invoke.texi: Document it.
* value-prof.c (dump_histogram_value):
Document and support behavior for counters[0]
being a negative value.
(get_nth_most_common_value): Handle negative
counters[0] in respect to flag_profile_reproducible.
PR ipa/92924
* libgcov-merge.c (merge_topn_values_set): Record
when a TOP N counter becomes invalid. When merging
remove a smallest value if the space is needed.
2019-07-03 Martin Liska <mliska@suse.cz>
* gcc.dg/tree-prof/val-prof-2.c: Update scanned pattern
as we do now better.
2019-07-03 Martin Liska <mliska@suse.cz>
* libgcov-merge.c (merge_single_value_set): Support N values.
* libgcov-profiler.c (__gcov_one_value_profiler_body): Likewise.
From-SVN: r273004
2014-07-10 Rong Xu <xur@google.com>
Add gcov-tool: an offline gcda profile processing tool
Support.
* gcc/gcov-io.c (gcov_position): Make avaialble to gcov-tool.
(gcov_is_error): Ditto.
(gcov_read_string): Ditto.
(gcov_read_sync): Ditto.
* gcc/gcov-io.h: Move counter defines to gcov-counter.def.
* gcc/gcov-dump.c (tag_counters): Use gcov-counter.def.
* gcc/coverage.c: Ditto.
* gcc/gcov-tool.c: Offline gcda profile processing tool.
(unlink_gcda_file): Remove one gcda file.
(unlink_profile_dir): Remove gcda files from the profile path.
(gcov_output_files): Output gcda files to an output dir.
(profile_merge): Merge two profiles in directory.
(print_merge_usage_message): Print merge usage.
(merge_usage): Print merge usage and exit.
(do_merge): Driver for profile merge sub-command.
(profile_rewrite): Rewrite profile.
(print_rewrite_usage_message): Print rewrite usage.
(rewrite_usage): Print rewrite usage and exit.
(do_rewrite): Driver for profile rewrite sub-command.
(print_usage): Print gcov-info usage and exit.
(print_version): Print gcov-info version.
(process_args): Process arguments.
(main): Main routine for gcov-tool.
* gcc/Makefile.in: Build and install gcov-tool.
* gcc/gcov-counter.def: New file split from gcov-io.h.
* libgcc/libgcov-driver.c (gcov_max_filename): Make available
to gcov-tool.
* libgcc/libgcov-merge.c (__gcov_merge_add): Replace
gcov_read_counter() with a Macro.
(__gcov_merge_ior): Ditto.
(__gcov_merge_time_profile): Ditto.
(__gcov_merge_single): Ditto.
(__gcov_merge_delta): Ditto.
* libgcc/libgcov-util.c (void gcov_set_verbose): Set the verbose flag
in the utility functions.
(set_fn_ctrs): Utility function for reading gcda files to in-memory
gcov_list object link lists.
(tag_function): Ditto.
(tag_blocks): Ditto.
(tag_arcs): Ditto.
(tag_lines): Ditto.
(tag_counters): Ditto.
(tag_summary): Ditto.
(read_gcda_finalize): Ditto.
(read_gcda_file): Ditto.
(ftw_read_file): Ditto.
(read_profile_dir_init): Ditto.
(gcov_read_profile_dir): Ditto.
(gcov_read_counter_mem): Ditto.
(gcov_get_merge_weight): Ditto.
(merge_wrapper): A wrapper function that calls merging handler.
(gcov_merge): Merge two gcov_info objects with weights.
(find_match_gcov_info): Find the matched gcov_info in the list.
(gcov_profile_merge): Merge two gcov_info object lists.
(__gcov_add_counter_op): Process edge profile counter values.
(__gcov_ior_counter_op): Process IOR profile counter values.
(__gcov_delta_counter_op): Process delta profile counter values.
(__gcov_single_counter_op): Process single profile counter values.
(fp_scale): Callback function for float-point scaling.
(int_scale): Callback function for integer fraction scaling.
(gcov_profile_scale): Scaling profile counters.
(gcov_profile_normalize): Normalize profile counters.
* libgcc/libgcov.h: Add headers and functions for gcov-tool use.
(gcov_get_counter): New.
(gcov_get_counter_target): Ditto.
(struct gcov_info): Make the functions field mutable in gcov-tool
compilation.
* gcc/doc/gcc.texi: Include gcov-tool.texi.
* gcc/doc/gcov-tool.texi: Document for gcov-tool.
From-SVN: r212448
2014-01-08 Rong Xu <xur@google.com>
* gcc/gcov-io.c (gcov_var): Move from gcov-io.h.
(gcov_position): Ditto.
(gcov_is_error): Ditto.
(gcov_rewrite): Ditto.
* gcc/gcov-io.h: Refactor. Move gcov_var to gcov-io.h, and libgcov
only part to libgcc/libgcov.h.
* libgcc/libgcov-driver.c: Use libgcov.h.
(buffer_fn_data): Use xmalloc instead of malloc.
(gcov_exit_merge_gcda): Ditto.
* libgcc/libgcov-driver-system.c (allocate_filename_struct): Ditto.
* libgcc/libgcov.h: New common header files for libgcov-*.h.
* libgcc/libgcov-interface.c: Use libgcov.h
* libgcc/libgcov-merge.c: Ditto.
* libgcc/libgcov-profiler.c: Ditto.
* libgcc/Makefile.in: Add dependence to libgcov.h
From-SVN: r206435
