Existing text output in GCC has to be implemented by writing
sequentially to a pretty_printer instance. This makes it
hard to implement some kinds of diagnostic output (see e.g.
diagnostic-show-locus.cc).
This patch adds more flexible ways of creating text output:
- a canvas class, which can be "painted" to via random-access (rather
that sequentially)
- a table class for 2D grid layout, supporting items that span
multiple rows/columns
- a widget class for organizing diagrams hierarchically.
The patch also expands GCC's diagnostics subsystem so that diagnostics
can have "text art" diagrams - think ASCII art, but potentially
including some Unicode characters, such as box-drawing chars.
The new code is in a new "gcc/text-art" subdirectory and "text_art"
namespace.
The patch adds a new "-fdiagnostics-text-art-charset=VAL" option, with
values:
- "none": don't emit diagrams (added to -fdiagnostics-plain-output)
- "ascii": use pure ASCII in diagrams
- "unicode": allow for conservative use of unicode drawing characters
(such as box-drawing characters).
- "emoji" (the default): as "unicode", but potentially allow for
conservative use of emoji in the output (such as U+26A0 WARNING SIGN).
I made it possible to disable emoji separately from unicode as I believe
there's a generation gap in acceptance of these characters (some older
programmers have a visceral reaction against them, whereas younger
programmers may have no problem with them).
Diagrams are emitted to stderr by default. With SARIF output they are
captured as a location in "relatedLocations", with the diagram as a
code block in Markdown within a "markdown" property of a message.
This patch doesn't add any such diagram usage to GCC, saving that for
followups, apart from adding a plugin to the test suite to exercise the
functionality.
contrib/ChangeLog:
* unicode/gen-box-drawing-chars.py: New file.
* unicode/gen-combining-chars.py: New file.
* unicode/gen-printable-chars.py: New file.
gcc/ChangeLog:
* Makefile.in (OBJS-libcommon): Add text-art/box-drawing.o,
text-art/canvas.o, text-art/ruler.o, text-art/selftests.o,
text-art/style.o, text-art/styled-string.o, text-art/table.o,
text-art/theme.o, and text-art/widget.o.
* color-macros.h (COLOR_FG_BRIGHT_BLACK): New.
(COLOR_FG_BRIGHT_RED): New.
(COLOR_FG_BRIGHT_GREEN): New.
(COLOR_FG_BRIGHT_YELLOW): New.
(COLOR_FG_BRIGHT_BLUE): New.
(COLOR_FG_BRIGHT_MAGENTA): New.
(COLOR_FG_BRIGHT_CYAN): New.
(COLOR_FG_BRIGHT_WHITE): New.
(COLOR_BG_BRIGHT_BLACK): New.
(COLOR_BG_BRIGHT_RED): New.
(COLOR_BG_BRIGHT_GREEN): New.
(COLOR_BG_BRIGHT_YELLOW): New.
(COLOR_BG_BRIGHT_BLUE): New.
(COLOR_BG_BRIGHT_MAGENTA): New.
(COLOR_BG_BRIGHT_CYAN): New.
(COLOR_BG_BRIGHT_WHITE): New.
* common.opt (fdiagnostics-text-art-charset=): New option.
(diagnostic-text-art.h): New SourceInclude.
(diagnostic_text_art_charset) New Enum and EnumValues.
* configure: Regenerate.
* configure.ac (gccdepdir): Add text-art to loop.
* diagnostic-diagram.h: New file.
* diagnostic-format-json.cc (json_emit_diagram): New.
(diagnostic_output_format_init_json): Wire it up to
context->m_diagrams.m_emission_cb.
* diagnostic-format-sarif.cc: Include "diagnostic-diagram.h" and
"text-art/canvas.h".
(sarif_result::on_nested_diagnostic): Move code to...
(sarif_result::add_related_location): ...this new function.
(sarif_result::on_diagram): New.
(sarif_builder::emit_diagram): New.
(sarif_builder::make_message_object_for_diagram): New.
(sarif_emit_diagram): New.
(diagnostic_output_format_init_sarif): Set
context->m_diagrams.m_emission_cb to sarif_emit_diagram.
* diagnostic-text-art.h: New file.
* diagnostic.cc: Include "diagnostic-text-art.h",
"diagnostic-diagram.h", and "text-art/theme.h".
(diagnostic_initialize): Initialize context->m_diagrams and
call diagnostics_text_art_charset_init.
(diagnostic_finish): Clean up context->m_diagrams.m_theme.
(diagnostic_emit_diagram): New.
(diagnostics_text_art_charset_init): New.
* diagnostic.h (text_art::theme): New forward decl.
(class diagnostic_diagram): Likewise.
(diagnostic_context::m_diagrams): New field.
(diagnostic_emit_diagram): New decl.
* doc/invoke.texi (Diagnostic Message Formatting Options): Add
-fdiagnostics-text-art-charset=.
(-fdiagnostics-plain-output): Add
-fdiagnostics-text-art-charset=none.
* gcc.cc: Include "diagnostic-text-art.h".
(driver_handle_option): Handle OPT_fdiagnostics_text_art_charset_.
* opts-common.cc (decode_cmdline_options_to_array): Add
"-fdiagnostics-text-art-charset=none" to expanded_args for
-fdiagnostics-plain-output.
* opts.cc: Include "diagnostic-text-art.h".
(common_handle_option): Handle OPT_fdiagnostics_text_art_charset_.
* pretty-print.cc (pp_unicode_character): New.
* pretty-print.h (pp_unicode_character): New decl.
* selftest-run-tests.cc: Include "text-art/selftests.h".
(selftest::run_tests): Call text_art_tests.
* text-art/box-drawing-chars.inc: New file, generated by
contrib/unicode/gen-box-drawing-chars.py.
* text-art/box-drawing.cc: New file.
* text-art/box-drawing.h: New file.
* text-art/canvas.cc: New file.
* text-art/canvas.h: New file.
* text-art/ruler.cc: New file.
* text-art/ruler.h: New file.
* text-art/selftests.cc: New file.
* text-art/selftests.h: New file.
* text-art/style.cc: New file.
* text-art/styled-string.cc: New file.
* text-art/table.cc: New file.
* text-art/table.h: New file.
* text-art/theme.cc: New file.
* text-art/theme.h: New file.
* text-art/types.h: New file.
* text-art/widget.cc: New file.
* text-art/widget.h: New file.
gcc/testsuite/ChangeLog:
* gcc.dg/plugin/diagnostic-test-text-art-ascii-bw.c: New test.
* gcc.dg/plugin/diagnostic-test-text-art-ascii-color.c: New test.
* gcc.dg/plugin/diagnostic-test-text-art-none.c: New test.
* gcc.dg/plugin/diagnostic-test-text-art-unicode-bw.c: New test.
* gcc.dg/plugin/diagnostic-test-text-art-unicode-color.c: New test.
* gcc.dg/plugin/diagnostic_plugin_test_text_art.c: New test plugin.
* gcc.dg/plugin/plugin.exp (plugin_test_list): Add them.
libcpp/ChangeLog:
* charset.cc (get_cppchar_property): New function template, based
on...
(cpp_wcwidth): ...this function. Rework to use the above.
Include "combining-chars.inc".
(cpp_is_combining_char): New function
Include "printable-chars.inc".
(cpp_is_printable_char): New function
* combining-chars.inc: New file, generated by
contrib/unicode/gen-combining-chars.py.
* include/cpplib.h (cpp_is_combining_char): New function decl.
(cpp_is_printable_char): New function decl.
* printable-chars.inc: New file, generated by
contrib/unicode/gen-printable-chars.py.
Signed-off-by: David Malcolm <dmalcolm@redhat.com>
When parsing a summary or manifest file, if we're not either after a tool
line (e.g. "=== gdb tests ===") or before a summary line (e.g.,
"=== gdb Summary ===") then the current line can't be a valid result line
so ignore it.
This addresses a problem we're seeing when running the GDB testsuite in
our CI environment where it produces a valid summary file, but then after
the "=== gdb Summary ===" section it outputs a series of Tcl errors that
match _VALID_TEST_RESULTS_REX and thus confuse the parsing logic:
05: 14:32 .sum file seems to be broken: tool="None", exp="None", summary_line="ERROR: -------------------------------------------"
05: 14:32 Traceback (most recent call last):
05: 14:32 File "/path/to/gcc/contrib/testsuite-management/validate_failures.py", line 706, in <module>
05: 14:32 retval = Main(sys.argv)
05: 14:32 File "/path/to/gcc/contrib/testsuite-management/validate_failures.py", line 697, in Main
05: 14:32 retval = CheckExpectedResults()
05: 14:32 File "/path/to/gcc/contrib/testsuite-management/validate_failures.py", line 572, in CheckExpectedResults
05: 14:32 actual = GetResults(sum_files)
05: 14:32 File "/path/to/gcc/contrib/testsuite-management/validate_failures.py", line 447, in GetResults
05: 14:32 build_results.update(ParseSummary(sum_fname))
05: 14:32 File "/path/to/gcc/contrib/testsuite-management/validate_failures.py", line 389, in ParseSummary
05: 14:32 result = result_set.MakeTestResult(line, ordinal)
05: 14:32 File "/path/to/gcc/contrib/testsuite-management/validate_failures.py", line 236, in MakeTestResult
05: 14:32 return TestResult(summary_line, ordinal,
05: 14:32 File "/path/to/gcc/contrib/testsuite-management/validate_failures.py", line 148, in __init__
05: 14:32 raise
contrib/ChangeLog:
* testsuite-management/validate_failures.py (IsInterestingResult):
Add result_set argument and use it. Adjust callers.
This patch simplifies comparison of results that have filesystem
paths. E.g., (assuming different values of <N>):
<cut>
Running /home/user/gcc-N/gcc/testsuite/gcc.target/aarch64/sve/acle/aarch64-sve-acle-asm.exp ...
ERROR: tcl error sourcing /home/user/gcc-N/gcc/testsuite/gcc.target/aarch64/sve/acle/aarch64-sve-acle-asm.exp.
</cut>
We add "--srcpath <regex>", option, and set it by default to
"[^ ]+/testsuite/", which works well for all components of the GNU
Toolchain. We then remove substrings matching <regex> from paths of
.exp files and from occasional "ERROR:" results.
contrib/ChangeLog:
* testsuite-management/validate_failures.py (TestResult,)
(ParseManifestWorker, ParseSummary, Main): Handle new option
"--srcpath <regex>".
This option sets "today" date to compare expiration entries against.
Setting expiration date into the future allows re-detection of flaky
tests and creating fresh entries for them before the current flaky
entries expire.
contrib/ChangeLog:
* testsuite-management/validate_failures.py (TestResult): Update.
(Main): Handle new option "--expiry_date YYYYMMDD".
This option is used to detect flaky tests that FAILed in the clean
build (or manifest), but PASSed in the current build (or manifest).
The option inverts output logic similar to what "-v/--invert-match"
does for grep.
contrib/ChangeLog:
* testsuite-management/validate_failures.py (ResultSet.update,)
(ResultSet.HasTestsuite): New methods.
(GetResults): Update.
(ParseSummary, CompareResults, PerformComparison, Main): Handle new
option --invert_match.
- Print message in case of broken sum file error.
- Print error messages to stderr. The script's stdout is, usually,
redirected to a file, and error messages shouldn't go there.
contrib/ChangeLog:
* testsuite-management/validate_failures.py (TestResult): Improve error
output.
This makes it easier to extract the $tool:$exp pair when iterating
over failures/flaky tests, which, in turn, simplifies re-running
testsuite parts that have unexpected failures or passes.
contrib/ChangeLog:
* testsuite-management/validate_failures.py (_EXP_LINE_FORMAT,)
(_EXP_LINE_REX, ResultSet): Support "$tool:" prefix in exp names.
... in the results. Python exits with code "1" on exceptions and
internal errors, which we use to detect failure to parse results.
contrib/ChangeLog:
* testsuite-management/validate_failures.py (Main): Use exit code "2"
to indicate regression.
Before this patch we would identify malformed line
"UNRESOLVEDTest run by tcwg-buildslave on Mon Aug 23 10:17:50 2021"
as an interesting result, only to fail in TestResult:__init__ due
to missing ":" after UNRESOLVED.
This patch makes all places that parse result lines use a single
compiled regex.
contrib/ChangeLog:
* testsuite-management/validate_failures.py (_VALID_TEST_RESULTS_REX):
Update.
(TestResult): Use _VALID_TEST_RESULTS_REX.
... and don't require a valid build directory when no data from it
is necessary.
contrib/ChangeLog:
* testsuite-management/validate_failures.py: Simplify GetManifestPath().
This allows comparison of two build directories with a manifest
listing known flaky tests on the side.
contrib/ChangeLog:
* testsuite-management/validate_failures.py (GetResults): Update.
(CompareBuilds): Read in manifest.
contrib/ChangeLog:
* testsuite-management/validate_failures.py (ParseManifestWorker):
Support expiry attributes in manifests.
(ParseSummary): Add a comment.
This patch adds tracking of current testsuite "tool" and "exp"
to the processing of .sum files. This avoids aliasing between
tests from different testsuites with same name+description.
E.g., this is necessary for testsuite/c-c++-common, which is ran
for both gcc and g++ "tools".
This patch changes manifest format from ...
<cut>
FAIL: gcc_test
FAIL: g++_test
</cut>
... to ...
<cut>
=== gcc tests ===
Running gcc/foo.exp ...
FAIL: gcc_test
=== gcc Summary ==
=== g++ tests ===
Running g++/bar.exp ...
FAIL: g++_test
=== g++ Summary ==
</cut>.
The new format uses same formatting as DejaGnu's .sum files
to specify which "tool" and "exp" the test belongs to.
contrib/ChangeLog:
* testsuite-management/validate_failures.py: Avoid testsuite
aliasing.
- Fix gen_autofdo_event: The download URL for the Intel Perfmon Event
list has changed, as well as the JSON format.
Also it now uses pattern matching to match CPUs. Update the script to support all of this.
- Regenerate gcc-auto-profile with the latest published Intel model
numbers, so it works with recent systems.
- So far it's still broken on hybrid systems
contrib/ChangeLog:
* gen_autofdo_event.py: Update for download server changes
gcc/ChangeLog
* config/i386/gcc-auto-profile: Regenerate.
POSIX sh does not support the == for string comparisons, use = instead.
These contrib scripts all use a bash shebang so == does work, but
there's no reason they can't just use the more portable form anyway.
PR bootstrap/105831
contrib/ChangeLog:
* bench-stringop: Use = operator instead of ==.
* repro_fail: Likewise.
contrib/reghunt/ChangeLog:
* bin/reg-hunt: Use = operator instead of ==.
2023-04-17 Jakub Jelinek <jakub@redhat.com>
maintainer-scripts/
* crontab: Snapshots from trunk are now GCC 14 related.
Add GCC 13 snapshots from the respective branch.
contrib/
* gcc-changelog/git_update_version.py (active_refs): Add
releases/gcc-13.
I've noticed we have instructions on how to update from newer Unicode
standard, but it didn't mention uname2c.h regeneration.
The following patch mentions that, also mentions that the Copyright years
of Unicode should be updated and adds a copy of NameAliases.txt which
is used for uname2c.h generation.
2023-03-16 Jakub Jelinek <jakub@redhat.com>
* unicode/README: Update to mention also makeuname2c.
* unicode/NameAliases.txt: New file.
Already in the first revision of 'contrib/config-list.mk', i686-symbolics-gnu
has been present, but it's not clear to me whether that was meant to be
Symbolics as in the manufacturer, <https://en.wikipedia.org/wiki/Symbolics>,
with GNU (that is, GNU/Hurd) kernel/operating system (user land), or Symbolics
kernel with GNU operating system (user land)?
I can't find any mention of "Symbolics" in the history of 'config.sub'
upstream.
Either way, GCC configures i686-symbolics-gnu exactly the same as i686-gnu:
$ sed -n -e '/Using .* host machine hooks\.$/q' -e '/^Using the following target machine macro files:$/,$p' log/i686-gnu-make.out
Using the following target machine macro files:
[...]/gcc/config/vxworks-dummy.h
[...]/gcc/config/i386/i386.h
[...]/gcc/config/i386/unix.h
[...]/gcc/config/i386/att.h
[...]/gcc/config/elfos.h
[...]/gcc/config/gnu-user.h
[...]/gcc/config/glibc-stdint.h
[...]/gcc/config/i386/gnu-user-common.h
[...]/gcc/config/i386/gnu-user.h
[...]/gcc/config/gnu.h
[...]/gcc/config/i386/gnu.h
[...]/gcc/config/initfini-array.h
..., so let's clarify i686-symbolics-gnu to i686-gnu.
contrib/
* config-list.mk (LIST): Clarify i686-symbolics-gnu to i686-gnu.
Petr informed me that perl 5.32 bundled with Solaris 11.4 warns about
make_sunver.pl:
Unescaped left brace in regex is passed through in regex; marked by <-- HERE in m/^([ \t]*){ <-- HERE $/ at /vol/gcc/src/hg/master/local/libgomp/../contrib/make_sunver.pl line 216.
I didn't notice since I'm using a common installation of perl 5.12
across Solaris versions that doesn't show that warning.
His patch fixes the issue. Tested on Solaris 11.3 (perl 5.12) and 11.4
(perl 5.32).
2023-01-20 Petr Sumbera <petr.sumbera@oracle.com>
contrib:
* make_sunver.pl: Escape brace.
I use objs-gcc.sh as a preparatory step before calling
btest-gcc.sh in my scripts, for example my cris-elf
autotester. I thought, why not use it for native builds
too. Except that use, with binutils release-style tarballs
and a x86_64-pc-linux-gnu host, was broken. Now that I look
at it, the script seems to have aged poorly... Still,
there's a need for such a script to install stuff needed for
btest-gcc.sh (and to fix up stuff if needed), and this can
still be that script. So, I prefer to fix show-stoppers for
common uses, while taking care to retain compatibility for
use that could possibly still work, with current sources.
A long time ago (before 2011, but after this script was
created in 2002, and used for a few years), the binutils
(and gdb and gcc) toplevel Makefile may have had a bootstrap
target that worked with binutils but didn't require gcc
sources to be present. Now, you'll get an error (see
configure.ac line 1366 and on). Let's just build the
default make-target when "bootstrap" is known to fail.
An alternative would be to fold this native
non-i686-pc-linux-gnu clause into the native
i686-pc-linux-gnu clause, as that seems to have been
originally intended as *the* single native clause, but
that'd require further edits (e.g. to remove install-dejagnu
and make gdb build conditional on gdb sources presence, to
work with binutils tarballs, and I'd also then prefer to
build not just ld, but also gas and binutils).
As it's a minimal obvious change required for current native
use with release-tarballs and git-checkout use(*), I'm
installing this as obvious.
*) Native i686-pc-linux-gnu remains broken for other use
than specially constructed combined trees where dejagnu is
included at the toplevel (i.e. historic Cygnus devo-type).
contrib/regression:
* objs-gcc.sh: Only bootstrap if source-directory contains gcc.
This patch adds the dependencies for automatically generated files used
by the Modula-2 front end.
contrib/ChangeLog:
* gcc_update (files_and_dependencies): Add dependencies for
gcc/m2/gm2config.h.in, gcc/m2/configure,
gcc/m2/gm2-libs/config-host, libgm2/Makefile.in,
libgm2/aclocal.m4, libgm2/libm2cor/Makefile.in,
libgm2/libm2pim/Makefile.in, libgm2/libm2iso/Makefile.in,
libgm2/libm2log/Makefile.in and libgm2/libm2min/Makefile.in.
Signed-off-by: Gaius Mulley <gaiusmod2@gmail.com>
As mentioned on IRC or in PR108413, the last update-copyright.py --this year
failed and that is why we are in a strange state where some copyrights have
been updated and others have not.
The full list of errors I got was I think:
gcc/m2/mc-boot/GmcOptions.c: unrecognised copyright: comment (f, (const char *) "Copyright (C) ''2021'' Free Software Foundation, Inc.", 53);
gcc/m2/mc-boot/GmcOptions.c: unrecognised copyright: comment (f, (const char *) "Copyright (C) ''2021'' Free Software Foundation, Inc.", 53);
gcc/testsuite/gm2/switches/pedantic-params/pass/Strings.mod: unrecognised copyright holder: Faculty of Information Technology,
gcc/testsuite/gm2/switches/pedantic-params/pass/Strings2.mod: unrecognised copyright holder: Faculty of Information Technology,
libphobos/libdruntime/__builtins.di: unrecognised copyright: * Copyright: Copyright Digital Mars 2022
libstdc++-v3/src/c++17/fast_float/fast_float.h: unrecognised copyright holder: The fast_float authors
libstdc++-v3/include/c_compatibility/stdatomic.h: unrecognised copyright holder: The GCC developers
The following patch deals with the gcc/testsuite/gm2 ones and
with the fast_float.h one, ok for trunk?
Not really sure what we should do in the GmcOptions.c case
(perhaps obfuscate it in the source somehow by splitting
the string literals into different substrings
Perhaps "Copy" "right (" "C) ''..." would do it? Or do we want
to bump there each year (manually or by the script)?
E.g. in gcc.cc we have
printf ("Copyright %s 2023 Free Software Foundation, Inc.\n",
_("(C)"));
which also prints (C) nicer in Unicode if possible and is updated
by hand each year.
I have no idea about the libphobos case, we have tons of
libphobos/src/std/format/spec.d:Copyright: Copyright The D Language Foundation 2000-2013.
libphobos/src/std/random.d:Copyright: Copyright Andrei Alexandrescu 2008 - 2009, Joseph Rushton Wakeling 2012.
etc. lines and those aren't reported as errors.
And the last one is that I think for The GCC developers we should treat it
similarly like FSF and bump copyright on it.
Would
canon_gcc = 'The GCC developers'
self.add_package_author ('The GCC developers', canon_gcc)
self.add_package_author ('The GCC Developers', canon_gcc)
or something similar do the trick?
2023-01-16 Jakub Jelinek <jakub@redhat.com>
PR other/108413
* update-copyright.py (TestsuiteFilter): Add .mod and .rs extensions.
(GCCCopyright): Add 'The fast_float authors' as external author.
