The newer update-copyright.py fixes file encoding too, removing cr/lf
on binutils/bfdtest2.c and ld/testsuite/ld-cygwin/exe-export.exp, and
embedded cr in binutils/testsuite/binutils-all/ar.exp string match.
The result of running etc/update-copyright.py --this-year, fixing all
the files whose mode is changed by the script, plus a build with
--enable-maintainer-mode --enable-cgen-maint=yes, then checking
out */po/*.pot which we don't update frequently.
The copy of cgen was with commit d1dd5fcc38ead reverted as that commit
breaks building of bfp opcodes files.
These warnings are all off by default, but if they do fire you get
spurious ERRORs when running make check-libctf.
libctf/ChangeLog
2021-09-27 Nick Alcock <nick.alcock@oracle.com>
* testsuite/libctf-lookup/enum-symbol.c: Remove unused label.
* testsuite/libctf-lookup/conflicting-type-syms.c: Remove unused
variables.
* testsuite/libctf-regression/pptrtab.c: Likewise.
* testsuite/libctf-regression/type-add-unnamed-struct.c: Likewise.
* testsuite/libctf-writable/pptrtab.c: Likewise.
* testsuite/libctf-writable/reserialize-strtab-corruption.c:
Likewise.
* testsuite/libctf-regression/nonstatic-var-section-ld-r.c: Fix
format string.
* testsuite/libctf-regression/nonstatic-var-section-ld.c:
Likewise.
* testsuite/libctf-regression/nonstatic-var-section-ld.lk: Adjust.
* testsuite/libctf-writable/symtypetab-nonlinker-writeout.c: Fix
initializer.
This is the first tricky one, the first complex multi-entry vlen
containing strings. To handle this in vlen form, we have to handle
pending refs moving around on realloc.
We grow vlen regions using a new ctf_grow_vlen function, and iterate
through the existing enums every time a grow happens, telling the string
machinery the distance between the old and new vlen region and letting
it adjust the pending refs accordingly. (This avoids traversing all
outstanding refs to find the refs that need adjusting, at the cost of
having to traverse one enum: an obvious major performance win.)
Addition of enums themselves (and also structs/unions later) is a bit
trickier than earlier forms, because the type might be being promoted
from a forward, and forwards have no vlen: so we have to spot that and
create it if needed.
Serialization of enums simplifies down to just telling the string
machinery about the string refs; all the enum type-lookup code loses all
its dynamic member lookup complexity entirely.
A new test is added that iterates over (and gets values of) an enum with
enough members to force a round of vlen growth.
libctf/ChangeLog
2021-03-18 Nick Alcock <nick.alcock@oracle.com>
* ctf-impl.h (ctf_dtdef_t) <dtd_vlen_alloc>: New.
(ctf_str_move_pending): Declare.
* ctf-string.c (ctf_str_add_ref_internal): Fix error return.
(ctf_str_move_pending): New.
* ctf-create.c (ctf_grow_vlen): New.
(ctf_dtd_delete): Zero out the vlen_alloc after free. Free the
vlen later: iterate over it and free enum name refs first.
(ctf_add_generic): Populate dtd_vlen_alloc from vlen.
(ctf_add_enum): populate the vlen; do it by hand if promoting
forwards.
(ctf_add_enumerator): Set up the vlen rather than the dmd. Expand
it as needed, repointing string refs via ctf_str_move_pending. Add
the enumerand names as pending strings.
* ctf-serialize.c (ctf_copy_emembers): Remove.
(ctf_emit_type_sect): Copy the vlen into place and ref the
strings.
* ctf-types.c (ctf_enum_next): The dynamic portion now uses
the same code as the non-dynamic.
(ctf_enum_name): Likewise.
(ctf_enum_value): Likewise.
* testsuite/libctf-lookup/enum-many-ctf.c: New test.
* testsuite/libctf-lookup/enum-many.lk: New test.
ctf-link.c is unnecessarily confusing because ctf_link_lazy_open is
positioned near functions that have nothing to do with opening files.
Move it around, and fix some tabdamage that's crept in lately.
libctf/ChangeLog
2021-03-18 Nick Alcock <nick.alcock@oracle.com>
* ctf-link.c (ctf_link_lazy_open): Move up in the file, to near
ctf_link_add_ctf.
* ctf-lookup.c (ctf_lookup_symbol_idx): Repair tabdamage.
(ctf_lookup_by_sym_or_name): Likewise.
* testsuite/libctf-lookup/struct-iteration.c: Likewise.
* testsuite/libctf-regression/type-add-unnamed-struct.c: Likewise.
The existing ctf_lookup_by_symbol and ctf_arc_lookup_symbol functions
suffice to look up the types of symbols if the caller already has a
symbol number. But the caller often doesn't have one of those and only
knows the name of the symbol: also, in object files, the caller might
not have a useful symbol number in any sense (and neither does libctf:
the 'symbol number' we use in that case literally starts at 0 for the
lexicographically first-sorted symbol in the symtypetab and counts those
symbols, so it corresponds to nothing useful).
This means that even though object files have a symtypetab (generated by
the compiler or by ld -r), the only way we can look up anything in it is
to iterate over all symbols in turn with ctf_symbol_next until we find
the one we want.
This is unhelpful and pointlessly inefficient.
So add a pair of functions to look up symbols by name in a dict and in a
whole archive: ctf_lookup_by_symbol_name and ctf_arc_lookup_symbol_name.
These are identical to the existing functions except that they take
symbol names rather than symbol numbers.
To avoid insane repetition, we do some refactoring in the process, so
that both ctf_lookup_by_symbol and ctf_arc_lookup_symbol turn into thin
wrappers around internal functions that do both lookup by symbol index
and lookup by name. This massively reduces code duplication because
even the existing lookup-by-index stuff wants to use a name sometimes
(when looking up in indexed sections), and the new lookup-by-name stuff
has to turn it into an index sometimes (when looking up in non-indexed
sections): doing it this way lets us share most of that.
The actual name->index lookup is done by ctf_lookup_symbol_idx. We do
not anticipate this lookup to be as heavily used as ld.so symbol lookup
by many orders of magnitude, so using the ELF symbol hashes would
probably take more time to read them than is saved by using the hashes,
and it adds a lot of complexity. Instead, do a linear search for the
symbol name, caching all the name -> index mappings as we go, so that
future searches are likely to hit in the cache. To avoid having to
repeat this search over and over in a CTF archive when
ctf_arc_lookup_symbol_name is used, have cached archive lookups (the
sort done by ctf_arc_lookup_symbol* and the ctf_archive_next iterator)
pick out the first dict they cache in a given archive and store it in a
new ctf_archive field, ctfi_crossdict_cache. This can be used to store
cross-dictionary cached state that depends on things like the ELF symbol
table rather than the contents of any one dict. ctf_lookup_symbol_idx
then caches its name->index mappings in the dictionary named in the
crossdict cache, if any, so that ctf_lookup_symbol_idx in other dicts
in the same archive benefit from the previous linear search, and the
symtab only needs to be scanned at most once.
(Note that if you call ctf_lookup_by_symbol_name in one specific dict,
and then follow it with a ctf_arc_lookup_symbol_name, the former will
not use the crossdict cache because it's only populated by the dict
opens in ctf_arc_lookup_symbol_name. This is harmless except for a small
one-off waste of memory and time: it's only a cache, after all. We can
fix this later by using the archive caching machinery more
aggressively.)
In ctf-archive, we do similar things, turning ctf_arc_lookup_symbol into
a wrapper around a new function that does both index -> ID and name ->
ID lookups across all dicts in an archive. We add a new
ctfi_symnamedicts cache that maps symbol names to the ctf_dict_t * that
it was found in (so that linear searches for symbols don't need to be
repeated): but we also *remove* a cache, the ctfi_syms cache that was
memoizing the actual ctf_id_t returned from every call to
ctf_arc_lookup_symbol. This is pointless: all it saves is one call to
ctf_lookup_by_symbol, and that's basically an array lookup and nothing
more so isn't worth caching. (Equally, given that symbol -> index
mappings are cached by ctf_lookup_by_symbol_name, those calls are nearly
free after the first call, so there's no point caching the ctf_id_t in
that case either.)
We fix up one test that was doing manual symbol lookup to use
ctf_arc_lookup_symbol instead, and enhance it to check that the caching
layer is not totally broken: we also add a new test to do lookups in a
.o file, and another to do lookups in an archive with conflicted types
and make sure that sort of multi-dict lookup is actually working.
include/ChangeLog
2021-02-17 Nick Alcock <nick.alcock@oracle.com>
* ctf-api.h (ctf_arc_lookup_symbol_name): New.
(ctf_lookup_by_symbol_name): Likewise.
libctf/ChangeLog
2021-02-17 Nick Alcock <nick.alcock@oracle.com>
* ctf-impl.h (ctf_dict_t) <ctf_symhash>: New.
<ctf_symhash_latest>: Likewise.
(struct ctf_archive_internal) <ctfi_crossdict_cache>: New.
<ctfi_symnamedicts>: New.
<ctfi_syms>: Remove.
(ctf_lookup_symbol_name): Remove.
* ctf-lookup.c (ctf_lookup_symbol_name): Propagate errors from
parent properly. Make static.
(ctf_lookup_symbol_idx): New, linear search for the symbol name,
cached in the crossdict cache's ctf_symhash (if available), or
this dict's (otherwise).
(ctf_try_lookup_indexed): Allow the symname to be passed in.
(ctf_lookup_by_symbol): Turn into a wrapper around...
(ctf_lookup_by_sym_or_name): ... this, supporting name lookup too,
using ctf_lookup_symbol_idx in non-writable dicts. Special-case
name lookup in dynamic dicts without reported symbols, which have
no symtab or dynsymidx but where name lookup should still work.
(ctf_lookup_by_symbol_name): New, another wrapper.
* ctf-archive.c (enosym): Note that this is present in
ctfi_symnamedicts too.
(ctf_arc_close): Adjust for removal of ctfi_syms. Free the
ctfi_symnamedicts.
(ctf_arc_flush_caches): Likewise.
(ctf_dict_open_cached): Memoize the first cached dict in the
crossdict cache.
(ctf_arc_lookup_symbol): Turn into a wrapper around...
(ctf_arc_lookup_sym_or_name): ... this. No longer cache
ctf_id_t lookups: just call ctf_lookup_by_symbol as needed (but
still cache the dicts those lookups succeed in). Add
lookup-by-name support, with dicts of successful lookups cached in
ctfi_symnamedicts. Refactor the caching code a bit.
(ctf_arc_lookup_symbol_name): New, another wrapper.
* ctf-open.c (ctf_dict_close): Free the ctf_symhash.
* libctf.ver (LIBCTF_1.2): New version. Add
ctf_lookup_by_symbol_name, ctf_arc_lookup_symbol_name.
* testsuite/libctf-lookup/enum-symbol.c (main): Use
ctf_arc_lookup_symbol rather than looking up the name ourselves.
Fish it out repeatedly, to make sure that symbol caching isn't
broken.
(symidx_64): Remove.
(symidx_32): Remove.
* testsuite/libctf-lookup/enum-symbol-obj.lk: Test symbol lookup
in an unlinked object file (indexed symtypetab sections only).
* testsuite/libctf-writable/symtypetab-nonlinker-writeout.c
(try_maybe_reporting): Check symbol types via
ctf_lookup_by_symbol_name as well as ctf_symbol_next.
* testsuite/libctf-lookup/conflicting-type-syms.*: New test of
lookups in a multi-dict archive.
This returns an int, not a long int or an ssize_t (as one test was
inconsistently assuming).
libctf/ChangeLog
2021-01-05 Nick Alcock <nick.alcock@oracle.com>
* testsuite/libctf-lookup/struct-iteration.c (main):
ctf_member_count returns an int.
We never actually check to see if the compiler supports CTF,
or even if a suitable compiler exists.
libctf/ChangeLog
2021-01-05 Nick Alcock <nick.alcock@oracle.com>
* Makefile.am (BASEDIR): New.
(BFDDIR): Likewise.
(check-DEJAGNU): Add development.exp to prerequisites.
(development.exp): New.
(CONFIG_STATUS_DEPENDENCIES): New.
(EXTRA_DEJAGNU_SITE_CONFIG): Likewise.
(DISTCLEANFILES): Likewise.
* Makefile.in: Regenerated.
* testsuite/lib/ctf-lib.exp (check_ctf_available): Return boolean.
* testsuite/libctf-lookup/lookup.exp: Call check_ctf_available.
* testsuite/libctf-regression/regression.exp: Likewise.
libctf has no intrinsic support for the GCC unnamed structure member
extension. This principally means that you can't look up named members
inside unnamed struct or union members via ctf_member_info: you have to
tiresomely find out the type ID of the unnamed members via iteration,
then look in each of these.
This is ridiculous. Fix it by extending ctf_member_info so that it
recurses into unnamed members for you: this is still unambiguous because
GCC won't let you create ambiguously-named members even in the presence
of this extension.
For consistency, and because the release hasn't happened and we can
still do this, break the ctf_member_next API and add flags: we specify
one flag, CTF_MN_RECURSE, which if set causes ctf_member_next to
automatically recurse into unnamed members for you, returning not only
the members themselves but all their contained members, so that you can
use ctf_member_next to identify every member that it would be valid to
call ctf_member_info with.
New lookup tests are added for all of this.
include/ChangeLog
2021-01-05 Nick Alcock <nick.alcock@oracle.com>
* ctf-api.h (CTF_MN_RECURSE): New.
(ctf_member_next): Add flags argument.
libctf/ChangeLog
2021-01-05 Nick Alcock <nick.alcock@oracle.com>
* ctf-impl.h (struct ctf_next) <u.ctn_next>: Move to...
<ctn_next>: ... here.
* ctf-util.c (ctf_next_destroy): Unconditionally destroy it.
* ctf-lookup.c (ctf_symbol_next): Adjust accordingly.
* ctf-types.c (ctf_member_iter): Reimplement in terms of...
(ctf_member_next): ... this. Support recursive unnamed member
iteration (off by default).
(ctf_member_info): Look up members in unnamed sub-structs.
* ctf-dedup.c (ctf_dedup_rhash_type): Adjust ctf_member_next call.
(ctf_dedup_emit_struct_members): Likewise.
* testsuite/libctf-lookup/struct-iteration-ctf.c: Test empty unnamed
members, and a normal member after the end.
* testsuite/libctf-lookup/struct-iteration.c: Verify that
ctf_member_count is consistent with the number of successful returns
from a non-recursive ctf_member_next.
* testsuite/libctf-lookup/struct-iteration-*: New, test iteration
over struct members.
* testsuite/libctf-lookup/struct-lookup.c: New test.
* testsuite/libctf-lookup/struct-lookup.lk: New test.
I had reports that this doesn't work. This test shows it working (and
also shows how annoying it is to do symbol lookup by name with the
present API: we need a ctf_arc_lookup_symbol_name for users that don't
already have a symtab handy).
libctf/ChangeLog
2021-01-05 Nick Alcock <nick.alcock@oracle.com>
* testsuite/libctf-lookup/enum-symbol.lk: New symbol-lookup test.
* testsuite/libctf-lookup/enum-symbol-ctf.c: New CTF input.
* testsuite/libctf-lookup/enum-symbol.c: New lookup test.
This introduces a new lookup testsuite under libctf, which operates by
compiling (with libtool) a "lookup" .c file that uses libctf to analyze
some other program, then compiling some number of test object files with
CTF and optionally linking them together and running the lookup program
on the test object files (or linked test binary), before diffing the
result much as run_dump_test does.
This lets us test the portions of libctf that are not previously
testable, notably the portions that do lookup on linked output and
that create dynamic dictionaries and then do lookup on them before
writing them out, something that is not tested by the ld-ctf testsuite
because the linker never does this.
A couple of simple tests are added: one testing the functionality of
enum lookups, and one testing that the recently-added commit adding
extra paranoia to incomplete type handling doesn't break linking and
that the result of the link is an (otherwise-impossible) array of
forward type in the shared CTF dict.
ChangeLog
2021-01-05 Nick Alcock <nick.alcock@oracle.com>
* Makefile.def (libctf): No longer no_check. Checking depends on
all-ld.
* Makefile.in: Regenerated.
libctf/ChangeLog
2021-01-05 Nick Alcock <nick.alcock@oracle.com>
* Makefile.am (EXPECT): New.
(RUNTEST): Likewise.
(RUNTESTFLAGS): Likewise.
(CC_FOR_TARGET): Likewise.
(check-DEJAGNU): Likewise.
(AUTOMAKE_OPTIONS): Add dejagnu.
* Makefile.in: Regenerated.
* testsuite/config/default.exp: New.
* testsuite/lib/ctf-lib.exp: Likewise.
* testsuite/libctf-lookup/enum.lk: New test.
* testsuite/libctf-lookup/enum-ctf.c: New CTF input.
* testsuite/libctf-lookup/enum.c: New lookup test.
* testsuite/libctf-lookup/ambiguous-struct*.c: New test.
* testsuite/libctf-lookup/lookup.exp: New.
