PR 29389
bfd * bfd.c (BFD_CLOSED_BY_CACHE): New bfd flag.
* cache.c (bfd_cache_delete): Set BFD_CLOSED_BY_DELETE on the
closed bfd.
(bfd_cache_lookup_worker): Clear BFD_CLOSED_BY_DELETE on the newly
reopened bfd.
* opncls.c (bfd_set_filename): Refuse to change the name of a bfd
that has been closed by bfd_cache_delete. Mark changed bfds as
uncacheable.
* bfd-in2.h: Regenerate.
ld * ldlang.h (lang_input_statement_struct): Add sort_key field.
* emultempl/pe.em (after_open): If multiple import libraries refer
to the same bfd, store their names in the sort_key field.
* emultempl/pep.em (after_open): Likewise.
* ldlang.c (sort_filename): New function. Returns the filename to
be used when sorting input files.
(wild_sort): Use the sort_filename function.
These two macros print either a 16 digit hex number or an 8 digit
hex number. Unfortunately they depend on both target and host, which
means that the output for 32-bit targets may be either 8 or 16 hex
digits.
Replace them in most cases with code that prints a bfd_vma using
PRIx64. In some cases, deliberately lose the leading zeros.
This change some output, notably in base/offset fields of m68k
disassembly which I think looks better that way, and in error
messages. I've kept leading zeros in symbol dumps (objdump -t)
and in PE header dumps.
bfd/
* bfd-in.h (fprintf_vma, sprintf_vma, printf_vma): Delete.
* bfd-in2.h: Regenerate.
* bfd.c (bfd_sprintf_vma): Don't use sprintf_vma.
(bfd_fprintf_vma): Don't use fprintf_vma.
* coff-rs6000.c (xcoff_reloc_type_tls): Don't use sprintf_vma.
Instead use PRIx64 to print bfd_vma values.
(xcoff_ppc_relocate_section): Likewise.
* cofflink.c (_bfd_coff_write_global_sym): Likewise.
* mmo.c (mmo_write_symbols_and_terminator): Likewise.
* srec.c (srec_write_symbols): Likewise.
* elf32-xtensa.c (print_r_reloc): Similarly for fprintf_vma.
* pei-x86_64.c (pex64_dump_xdata): Likewise.
(pex64_bfd_print_pdata_section): Likewise.
* som.c (som_print_symbol): Likewise.
* ecoff.c (_bfd_ecoff_print_symbol): Use bfd_fprintf_vma.
opcodes/
* dis-buf.c (perror_memory, generic_print_address): Don't use
sprintf_vma. Instead use PRIx64 to print bfd_vma values.
* i386-dis.c (print_operand_value, print_displacement): Likewise.
* m68k-dis.c (print_base, print_indexed): Likewise.
* ns32k-dis.c (print_insn_arg): Likewise.
* ia64-gen.c (_opcode_int64_low, _opcode_int64_high): Delete.
(opcode_fprintf_vma): Delete.
(print_main_table): Use PRIx64 to print opcode.
binutils/
* od-macho.c: Replace all uses of printf_vma with bfd_printf_vma.
* objcopy.c (copy_object): Don't use sprintf_vma. Instead use
PRIx64 to print bfd_vma values.
(copy_main): Likewise.
* readelf.c (CHECK_ENTSIZE_VALUES): Likewise.
(dynamic_section_mips_val): Likewise.
(print_vma): Don't use printf_vma. Instead use PRIx64 to print
bfd_vma values.
(dump_ia64_vms_dynamic_fixups): Likewise.
(process_version_sections): Likewise.
* rddbg.c (stab_context): Likewise.
gas/
* config/tc-i386.c (offset_in_range): Don't use sprintf_vma.
Instead use PRIx64 to print bfd_vma values.
(md_assemble): Likewise.
* config/tc-mips.c (load_register, macro): Likewise.
* messages.c (as_internal_value_out_of_range): Likewise.
* read.c (emit_expr_with_reloc): Likewise.
* config/tc-ia64.c (note_register_values): Don't use fprintf_vma.
Instead use PRIx64 to print bfd_vma values.
(print_dependency): Likewise.
* listing.c (list_symbol_table): Use bfd_sprintf_vma.
* symbols.c (print_symbol_value_1): Use %p to print pointers.
(print_binary): Likewise.
(print_expr_1): Use PRIx64 to print bfd_vma values.
* write.c (print_fixup): Use %p to print pointers. Don't use
fprintf_vma.
* testsuite/gas/all/overflow.l: Update expected output.
* testsuite/gas/m68k/mcf-mov3q.d: Likewise.
* testsuite/gas/m68k/operands.d: Likewise.
* testsuite/gas/s12z/truncated.d: Likewise.
ld/
* deffilep.y (def_file_print): Don't use fprintf_vma. Instead
use PRIx64 to print bfd_vma values.
* emultempl/armelf.em (gld${EMULATION_NAME}_finish): Don't use
sprintf_vma. Instead use PRIx64 to print bfd_vma values.
* emultempl/pe.em (gld${EMULATION_NAME}_finish): Likewise.
* ldlang.c (lang_map): Use %V to print region origin.
(lang_one_common): Don't use sprintf_vma.
* ldmisc.c (vfinfo): Don't use fprintf_vma or sprintf_vma.
* pe-dll.c (pe_dll_generate_def_file): Likewise.
gdb/
* remote.c (remote_target::trace_set_readonly_regions): Replace
uses of sprintf_vma with bfd_sprintf_vma.
On seeing PR29369 my suspicion was naturally on a recent powerpc64
change, commit 0ab80031430e. Without a reproducer, I spent time
wondering what could have gone wrong, and while I doubt this patch
would have fixed the PR, there are some improvements that can be made
to cater for user silliness.
I also noticed that when -z relro -z now sections are created out of
order, with .got before .plt in the section headers but .got is laid
out at a higher address. That's due to the address expression for
.branch_lt referencing SIZEOF(.got) and so calling init_os (which
creates a bfd section) for .got before the .plt section is created.
Fix that by ignoring SIZEOF in exp_init_os. Unlike ADDR and LOADADDR
which need to reference section vma and lma respectively, SIZEOF can
and does cope with a missing bfd section by returning zero for its
size, which of course is correct.
PR 29369
* ldlang.c (exp_init_os): Don't create a bfd section for SIZEOF.
* emulparams/elf64ppc.sh (OTHER_RELRO_SECTIONS_2): Revise
.branch_lt address to take into account possible user sections
with alignment larger than 8 bytes.
A number of targets use assignments like:
. = DATA_SEGMENT_RELRO_END (SIZEOF (.got.plt) >= 12 ? 12 : 0, .);
(from i386) in linker scripts to put the end of the relro segment past
the header in .got.plt. Examination of testcases like those edited by
this patch instead sees the end of the relro segment being placed at
the start of .got.plt. For the i386 pie1 test:
[ 9] .got.plt PROGBITS 00002000 001000 00000c 04 WA 0 0 4
GNU_RELRO 0x000f90 0x00001f90 0x00001f90 0x00070 0x00070 R 0x1
A map file shows:
.dynamic 0x0000000000001f90 0x70
*(.dynamic)
.dynamic 0x0000000000001f90 0x70 tmpdir/pie1.o
0x0000000000001f90 _DYNAMIC
.got 0x0000000000002000 0x0
*(.got)
.got 0x0000000000002000 0x0 tmpdir/pie1.o
*(.igot)
0x0000000000002ff4 . = DATA_SEGMENT_RELRO_END (., (SIZEOF (.got.plt) >= 0xc)?0xc:0x0)
.got.plt 0x0000000000002000 0xc
*(.got.plt)
.got.plt 0x0000000000002000 0xc tmpdir/pie1.o
0x0000000000002000 _GLOBAL_OFFSET_TABLE_
The DATA_SEGMENT_RELRO_END value in the map file is weird too. All of
this is triggered by SIZEOF (.got.plt) being evaluated wrongly as
zero. Fix it by taking into account the action of
lang_reset_memory_regions during relaxation.
* ldexp.c (fold_name <SIZEOF>): Use rawsize if size has been reset.
* ldlang.c (lang_size_sections_1): Don't reset processed_vma here.
* testsuite/ld-i386/pie1.d: Adjust to suit.
* testsuite/ld-x86-64/pr20830a.d: Likewise.
* testsuite/ld-x86-64/pr20830b.d: Likewise.
* testsuite/ld-x86-64/pr21038a.d: Likewise.
* testsuite/ld-x86-64/pr21038b.d: Likewise.
* testsuite/ld-x86-64/pr21038c.d: Likewise.
The better to see any code that accesses expld.dataseg.
* ldexp.c (fold_segment_end): Remove seg parameter. Adjust calls.
(fold_segment_align, fold_segment_relro_end): Likewise.
* ldlang.c (lang_size_segment): Likewise.
(lang_size_relro_segment_1, lang_find_relro_sections_1): Likewise.
Background
==========
There are constraints on layout of binaries to meet demand paging and
memory protection requirements. Demand paged binaries must have file
offset mod pagesize equal to vma mod pagesize. Memory protection
(executable, read, write status) can only change at page boundaries.
The linker's MAXPAGESIZE variable gives the page size for these layout
constraints.
In a typical basic executable with two memory segments, text (RE) and
data (RW), the data segment must start on a different page to the
last text segment page. For example, with 64k pages and a small
executable of 48k text and 1k data, the text segment might start at
address 0x10000 and data at 0x20000 for a total of two 64k memory
pages. Demand paging would require the image on disk to be 64k+1k
in size. We can do better than that. If the data segment instead
starts at 0x2c000 (the end of the text segment plus one 64k page) then
there are still only two memory pages, but the disk image is now
smaller, 48k+1k in size. This is why the linker normally starts the
data segment at the end of the text segment plus one page. That
simple heuristic isn't ideal in all cases. Changing our simple
example to one with 64k-1 text size, following that heuristic would
result in data starting at 0x2ffff. Now we have two 64k memory data
pages for a data segment of 1k! If the data segment instead started
at 0x30000 we'd get a single data segment page at the cost of 1 byte
extra in the disk image, which is likely a good trade-off. So the
linker does adjust the simple heuristic. Just how much disk image
size increase is allowed is controlled by the linker's COMMONPAGESIZE
variable.
A PT_GNU_RELRO segment overlays the initial part of the data segment,
saying that those pages should be made read-only after relocation by
the dynamic loader. Page granularity for memory protection means that
the end of the relro segment must be at a page boundary.
The problem
===========
Unfortunately most targets currently only align the end of the relro
segment to COMMONPAGESIZE. That results in only partial relro
protection if an executable is running with MAXPAGESIZE pages, since
any part of the relro segment past the last MAXPAGESIZE boundary can't
be made read-only without also affecting sections past the end of the
relro segment. I believe this problem arose because x86 always runs
with 4k (COMMPAGESIZE) memory pages, and therefore using a larger
MAXPAGESIZE on x86 is for reasons other than the demand paging and
memory page protection boundary requirements.
The solution
============
Always end the relro segment on a MAXPAGESIZE boundary, except for
x86. Note that the relro segment, comprising of sections at the start
of the data segment, is sized according to how those sections are laid
out. That means the start of the relro segment is fixed relative to
its end. Which also means the start of the data segment must be at a
fixed address mod MAXPAGESIZE. So for relro the linker can't play
games with the start of the data segment to save disk space. At
least, not without introducing gaps between the relro sections. In
fact, because the linker was starting layout using its simple
heuristic of starting the data segment at the end of the text segment
plus one page, it was sometimes introducing page gaps for no reason.
See pr28743.
PR 28824
PR 28734
* ldexp.c (fold_segment_align): When relro, don't adjust up by
offset within page. Set relropagesize.
(fold_segment_relro_end): Align to relropagesize.
* ldexp.h (seg_align_type): Rename pagesize to commonpagesize.
Add relropagesize. Comment.
* ldlang.c (lang_size_segment): Adjust to suit field renaming.
(lang_size_relro_segment_1): Align relro_end using relropagesize.
This is just a tidy, making the __ehdr_start symbol flag tweaks all in
one place.
* ldelf.c (ldelf_before_allocation): Don't set rel_from_abs
for __ehdr_start.
* ldlang.c (lang_symbol_tweaks): Set it here instead.
1. Compute the desired PT_GNU_RELRO segment base and find the maximum
section alignment of sections starting from the PT_GNU_RELRO segment.
2. Find the first preceding load section.
3. Don't add the 1-page gap between the first preceding load section and
the relro segment if the maximum page size >= the maximum section
alignment. Align the PT_GNU_RELRO segment first. Subtract the maximum
page size if therer is still a 1-page gap.
PR ld/28743
PR ld/28819
* ldlang.c (lang_size_relro_segment_1): Rewrite.
* testsuite/ld-x86-64/pr28743-1.d: New file.
* testsuite/ld-x86-64/pr28743-1.s: Likewise.
* testsuite/ld-x86-64/x86-64.exp: Run pr28743-1.
bfd/
* elf-bfd.h (UNDEFWEAK_NO_DYNAMIC_RELOC): Test linker_def.
ld/
* ldelf.c (ldelf_before_allocation): Don't force __ehdr_start
local and hidden here..
* ldlang.c (lang_symbol_tweaks): ..do so here instead and set
def_regular and linker_def for check_relocs. New function
extracted from lang_process.
Add a -z pack-relative-relocs option to enable DT_RELR and create a
relr.dyn section for DT_RELR. DT_RELR is implemented with the linker
relaxation infrastructure, but it doesn't require the --relax option
enabled. -z pack-relative-relocs implies -z combreloc. -z nocombreloc
implies -z nopack-relative-relocs.
-z pack-relative-relocs is chosen over the similar option in lld,
--pack-dyn-relocs=relr, to implement a glibc binary lockout mechanism
with a special glibc version symbol, to avoid random crashes of DT_RELR
binaries with the existing glibc binaries.
bfd/
* elf-bfd.h (elf_link_hash_table): Add srelrdyn.
* elflink.c (_bfd_elf_link_create_dynamic_sections): Create a
.relr.dyn section for DT_RELR.
include/
* bfdlink.h (bfd_link_info): Add enable_dt_relr.
ld/
* News: Mention -z pack-relative-relocs and
-z nopack-relative-relocs.
* ld.texi: Document -z pack-relative-relocs and
-z nopack-relative-relocs.
* ldelf.c (ldelf_after_parse): Disable DT_RELR if not building
PIE nor shared library. Add 3 spare dynamic tags for DT_RELR,
DT_RELRSZ and DT_RELRENT.
* ldlang.c (lang_relax_sections): Also enable relaxation if
DT_RELR is enabled.
* emulparams/elf32_x86_64.sh: Source dt-relr.sh.
* emulparams/elf_i386.sh: Likewise.
* emulparams/elf_x86_64.sh: Likewise.
* emulparams/dt-relr.sh: New file.
* scripttempl/elf.sc: Support .relr.dyn.
For the sake of DT_RELR.
bfd/
* elflink.c (elf_link_input_bfd): Don't set SEC_ELF_REVERSE_COPY
here. Move sanity checks to reverse copying code.
ld/
* ldlang.c (lang_add_section): Set SEC_ELF_REVERSE_COPY for
.ctors/.dtors in .init_array/.fini_array.
The result of running etc/update-copyright.py --this-year, fixing all
the files whose mode is changed by the script, plus a build with
--enable-maintainer-mode --enable-cgen-maint=yes, then checking
out */po/*.pot which we don't update frequently.
The copy of cgen was with commit d1dd5fcc38ead reverted as that commit
breaks building of bfp opcodes files.
It's not foolproof, for example we don't catch output to a linker
script, to a library specified with -l, or to an element of a thin
archive.
* ldlang.c (open_output): Exit with error on output file matching
an input file.
* testsuite/ld-misc/just-symbols.exp: Adjust ld -r test to suit.
* ld.texi: Document new output section type.
* ldgram.y: Add new token.
* ldlang.c: Handle the new flag.
* ldlang.h: Add readonly_section to list of section types.
* ldlex.l: Add a new identifier.
* testsuite/ld-scripts/output-section-types.t: New example linker script.
* testsuite/ld-scripts/output-section-types.d: Test driver.
* testsyute/ld-scripts/script.exp: Run the new test.
It's not enough to test that the output is ELF before casting
bfd_link_hash_entry to elf_link_hash_entry. Some ELF targets (d30v,
dlx, pj, s12z, xgate) use the generic linker support in bfd/linker.c
and thus their symbols are of type generic_link_hash_entry.
Not all of the places this patch touches can result in wrong accesses,
but I thought it worth ensuring that all occurrences of
elf_link_hash_entry in ld/ were obviously correct.
PR 27719
* ldlang.c (lang_mark_undefineds, undef_start_stop): Test that
the symbol hash table is the correct type before accessing
elf_link_hash_entry symbols.
* plugin.c (is_visible_from_outside): Likewise.
* emultempl/armelf.em (ld${EMULATION_NAME}_finish): Likewise.
* emultempl/solaris2.em (elf_solaris2_before_allocation): Likewise.
The exception for debug sections in clearing SEC_EXCLUDE when
relocatable was really for one specific debug section, so let's make
it do just that.
bfd/
PR 27590
* elf.c (_bfd_elf_make_section_from_shdr): Remove SHF_EXCLUDE
test for .gnu.debuglto*.
ld/
PR 27590
* ldlang.c (lang_gc_sections): Clear SEC_EXCLUDE when relocatable
for all sections except .stabstr.
The testcases added here show situations where synthesized start/stop
symbols don't cause their associated input sections to be marked.
Fixed with the elflink.c and ldlang.c changes.
bfd/
PR 27500
* elflink.c (_bfd_elf_gc_mark_rsec): Do special start/stop
processing not when start/stop symbol section is unmarked but
on first time a start/stop symbol is processed.
ld/
* ldlang.c (insert_undefined): Don't mark symbols here.
(lang_mark_undefineds): Do so here instead, new function.
(lang_process): Call lang_mark_undefineds.
* testsuite/ld-gc/start3.d,
* testsuite/ld-gc/start3.s: New test.
* testsuite/ld-gc/start4.d,
* testsuite/ld-gc/start4.s: New test.
* testsuite/ld-gc/gc.exp: Run them.
If a weak reference to a __start_foo or __stop_foo symbol ends up
having no definition due to all the foo sections being removed for
some reason, undef_start_stop currently makes the symbol strong
undefined. That risks a linker undefined symbol error. Fix that by
making the symbol undefweak and also undo some dynamic symbol state.
Note that saving the state of the symbol type at the time
lang_init_start_stop runs is not sufficient. The linker may have
merged in a shared library reference by that point and made what was
an undefweak in regular objects, a strong undefined. So it is
necessary to look at the ELF symbol flags to decide whether an
undefweak is the proper resolution.
Something probably should be done for COFF/PE too, but I'm unsure how
to do go about that.
* ldlang.c (undef_start_stop): For ELF make undefined start/stop
symbols undefweak if that was how they were referenced. Undo
dynamic state too.
The variable section in a CTF dict is meant to contain the types of
variables that do not appear in the symbol table (mostly file-scope
static declarations). We implement this by having the compiler emit
all potential data symbols into both sections, then delete those
symbols from the variable section that correspond to data symbols the
linker has reported.
Unfortunately, the check for this in ctf_serialize is wrong: rather than
checking the set of linker-reported symbols, we check the set of names
in the data object symtypetab section: if the linker has reported no
symbols at all (usually if ld -r has been run, or if a non-linker
program that does not use symbol tables is calling ctf_link) this will
include every single symbol, emptying the variable section completely.
Worse, when ld -r is in use, we want to force writeout of every
symtypetab entry on the inputs, in an indexed section, whether or not
the linker has reported them, since this isn't a final link yet and the
symbol table is not finalized (and may grow more symbols than the linker
has yet reported). But the check for this is flawed too: we were
relying on ctf_link_shuffle_syms not having been called if no symbols
exist, but that function is *always* called by ld even when ld -r is in
use: ctf_link_add_linker_symbol is the one that's not called when there
are no symbols.
We clearly need to rethink this. Using the emptiness of the set of
reported symbols as a test for ld -r is just ugly: the linker already
knows if ld -r is underway and can just tell us. So add a new linker
flag CTF_LINK_NO_FILTER_REPORTED_SYMS that is set to stop the linker
filtering the symbols in the symtypetab sections using the set that the
linker has reported: use the presence or absence of this flag to
determine whether to emit unindexed symtabs: we only remove entries from
the variable section when filtering symbols, and we only remove them if
they are in the reported symbol set, fixing the case where no symbols
are reported by the linker at all.
(The negative sense of the new CTF_LINK flag is intentional: the common
case, both for ld and for simple tools that want to do a ctf_link with
no ELF symbol table in sight, is probably to filter out symbols that no
linker has reported: i.e., for the simple tools, all of them.)
There's another wrinkle, though. It is quite possible for a non-linker
to add symbols to a dict via ctf_add_*_sym and then write it out via the
ctf_write APIs: perhaps it's preparing a dict for a later linker
invocation. Right now this would not lead to anything terribly
meaningful happening: ctf_serialize just assumes it was called via
ctf_link if symbols are present. So add an (internal-to-libctf) flag
that indicates that a writeout is happening via ctf_link_write, and set
it there (propagating it to child dicts as needed). ctf_serialize can
then spot when it is not being called by a linker, and arrange to always
write out an indexed, sorted symtypetab for fastest possible future
symbol lookup by name in that case. (The writeouts done by ld -r are
unsorted, because the only thing likely to use those symtabs is the
linker, which doesn't benefit from symtypetab sorting.)
Tests added for all three linking cases (ld -r, ld -shared, ld), with a
bit of testsuite framework enhancement to stop it unconditionally
linking the CTF to be checked by the lookup program with -shared, so
tests can now examine CTF linked with -r or indeed with no flags at all,
though the output filename is still foo.so even in this case.
Another test added for the non-linker case that endeavours to determine
whether the symtypetab is sorted by examining the order of entries
returned from ctf_symbol_next: nobody outside libctf should rely on
this ordering, but this test is not outside libctf :)
include/ChangeLog
2021-01-26 Nick Alcock <nick.alcock@oracle.com>
* ctf-api.h (CTF_LINK_NO_FILTER_REPORTED_SYMS): New.
ld/ChangeLog
2021-01-26 Nick Alcock <nick.alcock@oracle.com>
* ldlang.c (lang_merge_ctf): Set CTF_LINK_NO_FILTER_REPORTED_SYMS
when appropriate.
libctf/ChangeLog
2021-01-27 Nick Alcock <nick.alcock@oracle.com>
* ctf-impl.c (_libctf_nonnull_): Add parameters.
(LCTF_LINKING): New flag.
(ctf_dict_t) <ctf_link_flags>: Mention it.
* ctf-link.c (ctf_link): Keep LCTF_LINKING set across call.
(ctf_write): Likewise, including in child dictionaries.
(ctf_link_shuffle_syms): Make sure ctf_dynsyms is NULL if there
are no reported symbols.
* ctf-create.c (symtypetab_delete_nonstatic_vars): Make sure
the variable has been reported as a symbol by the linker.
(symtypetab_skippable): Mention relationship between SYMFP and the
flags.
(symtypetab_density): Adjust nonnullity. Exit early if no symbols
were reported and force-indexing is off (i.e., we are doing a
final link).
(ctf_serialize): Handle the !LCTF_LINKING case by writing out an
indexed, sorted symtypetab (and allow SYMFP to be NULL in this
case). Turn sorting off if this is a non-final link. Only delete
nonstatic vars if we are filtering symbols and the linker has
reported some.
* testsuite/libctf-regression/nonstatic-var-section-ld-r*:
New test of variable and symtypetab section population when
ld -r is used.
* testsuite/libctf-regression/nonstatic-var-section-ld-executable.lk:
Likewise, when ld of an executable is used.
* testsuite/libctf-regression/nonstatic-var-section-ld.lk:
Likewise, when ld -shared alone is used.
* testsuite/libctf-regression/nonstatic-var-section-ld*.c:
Lookup programs for the above.
* testsuite/libctf-writable/symtypetab-nonlinker-writeout.*: New
test, testing survival of symbols across ctf_write paths.
* testsuite/lib/ctf-lib.exp (run_lookup_test): New option,
nonshared, suppressing linking of the SOURCE with -shared.
This changes the initialisation of output sections so that it is
possible to create read-only sections fed only from linker script
BYTE, SHORT, LONG or QUAD. That currently isn't possible even for one
of the well-known ELF sections like .rodata, because once a section is
marked read/write that sticks. On the other hand if we start
read-only, well-known ELF sections end up read/write as appropriate.
For example .tdata will still be SHF_ALLOC + SHF_WRITE + SHF_TLS.
PR 26378
* ldlang.c (map_input_to_output_sections): Start with a read-only
section for data statements.
* testsuite/ld-elf/size-2.d: Adjust to suit.
This moves the SHF_LINK_ORDER sorting from bfd_elf_final_link to
the linker which means generic ELF targets now support SHF_LINK_ORDER
and we cope with odd cases that require resizing of output sections.
The patch also fixes two bugs in the current implementation,
introduced by commit cd6d537c48. The pattern test used by that
commit meant that sections matching something like
"*(.IA_64.unwind* .gnu.linkonce.ia64unw.*)" would not properly sort a
mix of sections matching the two wildcards. That commit also assumed
a stable qsort.
bfd/
PR 27160
* section.c (struct bfd_section): Remove pattern field.
(BFD_FAKE_SECTION): Adjust to suit.
* bfd-in2.h: Regenerate.
* elflink.c (compare_link_order, elf_fixup_link_order): Delete.
(bfd_elf_final_link): Don't call elf_fixup_link_order.
ld/
PR 27160
* ldlang.h (lang_output_section_statement_type): Add data field.
(lang_input_section_type, lang_section_bst_type): Add pattern field.
(statement_list): Declare.
(lang_add_section): Adjust prototype.
* emultempl/aarch64elf.em: Adjust lang_add_section calls.
* emultempl/armelf.em: Likewise.
* emultempl/beos.em: Likewise.
* emultempl/cskyelf.em: Likewise.
* emultempl/hppaelf.em: Likewise.
* emultempl/m68hc1xelf.em: Likewise.
* emultempl/metagelf.em: Likewise.
* emultempl/mipself.em: Likewise.
* emultempl/mmo.em: Likewise.
* emultempl/msp430.em: Likewise.
* emultempl/nios2elf.em: Likewise.
* emultempl/pe.em: Likewise.
* emultempl/pep.em: Likewise.
* emultempl/ppc64elf.em: Likewise.
* emultempl/spuelf.em: Likewise.
* emultempl/vms.em: Likewise.
* ldelf.c: Likewise.
* ldelfgen.c: Include ldctor.h.
(struct os_sections): New.
(add_link_order_input_section, link_order_scan): New functions.
(compare_link_order, fixup_link_order): New functions.
(ldelf_map_segments): Call link_order_scan and fixup_link_order.
* ldlang.c (statement_list): Make global.
(output_section_callback_fast): Save pattern in tree node.
(lang_add_section): Add pattern parameter, save in lang_input_section.
(output_section_callback_tree_to_list): Adjust lang_add_section calls.
(lang_insert_orphan, output_section_callback): Likewise.
(ldlang_place_orphan): Likewise.
(gc_section_callback): Don't set section->pattern
* testsuite/ld-elf/pr26256-2a.d: Don't xfail generic.
* testsuite/ld-elf/pr26256-3b.d: Likewise.
* testsuite/ld-elf/pr26256-2b.d: Likewise. notarget xgate.
For non-relocatable link with SHF_LINK_ORDER inputs, allow mixed indirect
and data inputs with ordered and unordered inputs:
1. Add pattern to bfd_section for the matching section name pattern in
linker script and update BFD_FAKE_SECTION.
2. Sort the consecutive bfd_indirect_link_order sections with the same
pattern to allow linker script to overdide input section order.
3. Place unordered sections before ordered sections.
4. Change the offsets of the indirect input sections only.
bfd/
PR ld/26256
* elflink.c (compare_link_order): Place unordered sections before
ordered sections.
(elf_fixup_link_order): Add a link info argument. Allow mixed
ordered and unordered input sections for non-relocatable link.
Sort the consecutive bfd_indirect_link_order sections with the
same pattern. Change the offsets of the bfd_indirect_link_order
sections only.
(bfd_elf_final_link): Pass info to elf_fixup_link_order.
* section.c (bfd_section): Add pattern.
(BFD_FAKE_SECTION): Initialize pattern to NULL.
* bfd-in2.h: Regenerated.
gas/
PR ld/26256
* config/obj-elf.c (obj_elf_change_section): Also filter out
SHF_LINK_ORDER.
ld/
PR ld/26256
* ldlang.c (gc_section_callback): Set pattern.
* testsuite/ld-elf/pr26256-1.s: New file.
* testsuite/ld-elf/pr26256-1.t: Likewise.
* testsuite/ld-elf/pr26256-1a.d: Likewise.
* testsuite/ld-elf/pr26256-1b.d: Likewise.
* testsuite/ld-elf/pr26256-2.s: Likewise.
* testsuite/ld-elf/pr26256-2.t: Likewise.
* testsuite/ld-elf/pr26256-2a.d: Likewise.
* testsuite/ld-elf/pr26256-2b-alt.d: Likewise.
* testsuite/ld-elf/pr26256-2b.d: Likewise.
* testsuite/ld-elf/pr26256-3.s: Likewise.
* testsuite/ld-elf/pr26256-3a.d: Likewise.
* testsuite/ld-elf/pr26256-3a.t: Likewise.
* testsuite/ld-elf/pr26256-3b.d: Likewise.
* testsuite/ld-elf/pr26256-3b.t: Likewise.
The failure on this PR is due to using the same bfd section for
multiple output sections. Commit 21401fc7bf managed to create
duplicate linker script output section statements, but not the actual
bfd sections.
PR 27100
* ldlang.h (lang_output_section_statement_type): Add dup_output.
* ldlang.c (lang_output_section_statement_lookup): Set dup_output.
(init_os): Test dup_output rather than constraint.
* testsuite/ld-scripts/pr27100.d,
* testsuite/ld-scripts/pr27100.s,
* testsuite/ld-scripts/pr27100.t: New test.
* testsuite/ld-scripts/data.exp: Run it. Don't exclude aout here.
* testsuite/ld-scripts/data.d: Do so here instead.
* testsuite/ld-scripts/fill.d: Likewise.
* testsuite/ld-scripts/fill16.d: Likewise.
Previously, ld merged duplicate output sections if such existed in
scripts, except for those with a constraint of SPECIAL. This makes
scripts with duplicate output section statements create duplicate
output sections in the linker output file.
* ldlang.c (lang_output_section_statement_lookup): Change "create"
parameter to a tristate, if 2 then always create a new output
section statement. Update all callers, with
lang_enter_output_section_statement using "2".
(map_input_to_output_sections): Don't ignore SPECIAL constraint
here.
* ldlang.h (lang_output_section_statement_type): Update prototype.
(lang_output_section_find): Update.
This is embarrassing.
The whole point of CTF is that it remains intact even after a binary is
stripped, providing a compact mapping from symbols to types for
everything in the externally-visible interface of an ELF object: it has
connections to the symbol table for that purpose, and to the string
table to avoid duplicating symbol names. So it's a shame that the hooks
I implemented last year served to hook it up to the .symtab and .strtab,
which obviously disappear on strip, leaving any accompanying the CTF
dict containing references to strings (and, soon, symbols) which don't
exist any more because their containing strtab has been vaporized. The
original Solaris design used .dynsym and .dynstr (well, actually,
.ldynsym, which has more symbols) which do not disappear. So should we.
Thankfully the work we did before serves as guide rails, and adjusting
things to use the .dynstr and .dynsym was fast and easy. The only
annoyance is that the dynsym is assembled inside elflink.c in a fairly
piecemeal fashion, so that the easiest way to get the symbols out was to
hook in before every call to swap_symbol_out (we also leave in a hook in
front of symbol additions to the .symtab because it seems plausible that
we might want to hook them in future too: for now that hook is unused).
We adjust things so that rather than being offered a whole hash table of
symbols at once, libctf is now given symbols one at a time, with st_name
indexes already resolved and pointing at their final .dynstr offsets:
it's now up to libctf to resolve these to names as needed using the
strtab info we pass it separately.
Some bits might be contentious. The ctf_new_dynstr callback takes an
elf_internal_sym, and this remains an elf_internal_sym right down
through the generic emulation layers into ldelfgen. This is no worse
than the elf_sym_strtab we used to pass down, but in the future when we
gain non-ELF CTF symtab support we might want to lower the
elf_internal_sym to some other representation (perhaps a
ctf_link_symbol) in bfd or in ldlang_ctf_new_dynsym. We rename the
'apply_strsym' hooks to 'acquire_strings' instead, becuse they no longer
have anything to do with symbols.
There are some API changes to pieces of API which are technically public
but actually totally unused by anything and/or unused by anything but ld
so they can change freely: the ctf_link_symbol gains new fields to allow
symbol names to be given as strtab offsets as well as strings, and a
symidx so that the symbol index can be passed in. ctf_link_shuffle_syms
loses its callback parameter: the idea now is that linkers call the new
ctf_link_add_linker_symbol for every symbol in .dynsym, feed in all the
strtab entries with ctf_link_add_strtab, and then a call to
ctf_link_shuffle_syms will apply both and arrange to use them to reorder
the CTF symtab at CTF serialization time (which is coming in the next
commit).
Inside libctf we have a new preamble flag CTF_F_DYNSTR which is always
set in v3-format CTF dicts from this commit forwards: CTF dicts without
this flag are associated with .strtab like they used to be, so that old
dicts' external strings don't turn to garbage when loaded by new libctf.
Dicts with this flag are associated with .dynstr and .dynsym instead.
(The flag is not the next in sequence because this commit was written
quite late: the missing flags will be filled in by the next commit.)
Tests forthcoming in a later commit in this series.
bfd/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* elflink.c (elf_finalize_dynstr): Call examine_strtab after
dynstr finalization.
(elf_link_swap_symbols_out): Don't call it here. Call
ctf_new_symbol before swap_symbol_out.
(elf_link_output_extsym): Call ctf_new_dynsym before
swap_symbol_out.
(bfd_elf_final_link): Likewise.
* elf.c (swap_out_syms): Pass in bfd_link_info. Call
ctf_new_symbol before swap_symbol_out.
(_bfd_elf_compute_section_file_positions): Adjust.
binutils/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* readelf.c (dump_section_as_ctf): Use .dynsym and .dynstr, not
.symtab and .strtab.
include/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* bfdlink.h (struct elf_sym_strtab): Replace with...
(struct elf_internal_sym): ... this.
(struct bfd_link_callbacks) <examine_strtab>: Take only a
symstrtab argument.
<ctf_new_symbol>: New.
<ctf_new_dynsym>: Likewise.
* ctf-api.h (struct ctf_link_sym) <st_symidx>: New.
<st_nameidx>: Likewise.
<st_nameidx_set>: Likewise.
(ctf_link_iter_symbol_f): Removed.
(ctf_link_shuffle_syms): Remove most parameters, just takes a
ctf_dict_t now.
(ctf_link_add_linker_symbol): New, split from
ctf_link_shuffle_syms.
* ctf.h (CTF_F_DYNSTR): New.
(CTF_F_MAX): Adjust.
ld/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* ldelfgen.c (struct ctf_strsym_iter_cb_arg): Rename to...
(struct ctf_strtab_iter_cb_arg): ... this, changing fields:
<syms>: Remove.
<symcount>: Remove.
<symstrtab>: Rename to...
<strtab>: ... this.
(ldelf_ctf_strtab_iter_cb): Adjust.
(ldelf_ctf_symbols_iter_cb): Remove.
(ldelf_new_dynsym_for_ctf): New, tell libctf about a single
symbol.
(ldelf_examine_strtab_for_ctf): Rename to...
(ldelf_acquire_strings_for_ctf): ... this, only doing the strtab
portion and not symbols.
* ldelfgen.h: Adjust declarations accordingly.
* ldemul.c (ldemul_examine_strtab_for_ctf): Rename to...
(ldemul_acquire_strings_for_ctf): ... this.
(ldemul_new_dynsym_for_ctf): New.
* ldemul.h: Adjust declarations accordingly.
* ldlang.c (ldlang_ctf_apply_strsym): Rename to...
(ldlang_ctf_acquire_strings): ... this.
(ldlang_ctf_new_dynsym): New.
(lang_write_ctf): Call ldemul_new_dynsym_for_ctf with NULL to do
the actual symbol shuffle.
* ldlang.h (struct elf_strtab_hash): Adjust accordingly.
* ldmain.c (bfd_link_callbacks): Wire up new/renamed callbacks.
libctf/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* ctf-link.c (ctf_link_shuffle_syms): Adjust.
(ctf_link_add_linker_symbol): New, unimplemented stub.
* libctf.ver: Add it.
* ctf-create.c (ctf_serialize): Set CTF_F_DYNSTR on newly-serialized
dicts.
* ctf-open-bfd.c (ctf_bfdopen_ctfsect): Check for the flag: open the
symtab/strtab if not present, dynsym/dynstr otherwise.
* ctf-archive.c (ctf_arc_bufpreamble): New, get the preamble from
some arbitrary member of a CTF archive.
* ctf-impl.h (ctf_arc_bufpreamble): Declare it.
The naming of the ctf_file_t type in libctf is a historical curiosity.
Back in the Solaris days, CTF dictionaries were originally generated as
a separate file and then (sometimes) merged into objects: hence the
datatype was named ctf_file_t, and known as a "CTF file". Nowadays, raw
CTF is essentially never written to a file on its own, and the datatype
changed name to a "CTF dictionary" years ago. So the term "CTF file"
refers to something that is never a file! This is at best confusing.
The type has also historically been known as a 'CTF container", which is
even more confusing now that we have CTF archives which are *also* a
sort of container (they contain CTF dictionaries), but which are never
referred to as containers in the source code.
So fix this by completing the renaming, renaming ctf_file_t to
ctf_dict_t throughout, and renaming those few functions that refer to
CTF files by name (keeping compatibility aliases) to refer to dicts
instead. Old users who still refer to ctf_file_t will see (harmless)
pointer-compatibility warnings at compile time, but the ABI is unchanged
(since C doesn't mangle names, and ctf_file_t was always an opaque type)
and things will still compile fine as long as -Werror is not specified.
All references to CTF containers and CTF files in the source code are
fixed to refer to CTF dicts instead.
Further (smaller) renamings of annoyingly-named functions to come, as
part of the process of souping up queries across whole archives at once
(needed for the function info and data object sections).
binutils/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* objdump.c (dump_ctf_errs): Rename ctf_file_t to ctf_dict_t.
(dump_ctf_archive_member): Likewise.
(dump_ctf): Likewise. Use ctf_dict_close, not ctf_file_close.
* readelf.c (dump_ctf_errs): Rename ctf_file_t to ctf_dict_t.
(dump_ctf_archive_member): Likewise.
(dump_section_as_ctf): Likewise. Use ctf_dict_close, not
ctf_file_close.
gdb/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* ctfread.c: Change uses of ctf_file_t to ctf_dict_t.
(ctf_fp_info::~ctf_fp_info): Call ctf_dict_close, not ctf_file_close.
include/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* ctf-api.h (ctf_file_t): Rename to...
(ctf_dict_t): ... this. Keep ctf_file_t around for compatibility.
(struct ctf_file): Likewise rename to...
(struct ctf_dict): ... this.
(ctf_file_close): Rename to...
(ctf_dict_close): ... this, keeping compatibility function.
(ctf_parent_file): Rename to...
(ctf_parent_dict): ... this, keeping compatibility function.
All callers adjusted.
* ctf.h: Rename references to ctf_file_t to ctf_dict_t.
(struct ctf_archive) <ctfa_nfiles>: Rename to...
<ctfa_ndicts>: ... this.
ld/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* ldlang.c (ctf_output): This is a ctf_dict_t now.
(lang_ctf_errs_warnings): Rename ctf_file_t to ctf_dict_t.
(ldlang_open_ctf): Adjust comment.
(lang_merge_ctf): Use ctf_dict_close, not ctf_file_close.
* ldelfgen.h (ldelf_examine_strtab_for_ctf): Rename ctf_file_t to
ctf_dict_t. Change opaque declaration accordingly.
* ldelfgen.c (ldelf_examine_strtab_for_ctf): Adjust.
* ldemul.h (examine_strtab_for_ctf): Likewise.
(ldemul_examine_strtab_for_ctf): Likewise.
* ldeuml.c (ldemul_examine_strtab_for_ctf): Likewise.
libctf/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* ctf-impl.h: Rename ctf_file_t to ctf_dict_t: all declarations
adjusted.
(ctf_fileops): Rename to...
(ctf_dictops): ... this.
(ctf_dedup_t) <cd_id_to_file_t>: Rename to...
<cd_id_to_dict_t>: ... this.
(ctf_file_t): Fix outdated comment.
<ctf_fileops>: Rename to...
<ctf_dictops>: ... this.
(struct ctf_archive_internal) <ctfi_file>: Rename to...
<ctfi_dict>: ... this.
* ctf-archive.c: Rename ctf_file_t to ctf_dict_t.
Rename ctf_archive.ctfa_nfiles to ctfa_ndicts.
Rename ctf_file_close to ctf_dict_close. All users adjusted.
* ctf-create.c: Likewise. Refer to CTF dicts, not CTF containers.
(ctf_bundle_t) <ctb_file>: Rename to...
<ctb_dict): ... this.
* ctf-decl.c: Rename ctf_file_t to ctf_dict_t.
* ctf-dedup.c: Likewise. Rename ctf_file_close to
ctf_dict_close. Refer to CTF dicts, not CTF containers.
* ctf-dump.c: Likewise.
* ctf-error.c: Likewise.
* ctf-hash.c: Likewise.
* ctf-inlines.h: Likewise.
* ctf-labels.c: Likewise.
* ctf-link.c: Likewise.
* ctf-lookup.c: Likewise.
* ctf-open-bfd.c: Likewise.
* ctf-string.c: Likewise.
* ctf-subr.c: Likewise.
* ctf-types.c: Likewise.
* ctf-util.c: Likewise.
* ctf-open.c: Likewise.
(ctf_file_close): Rename to...
(ctf_dict_close): ...this.
(ctf_file_close): New trivial wrapper around ctf_dict_close, for
compatibility.
(ctf_parent_file): Rename to...
(ctf_parent_dict): ... this.
(ctf_parent_file): New trivial wrapper around ctf_parent_dict, for
compatibility.
* libctf.ver: Add ctf_dict_close and ctf_parent_dict.
I wanted to write a linker script like this:
PROVIDE(mem_origin = 0x1000);
PROVIDE(mem_length = 0x1000);
MEMORY
{
REGION : ORIGIN = mem_origin, LENGTH = mem_length
}
....
Then when I link using this script I can optionally supply:
--defsym=mem_origin=..... --defsym=mem_length=....
to override the defaults.
And though passing `--defsym' does work, if I remove the use of
`--defsym' and just rely on the defaults I get an error:
ld-new: invalid origin for memory region REGION
Interestingly, if I make the above error non-fatal and dump a linker
map file I see that (a) REGION has origin 0x0, and length 0xffff...,
and (b) the symbol from the PROVIDE is provided.
An examination of ldlang.c:lang_process shows us what the issue is,
the origin and length of all memory regions are set as a result of a
single call to lang_do_memory_regions, this call is done after calling
open_input_bfds.
During the open_input_bfds call provide statements can be converted to
provided statements if we know that the assigned symbol is needed, but
for symbols that are only used in the memory regions we are unaware
that we need these symbols.
What I propose in this patch is to make two calls to
lang_do_memory_regions, in the first call we process the expressions
for the origin and length fields of each region, however, errors,
especially undefined symbols, will be ignored. The origin and length
values are not updated. However, by evaluating the expressions any
symbols we need will be added to the symbol table.
Now when we call open_input_bfds, when we process the provide
statements, we will see that the assigned symbol is needed add its new
value to the symbol table.
Finally we reach the original call to lang_do_memory_regions, in
this (now second) call we again process the expressions, and this time
update the origin and length values. Any errors encountered now are
reported to the user.
ld/ChangeLog:
* ldlang.c (lang_process): Add extra call to
lang_do_memory_regions, and pass parameter.
(lang_do_memory_regions): Add parameter, only define origin and
length when requested. Reindent.
* testsuite/ld-scripts/provide-10.d: New file.
* testsuite/ld-scripts/provide-10.map: New file.
* testsuite/ld-scripts/provide-11.d: New file.
* testsuite/ld-scripts/provide-11.map: New file.
* testsuite/ld-scripts/provide-12.d: New file.
* testsuite/ld-scripts/provide-12.map: New file.
* testsuite/ld-scripts/provide-9.d: New file.
* testsuite/ld-scripts/provide-9.map: New file.
* testsuite/ld-scripts/provide-9.t: New file.
This commit follows on from the earlier commit "libctf, ld, binutils:
add textual error/warning reporting for libctf" and converts every error
in libctf that was reported using ctf_dprintf to use ctf_err_warn
instead, gettextizing them in the process, using N_() where necessary to
avoid doing gettext calls unless an error message is actually generated,
and rephrasing some error messages for ease of translation.
This requires a slight change in the ctf_errwarning_next API: this API
is public but has not been in a release yet, so can still change freely.
The problem is that many errors are emitted at open time (whether
opening of a CTF dict, or opening of a CTF archive): the former of these
throws away its incompletely-initialized ctf_file_t rather than return
it, and the latter has no ctf_file_t at all. So errors and warnings
emitted at open time cannot be stored in the ctf_file_t, and have to go
elsewhere.
We put them in a static local in ctf-subr.c (which is not very
thread-safe: a later commit will improve things here): ctf_err_warn with
a NULL fp adds to this list, and the public interface
ctf_errwarning_next with a NULL fp retrieves from it.
We need a slight exception from the usual iterator rules in this case:
with a NULL fp, there is nowhere to store the ECTF_NEXT_END "error"
which signifies the end of iteration, so we add a new err parameter to
ctf_errwarning_next which is used to report such iteration-related
errors. (If an fp is provided -- i.e., if not reporting open errors --
this is optional, but even if it's optional it's still an API change.
This is actually useful from a usability POV as well, since
ctf_errwarning_next is usually called when there's been an error, so
overwriting the error code with ECTF_NEXT_END is not very helpful!
So, unusually, ctf_errwarning_next now uses the passed fp for its
error code *only* if no errp pointer is passed in, and leaves it
untouched otherwise.)
ld, objdump and readelf are adapted to call ctf_errwarning_next with a
NULL fp to report open errors where appropriate.
The ctf_err_warn API also has to change, gaining a new error-number
parameter which is used to add the error message corresponding to that
error number into the debug stream when LIBCTF_DEBUG is enabled:
changing this API is easy at this point since we are already touching
all existing calls to gettextize them. We need this because the debug
stream should contain the errno's message, but the error reported in the
error/warning stream should *not*, because the caller will probably
report it themselves at failure time regardless, and reporting it in
every error message that leads up to it leads to a ridiculous chattering
on failure, which is likely to end up as ridiculous chattering on stderr
(trimmed a bit):
CTF error: `ld/testsuite/ld-ctf/A.c (0): lookup failure for type 3: flags 1: The parent CTF dictionary is unavailable'
CTF error: `ld/testsuite/ld-ctf/A.c (0): struct/union member type hashing error during type hashing for type 80000001, kind 6: The parent CTF dictionary is unavailable'
CTF error: `deduplicating link variable emission failed for ld/testsuite/ld-ctf/A.c: The parent CTF dictionary is unavailable'
ld/.libs/lt-ld-new: warning: CTF linking failed; output will have no CTF section: `The parent CTF dictionary is unavailable'
We only need to be told that the parent CTF dictionary is unavailable
*once*, not over and over again!
errmsgs are still emitted on warning generation, because warnings do not
usually lead to a failure propagated up to the caller and reported
there.
Debug-stream messages are not translated. If translation is turned on,
there will be a mixture of English and translated messages in the debug
stream, but rather that than burden the translators with debug-only
output.
binutils/ChangeLog
2020-08-27 Nick Alcock <nick.alcock@oracle.com>
* objdump.c (dump_ctf_archive_member): Move error-
reporting...
(dump_ctf_errs): ... into this separate function.
(dump_ctf): Call it on open errors.
* readelf.c (dump_ctf_archive_member): Move error-
reporting...
(dump_ctf_errs): ... into this separate function. Support
calls with NULL fp. Adjust for new err parameter to
ctf_errwarning_next.
(dump_section_as_ctf): Call it on open errors.
include/ChangeLog
2020-08-27 Nick Alcock <nick.alcock@oracle.com>
* ctf-api.h (ctf_errwarning_next): New err parameter.
ld/ChangeLog
2020-08-27 Nick Alcock <nick.alcock@oracle.com>
* ldlang.c (lang_ctf_errs_warnings): Support calls with NULL fp.
Adjust for new err parameter to ctf_errwarning_next. Only
check for assertion failures when fp is non-NULL.
(ldlang_open_ctf): Call it on open errors.
* testsuite/ld-ctf/ctf.exp: Always use the C locale to avoid
breaking the diags tests.
libctf/ChangeLog
2020-08-27 Nick Alcock <nick.alcock@oracle.com>
* ctf-subr.c (open_errors): New list.
(ctf_err_warn): Calls with NULL fp append to open_errors. Add err
parameter, and use it to decorate the debug stream with errmsgs.
(ctf_err_warn_to_open): Splice errors from a CTF dict into the
open_errors.
(ctf_errwarning_next): Calls with NULL fp report from open_errors.
New err param to report iteration errors (including end-of-iteration)
when fp is NULL.
(ctf_assert_fail_internal): Adjust ctf_err_warn call for new err
parameter: gettextize.
* ctf-impl.h (ctfo_get_vbytes): Add ctf_file_t parameter.
(LCTF_VBYTES): Adjust.
(ctf_err_warn_to_open): New.
(ctf_err_warn): Adjust.
(ctf_bundle): Used in only one place: move...
* ctf-create.c: ... here.
(enumcmp): Use ctf_err_warn, not ctf_dprintf, passing the err number
down as needed. Don't emit the errmsg. Gettextize.
(membcmp): Likewise.
(ctf_add_type_internal): Likewise.
(ctf_write_mem): Likewise.
(ctf_compress_write): Likewise. Report errors writing the header or
body.
(ctf_write): Likewise.
* ctf-archive.c (ctf_arc_write_fd): Use ctf_err_warn, not
ctf_dprintf, and gettextize, as above.
(ctf_arc_write): Likewise.
(ctf_arc_bufopen): Likewise.
(ctf_arc_open_internal): Likewise.
* ctf-labels.c (ctf_label_iter): Likewise.
* ctf-open-bfd.c (ctf_bfdclose): Likewise.
(ctf_bfdopen): Likewise.
(ctf_bfdopen_ctfsect): Likewise.
(ctf_fdopen): Likewise.
* ctf-string.c (ctf_str_write_strtab): Likewise.
* ctf-types.c (ctf_type_resolve): Likewise.
* ctf-open.c (get_vbytes_common): Likewise. Pass down the ctf dict.
(get_vbytes_v1): Pass down the ctf dict.
(get_vbytes_v2): Likewise.
(flip_ctf): Likewise.
(flip_types): Likewise. Use ctf_err_warn, not ctf_dprintf, and
gettextize, as above.
(upgrade_types_v1): Adjust calls.
(init_types): Use ctf_err_warn, not ctf_dprintf, as above.
(ctf_bufopen_internal): Likewise. Adjust calls. Transplant errors
emitted into individual dicts into the open errors if this turns
out to be a failed open in the end.
* ctf-dump.c (ctf_dump_format_type): Adjust ctf_err_warn for new err
argument. Gettextize. Don't emit the errmsg.
(ctf_dump_funcs): Likewise. Collapse err label into its only case.
(ctf_dump_type): Likewise.
* ctf-link.c (ctf_create_per_cu): Adjust ctf_err_warn for new err
argument. Gettextize. Don't emit the errmsg.
(ctf_link_one_type): Likewise.
(ctf_link_lazy_open): Likewise.
(ctf_link_one_input_archive): Likewise.
(ctf_link_deduplicating_count_inputs): Likewise.
(ctf_link_deduplicating_open_inputs): Likewise.
(ctf_link_deduplicating_close_inputs): Likewise.
(ctf_link_deduplicating): Likewise.
(ctf_link): Likewise.
(ctf_link_deduplicating_per_cu): Likewise. Add some missed
ctf_set_errnos to obscure error cases.
* ctf-dedup.c (ctf_dedup_rhash_type): Adjust ctf_err_warn for new
err argument. Gettextize. Don't emit the errmsg.
(ctf_dedup_populate_mappings): Likewise.
(ctf_dedup_detect_name_ambiguity): Likewise.
(ctf_dedup_init): Likewise.
(ctf_dedup_multiple_input_dicts): Likewise.
(ctf_dedup_conflictify_unshared): Likewise.
(ctf_dedup): Likewise.
(ctf_dedup_rwalk_one_output_mapping): Likewise.
(ctf_dedup_id_to_target): Likewise.
(ctf_dedup_emit_type): Likewise.
(ctf_dedup_emit_struct_members): Likewise.
(ctf_dedup_populate_type_mapping): Likewise.
(ctf_dedup_populate_type_mappings): Likewise.
(ctf_dedup_emit): Likewise.
(ctf_dedup_hash_type): Likewise. Fix a bit of messed-up error
status setting.
(ctf_dedup_rwalk_one_output_mapping): Likewise. Don't hide
unknown-type-kind messages (which signify file corruption).
* ldlang.c (lang_check): Don't complain about relocs or merge
attributes from --just-symbols input.
* testsuite/ld-misc/just-symbols.exp: Just dump .data section.
Don't run test on a number of targets.
The trick we use to prevent ld doing as it does for almost all other
sections and copying the input CTF section into the output has recently
broken, causing output to be produced with a valid CTF section followed
by massive numbers of CTF sections, one per .ctf in the input (minus
one, for the one that was filled out by ctf_link). Their size is being
forcibly set to zero, but they're still present, wasting space and
looking ridiculous.
This is not right:
ld/ld-new :
section size addr
.interp 28 4194984
[...]
.bss 21840 6788544
.comment 92 0
.ctf 87242 0
.ctf 0 0
.ctf 0 0
[snip 131 more empty sections]
.gnu.build.attributes 7704 6818576
.debug_aranges 6592 0
.debug_info 4488859 0
.debug_abbrev 150099 0
.debug_line 796759 0
.debug_str 237926 0
.debug_loc 2247302 0
.debug_ranges 237920 0
Total 10865285
The fix is to exclude these unwanted input sections from being present
in the output. We tried this before and it broke things, because if you
exclude all the .ctf sections there isn't going to be one in the output
so there is nowhere to put the deduplicated CTF. The solution to that is
really simple: set SEC_EXCLUDE on *all but one* CTF section. We don't
care which one (they're all the same once their size has been zeroed),
so just pick the first we see.
ld/
* ldlang.c (ldlang_open_ctf): Set SEC_EXCLUDE on all but the
first input .ctf section.
libctf recently changed to make it possible to not emit the CTF
variables section. Make this the default for ld: the variables section
is a simple name -> type mapping, and the names can be quite voluminous.
Nothing in the variables section appears in the symbol table, by
definition, so GDB cannot make use of them: special-purpose projects
that implement their own analogues of symbol table lookup can do so, but
they'll need to tell the linker to emit the variables section after all.
The new --ctf-variables option does this.
The --ctf-share-types option (valid values "share-duplicated" and
"share-unconflicted") allow the caller to specify the CTF link mode.
Most users will want share-duplicated, since it allows for more
convenient debugging: but very large projects composed of many decoupled
components may want to use share-unconflicted mode, which places types
that appear in only one TU into per-TU dicts. (They may also want to
relink the CTF using the ctf_link API and cu-mapping, to make their
"components" larger than a single TU. Right now the linker does not
expose the CU-mapping machinery. Perhaps it should in future to make
this use case easier.)
For now, giving the linker the ability to emit share-duplicated CTF lets
us add testcases for that mode to the testsuite.
ld/
* ldlex.h (option_values) <OPTION_CTF_VARIABLES,
OPTION_NO_CTF_VARIABLES, OPTION_CTF_SHARE_TYPES>: New.
* ld.h (ld_config_type) <ctf_variables, ctf_share_duplicated>:
New fields.
* ldlang.c (lang_merge_ctf): Use them.
* lexsup.c (ld_options): Add ctf-variables, no-ctf-variables,
ctf-share-types.
(parse_args) <OPTION_CTF_VARIABLES, OPTION_NO_CTF_VARIABLES,
OPTION_CTF_SHARE_TYPES>: New cases.
* ld.texi: Document new options.
* NEWS: Likewise.
ld/
* ldlang.c (lang_merge_ctf): Turn errors into warnings.
Fix a comment typo.
(lang_write_ctf): Turn an error into a warning.
(ldlang_open_ctf): Reformat warnings. Fix printing file names.
Reviewed-by: Nick Alcock <nick.alcock@oracle.com>
This commit adds a long-missing piece of infrastructure to libctf: the
ability to report errors and warnings using all the power of printf,
rather than being restricted to one errno value. Internally, libctf
calls ctf_err_warn() to add errors and warnings to a list: a new
iterator ctf_errwarning_next() then consumes this list one by one and
hands it to the caller, which can free it. New errors and warnings are
added until the list is consumed by the caller or the ctf_file_t is
closed, so you can dump them at intervals. The caller can of course
choose to print only those warnings it wants. (I am not sure whether we
want objdump, readelf or ld to print warnings or not: right now I'm
printing them, but maybe we only want to print errors? This entirely
depends on whether warnings are voluminous things describing e.g. the
inability to emit single types because of name clashes or something.
There are no users of this infrastructure yet, so it's hard to say.)
There is no internationalization here yet, but this at least adds a
place where internationalization can be added, to one of
ctf_errwarning_next or ctf_err_warn.
We also provide a new ctf_assert() function which uses this
infrastructure to provide non-fatal assertion failures while emitting an
assert-like string to the caller: to save space and avoid needlessly
duplicating unchanging strings, the assertion test is inlined but the
print-things-out failure case is not. All assertions in libctf will be
converted to use this machinery in future commits and propagate
assertion-failure errors up, so that the linker in particular cannot be
killed by libctf assertion failures when it could perfectly well just
print warnings and drop the CTF section.
include/
* ctf-api.h (ECTF_INTERNAL): Adjust error text.
(ctf_errwarning_next): New.
libctf/
* ctf-impl.h (ctf_assert): New.
(ctf_err_warning_t): Likewise.
(ctf_file_t) <ctf_errs_warnings>: Likewise.
(ctf_err_warn): New prototype.
(ctf_assert_fail_internal): Likewise.
* ctf-inlines.h (ctf_assert_internal): Likewise.
* ctf-open.c (ctf_file_close): Free ctf_errs_warnings.
* ctf-create.c (ctf_serialize): Copy it on serialization.
* ctf-subr.c (ctf_err_warn): New, add an error/warning.
(ctf_errwarning_next): New iterator, free and pass back
errors/warnings in succession.
* libctf.ver (ctf_errwarning_next): Add.
ld/
* ldlang.c (lang_ctf_errs_warnings): New, print CTF errors
and warnings. Assert when libctf asserts.
(lang_merge_ctf): Call it.
(land_write_ctf): Likewise.
binutils/
* objdump.c (ctf_archive_member): Print CTF errors and warnings.
* readelf.c (dump_ctf_archive_member): Likewise.
We change the previous definition in the IR object to undefweak only
after all LTO symbols have been read.
include/
PR ld/26262
PR ld/26267
* bfdlink.h (bfd_link_info): Add lto_all_symbols_read.
ld/
PR ld/26262
PR ld/26267
* ldlang.c (lang_process): Set lto_all_symbols_read after all
LTO IR symbols have been read.
* plugin.c (plugin_notice): Override the IR definition only if
all LTO IR symbols have been read or the new definition is
non-weak and the the IR definition is weak
* testsuite/ld-plugin/lto.exp: Run PR ld/26262 and ld/26267
tests.
* testsuite/ld-plugin/pr26262a.c: New file.
* testsuite/ld-plugin/pr26262b.c: Likewise.
* testsuite/ld-plugin/pr26262c.c: Likewise.
* testsuite/ld-plugin/pr26267.err: Likewise.
* testsuite/ld-plugin/pr26267a.c: Likewise.
* testsuite/ld-plugin/pr26267b.c: Likewise.
* testsuite/ld-plugin/pr26267c.c: Likewise.
