Now that filtered and unfiltered output can be treated identically, we
can unify the printf family of functions. This is done under the name
"gdb_printf". Most of this patch was written by script.
New in this version:
- Rebase on master, fix a few more issues that appeared.
python-internal.h contains a number of macros that helped make the code
work with both Python 2 and 3. Remove them and adjust the code to use
the Python 3 functions.
Change-Id: I99a3d80067fb2d65de4f69f6473ba6ffd16efb2d
New in this version:
- Add a PY_MAJOR_VERSION check in configure.ac / AC_TRY_LIBPYTHON. If
the user passes --with-python=python2, this will cause a configure
failure saying that GDB only supports Python 3.
Support for Python 2 is a maintenance burden for any patches touching
Python support. Among others, the differences between Python 2 and 3
string and integer types are subtle. It requires a lot of effort and
thinking to get something that behaves correctly on both. And that's if
the author and reviewer of the patch even remember to test with Python
2.
See this thread for an example:
https://sourceware.org/pipermail/gdb-patches/2021-December/184260.html
So, remove Python 2 support. Update the documentation to state that GDB
can be built against Python 3 (as opposed to Python 2 or 3).
Update all the spots that use:
- sys.version_info
- IS_PY3K
- PY_MAJOR_VERSION
- gdb_py_is_py3k
... to only keep the Python 3 portions and drop the use of some
now-removed compatibility macros.
I did not update the configure script more than just removing the
explicit references to Python 2. We could maybe do more there, like
check the Python version and reject it if that version is not
supported. Otherwise (with this patch), things will only fail at
compile time, so it won't really be clear to the user that they are
trying to use an unsupported Python version. But I'm a bit lost in the
configure code that checks for Python, so I kept that for later.
Change-Id: I75b0f79c148afbe3c07ac664cfa9cade052c0c62
Add a new function, gdb.format_address, which is a wrapper around
GDB's print_address function.
This method takes an address, and returns a string with the format:
ADDRESS <SYMBOL+OFFSET>
Where, ADDRESS is the original address, formatted as hexadecimal,
SYMBOL is a symbol with an address lower than ADDRESS, and OFFSET is
the offset from SYMBOL to ADDRESS in decimal.
If there's no SYMBOL suitably close to ADDRESS then the
<SYMBOL+OFFSET> part is not included.
This is useful if a user wants to write a Python script that
pretty-prints addresses, the user no longer needs to do manual symbol
lookup, or worry about correctly formatting addresses.
Additionally, there are some settings that effect how GDB picks
SYMBOL, and whether the file name and line number should be included
with the SYMBOL name, the gdb.format_address function ensures that the
users Python script also benefits from these settings.
The gdb.format_address by default selects SYMBOL from the current
inferiors program space, and address is formatted using the
architecture for the current inferior. However, a user can also
explicitly pass a program space and architecture like this:
gdb.format_address(ADDRESS, PROGRAM_SPACE, ARCHITECTURE)
In order to format an address for a different inferior.
Notes on the implementation:
In py-arch.c I extended arch_object_to_gdbarch to add an assertion for
the type of the PyObject being worked on. Prior to this commit all
uses of arch_object_to_gdbarch were guaranteed to pass this function a
gdb.Architecture object, but, with this commit, this might not be the
case.
So, with this commit I've made it a requirement that the PyObject be a
gdb.Architecture, and this is checked with the assert. And in order
that callers from other files can check if they have a
gdb.Architecture object, I've added the new function
gdbpy_is_architecture.
In py-progspace.c I've added two new function, the first
progspace_object_to_program_space, converts a PyObject of type
gdb.Progspace to the associated program_space pointer, and
gdbpy_is_progspace checks if a PyObject is a gdb.Progspace or not.
The motivation for this patch is the fact that py-micmd.c doesn't build
with Python 2, due to PyDict_GetItemWithError being a Python 3-only
function:
CXX python/py-micmd.o
/home/smarchi/src/binutils-gdb/gdb/python/py-micmd.c: In function ‘int micmdpy_uninstall_command(micmdpy_object*)’:
/home/smarchi/src/binutils-gdb/gdb/python/py-micmd.c:430:20: error: ‘PyDict_GetItemWithError’ was not declared in this scope; did you mean ‘PyDict_GetItemString’?
430 | PyObject *curr = PyDict_GetItemWithError (mi_cmd_dict.get (),
| ^~~~~~~~~~~~~~~~~~~~~~~
| PyDict_GetItemString
A first solution to fix this would be to try to replace
PyDict_GetItemWithError equivalent Python 2 code. But I looked at why
we are doing this in the first place: it is to maintain the
`gdb._mi_commands` Python dictionary that we use as a `name ->
gdb.MICommand object` map. Since the `gdb._mi_commands` dictionary is
never actually used in Python, it seems like a lot of trouble to use a
Python object for this.
My first idea was to replace it with a C++ map
(std::unordered_map<std::string, gdbpy_ref<micmdpy_object>>). While
implementing this, I realized we don't really need this map at all. The
mi_command_py objects registered in the main MI command table can own
their backing micmdpy_object (that's a gdb.MICommand, but seen from the
C++ code). To know whether an mi_command is an mi_command_py, we can
use a dynamic cast. Since there's one less data structure to maintain,
there are less chances of messing things up.
- Change mi_command_py::m_pyobj to a gdbpy_ref, the mi_command_py is
now what keeps the MICommand alive.
- Set micmdpy_object::mi_command in the constructor of mi_command_py.
If mi_command_py manages setting/clearing that field in
swap_python_object, I think it makes sense that it also takes care of
setting it initially.
- Move a bunch of checks from micmdpy_install_command to
swap_python_object, and make them gdb_asserts.
- In micmdpy_install_command, start by doing an mi_cmd_lookup. This is
needed to know whether there's a Python MI command already registered
with that name. But we can already tell if there's a non-Python
command registered with that name. Return an error if that happens,
rather than waiting for insert_mi_cmd_entry to fail. Change the
error message to "name is already in use" rather than "may already be
in use", since it's more precise.
I asked Andrew about the original intent of using a Python dictionary
object to hold the command objects. The reason was to make sure the
objects get destroyed when the Python runtime gets finalized, not later.
Holding the objects in global C++ data structures and not doing anything
more means that the held Python objects will be decref'd after the
Python interpreter has been finalized. That's not desirable. I tried
it and it indeed segfaults.
Handle this by adding a gdbpy_finalize_micommands function called in
finalize_python. This is the mirror of gdbpy_initialize_micommands
called in do_start_initialization. In there, delete all Python MI
commands. I think it makes sense to do it this way: if it was somehow
possible to unload Python support from GDB in the middle of a session
we'd want to unregister any Python MI command. Otherwise, these MI
commands would be backed with a stale PyObject or simply nothing.
Delete tests that were related to `gdb._mi_commands`.
Co-Authored-By: Andrew Burgess <aburgess@redhat.com>
Change-Id: I060d5ebc7a096c67487998a8a4ca1e8e56f12cd3
This commit allows a user to create custom MI commands using Python
similarly to what is possible for Python CLI commands.
A new subclass of mi_command is defined for Python MI commands,
mi_command_py. A new file, gdb/python/py-micmd.c contains the logic
for Python MI commands.
This commit is based on work linked too from this mailing list thread:
https://sourceware.org/pipermail/gdb/2021-November/049774.html
Which has also been previously posted to the mailing list here:
https://sourceware.org/pipermail/gdb-patches/2019-May/158010.html
And was recently reposted here:
https://sourceware.org/pipermail/gdb-patches/2022-January/185190.html
The version in this patch takes some core code from the previously
posted patches, but also has some significant differences, especially
after the feedback given here:
https://sourceware.org/pipermail/gdb-patches/2022-February/185767.html
A new MI command can be implemented in Python like this:
class echo_args(gdb.MICommand):
def invoke(self, args):
return { 'args': args }
echo_args("-echo-args")
The 'args' parameter (to the invoke method) is a list
containing (almost) all command line arguments passed to the MI
command (--thread and --frame are handled before the Python code is
called, and removed from the args list). This list can be empty if
the MI command was passed no arguments.
When used within gdb the above command produced output like this:
(gdb)
-echo-args a b c
^done,args=["a","b","c"]
(gdb)
The 'invoke' method of the new command must return a dictionary. The
keys of this dictionary are then used as the field names in the mi
command output (e.g. 'args' in the above).
The values of the result returned by invoke can be dictionaries,
lists, iterators, or an object that can be converted to a string.
These are processed recursively to create the mi output. And so, this
is valid:
class new_command(gdb.MICommand):
def invoke(self,args):
return { 'result_one': { 'abc': 123, 'def': 'Hello' },
'result_two': [ { 'a': 1, 'b': 2 },
{ 'c': 3, 'd': 4 } ] }
Which produces output like:
(gdb)
-new-command
^done,result_one={abc="123",def="Hello"},result_two=[{a="1",b="2"},{c="3",d="4"}]
(gdb)
I have required that the fields names used in mi result output must
match the regexp: "^[a-zA-Z][-_a-zA-Z0-9]*$" (without the quotes).
This restriction was never written down anywhere before, but seems
sensible to me, and we can always loosen this rule later if it proves
to be a problem. Much harder to try and add a restriction later, once
people are already using the API.
What follows are some details about how this implementation differs
from the original patch that was posted to the mailing list.
In this patch, I have changed how the lifetime of the Python
gdb.MICommand objects is managed. In the original patch, these object
were kept alive by an owned reference within the mi_command_py object.
As such, the Python object would not be deleted until the
mi_command_py object itself was deleted.
This caused a problem, the mi_command_py were held in the global mi
command table (in mi/mi-cmds.c), which, as a global, was not cleared
until program shutdown. By this point the Python interpreter has
already been shutdown. Attempting to delete the mi_command_py object
at this point was causing GDB to try and invoke Python code after
finalising the Python interpreter, and we would crash.
To work around this problem, the original patch added code in
python/python.c that would search the mi command table, and delete the
mi_command_py objects before the Python environment was finalised.
In contrast, in this patch, I have added a new global dictionary to
the gdb module, gdb._mi_commands. We already have several such global
data stores related to pretty printers, and frame unwinders.
The MICommand objects are placed into the new gdb.mi_commands
dictionary, and it is this reference that keeps the objects alive.
When GDB's Python interpreter is shut down gdb._mi_commands is deleted,
and any MICommand objects within it are deleted at this point.
This change avoids having to make the mi_cmd_table global, and walk
over it from within GDB's python related code.
This patch handles command redefinition entirely within GDB's python
code, though this does impose one small restriction which is not
present in the original code (detailed below), I don't think this is a
big issue. However, the original patch relied on being able to
finish executing the mi_command::do_invoke member function after the
mi_command object had been deleted. Though continuing to execute a
member function after an object is deleted is well defined, it is
also (IMHO) risky, its too easy for someone to later add a use of the
object without realising that the object might sometimes, have been
deleted. The new patch avoids this issue.
The one restriction that is added to avoid this, is that an MICommand
object can't be reinitialised with a different command name, so:
(gdb) python cmd = MyMICommand("-abc")
(gdb) python cmd.__init__("-def")
can't reinitialize object with a different command name
This feels like a pretty weird edge case, and I'm happy to live with
this restriction.
I have also changed how the memory is managed for the command name.
In the most recently posted patch series, the command name is moved
into a subclass of mi_command, the python mi_command_py, which
inherits from mi_command is then free to use a smart pointer to manage
the memory for the name.
In this patch, I leave the mi_command class unchanged, and instead
hold the memory for the name within the Python object, as the lifetime
of the Python object always exceeds the c++ object stored in the
mi_cmd_table. This adds a little more complexity in py-micmd.c, but
leaves the mi_command class nice and simple.
Next, this patch adds some extra functionality, there's a
MICommand.name read-only attribute containing the name of the command,
and a read-write MICommand.installed attribute that can be used to
install (make the command available for use) and uninstall (remove the
command from the mi_cmd_table so it can't be used) the command. This
attribute will be automatically updated if a second command replaces
an earlier command.
This patch adds additional error handling, and makes more use the
gdbpy_handle_exception function.
Co-Authored-By: Jan Vrany <jan.vrany@labware.com>
This commit moves the two Python functions that are used for styling
into a new module, gdb.styling, there's then a small update in
python.c so GDB can find the functions in their new location.
The motivation for this change is purely to try and reduce the clutter
in the top-level gdb module, and encapsulate related functions into
modules. I did ponder documenting these functions as part of the
Python API, however, doing so would effectively "fix" the API, and I'm
still wondering if there's improvements that could be made, also, the
colorize function is only called in some cases now that GDB prefers
libsource-highlight, so it's not entirely sure how this would work as
part of a user facing API.
Still, despite these functions never having been part of a documented
API, it is possible that a user out there has overridden these to, in
some way, customize how GDB performs styling. Moving the function as
I propose in this patch could break things for that user, however,
fixing this breakage is trivial, and, as these functions were never
documented, I don't think we should be obliged to not break user code
that relies on them.
This commit adds styling support to the disassembler output, as such
two new commands are added to GDB:
set style disassembler enabled on|off
show style disassembler enabled
In this commit I make use of the Python Pygments package to provide
the styling. I did investigate making use of libsource-highlight,
however, I found the highlighting results to be inferior to those of
Pygments; only some mnemonics were highlighted, and highlighting of
register names such as r9d and r8d (on x86-64) was incorrect.
To enable disassembler highlighting via Pygments, I've added a new
extension language hook, which is then implemented for Python. This
hook is very similar to the existing hook for source code
colorization.
One possibly odd choice I made with the new hook is to pass a
gdb.Architecture through, even though this is currently unused. The
reason this argument is not used is that, currently, styling is
performed identically for all architectures.
However, even though the Python function used to perform styling of
disassembly output is not part of any documented API, I don't want
to close the door on a user overriding this function to provide
architecture specific styling. To do this, the user would inevitably
require access to the gdb.Architecture, and so I decided to add this
field now.
The styling is applied within gdb_disassembler::print_insn, to achieve
this, gdb_disassembler now writes its output into a temporary buffer,
styling is then applied to the contents of this buffer. Finally the
gdb_disassembler buffer is copied out to its final destination stream.
There's a new test to check that the disassembler output includes some
escape sequences, though I don't check for specific colours; the
precise colors will depend on which instructions are in the
disassembler output, and, I guess, how pygments is configured.
The only negative change with this commit is how we currently style
addresses in GDB.
Currently, when the disassembler wants to print an address, we call
back into GDB, and GDB prints the address value using the `address`
styling, and the symbol name using `function` styling. After this
commit, if pygments is used, then all disassembler styling is done
through pygments, and this include the address and symbol name parts
of the disassembler output.
I don't know how much of an issue this will be for people. There's
already some precedent for this in GDB when we look at source styling.
For example, function names in styled source listings are not styled
using the `function` style, but instead, either GNU Source Highlight,
or pygments gets to decide how the function name should be styled.
If the Python pygments library is not present then GDB will continue
to behave as it always has, the disassembler output is mostly
unstyled, but the address and symbols are styled using the `address`
and `function` styles, as they are today.
However, if the user does `set style disassembler enabled off`, then
all disassembler styling is switched off. This obviously covers the
use of pygments, but also includes the minimal styling done by GDB
when pygments is not available.
This commit adds support for source files that contain non utf-8
characters when performing source styling using the Python pygments
package. This does not change the behaviour of GDB when the GNU
Source Highlight library is used.
For the following problem description, assume that either GDB is built
without GNU Source Highlight support, of that this has been disabled
using 'maintenance set gnu-source-highlight enabled off'.
The initial problem reported was that a source file containing non
utf-8 characters would cause GDB to print a Python exception, and then
display the source without styling, e.g.:
Python Exception <class 'UnicodeDecodeError'>: 'utf-8' codec can't decode byte 0xc0 in position 142: invalid start byte
/* Source code here, without styling... */
Further, as the user steps through different source files, each time
the problematic source file was evicted from the source cache, and
then later reloaded, the exception would be printed again.
Finally, this problem is only present when using Python 3, this issue
is not present for Python 2.
What makes this especially frustrating is that GDB can clearly print
the source file contents, they're right there... If we disable
styling completely, or make use of the GNU Source Highlight library,
then everything is fine. So why is there an error when we try to
apply styling using Python?
The problem is the use of PyString_FromString (which is an alias for
PyUnicode_FromString in Python 3), this function converts a C string
into a either a Unicode object (Py3) or a str object (Py2). For
Python 2 there is no unicode encoding performed during this function
call, but for Python 3 the input is assumed to be a uft-8 encoding
string for the purpose of the conversion. And here of course, is the
problem, if the source file contains non utf-8 characters, then it
should not be treated as utf-8, but that's what we do, and that's why
we get an error.
My first thought when looking at this was to spot when the
PyString_FromString call failed with a UnicodeDecodeError and silently
ignore the error. This would mean that GDB would print the source
without styling, but would also avoid the annoying exception message.
However, I also make use of `pygmentize`, a command line wrapper
around the Python pygments module, which I use to apply syntax
highlighting in the output of `less`. And this command line wrapper
is quite happy to syntax highlight my source file that contains non
utf-8 characters, so it feels like the problem should be solvable.
It turns out that inside the pygments module there is already support
for guessing the encoding of the incoming file content, if the
incoming content is not already a Unicode string. This is what
happens for Python 2 where the incoming content is of `str` type.
We could try and make GDB smarter when it comes to converting C
strings into Python Unicode objects; this would probably require us to
just try a couple of different encoding schemes rather than just
giving up after utf-8.
However, I figure, why bother? The pygments module already does this
for us, and the colorize API is not part of the documented external
API of GDB. So, why not just change the colorize API, instead of the
content being a Unicode string (for Python 3), lets just make the
content be a bytes object. The pygments module can then take
responsibility for guessing the encoding.
So, currently, the colorize API receives a unicode object, and returns
a unicode object. I propose that the colorize API receive a bytes
object, and return a bytes object.
Add a new function gdb.history_count to the Python api, this function
returns an integer, the number of items in GDB's value history.
This is useful if you want to pull items from the history by their
absolute number, for example, if you wanted to show a complete history
list. Previously we could figure out how many items are in the
history list by trying to fetch the items, and then catching the
exception when the item is not available, but having this function
seems nicer.
Currently, gdb's Python layer captures the current architecture and
language when "entering" Python code. This has some undesirable
effects, and so this series changes how this is handled.
First, there is code like this:
gdbpy_enter enter_py (python_gdbarch, python_language);
This is incorrect, because both of these are NULL when not otherwise
assigned. This can cause crashes in some cases -- I've added one to
the test suite. (Note that this crasher is just an example, other
ones along the same lines are possible.)
Second, when the language is captured in this way, it means that
Python code cannot affect the current language for its own purposes.
It's reasonable to want to write code like this:
gdb.execute('set language mumble')
... stuff using the current language
gdb.execute('set language previous-value')
However, this won't actually work, because the language is captured on
entry. I've added a test to show this as well.
This patch changes gdb to try to avoid capturing the current values.
The Python concept of the current gdbarch is only set in those few
cases where a non-default value is computed or needed; and the
language is not captured at all -- instead, in the cases where it's
required, the current language is temporarily changed.
This commit brings all the changes made by running gdb/copyright.py
as per GDB's Start of New Year Procedure.
For the avoidance of doubt, all changes in this commits were
performed by the script.
This commit adds a new object type gdb.TargetConnection. This new
type represents a connection within GDB (a connection as displayed by
'info connections').
There's three ways to find a gdb.TargetConnection, there's a new
'gdb.connections()' function, which returns a list of all currently
active connections.
Or you can read the new 'connection' property on the gdb.Inferior
object type, this contains the connection for that inferior (or None
if the inferior has no connection, for example, it is exited).
Finally, there's a new gdb.events.connection_removed event registry,
this emits a new gdb.ConnectionEvent whenever a connection is removed
from GDB (this can happen when all inferiors using a connection exit,
though this is not always the case, depending on the connection type).
The gdb.ConnectionEvent has a 'connection' property, which is the
gdb.TargetConnection being removed from GDB.
The gdb.TargetConnection has an 'is_valid()' method. A connection
object becomes invalid when the underlying connection is removed from
GDB (as discussed above, this might be when all inferiors using a
connection exit, or it might be when the user explicitly replaces a
connection in GDB by issuing another 'target' command).
The gdb.TargetConnection has the following read-only properties:
'num': The number for this connection,
'type': e.g. 'native', 'remote', 'sim', etc
'description': The longer description as seen in the 'info
connections' command output.
'details': A string or None. Extra details for the connection, for
example, a remote connection's details might be
'hostname:port'.
In this commit:
commit c6a6aad52d
Date: Mon Oct 25 17:25:45 2021 +0100
gdb/python: make some global variables static
building without Python was broken. The extension_language_python
global was moved from being always defined, to only being defined when
the HAVE_PYTHON macro was defined. As a consequence, building without
Python support would result in errors like:
/usr/bin/ld: extension.o:(.rodata+0x120): undefined reference to `extension_language_python'
This commit fixes the problem by moving the definition of
extension_language_python outside of the HAVE_PYTHON macro protection.
Make a couple of global variables static in python/python.c. To do
this I had to move the definition of extension_language_python to
later in the file.
There should be no user visible changes after this commit.
There's a common pattern to call add_basic_prefix_cmd and
add_show_prefix_cmd to add matching set and show commands. Add the
add_setshow_prefix_cmd function to factor that out and use it at a few
places.
Change-Id: I6e9e90a30e9efb7b255bf839cac27b85d7069cfd
In a future commit I'm going to be creating gdb.Membuf objects from a
new file within gdb/python/py*.c. Currently all gdb.Membuf objects
are created directly within infpy_read_memory (as a result of calling
gdb.Inferior.read_memory()).
Initially I split out the Membuf creation code into a new function,
and left the new function in gdb/python/py-inferior.c, however, it
felt a little random that the Membuf creation code should live with
the inferior handling code.
So, then I moved all of the Membuf related code out into a new file,
gdb/python/py-membuf.c, the interface is gdbpy_buffer_to_membuf, which
wraps an array of bytes into a gdb.Membuf object.
Most of the code is moved directly from py-inferior.c with only minor
tweaks to layout and replacing NULL with nullptr, hence, I've left the
copyright date on py-membuf.c as 2009-2021 to match py-inferior.c.
Currently, the only user of this code is still py-inferior.c, but in
later commits this will change.
There should be no user visible changes after this commit.
Add a new function to the Python API, gdb.architecture_names(). This
function returns a list containing all of the supported architecture
names within the current build of GDB.
The values returned in this list are all of the possible values that
can be returned from gdb.Architecture.name().
The test-case gdb.gdb/python-interrupts.exp:
- runs to captured_command_loop
- sets a breakpoint at set_active_ext_lang
- calls a python command
- verifies the command triggers the breakpoint
- sends a signal and verifies the result
The test-case is fragile, because (f.i. with -flto) it cannot be guaranteed
that captured_command_loop and set_active_ext_lang are available for setting
breakpoints.
Reimplement the test-case as unittest, using:
- execute_command_to_string to capture the output
- try/catch to catch the "Error while executing Python code" exception
- a new hook selftests::hook_set_active_ext_lang to raise the signal
Tested on x86_64-linux.
The pattern for using execute_command_to_string is:
...
std::string output;
output = execute_fn_to_string (fn, term_out);
...
This results in a problem when using it in a try/catch:
...
try
{
output = execute_fn_to_string (fn, term_out)
}
catch (const gdb_exception &e)
{
/* Use output. */
}
...
If an expection was thrown during execute_fn_to_string, then the output
remains unassigned, while it could be worthwhile to known what output was
generated by gdb before the expection was thrown.
Fix this by returning the string using a parameter instead:
...
execute_fn_to_string (output, fn, term_out)
...
Also add a variant without string parameter, to support places where the
function is used while ignoring the result:
...
execute_fn_to_string (fn, term_out)
...
Tested on x86_64-linux.
Add a new event, gdb.events.gdb_exiting, which is called once GDB
decides it is going to exit.
This event is not triggered in the case that GDB performs a hard
abort, for example, when handling an internal error and the user
decides to quit the debug session, or if GDB hits an unexpected,
fatal, signal.
This event is triggered if the user just types 'quit' at the command
prompt, or if GDB is run with '-batch' and has processed all of the
required commands.
The new event type is gdb.GdbExitingEvent, and it has a single
attribute exit_code, which is the value that GDB is about to exit
with.
The event is triggered before GDB starts dismantling any of its own
internal state, so, my expectation is that most Python calls should
work just fine at this point.
When considering this functionality I wondered about using the
'atexit' Python module. However, this is triggered when the Python
environment is shut down, which is done from a final cleanup. At
this point we don't know for sure what other GDB state has already
been cleaned up.
String-like settings (var_string, var_filename, var_optional_filename,
var_string_noescape) currently take a pointer to a `char *` storage
variable (typically global) that holds the setting's value. I'd like to
"mordernize" this by changing them to use an std::string for storage.
An obvious reason is that string operations on std::string are often
easier to write than with C strings. And they avoid having to do any
manual memory management.
Another interesting reason is that, with `char *`, nullptr and an empty
string often both have the same meaning of "no value". String settings
are initially nullptr (unless initialized otherwise). But when doing
"set foo" (where `foo` is a string setting), the setting now points to
an empty string. For example, solib_search_path is nullptr at startup,
but points to an empty string after doing "set solib-search-path". This
leads to some code that needs to check for both to check for "no value".
Or some code that converts back and forth between NULL and "" when
getting or setting the value. I find this very error-prone, because it
is very easy to forget one or the other. With std::string, we at least
know that the variable is not "NULL". There is only one way of
representing an empty string setting, that is with an empty string.
I was wondering whether the distinction between NULL and "" would be
important for some setting, but it doesn't seem so. If that ever
happens, it would be more C++-y and self-descriptive to use
optional<string> anyway.
Actually, there's one spot where this distinction mattered, it's in
init_history, for the test gdb.base/gdbinit-history.exp. init_history
sets the history filename to the default ".gdb_history" if it sees that
the setting was never set - if history_filename is nullptr. If
history_filename is an empty string, it means the setting was explicitly
cleared, so it leaves it as-is. With the change to std::string, this
distinction doesn't exist anymore. This can be fixed by moving the code
that chooses a good default value for history_filename to
_initialize_top. This is ran before -ex commands are processed, so an
-ex command can then clear that value if needed (what
gdb.base/gdbinit-history.exp tests).
Another small improvement, in my opinion is that we can now easily
give string parameters initial values, by simply initializing the global
variables, instead of xstrdup-ing it in the _initialize function.
In Python and Guile, when registering a string-like parameter, we
allocate (with new) an std::string that is owned by the param_smob (in
Guile) and the parmpy_object (in Python) objects.
This patch started by changing all relevant add_setshow_* commands to
take an `std::string *` instead of a `char **` and fixing everything
that failed to build. That includes of course all string setting
variable and their uses.
string_option_def now uses an std::string also, because there's a
connection between options and settings (see
add_setshow_cmds_for_options).
The add_path function in source.c is really complex and twisted, I'd
rather not try to change it to work on an std::string right now.
Instead, I added an overload that copies the std:string to a `char *`
and back. This means more copying, but this is not used in a hot path
at all, so I think it is acceptable.
Change-Id: I92c50a1bdd8307141cdbacb388248e4e4fc08c93
Co-authored-by: Lancelot SIX <lsix@lancelotsix.com>
cmd_list_element can contain a pointer to data that can be set and / or
shown. This is achieved with the void* VAR member which points to the
data that can be accessed, while the VAR_TYPE member (of type enum
var_types) indicates how to interpret the data pointed to.
With this pattern, the user of the cmd_list_element needs to know what
is the storage type associated with a given VAR_TYPES in order to do
the proper casting. No automatic safeguard is available to prevent
miss-use of the pointer. Client code typically looks something like:
switch (c->var_type)
{
case var_zuinteger:
unsigned int v = *(unsigned int*) c->var;
...
break;
case var_boolean:
bool v = *(bool *) c->var;
...
break;
...
}
This patch proposes to add an abstraction around the var_types and void*
pointer pair. The abstraction is meant to prevent the user from having
to handle the cast and verify that the data is read or written as a type
that is coherent with the setting's var_type. This is achieved by
introducing the struct setting which exposes a set of templated get /
set member functions. The template parameter is the type of the
variable that holds the referred variable.
Using those accessors allows runtime checks to be inserted in order to
ensure that the data pointed to has the expected type. For example,
instantiating the member functions with bool will yield something
similar to:
const bool &get<bool> () const
{
gdb_assert (m_var_type == var_boolean);
gdb_assert (m_var != nullptr);
return *static_cast<bool *> (m_var);
}
void set<bool> (const bool &var)
{
gdb_assert (m_var_type == var_boolean);
gdb_assert (m_var != nullptr);
*static_cast<bool *> (m_var) = var;
}
Using the new abstraction, our initial example becomes:
switch (c->var_type)
{
case var_zuinteger:
unsigned int v = c->var->get<unsigned int> ();
...
break;
case var_boolean:
bool v = c->var->get<bool> ();
...
break;
...
}
While the call site is still similar, the introduction of runtime checks
help ensure correct usage of the data.
In order to avoid turning the bulk of add_setshow_cmd_full into a
templated function, and following a suggestion from Pedro Alves, a
setting can be constructed from a pre validated type erased reference to
a variable. This is what setting::erased_args is used for.
Introducing an opaque abstraction to describe a setting will also make
it possible to use callbacks to retrieve or set the value of the setting
on the fly instead of pointing to a static chunk of memory. This will
be done added in a later commit.
Given that a cmd_list_element may or may not reference a setting, the
VAR and VAR_TYPES members of the struct are replaced with a
gdb::optional<setting> named VAR.
Few internal function signatures have been modified to take into account
this new abstraction:
-The functions value_from_setting, str_value_from_setting and
get_setshow_command_value_string used to have a 'cmd_list_element *'
parameter but only used it for the VAR and VAR_TYPE member. They now
take a 'const setting &' parameter instead.
- Similarly, the 'void *' and a 'enum var_types' parameters of
pascm_param_value and gdbpy_parameter_value have been replaced with a
'const setting &' parameter.
No user visible change is expected after this patch.
Tested on GNU/Linux x86_64, with no regression noticed.
Co-authored-by: Simon Marchi <simon.marchi@polymtl.ca>
Change-Id: Ie1d08c3ceb8b30b3d7bf1efe036eb8acffcd2f34
A build without Python will currently fail, because
selftests::test_python uses gdb_python_initialized, which is only
conditionally defined.
This patch fixes the build by making test_python also be conditionally
defined. I chose this approach because the selftest will fail if
Python is not enabled, so it didn't seem useful to leave it defined.
The test-case gdb.gdb/python-selftest.exp:
- patches the gdb_python_initialized variable in gdb to 0
- checks that the output of a python command is "Python not initialized"
Reimplement gdb.gdb/python-selftest.exp as unittest, using:
- execute_command_to_string to capture the output
- try/catch to catch the "Python not initialized" exception.
Tested on x86_64-linux.
The guile API has (history-append! <value>) to add values into GDB's
history list. There is currently no equivalent in the Python API.
This commit adds gdb.add_history(<value>) to the Python API, this
function takes <value> a gdb.Value (or anything that can be passed to
the constructor of gdb.Value), and adds the value it represents to
GDB's history list. The index of the newly added value is returned.
The alias creation functions currently accept a name to specify the
target command. They pass this to add_alias_cmd, which needs to lookup
the target command by name.
Given that:
- We don't support creating an alias for a command before that command
exists.
- We always use add_info_alias just after creating that target command,
and therefore have access to the target command's cmd_list_element.
... change add_com_alias to accept the target command as a
cmd_list_element (other functions are done in subsequent patches). This
ensures we don't create the alias before the target command, because you
need to get the cmd_list_element from somewhere when you call the alias
creation function. And it avoids an unecessary command lookup. So it
seems better to me in every aspect.
gdb/ChangeLog:
* command.h (add_com_alias): Accept target as
cmd_list_element. Update callers.
Change-Id: I24bed7da57221cc77606034de3023fedac015150
Previously, the prefixname field of struct cmd_list_element was manually
set for prefix commands. This seems verbose and error prone as it
required every single call to functions adding prefix commands to
specify the prefix name while the same information can be easily
generated.
Historically, this was not possible as the prefix field was null for
many commands, but this was fixed in commit
3f4d92ebdf by Philippe Waroquiers, so
we can rely on the prefix field being set when generating the prefix
name.
This commit also fixes a use after free in this scenario:
* A command gets created via Python (using the gdb.Command class).
The prefix name member is dynamically allocated.
* An alias to the new command is created. The alias's prefixname is set
to point to the prefixname for the original command with a direct
assignment.
* A new command with the same name as the Python command is created.
* The object for the original Python command gets freed and its
prefixname gets freed as well.
* The alias is updated to point to the new command, but its prefixname
is not updated so it keeps pointing to the freed one.
gdb/ChangeLog:
* command.h (add_prefix_cmd): Remove the prefixname argument as
it can now be generated automatically. Update all callers.
(add_basic_prefix_cmd): Ditto.
(add_show_prefix_cmd): Ditto.
(add_prefix_cmd_suppress_notification): Ditto.
(add_abbrev_prefix_cmd): Ditto.
* cli/cli-decode.c (add_prefix_cmd): Ditto.
(add_basic_prefix_cmd): Ditto.
(add_show_prefix_cmd): Ditto.
(add_prefix_cmd_suppress_notification): Ditto.
(add_prefix_cmd_suppress_notification): Ditto.
(add_abbrev_prefix_cmd): Ditto.
* cli/cli-decode.h (struct cmd_list_element): Replace the
prefixname member variable with a method which generates the
prefix name at runtime. Update all code reading the prefix
name to use the method, and remove all code setting it.
* python/py-cmd.c (cmdpy_destroyer): Remove code to free the
prefixname member as it's now a method.
(cmdpy_function): Determine if the command is a prefix by
looking at prefixlist, not prefixname.
Add two new commands to GDB that can be placed into the early
initialization to control how Python starts up. The new options are:
set python ignore-environment on|off
set python dont-write-bytecode auto|on|off
show python ignore-environment
show python dont-write-bytecode
These can be used from GDB's startup file to control how the Python
extension language behaves. These options are equivalent to the -E
and -B flags to python respectively, their descriptions from the
Python man page:
-E Ignore environment variables like PYTHONPATH and PYTHONHOME
that modify the behavior of the interpreter.
-B Don't write .pyc files on import.
gdb/ChangeLog:
* NEWS: Mention new commands.
* python/python.c (python_ignore_environment): New static global.
(show_python_ignore_environment): New function.
(set_python_ignore_environment): New function.
(python_dont_write_bytecode): New static global.
(show_python_dont_write_bytecode): New function.
(set_python_dont_write_bytecode): New function.
(_initialize_python): Register new commands.
gdb/doc/ChangeLog:
* python.texinfo (Python Commands): Mention new commands.
gdb/testsuite/ChangeLog:
* gdb.python/py-startup-opt.exp: New file.
Now that both Python and Guile are fully initialized from their
respective finish_initialization methods, the "finish" in the method
name doesn't really make sense; initialization starts _and_ finishes
with that method.
As such, this commit renames finish_initialization to just initialize.
There should be no user visible changes after this commit.
gdb/ChangeLog:
* extension-priv.h (struct extension_language_ops): Rename
'finish_initialization' to 'initialize'.
* extension.c (finish_ext_lang_initialization): Renamed to...
(ext_lang_initialization): ...this, update comment, and updated
the calls to reflect the change in struct extension_language_ops.
* extension.h (finish_ext_lang_initialization): Renamed to...
(ext_lang_initialization): ...this.
* guile/guile.c (gdbscm_finish_initialization): Renamed to...
(gdbscm_initialize): ...this, update comment at definition.
(guile_extension_ops): Update.
* main.c (captured_main_1): Update call to
finish_ext_lang_initialization.
* python/python.c (gdbpy_finish_initialization): Rename to...
(gdbpy_initialize): ...this, update comment at definition, and
update call to do_finish_initialization.
(python_extension_ops): Update.
(do_finish_initialization): Rename to...
(do_initialize): ...this, and update comment.
Delay Python initialisation until gdbpy_finish_initialization.
This is mostly about splitting the existing gdbpy_initialize_*
functions in two, all the calls to register_objfile_data_with_cleanup,
gdbarch_data_register_post_init, etc are moved into new _initialize_*
functions, but everything else is left in the gdbpy_initialize_*
functions.
Then the call to do_start_initialization (in python/python.c) is moved
from the _initialize_python function into gdbpy_finish_initialization.
There should be no user visible changes after this commit.
gdb/ChangeLog:
* python/py-arch.c (_initialize_py_arch): New function.
(gdbpy_initialize_arch): Move code to _initialize_py_arch.
* python/py-block.c (_initialize_py_block): New function.
(gdbpy_initialize_blocks): Move code to _initialize_py_block.
* python/py-inferior.c (_initialize_py_inferior): New function.
(gdbpy_initialize_inferior): Move code to _initialize_py_inferior.
* python/py-objfile.c (_initialize_py_objfile): New function.
(gdbpy_initialize_objfile): Move code to _initialize_py_objfile.
* python/py-progspace.c (_initialize_py_progspace): New function.
(gdbpy_initialize_pspace): Move code to _initialize_py_progspace.
* python/py-registers.c (_initialize_py_registers): New function.
(gdbpy_initialize_registers): Move code to
_initialize_py_registers.
* python/py-symbol.c (_initialize_py_symbol): New function.
(gdbpy_initialize_symbols): Move code to _initialize_py_symbol.
* python/py-symtab.c (_initialize_py_symtab): New function.
(gdbpy_initialize_symtabs): Move code to _initialize_py_symtab.
* python/py-type.c (_initialize_py_type): New function.
(gdbpy_initialize_types): Move code to _initialize_py_type.
* python/py-unwind.c (_initialize_py_unwind): New function.
(gdbpy_initialize_unwind): Move code to _initialize_py_unwind.
* python/python.c (_initialize_python): Move call to
do_start_initialization to gdbpy_finish_initialization.
(gdbpy_finish_initialization): Add call to
do_start_initialization.
The current mechanism by which the Python gdb.current_objfile is
maintained does not allow for nested auto-load events. It is assumed
that once an auto-load script has finished loading then the current
objfile should be set back to NULL. In a nested situation, we should
be restoring the previous value.
We already have an RAII class to handle save/restore type behaviour,
so lets just switch to use that.
The test is a little contrived, but is simple enough, and triggers the
bug. The real use case might involve the auto-load script calling
functions (either in the just-loaded object file, or in the main
executable), which in turn trigger further auto-loads to occur.
gdb/ChangeLog:
* python/python.c (gdbpy_source_objfile_script): Use
make_scoped_restore to restore gdbpy_current_objfile.
(gdbpy_execute_objfile_script): Likewise.
gdb/testsuite/ChangeLog:
* gdb.python/py-auto-load-chaining-f1.c: New file.
* gdb.python/py-auto-load-chaining-f1.o-gdb.py: New file.
* gdb.python/py-auto-load-chaining-f2.c: New file.
* gdb.python/py-auto-load-chaining-f2.o-gdb.py: New file.
* gdb.python/py-auto-load-chaining.c: New file.
* gdb.python/py-auto-load-chaining.exp: New file.
While working on another patch I noticed an oddly formatted error
message in the Python code.
When 'set python print-stack message' is in effect then consider this
Python script:
class TestCommand (gdb.Command):
def __init__ (self):
gdb.Command.__init__ (self, "test-cmd", gdb.COMMAND_DATA)
def invoke(self, args, from_tty):
raise RuntimeError ("bad")
TestCommand ()
And this GDB session:
(gdb) source path/to/python/script.py
(gdb) test-cmd
Python Exception <class 'RuntimeError'> bad:
Error occurred in Python: bad
The line 'Python Exception <class 'RuntimeError'> bad:' doesn't look
terrible in this situation, the colon at the end of the first line
makes sense given the second line.
However, there are places in GDB where there is no second line
printed, for example consider this python script:
def stop_listener (e):
raise RuntimeError ("bad")
gdb.events.stop.connect (stop_listener)
Then this GDB session:
(gdb) file helloworld.exe
(gdb) start
Temporary breakpoint 1 at 0x40112a: file hello.c, line 6.
Starting program: helloworld.exe
Temporary breakpoint 1, main () at hello.c:6
6 printf ("Hello World\n");
Python Exception <class 'RuntimeError'> bad:
(gdb) si
0x000000000040112f 6 printf ("Hello World\n");
Python Exception <class 'RuntimeError'> bad:
In this case there is no auxiliary information displayed after the
warning, and the line ending in the colon looks weird to me.
A quick survey of the code seems to indicate that it is not uncommon
for there to be no auxiliary information line printed, its not just
the one case I found above.
I propose that the line that currently looks like this:
Python Exception <class 'RuntimeError'> bad:
Be reformatted like this:
Python Exception <class 'RuntimeError'>: bad
I think this looks fine then in either situation. The first now looks
like this:
(gdb) test-cmd
Python Exception <class 'RuntimeError'>: bad
Error occurred in Python: bad
And the second like this:
(gdb) si
0x000000000040112f 6 printf ("Hello World\n");
Python Exception <class 'RuntimeError'>: bad
There's just two tests that needed updating. Errors are checked for
in many more tests, but most of the time the pattern doesn't care
about the colon.
gdb/ChangeLog:
* python/python.c (gdbpy_print_stack): Reformat an error message.
gdb/testsuite/ChangeLog:
* gdb.python/py-framefilter.exp: Update expected results.
* gdb.python/python.exp: Update expected results.
This commits the result of running gdb/copyright.py as per our Start
of New Year procedure...
gdb/ChangeLog
Update copyright year range in copyright header of all GDB files.
I noticed that a few "#if"s could be removed from the Python code.
This patch is the result.
gdb/ChangeLog
2020-11-02 Tom Tromey <tromey@adacore.com>
* python/python.c: Consolidate two HAVE_PYTHON blocks.
(python_GdbModuleDef): Move earlier. Now static.
(do_start_initialization): Consolidate some IS_PY3K blocks.
This changes gdb to avoid PyLong_FromUnsignedLong, preferring
gdb_py_object_from_ulongest instead.
gdb/ChangeLog
2020-09-15 Tom Tromey <tromey@adacore.com>
* python/python.c (gdbpy_parameter_value): Use
gdb_py_object_from_ulongest.
This changes gdb to avoid PyLong_FromLong, preferring to
gdb_py_object_from_longest instead.
gdb/ChangeLog
2020-09-15 Tom Tromey <tromey@adacore.com>
* python/python.c (gdbpy_parameter_value): Use
gdb_py_object_from_longest.
* python/py-type.c (convert_field, typy_range): Use
gdb_py_object_from_longest.
* python/py-tui.c (gdbpy_tui_width, gdbpy_tui_height): Use
gdb_py_object_from_longest.
* python/py-lazy-string.c (stpy_get_length): Use
gdb_py_object_from_longest.
* python/py-infthread.c (thpy_get_num, thpy_get_global_num): Use
gdb_py_object_from_longest.
* python/py-infevents.c (create_memory_changed_event_object): Use
gdb_py_object_from_longest.
* python/py-inferior.c (infpy_get_num): Use
gdb_py_object_from_longest.
(infpy_get_pid): Likewise.
This commit adds a new method gdb.Architecture.registers that returns
an object of the new type gdb.RegisterDescriptorIterator. This
iterator returns objects of the new type gdb.RegisterDescriptor.
A RegisterDescriptor is not a way to read the value of a register,
this is already covered by Frame.read_register, a RegisterDescriptor
is simply a way to discover from Python, which registers are
available for a given architecture.
I did consider just returning a string, the name of each register,
instead of a RegisterDescriptor, however, I'm aware that it we don't
want to break the existing Python API in any way, so if I return just
a string now, but in the future we want more information about a
register then we would have to add a second API to get that
information. By going straight to a descriptor object now, it is easy
to add additional properties in the future should we wish to.
Right now the only property of a register that a user can access is
the name of the register.
In future we might want to be able to ask the register about is
register groups, or its type.
gdb/ChangeLog:
* Makefile.in (SUBDIR_PYTHON_SRCS): Add py-registers.c
* python/py-arch.c (archpy_registers): New function.
(arch_object_methods): Add 'registers' method.
* python/py-registers.c: New file.
* python/python-internal.h
(gdbpy_new_register_descriptor_iterator): Declare.
(gdbpy_initialize_registers): Declare.
* python/python.c (do_start_initialization): Call
gdbpy_initialize_registers.
* NEWS: Mention additions to the Python API.
gdb/testsuite/ChangeLog:
* gdb.python/py-arch-reg-names.exp: New file.
gdb/doc/ChangeLog:
* python.texi (Python API): Add new section the menu.
(Frames In Python): Add new @anchor.
(Architectures In Python): Document new registers method.
(Registers In Python): New section.
Python3.9b1 is now available on Rawhide. GDB w/ Python 3.9 support
can be built using the configure switch -with-python=/usr/bin/python3.9.
Attempting to run gdb/Python3.9 segfaults on startup:
#0 0x00007ffff7b0582c in PyEval_ReleaseLock () from /lib64/libpython3.9.so.1.0
#1 0x000000000069ccbf in do_start_initialization ()
at worktree-test1/gdb/python/python.c:1789
#2 _initialize_python ()
at worktree-test1/gdb/python/python.c:1877
#3 0x00000000007afb0a in initialize_all_files () at init.c:237
...
Consulting the the documentation...
https://docs.python.org/3/c-api/init.html
...we find that PyEval_ReleaseLock() has been deprecated since version
3.2. It recommends using PyEval_SaveThread or PyEval_ReleaseThread()
instead. In do_start_initialization, in gdb/python/python.c, we
can replace the calls to PyThreadState_Swap() and PyEval_ReleaseLock()
with a single call to PyEval_SaveThread. (Thanks to Keith Seitz
for working this out.)
With that in place, GDB gets a little bit further. It still dies
on startup, but the backtrace is different:
#0 0x00007ffff7b04306 in PyOS_InterruptOccurred ()
from /lib64/libpython3.9.so.1.0
#1 0x0000000000576e86 in check_quit_flag ()
at worktree-test1/gdb/extension.c:776
#2 0x0000000000576f8a in set_active_ext_lang (now_active=now_active@entry=0x983c00 <extension_language_python>)
at worktree-test1/gdb/extension.c:705
#3 0x000000000069d399 in gdbpy_enter::gdbpy_enter (this=0x7fffffffd2d0,
gdbarch=0x0, language=0x0)
at worktree-test1/gdb/python/python.c:211
#4 0x0000000000686e00 in python_new_inferior (inf=0xddeb10)
at worktree-test1/gdb/python/py-inferior.c:251
#5 0x00000000005d9fb9 in std::function<void (inferior*)>::operator()(inferior*) const (__args#0=<optimized out>, this=0xccad20)
at /usr/include/c++/10/bits/std_function.h:617
#6 gdb::observers::observable<inferior*>::notify (args#0=0xddeb10,
this=<optimized out>)
at worktree-test1/gdb/../gdbsupport/observable.h:106
#7 add_inferior_silent (pid=0)
at worktree-test1/gdb/inferior.c:113
#8 0x00000000005dbcb8 in initialize_inferiors ()
at worktree-test1/gdb/inferior.c:947
...
We checked with some Python Developers and were told that we should
acquire the GIL prior to calling any Python C API function. We
definitely don't have the GIL for calls of PyOS_InterruptOccurred().
I moved class_gdbpy_gil earlier in the file and use it in
gdbpy_check_quit_flag() to acquire (and automatically release) the
GIL.
With those changes in place, I was able to run to a GDB prompt. But,
when trying to quit, it segfaulted again due to due to some other
problems with gdbpy_check_quit_flag():
Thread 1 "gdb" received signal SIGSEGV, Segmentation fault.
0x00007ffff7bbab0c in new_threadstate () from /lib64/libpython3.9.so.1.0
(top-gdb) bt 8
#0 0x00007ffff7bbab0c in new_threadstate () from /lib64/libpython3.9.so.1.0
#1 0x00007ffff7afa5ea in PyGILState_Ensure.cold ()
from /lib64/libpython3.9.so.1.0
#2 0x000000000069b58c in gdbpy_gil::gdbpy_gil (this=<synthetic pointer>)
at worktree-test1/gdb/python/python.c:278
#3 gdbpy_check_quit_flag (extlang=<optimized out>)
at worktree-test1/gdb/python/python.c:278
#4 0x0000000000576e96 in check_quit_flag ()
at worktree-test1/gdb/extension.c:776
#5 0x000000000057700c in restore_active_ext_lang (previous=0xe9c050)
at worktree-test1/gdb/extension.c:729
#6 0x000000000088913a in do_my_cleanups (
pmy_chain=0xc31870 <final_cleanup_chain>,
old_chain=0xae5720 <sentinel_cleanup>)
at worktree-test1/gdbsupport/cleanups.cc:131
#7 do_final_cleanups ()
at worktree-test1/gdbsupport/cleanups.cc:143
In this case, we're trying to call a Python C API function after
Py_Finalize() has been called from finalize_python(). I made
finalize_python set gdb_python_initialized to false and then cause
check_quit_flag() to return early when it's false.
With these changes in place, GDB seems to be working again with
Python3.9b1. I think it likely that there are other problems lurking.
I wouldn't be surprised to find that there are other calls into Python
where we don't first make sure that we have the GIL. Further changes
may well be needed.
I see no regressions testing on Rawhide using a GDB built with the
default Python version (3.8.3) versus one built using Python 3.9b1.
I've also tested on Fedora 28, 29, 30, 31, and 32 (all x86_64) using
the default (though updated) system installed versions of Python on
those OSes. This means that I've tested against Python versions
2.7.15, 2.7.17, 2.7.18, 3.7.7, 3.8.2, and 3.8.3. In each case GDB
still builds without problem and shows no regressions after applying
this patch.
gdb/ChangeLog:
2020-MM-DD Kevin Buettner <kevinb@redhat.com>
Keith Seitz <keiths@redhat.com>
* python/python.c (do_start_initialization): For Python 3.9 and
later, call PyEval_SaveThread instead of PyEval_ReleaseLock.
(class gdbpy_gil): Move to earlier in file.
(finalize_python): Set gdb_python_initialized.
(gdbpy_check_quit_flag): Acquire GIL via gdbpy_gil. Return early
when not initialized.
I recently learned that move constructors generally should be marked
"noexcept". This ensures that standard containers will move objects
when possible, rather than copy them.
This patch fixes the cases I could find. Note that implicitly-defined
or defaulted move constructors will automatically do what you'd
expect; that is, they are noexcept if all the members have noexcept
move constructors.
While doing this, I noticed a couple of odd cases where the move
constructor seemed to assume that the object being constructed could
have state requiring destruction. I've fixed these as well. See
completion_result and scoped_mmap.
gdb/ChangeLog
2020-04-20 Tom Tromey <tromey@adacore.com>
* python/python.c (struct gdbpy_event): Mark move constructor as
noexcept.
* python/py-tui.c (class gdbpy_tui_window_maker): Mark move
constructor as noexcept.
* completer.h (struct completion_result): Mark move constructor as
noexcept.
* completer.c (completion_result::completion_result): Use
initialization style. Don't call reset_match_list.
gdbsupport/ChangeLog
2020-04-20 Tom Tromey <tromey@adacore.com>
* scoped_mmap.h (scoped_mmap): Mark move constructor as noexcept.
Use initialization style. Don't call destroy.
* scoped_fd.h (class scoped_fd): Mark move constructor as
noexcept.
* gdb_ref_ptr.h (class ref_ptr): Mark move constructor as
noexcept.
Currently there are many prefix commands that do nothing but call
either help_list or cmd_show_list. I happened to notice that one such
call, for "set print type", used the wrong command list parameter,
causing incorrect output.
Rather than fix this bug in isolation, I decided to eliminate this
possibility by adding two new ways to add prefix commands, which
simply route the call to help_list or cmd_show_list, as appropriate.
This makes it impossible for a mismatch to occur.
In some cases, a bit of output was removed; however, I don't think
this output in general was very useful. It seemed redundant with
what's already printed by help_list. A representative example is this
hunk, removed from ada-lang.c:
- printf_unfiltered (_(\
-"\"set ada\" must be followed by the name of a setting.\n"));
This simplified the CLI style set/show commands quite a bit, and
allowed the deletion of a macro.
This also cleans up some unusual code in windows-tdep.c.
Tested on x86-64 Fedora 30. Note that I have no way to build the
go32-nat.c change.
gdb/ChangeLog
2020-04-17 Tom Tromey <tromey@adacore.com>
* auto-load.c (show_auto_load_cmd): Remove.
(auto_load_show_cmdlist_get): Use add_show_prefix_cmd.
* arc-tdep.c (_initialize_arc_tdep): Use add_show_prefix_cmd.
(maintenance_print_arc_command): Remove.
* tui/tui-win.c (tui_command): Remove.
(tui_get_cmd_list): Use add_basic_prefix_cmd.
* tui/tui-layout.c (tui_layout_command): Remove.
(_initialize_tui_layout): Use add_basic_prefix_cmd.
* python/python.c (user_set_python, user_show_python): Remove.
(_initialize_python): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
* guile/guile.c (set_guile_command, show_guile_command): Remove.
(install_gdb_commands): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
(info_guile_command): Remove.
* dwarf2/read.c (set_dwarf_cmd, show_dwarf_cmd): Remove.
(_initialize_dwarf2_read): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
* cli/cli-style.h (class cli_style_option) <add_setshow_commands>:
Remove do_set and do_show parameters.
* cli/cli-style.c (set_style, show_style): Remove.
(_initialize_cli_style): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
(cli_style_option::add_setshow_commands): Remove do_set and
do_show parameters.
(cli_style_option::add_setshow_commands): Use
add_basic_prefix_cmd, add_show_prefix_cmd.
(STYLE_ADD_SETSHOW_COMMANDS): Remove macro.
(set_style_name): Remove.
* cli/cli-dump.c (dump_command, append_command): Remove.
(srec_dump_command, ihex_dump_command, verilog_dump_command)
(tekhex_dump_command, binary_dump_command)
(binary_append_command): Remove.
(_initialize_cli_dump): Use add_basic_prefix_cmd.
* windows-tdep.c (w32_prefix_command_valid): Remove global.
(init_w32_command_list): Remove; move into ...
(_initialize_windows_tdep): ... here. Use add_basic_prefix_cmd.
* valprint.c (set_print, show_print, set_print_raw)
(show_print_raw): Remove.
(_initialize_valprint): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
* typeprint.c (set_print_type, show_print_type): Remove.
(_initialize_typeprint): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
* record.c (set_record_command, show_record_command): Remove.
(_initialize_record): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
* cli/cli-cmds.c (_initialize_cli_cmds): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
(info_command, show_command, set_debug, show_debug): Remove.
* top.h (set_history, show_history): Don't declare.
* top.c (set_history, show_history): Remove.
* target-descriptions.c (set_tdesc_cmd, show_tdesc_cmd)
(unset_tdesc_cmd): Remove.
(_initialize_target_descriptions): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
* symtab.c (info_module_command): Remove.
(_initialize_symtab): Use add_basic_prefix_cmd.
* symfile.c (overlay_command): Remove.
(_initialize_symfile): Use add_basic_prefix_cmd.
* sparc64-tdep.c (info_adi_command): Remove.
(_initialize_sparc64_adi_tdep): Use add_basic_prefix_cmd.
* sh-tdep.c (show_sh_command, set_sh_command): Remove.
(_initialize_sh_tdep): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
* serial.c (serial_set_cmd, serial_show_cmd): Remove.
(_initialize_serial): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
* ser-tcp.c (set_tcp_cmd, show_tcp_cmd): Remove.
(_initialize_ser_tcp): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
* rs6000-tdep.c (set_powerpc_command, show_powerpc_command)
(_initialize_rs6000_tdep): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
* riscv-tdep.c (show_riscv_command, set_riscv_command)
(show_debug_riscv_command, set_debug_riscv_command): Remove.
(_initialize_riscv_tdep): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
* remote.c (remote_command, set_remote_cmd): Remove.
(_initialize_remote): Use add_basic_prefix_cmd.
* record-full.c (set_record_full_command)
(show_record_full_command): Remove.
(_initialize_record_full): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
* record-btrace.c (cmd_set_record_btrace)
(cmd_show_record_btrace, cmd_set_record_btrace_bts)
(cmd_show_record_btrace_bts, cmd_set_record_btrace_pt)
(cmd_show_record_btrace_pt): Remove.
(_initialize_record_btrace): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
* ravenscar-thread.c (set_ravenscar_command)
(show_ravenscar_command): Remove.
(_initialize_ravenscar): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
* mips-tdep.c (show_mips_command, set_mips_command)
(_initialize_mips_tdep): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
* maint.c (maintenance_command, maintenance_info_command)
(maintenance_check_command, maintenance_print_command)
(maintenance_set_cmd, maintenance_show_cmd): Remove.
(_initialize_maint_cmds): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
(show_per_command_cmd): Remove.
* maint-test-settings.c (maintenance_set_test_settings_cmd):
Remove.
(maintenance_show_test_settings_cmd): Remove.
(_initialize_maint_test_settings): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
* maint-test-options.c (maintenance_test_options_command):
Remove.
(_initialize_maint_test_options): Use add_basic_prefix_cmd.
* macrocmd.c (macro_command): Remove
(_initialize_macrocmd): Use add_basic_prefix_cmd.
* language.c (set_check, show_check): Remove.
(_initialize_language): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
* infcmd.c (unset_command): Remove.
(_initialize_infcmd): Use add_basic_prefix_cmd.
* i386-tdep.c (set_mpx_cmd, show_mpx_cmd): Remove.
(_initialize_i386_tdep): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
* go32-nat.c (go32_info_dos_command): Remove.
(_initialize_go32_nat): Use add_basic_prefix_cmd.
* cli/cli-decode.c (do_prefix_cmd, add_basic_prefix_cmd)
(do_show_prefix_cmd, add_show_prefix_cmd): New functions.
* frame.c (set_backtrace_cmd, show_backtrace_cmd): Remove.
(_initialize_frame): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
* dcache.c (set_dcache_command, show_dcache_command): Remove.
(_initialize_dcache): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
* cp-support.c (maint_cplus_command): Remove.
(_initialize_cp_support): Use add_basic_prefix_cmd.
* btrace.c (maint_btrace_cmd, maint_btrace_set_cmd)
(maint_btrace_show_cmd, maint_btrace_pt_set_cmd)
(maint_btrace_pt_show_cmd, _initialize_btrace): Use
add_basic_prefix_cmd, add_show_prefix_cmd.
* breakpoint.c (save_command): Remove.
(_initialize_breakpoint): Use add_basic_prefix_cmd.
* arm-tdep.c (set_arm_command, show_arm_command): Remove.
(_initialize_arm_tdep): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
* ada-lang.c (maint_set_ada_cmd, maint_show_ada_cmd)
(set_ada_command, show_ada_command): Remove.
(_initialize_ada_language): Use add_basic_prefix_cmd,
add_show_prefix_cmd.
* command.h (add_basic_prefix_cmd, add_show_prefix_cmd): Declare.
gdb/testsuite/ChangeLog
2020-04-17 Tom Tromey <tromey@adacore.com>
* gdb.cp/maint.exp (test_help): Simplify multiple_help_body.
Update tests.
* gdb.btrace/cpu.exp: Update tests.
* gdb.base/maint.exp: Update tests.
* gdb.base/default.exp: Update tests.
* gdb.base/completion.exp: Update tests.
This commit fixes a compilation warning/error when building GDB
with Python 3.9:
g++ -x c++ -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -DDNF_DEBUGINFO_INSTALL -I. -I../../gdb -I../../gdb/config -DLOCALEDIR="\"/usr/share/locale\"" -DHAVE_CONFIG_H -I../../gdb/../include/opcode -I../bfd -I../../gdb/../bfd -I../../gdb/../include -I../libdecnumber -I../../gdb/../libdecnumber -I../../gdb/../gnulib/import -I../gnulib/import -DTUI=1 -I/usr/include/guile/2.0 -pthread -I/usr/include/python3.9 -I/usr/include/python3.9 -I../../gdb/.. -pthread -Wall -Wpointer-arith -Wno-unused -Wunused-value -Wunused-variable -Wunused-function -Wno-switch -Wno-char-subscripts -Wempty-body -Wunused-but-set-parameter -Wunused-but-set-variable -Wno-sign-compare -Wno-error=maybe-uninitialized -Wno-mismatched-tags -Wsuggest-override -Wimplicit-fallthrough=3 -Wduplicated-cond -Wshadow=local -Wdeprecated-copy -Wdeprecated-copy-dtor -Wredundant-move -Wformat -Wformat-nonliteral -Wno-unused -Werror -c -o ser-tcp.o -MT ser-tcp.o -MMD -MP -MF ./.deps/ser-tcp.Tpo ../../gdb/ser-tcp.c
../../gdb/python/python.c: In function 'bool do_start_initialization()':
../../gdb/python/python.c:1621:23: error: 'void PyEval_InitThreads()' is deprecated [-Werror=deprecated-declarations]
1621 | PyEval_InitThreads ();
| ^
In file included from /usr/include/python3.9/Python.h:141,
from ../../gdb/python/python-internal.h:86,
from ../../gdb/python/python.c:92:
/usr/include/python3.9/ceval.h:132:37: note: declared here
132 | Py_DEPRECATED(3.9) PyAPI_FUNC(void) PyEval_InitThreads(void);
| ^~~~~~~~~~~~~~~~~~
Information about the deprecated function can be found here:
https://docs.python.org/3.9/whatsnew/3.9.html#deprecated
Specifically, with regard to PyEval_InitThreads(), it says:
The PyEval_InitThreads() and PyEval_ThreadsInitialized() functions
are now deprecated and will be removed in Python 3.11. Calling
PyEval_InitThreads() now does nothing. The GIL is initialized by
Py_Initialize() since Python 3.7. (Contributed by Victor Stinner
in bpo-39877.)
I chose to disable the call with a #if test using PY_VERSION_HEX.
There is precedent for use of PY_VERSION_HEX; it's used in two places
in python-internal.h. I noticed that under certain circumstances
python-internal.h defines PyEval_InitThreads to be nothing, which
accomplishes the same thing. I considered doing something similar for
this case, but decided against it because, at some point in the future,
the presence of PyEval_InitThreads() without some explanation will be
confusing to a reader who won't be able to find PyEval_InitThreads in
the current (future for us) Python API. IMO, use of the #if along
with an accompanying comment seemed more straightforward.
gdb/ChangeLog:
* python/python.c (do_start_initialization): Don't call
PyEval_InitThreads for Python 3.9 and beyond.
Change-Id: I0679fc10b6b76761a99538568f13188c6d8014e0
This patch adds support for writing new TUI windows in Python.
2020-02-22 Tom Tromey <tom@tromey.com>
* NEWS: Add entry for gdb.register_window_type.
* tui/tui-layout.h (window_factory): New typedef.
(tui_register_window): Declare.
* tui/tui-layout.c (saved_tui_windows): New global.
(tui_apply_current_layout): Use it.
(tui_register_window): New function.
* python/python.c (do_start_initialization): Call
gdbpy_initialize_tui.
(python_GdbMethods): Add "register_window_type" function.
* python/python-internal.h (gdbpy_register_tui_window)
(gdbpy_initialize_tui): Declare.
* python/py-tui.c: New file.
* Makefile.in (SUBDIR_PYTHON_SRCS): Add py-tui.c.
gdb/doc/ChangeLog
2020-02-22 Tom Tromey <tom@tromey.com>
* python.texi (Python API): Add menu item.
(TUI Windows In Python): New node.
gdb/testsuite/ChangeLog
2020-02-22 Tom Tromey <tom@tromey.com>
* gdb.python/tui-window.exp: New file.
* gdb.python/tui-window.py: New file.
Change-Id: I85fbfb923a1840450a00a7dce113a05d7f048baa
I noticed that do_start_initialization, in python.c, checks the result
of xmalloc. However, xmalloc cannot fail, so this check is useless.
This patch also changes the code to use XNEWVEC.
gdb/ChangeLog
2020-02-19 Tom Tromey <tromey@adacore.com>
* python/python.c (do_start_initialization): Use XNEWVEC. Remove
NULL check.
This is an update of this patch:
https://sourceware.org/ml/gdb-patches/2018-09/msg00884.html
This patch attempts to address PR gdb/23718 by re-enabling stdin
whenever an exception is caught during gdb.execute().
When Python gdb.execute() is called, an exception could occur (e.g. the
target disappearing), which is then converted into a Python exception. If
stdin was disabled before the exception is caught, it is not re-enabled,
because the exception doesn't propagate to the top level of the event loop,
whose catch block would otherwise enable it.
The result is that when execution of a Python script completes, GDB does
not prompt or accept input, and is effectively hung.
This change rectifies the issue by re-enabling stdin in the catch block of
execute_gdb_command, prior to converting the exception to a Python
exception.
Since this patch was originally posted I've added a test, and also I
converted the code to re-enable stdin from this:
SWITCH_THRU_ALL_UIS ()
{
async_enable_stdin ();
}
to simply this:
async_enable_stdin ();
My reasoning is that we only need the SWITCH_THRU_ALL_UIS if, at the time
the exception is caught, the current_ui might be different than at the time
we called async_disable_stdin. Within python's execute_gdb_command I think
it should be impossible to switch current_ui, so the SWITCH_THRU_ALL_UIS
isn't needed.
gdb/ChangeLog:
PR gdb/23718
* gdb/python/python.c (execute_gdb_command): Call
async_enable_stdin in catch block.
gdb/testsuite/ChangeLog:
PR gdb/23718
* gdb.server/server-kill-python.exp: New file.
Change-Id: I1cfc36ee9f8484cc1ed82be9be338353db6bc080
Tom Tromey