gdb: Look for compilation directory relative to directory search path

The 'directory' command allows the user to provide a list of filesystem
directories in which to search for source code.  The directories in this
search path are used as the base directory for the source filename from
the debug information (DW_AT_name).  Thus the directory search path
provides alternatives to the existing compilation directory from the
debug information (DW_AT_comp_dir).  Generally speaking, DW_AT_name
stores the filename argument passed to the compiler (including any
directory components), and DW_AT_comp_dir stores the current working
directory from which the compiler was executed.  For example:

    $ cd /path/to/project/subdir1
    $ gcc -c a/test.c -g

The corresponding debug information will look like this:

    DW_AT_name      : a/test.c
    DW_AT_comp_dir  : /path/to/project/subdir1

When compiling with the -fdebug-prefix-map GCC option, the compilation
directory can be arbitrarily rewritten.  In the above example, we may
rewrite the compilation directory as follows:

    $ gcc -c a/test.c -g -fdebug-prefix-map=/path/to/project=

In this case, the corresponding debug information will look like:

    DW_AT_name      : a/test.c
    DW_AT_comp_dir  : /subdir1

This prevents GDB from finding the corresponding source code based on
the debug information alone.  In some cases, a substitute-path command
can be used to re-map a consistent prefix in the rewritten compilation
directory to the real filesystem path.  However, there may not be a
consistent prefix remaining in the debug symbols (for example in a
project that has source code in many subdirectories under the project's
root), thereby requiring multiple substitute-path rules.  In this case,
it is easier to add the missing prefix to the directory search path via
the 'directory' command.

The function find_and_open_source currently searches in:

    SEARCH_PATH/FILENAME

where SEARCH_PATH corresponds to each individual entry in the directory
search path (which is guaranteed to contain the compilation directory
from the debug information, as well as the current working directory).
FILENAME corresponds to the source filename (DW_AT_name), which may have
directory components in it.  In addition, GDB searches in:

    SEARCH_PATH/FILE_BASENAME

where FILE_BASENAME is the basename of the DW_AT_name entry.

This change modifies find_and_open_source to additionally search in:

    SEARCH_PATH/COMP_DIR/FILENAME

where COMP_DIR is the compilation directory from the debug symbols.  In
the example given earlier, running:

    (gdb) directory /path/to/project

will now allow GDB to correctly locate the source code from the debug
information.

gdb/ChangeLog:

	* source.c (prepare_path_for_appending): New function.
	(openp): Make use of new function.
	(find_and_open_source): Search for the compilation directory and
	source file as a relative path beneath the directory search path.

gdb/doc/ChangeLog:

	* gdb.texinfo (Source Path): Additional text to better describe
	how the source path directory list is used when searching for
	source files.

gdb/testsuite/ChangeLog:

	* gdb.base/source-dir.exp: Add extra test for mapped compilation
	directory.

gdb/ChangeLog

@@ -1,3 +1,10 @@
+2019-09-17  Mike Gulick  <mgulick@mathworks.com>
+
+	* source.c (prepare_path_for_appending): New function.
+	(openp): Make use of new function.
+	(find_and_open_source): Search for the compilation directory and
+	source file as a relative path beneath the directory search path.
+
 2019-09-17  Andrew Burgess  <andrew.burgess@embecosm.com>
 
 	* source-cache.c (source_cache::get_line_charpos): Catch

gdb/doc/ChangeLog

@@ -1,3 +1,9 @@
+2019-09-17  Andrew Burgess  <andrew.burgess@embecosm.com>
+
+	* gdb.texinfo (Source Path): Additional text to better describe
+	how the source path directory list is used when searching for
+	source files.
+
 2019-09-12  Philippe Waroquiers  <philippe.waroquiers@skynet.be>
 
 	* gdb.texinfo (Ada Tasks): Tell the task name is printed, update

gdb/doc/gdb.texinfo

@@ -8954,11 +8954,21 @@ it tries all the directories in the list, in the order they are present
 in the list, until it finds a file with the desired name.
 
 For example, suppose an executable references the file
-@file{/usr/src/foo-1.0/lib/foo.c}, and our source path is
+@file{/usr/src/foo-1.0/lib/foo.c}, does not record a compilation
-@file{/mnt/cross}.  The file is first looked up literally; if this
+directory, and the @dfn{source path} is @file{/mnt/cross}.
-fails, @file{/mnt/cross/usr/src/foo-1.0/lib/foo.c} is tried; if this
+@value{GDBN} would look for the source file in the following
-fails, @file{/mnt/cross/foo.c} is opened; if this fails, an error
+locations:
-message is printed.  @value{GDBN} does not look up the parts of the
+
+@enumerate
+
+@item @file{/usr/src/foo-1.0/lib/foo.c}
+@item @file{/mnt/cross/usr/src/foo-1.0/lib/foo.c}
+@item @file{/mnt/cross/foo.c}
+
+@end enumerate
+
+If the source file is not present at any of the above locations then
+an error is printed.  @value{GDBN} does not look up the parts of the
 source file name, such as @file{/mnt/cross/src/foo-1.0/lib/foo.c}.
 Likewise, the subdirectories of the source path are not searched: if
 the source path is @file{/mnt/cross}, and the binary refers to
@@ -8966,11 +8976,91 @@ the source path is @file{/mnt/cross}, and the binary refers to
 @file{/mnt/cross/usr/src/foo-1.0/lib}.
 
 Plain file names, relative file names with leading directories, file
-names containing dots, etc.@: are all treated as described above; for
+names containing dots, etc.@: are all treated as described above,
-instance, if the source path is @file{/mnt/cross}, and the source file
+except that non-absolute file names are not looked up literally.  If
-is recorded as @file{../lib/foo.c}, @value{GDBN} would first try
+the @dfn{source path} is @file{/mnt/cross}, the source file is
-@file{../lib/foo.c}, then @file{/mnt/cross/../lib/foo.c}, and after
+recorded as @file{../lib/foo.c}, and no compilation directory is
-that---@file{/mnt/cross/foo.c}.
+recorded, then @value{GDBN} will search in the following locations:
+
+@enumerate
+
+@item @file{/mnt/cross/../lib/foo.c}
+@item @file{/mnt/cross/foo.c}
+
+@end enumerate
+
+@kindex cdir
+@kindex cwd
+@vindex $cdir@r{, convenience variable}
+@vindex $cwd@r{, convenience variable}
+@cindex compilation directory
+@cindex current directory
+@cindex working directory
+@cindex directory, current
+@cindex directory, compilation
+The @dfn{source path} will always include two special entries
+@samp{$cdir} and @samp{$cwd}, these refer to the compilation directory
+(if one is recorded) and the current working directory respectively.
+
+@samp{$cdir} causes @value{GDBN} to search within the compilation
+directory, if one is recorded in the debug information.  If no
+compilation directory is recorded in the debug information then
+@samp{$cdir} is ignored.
+
+@samp{$cwd} is not the same as @samp{.}---the former tracks the
+current working directory as it changes during your @value{GDBN}
+session, while the latter is immediately expanded to the current
+directory at the time you add an entry to the source path.
+
+If a compilation directory is recorded in the debug information, and
+@value{GDBN} has not found the source file after the first search
+using @dfn{source path}, then @value{GDBN} will combine the
+compilation directory and the filename, and then search for the source
+file again using the @dfn{source path}.
+
+For example, if the executable records the source file as
+@file{/usr/src/foo-1.0/lib/foo.c}, the compilation directory is
+recorded as @file{/project/build}, and the @dfn{source path} is
+@file{/mnt/cross:$cdir:$cwd} while the current working directory of
+the @value{GDBN} session is @file{/home/user}, then @value{GDBN} will
+search for the source file in the following loctions:
+
+@enumerate
+
+@item @file{/usr/src/foo-1.0/lib/foo.c}
+@item @file{/mnt/cross/usr/src/foo-1.0/lib/foo.c}
+@item @file{/project/build/usr/src/foo-1.0/lib/foo.c}
+@item @file{/home/user/usr/src/foo-1.0/lib/foo.c}
+@item @file{/mnt/cross/project/build/usr/src/foo-1.0/lib/foo.c}
+@item @file{/project/build/project/build/usr/src/foo-1.0/lib/foo.c}
+@item @file{/home/user/project/build/usr/src/foo-1.0/lib/foo.c}
+@item @file{/mnt/cross/foo.c}
+@item @file{/project/build/foo.c}
+@item @file{/home/user/foo.c}
+
+@end enumerate
+
+If the file name in the previous example had been recorded in the
+executable as a relative path rather than an absolute path, then the
+first look up would not have occurred, but all of the remaining steps
+would be similar.
+
+When searching for source files on MS-DOS and MS-Windows, where
+absolute paths start with a drive letter (e.g.
+@file{C:/project/foo.c}), @value{GDBN} will remove the drive letter
+from the file name before appending it to a search directory from
+@dfn{source path}; for instance if the executable references the
+source file @file{C:/project/foo.c} and @dfn{source path} is set to
+@file{D:/mnt/cross}, then @value{GDBN} will search in the following
+locations for the source file:
+
+@enumerate
+
+@item @file{C:/project/foo.c}
+@item @file{D:/mnt/cross/project/foo.c}
+@item @file{D:/mnt/cross/foo.c}
+
+@end enumerate
 
 Note that the executable search path is @emph{not} used to locate the
 source files.
@@ -8981,8 +9071,8 @@ each line is in the file.
 
 @kindex directory
 @kindex dir
-When you start @value{GDBN}, its source path includes only @samp{cdir}
+When you start @value{GDBN}, its source path includes only @samp{$cdir}
-and @samp{cwd}, in that order.
+and @samp{$cwd}, in that order.
 To add other directories, use the @code{directory} command.
 
 The search path is used to find both program source files and @value{GDBN}
@@ -9058,21 +9148,12 @@ part of absolute file names) or
 whitespace.  You may specify a directory that is already in the source
 path; this moves it forward, so @value{GDBN} searches it sooner.
 
-@kindex cdir
+The special strings @samp{$cdir} (to refer to the compilation
-@kindex cwd
+directory, if one is recorded), and @samp{$cwd} (to refer to the
-@vindex $cdir@r{, convenience variable}
+current working directory) can also be included in the list of
-@vindex $cwd@r{, convenience variable}
+directories @var{dirname}.  Though these will already be in the source
-@cindex compilation directory
+path they will be moved forward in the list so @value{GDBN} searches
-@cindex current directory
+them sooner.
-@cindex working directory
-@cindex directory, current
-@cindex directory, compilation
-You can use the string @samp{$cdir} to refer to the compilation
-directory (if one is recorded), and @samp{$cwd} to refer to the current
-working directory.  @samp{$cwd} is not the same as @samp{.}---the former
-tracks the current working directory as it changes during your @value{GDBN}
-session, while the latter is immediately expanded to the current
-directory at the time you add an entry to the source path.
 
 @item directory
 Reset the source path to its default value (@samp{$cdir:$cwd} on Unix systems).  This requires confirmation.

gdb/source.c

@@ -654,6 +654,36 @@ info_source_command (const char *ignore, int from_tty)
 }
 
 
+/* Helper function to remove characters from the start of PATH so that
+   PATH can then be appended to a directory name.  We remove leading drive
+   letters (for dos) as well as leading '/' characters and './'
+   sequences.  */
+
+const char *
+prepare_path_for_appending (const char *path)
+{
+  /* For dos paths, d:/foo -> /foo, and d:foo -> foo.  */
+  if (HAS_DRIVE_SPEC (path))
+    path = STRIP_DRIVE_SPEC (path);
+
+  const char *old_path;
+  do
+    {
+      old_path = path;
+
+      /* /foo => foo, to avoid multiple slashes that Emacs doesn't like.  */
+      while (IS_DIR_SEPARATOR(path[0]))
+	path++;
+
+      /* ./foo => foo */
+      while (path[0] == '.' && IS_DIR_SEPARATOR (path[1]))
+	path += 2;
+    }
+  while (old_path != path);
+
+  return path;
+}
+
 /* Open a file named STRING, searching path PATH (dir names sep by some char)
    using mode MODE in the calls to open.  You cannot use this function to
    create files (O_CREAT).
@@ -747,17 +777,9 @@ openp (const char *path, openp_flags opts, const char *string,
 	    goto done;
     }
 
-  /* For dos paths, d:/foo -> /foo, and d:foo -> foo.  */
+  /* Remove characters from the start of PATH that we don't need when PATH
-  if (HAS_DRIVE_SPEC (string))
+     is appended to a directory name.  */
-    string = STRIP_DRIVE_SPEC (string);
+  string = prepare_path_for_appending (string);
-
-  /* /foo => foo, to avoid multiple slashes that Emacs doesn't like.  */
-  while (IS_DIR_SEPARATOR(string[0]))
-    string++;
-
-  /* ./foo => foo */
-  while (string[0] == '.' && IS_DIR_SEPARATOR (string[1]))
-    string += 2;
 
   alloclen = strlen (path) + strlen (string) + 2;
   filename = (char *) alloca (alloclen);
@@ -1033,7 +1055,32 @@ find_and_open_source (const char *filename,
   openp_flags flags = OPF_SEARCH_IN_PATH;
   if (basenames_may_differ)
     flags |= OPF_RETURN_REALPATH;
+
+  /* Try to locate file using filename.  */
   result = openp (path, flags, filename, OPEN_MODE, fullname);
+  if (result < 0 && dirname != NULL)
+    {
+      /* Remove characters from the start of PATH that we don't need when
+	 PATH is appended to a directory name.  */
+      const char *filename_start = prepare_path_for_appending (filename);
+
+      /* Try to locate file using compilation dir + filename.  This is
+	 helpful if part of the compilation directory was removed,
+	 e.g. using gcc's -fdebug-prefix-map, and we have added the missing
+	 prefix to source_path.  */
+      std::string cdir_filename (dirname);
+
+      /* Remove any trailing directory separators.  */
+      while (IS_DIR_SEPARATOR (cdir_filename.back ()))
+	cdir_filename.pop_back ();
+
+      /* Add our own directory separator.  */
+      cdir_filename.append (SLASH_STRING);
+      cdir_filename.append (filename_start);
+
+      result = openp (path, flags, cdir_filename.c_str (), OPEN_MODE,
+		      fullname);
+    }
   if (result < 0)
     {
       /* Didn't work.  Try using just the basename.  */

gdb/testsuite/ChangeLog

@@ -1,3 +1,8 @@
+2019-09-17  Andrew Burgess  <andrew.burgess@embecosm.com>
+
+	* gdb.base/source-dir.exp: Add extra test for mapped compilation
+	directory.
+
 2019-09-17  Andrew Burgess  <andrew.burgess@embecosm.com>
 
 	* gdb.base/list-missing-source.exp: New file.

gdb/testsuite/gdb.base/source-dir.c

@@ -0,0 +1,22 @@
+/* This testcase is part of GDB, the GNU debugger.
+
+   Copyright 2019 Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+int
+main ()
+{
+  return 0;
+}

gdb/testsuite/gdb.base/source-dir.exp

@@ -15,9 +15,142 @@
 
 standard_testfile
 
-gdb_start
+# Take a list of directories DIRS, and return a regular expression
+# that will match against the output of the 'directory' command
+# assuming that DIRS are all of the directories that should appear in
+# the results.
+proc search_dir_list { dirs } {
+    set output "\r\nSource directories searched: "
+    append output [join $dirs "\[:;\]"]
 
-set foo "/nOtExStInG"
+    return ${output}
+}
 
-gdb_test "directory $foo/a $foo/b $foo/c" "\r\nSource directories searched: $foo/a\[:;\]$foo/b\[:;\]$foo/c\[:;\]\\\$cdir\[:;\]\\\$cwd"
+# Check that adding directories to the search path changes the order
-gdb_test "directory $foo/b $foo/d $foo/c" "\r\nSource directories searched: $foo/b\[:;\]$foo/d\[:;\]$foo/c\[:;\]$foo/a\[:;\]\\\$cdir\[:;\]\\\$cwd"
+# in which directories are searched.
+proc test_changing_search_directory {} {
+    gdb_start
+
+    set foo "/nOtExStInG"
+
+    gdb_test "directory $foo/a $foo/b $foo/c" \
+	[search_dir_list [list \
+			      "$foo/a" \
+			      "$foo/b" \
+			      "$foo/c" \
+			      "\\\$cdir" \
+			      "\\\$cwd"]]
+    gdb_test "directory $foo/b $foo/d $foo/c" \
+	[search_dir_list [list \
+			      "$foo/b" \
+			      "$foo/d" \
+			      "$foo/c" \
+			      "$foo/a" \
+			      "\\\$cdir" \
+			      "\\\$cwd"]]
+    gdb_exit
+}
+
+# Test that the compilation directory can also be extended with a
+# prefix from the directory search path in order to find source files.
+proc test_truncated_comp_dir {} {
+    global srcfile srcdir subdir binfile
+    global decimal
+
+    # When we run this test the current directory will be something
+    # like this:
+    #     /some/path/to/gdb/build/testsuite/
+    # We are going to copy the source file out of the source tree into
+    # a location like this:
+    #     /some/path/to/gdb/build/testsuite/output/gdb.base/soure-dir/
+    #
+    # We will then switch to this directory and compile the source
+    # file, however, we will ask GCC to remove this prefix from the
+    # compilation directory in the debug info:
+    #     /some/path/to/gdb/build/testsuite/output/
+    #
+    # As a result the debug information will look like this:
+    #
+    #     DW_AT_name        : source-dir.c
+    #     DW_AT_comp_dir    : /gdb.base/source-dir
+    #
+    # Finally we switch back to this directory:
+    #     /some/path/to/gdb/build/testsuite/
+    #
+    # and start GDB.  There was a time when GDB would be unable to
+    # find the source file no matter what we added to the directory
+    # search path, this should now be fixed.
+
+    set original_dir [pwd]
+    set working_dir [standard_output_file ""]
+    cd ${working_dir}
+
+    set strip_dir [file normalize "${working_dir}/../.."]
+
+    set new_srcfile [standard_output_file ${srcfile}]
+    set fd [open "$new_srcfile" w]
+    puts $fd "int
+    main ()
+    {
+      return 0;
+    }"
+    close $fd
+
+    set options \
+	"debug additional_flags=-fdebug-prefix-map=${strip_dir}="
+    if  { [gdb_compile "${srcfile}" "${binfile}" \
+	       executable ${options}] != "" } {
+	untested "failed to compile"
+	return -1
+    }
+
+    cd ${original_dir}
+
+    clean_restart ${binfile}
+
+    gdb_test_no_output "set directories \$cdir:\$cwd"
+    gdb_test "show directories" \
+	"\r\nSource directories searched: \\\$cdir\[:;\]\\\$cwd"
+
+    if ![runto_main] then {
+	fail "can't run to main"
+	return 0
+    }
+
+    gdb_test "info source" \
+    [multi_line \
+	 "Current source file is ${srcfile}" \
+	 "Compilation directory is \[^\n\r\]+" \
+	 "Source language is c." \
+	 "Producer is \[^\n\r\]+" \
+	 "Compiled with DWARF $decimal debugging format." \
+	 "Does not include preprocessor macro info." ] \
+	"info source before setting directory search list"
+
+    gdb_test "dir $strip_dir" \
+	[search_dir_list [list \
+			      "$strip_dir" \
+			      "\\\$cdir" \
+			      "\\\$cwd"]]
+    gdb_test "list" [multi_line \
+			 "1\[ \t\]+int" \
+			 "2\[ \t\]+main \\(\\)" \
+			 "3\[ \t\]+\\{" \
+			 "4\[ \t\]+return 0;" \
+			 "5\[ \t\]+\\}" ]
+
+    gdb_test "info source" \
+	[multi_line \
+	     "Current source file is ${srcfile}" \
+	     "Compilation directory is \[^\n\r\]+" \
+	     "Located in ${new_srcfile}" \
+	     "Contains 5 lines." \
+	     "Source language is c." \
+	     "Producer is \[^\n\r\]+" \
+	     "\[^\n\r\]+" \
+	     "\[^\n\r\]+" ] \
+	"info source after setting directory search list"
+}
+
+test_changing_search_directory
+test_truncated_comp_dir