This is an archive of the discontinued LLVM Phabricator instance.

Differential D55230

[lit] Multiple build outputs and default target bitness
ClosedPublic

Authored by zturner on Dec 3 2018, 12:06 PM.

Details

Reviewers
labath
stella.stamenova
aleksandr.urakov
Commits
rGcb6788d7b35b: [build.py] A few general improvements.
rLLDB348305: [build.py] A few general improvements.
rL348305: [build.py] A few general improvements.
Summary

This patch introduces two improvements to the build.py script.

First, it supports multiple build inputs and outputs. This is helpful if you want to run with mode=compile but you want to compile many source files at once. Now, you can specify an output directory and it will write all of them to that location.

Second, it changes the default architecture of test executables to be whatever the system is.

Both of these will help with porting DIA PDB tests over to the new build script, but are also generally useful features that most tests will probably want anyway.

Diff Detail

Repository
rL LLVM

Event Timeline

zturner created this revision.Dec 3 2018, 12:06 PM
Herald added a subscriber: mgorny. · View Herald TranscriptDec 3 2018, 12:06 PM
clayborg added a subscriber: clayborg.Dec 3 2018, 12:54 PM

What is the reason we aren't using cmake + ninja for this kind of stuff? Or is it using it under the covers?

zturner added a comment.Dec 3 2018, 1:04 PM
In D55230#1317320, @clayborg wrote:

What is the reason we aren't using cmake + ninja for this kind of stuff? Or is it using it under the covers?

CMake gives you a nice static configuration for how you want to build your host toolchain, but that doesn't always match up with how you want to build your test inferiors. You might even have a situation where two test inferiors need to be built with different toolchains than each other. I think dotest has a lot of the same issues here, that's why we use the Makefile system over there.

Eventually, we will want to be able to decouple the test suite from the local build entirely, so that you could invoke it from the command line similar to how you can currently invoke dotest from the command line. This would allow running the test suite multiple times with different toolchains or settings, which wouldn't be possible if there's a single upfront configuration done at CMake time.

It's also nice for test reproducibility to be able to look at a test and see the extact reproducer, e.g. (run this command line to build, then run lldb in this way and it repros).

stella.stamenova added inline comments.Dec 3 2018, 2:48 PM
lldb/lit/helper/build.py
664 ↗(On Diff #176447)

Why not? It makes sense that the test might still need to know and/or specify the name of the binary that was produced. The script could require that both -o and --outdir are specified, but I think -o is still needed.

aleksandr.urakov mentioned this in D54942: [PDB] Make PDB lit tests use the new builder.Dec 4 2018, 5:41 AM

Thanks! I've updated DIA PDB tests in D55230.

This revision was not accepted when it landed; it landed in state Needs Review.Dec 4 2018, 1:51 PM
Closed by commit rL348305: [build.py] A few general improvements. (authored by zturner). · Explain Why
This revision was automatically updated to reflect the committed changes.
Herald added a subscriber: llvm-commits. · View Herald TranscriptDec 4 2018, 1:51 PM
stella.stamenova added a comment.Dec 5 2018, 10:41 AM

LLDB :: BuildScript/toolchain-clang-cl.test is failing on Linux.

2018-12-05T17:59:42.1570845Z Command Output (stderr):
2018-12-05T17:59:42.1582271Z --
2018-12-05T17:59:42.1593369Z Traceback (most recent call last):
2018-12-05T17:59:42.1604689Z   File "/home/e2admin/vstsdrive/_work/51/s/llvm/tools/lldb/lit/helper/build.py", line 723, in <module>
2018-12-05T17:59:42.1615962Z     build(cmds)
2018-12-05T17:59:42.1627379Z   File "/home/e2admin/vstsdrive/_work/51/s/llvm/tools/lldb/lit/helper/build.py", line 621, in build
2018-12-05T17:59:42.1794738Z     print_environment(env)
2018-12-05T17:59:42.1806250Z   File "/home/e2admin/vstsdrive/_work/51/s/llvm/tools/lldb/lit/helper/build.py", line 154, in print_environment
2018-12-05T17:59:42.1817473Z     for e in env:
2018-12-05T17:59:42.1837821Z TypeError: 'NoneType' object is not iterable
2018-12-05T17:59:42.1850161Z /home/e2admin/vstsdrive/_work/51/s/llvm/tools/lldb/lit/BuildScript/toolchain-clang-cl.test:13:11: error: CHECK-32: expected string not found in input
2018-12-05T17:59:42.1861865Z CHECK-32: Output: {{.*}}toolchain-clang-cl.test.tmp\foo.exe
2018-12-05T17:59:42.1872878Z           ^
2018-12-05T17:59:42.1883892Z <stdin>:5:2: note: scanning from here
2018-12-05T17:59:42.1895337Z  Output: /home/e2admin/vstsdrive/_work/51/b/LLVMBuild/tools/lldb/lit/BuildScript/Output/toolchain-clang-cl.test.tmp/foo.exe
2018-12-05T17:59:42.1906629Z  ^
2018-12-05T17:59:42.1917610Z <stdin>:5:82: note: possible intended match here
2018-12-05T17:59:42.1929075Z  Output: /home/e2admin/vstsdrive/_work/51/b/LLVMBuild/tools/lldb/lit/BuildScript/Output/toolchain-clang-cl.test.tmp/foo.exe
2018-12-05T17:59:42.1940723Z                                                                                  ^
2018-12-05T17:59:42.1946391Z 
2018-12-05T17:59:42.1957310Z --

Revision Contents

PathSize
lldb/
trunk/
lit/
BuildScript/
35 lines
32 lines
49 lines
62 lines
7 lines
helper/
148 lines
5 lines
1 line

Diff 176707

lldb/trunk/lit/BuildScript/modes.test

RUN: %build -n --verbose --arch=32 --mode=compile --compiler=any -o %t/foo.out foobar.c \
RUN: | FileCheck --check-prefix=COMPILE %s
RUN: %build -n --verbose --arch=32 --mode=compile --compiler=any --outdir %t foo.c bar.c \
RUN: | FileCheck --check-prefix=COMPILE-MULTI %s
RUN: %build -n --verbose --arch=32 --mode=link --compiler=any -o %t/foo.exe foobar.obj \
RUN: | FileCheck --check-prefix=LINK %s
RUN: %build -n --verbose --arch=32 --mode=link --compiler=any -o %t/foobar.exe foo.obj bar.obj \
RUN: | FileCheck --check-prefix=LINK-MULTI %s
RUN: %build -n --verbose --arch=32 --mode=compile-and-link --compiler=any -o %t/foobar.exe foobar.c \
RUN: | FileCheck --check-prefix=BOTH %s
RUN: %build -n --verbose --arch=32 --mode=compile-and-link --compiler=any -o %t/foobar.exe foo.c bar.c \
RUN: | FileCheck --check-prefix=BOTH-MULTI %s
COMPILE: compiling foobar.c -> foo.out
COMPILE-MULTI: compiling foo.c -> foo.obj
COMPILE-MULTI: compiling bar.c -> bar.obj
LINK: linking foobar.obj -> foo.exe
LINK-MULTI: linking foo.obj+bar.obj -> foobar.exe
BOTH: compiling foobar.c -> foobar.obj
BOTH: linking foobar.obj -> foobar.exe
BOTH-MULTI: compiling foo.c -> foo.obj
BOTH-MULTI: compiling bar.c -> bar.obj
BOTH-MULTI: linking foo.obj+bar.obj -> foobar.exe

lldb/trunk/lit/BuildScript/script-args.test

RUN: %build -n --verbose --arch=32 --mode=compile --compiler=any -o %t/foo.out foobar.c \
RUN: | FileCheck %s
RUN: %build -n --verbose --arch=32 --mode=compile --compiler=any --outdir %t foo.c bar.c \
RUN: | FileCheck --check-prefix=MULTI-INPUT %s
CHECK: Script Arguments:
CHECK-NEXT: Arch: 32
CHECK: Compiler: any
CHECK: Outdir: {{.*}}script-args.test.tmp
CHECK: Output: {{.*}}script-args.test.tmp{{.}}foo.out
CHECK: Nodefaultlib: False
CHECK: Opt: none
CHECK: Mode: compile
CHECK: Clean: True
CHECK: Verbose: True
CHECK: Dryrun: True
CHECK: Inputs: foobar.c
MULTI-INPUT: Script Arguments:
MULTI-INPUT-NEXT: Arch: 32
MULTI-INPUT-NEXT: Compiler: any
MULTI-INPUT-NEXT: Outdir: {{.*}}script-args.test.tmp
MULTI-INPUT-NEXT: Output:
MULTI-INPUT-NEXT: Nodefaultlib: False
MULTI-INPUT-NEXT: Opt: none
MULTI-INPUT-NEXT: Mode: compile
MULTI-INPUT-NEXT: Clean: True
MULTI-INPUT-NEXT: Verbose: True
MULTI-INPUT-NEXT: Dryrun: True
MULTI-INPUT-NEXT: Inputs: foo.c
MULTI-INPUT-NEXT: bar.c
No newline at end of file

lldb/trunk/lit/BuildScript/toolchain-clang-cl.test

REQUIRES: lld
RUN: %build -n --verbose --arch=32 --compiler=clang-cl --mode=compile-and-link -o %t/foo.exe foobar.c \
RUN: | FileCheck --check-prefix=CHECK-32 %s
RUN: %build -n --verbose --arch=64 --compiler=clang-cl --mode=compile-and-link -o %t/foo.exe foobar.c \
RUN: | FileCheck --check-prefix=CHECK-64 %s
CHECK-32: Script Arguments:
CHECK-32: Arch: 32
CHECK-32: Compiler: clang-cl
CHECK-32: Outdir: {{.*}}
CHECK-32: Output: {{.*}}toolchain-clang-cl.test.tmp\foo.exe
CHECK-32: Nodefaultlib: False
CHECK-32: Opt: none
CHECK-32: Mode: compile
CHECK-32: Clean: True
CHECK-32: Verbose: True
CHECK-32: Dryrun: True
CHECK-32: Inputs: foobar.c
CHECK-32: Cleaning {{.*}}toolchain-clang-cl.test.tmp{{.}}foobar.ilk
CHECK-32: Cleaning {{.*}}toolchain-clang-cl.test.tmp{{.}}foobar.obj
CHECK-32: Cleaning {{.*}}toolchain-clang-cl.test.tmp{{.}}foo.pdb
CHECK-32: Cleaning {{.*}}toolchain-clang-cl.test.tmp{{.}}foo.exe
CHECK-32: compiling foobar.c -> foobar.obj
CHECK-32: {{.*}}clang-cl{{(.exe)?}} -m32
CHECK-32: linking foobar.obj -> foo.exe
CHECK-32: {{.*}}lld-link
CHECK-64: Script Arguments:
CHECK-64: Arch: 64
CHECK-64: Compiler: clang-cl
CHECK-64: Outdir: {{.*}}
CHECK-64: Output: {{.*}}toolchain-clang-cl.test.tmp\foo.exe
CHECK-64: Nodefaultlib: False
CHECK-64: Opt: none
CHECK-64: Mode: compile
CHECK-64: Clean: True
CHECK-64: Verbose: True
CHECK-64: Dryrun: True
CHECK-64: Inputs: foobar.c
CHECK-64: Cleaning {{.*}}toolchain-clang-cl.test.tmp{{.}}foobar.ilk
CHECK-64: Cleaning {{.*}}toolchain-clang-cl.test.tmp{{.}}foobar.obj
CHECK-64: Cleaning {{.*}}toolchain-clang-cl.test.tmp{{.}}foo.pdb
CHECK-64: Cleaning {{.*}}toolchain-clang-cl.test.tmp{{.}}foo.exe
CHECK-64: compiling foobar.c -> foobar.obj
CHECK-64: {{.*}}clang-cl{{(.exe)?}} -m64
CHECK-64: linking foobar.obj -> foo.exe
CHECK-64: {{.*}}lld-link{{(.exe)?}}

lldb/trunk/lit/BuildScript/toolchain-msvc.test

REQUIRES: system-windows, msvc
RUN: %build -n --verbose --arch=32 --compiler=msvc --mode=compile-and-link -o %t/foo.exe foobar.c \
RUN: | FileCheck --check-prefix=X86 %s
RUN: %build -n --verbose --arch=64 --compiler=msvc --mode=compile-and-link -o %t/foo.exe foobar.c \
RUN: | FileCheck --check-prefix=X64 %s
X86: Script Arguments:
X86: Arch: 32
X86: Compiler: msvc
X86: Outdir: {{.*}}
X86: Output: {{.*}}toolchain-msvc.test.tmp\foo.exe
X86: Nodefaultlib: False
X86: Opt: none
X86: Mode: compile
X86: Clean: True
X86: Verbose: True
X86: Dryrun: True
X86: Inputs: foobar.c
X86: Cleaning {{.*}}toolchain-msvc.test.tmp\foobar.ilk
X86: Cleaning {{.*}}toolchain-msvc.test.tmp\foobar.obj
X86: Cleaning {{.*}}toolchain-msvc.test.tmp\foo.pdb
X86: Cleaning {{.*}}toolchain-msvc.test.tmp\foo.exe
X86: compiling foobar.c -> foobar.obj
X86: Command Line: {{.*}}\{{[Hh]ost[Xx]64}}\x86\cl.exe
X86: linking foobar.obj -> foo.exe
X86: Command Line: {{.*}}\{{[Hh]ost[Xx]64}}\x86\link.exe
X86: Env
X86: LIB = {{.*}}\ATLMFC\lib\x86
X86: {{.*}}\lib\x86
X86: {{.*}}\ucrt\x86
X86: {{.*}}\um\x86
X86: PATH = {{.*}}\bin\{{[Hh]ost[Xx]64}}\x64
X64: Script Arguments:
X64: Arch: 64
X64: Compiler: msvc
X64: Outdir: {{.*}}
X64: Output: {{.*}}toolchain-msvc.test.tmp\foo.exe
X64: Nodefaultlib: False
X64: Opt: none
X64: Mode: compile
X64: Clean: True
X64: Verbose: True
X64: Dryrun: True
X64: Inputs: foobar.c
X64: Cleaning {{.*}}toolchain-msvc.test.tmp\foobar.ilk
X64: Cleaning {{.*}}toolchain-msvc.test.tmp\foobar.obj
X64: Cleaning {{.*}}toolchain-msvc.test.tmp\foo.pdb
X64: Cleaning {{.*}}toolchain-msvc.test.tmp\foo.exe
X64: compiling foobar.c -> foobar.obj
X64: Command Line: {{.*}}\{{[Hh]ost[Xx]64}}\x64\cl.exe
X64: linking foobar.obj -> foo.exe
X64: Command Line: {{.*}}\{{[Hh]ost[Xx]64}}\x64\link.exe
X64: Env
X64: LIB = {{.*}}\ATLMFC\lib\x64
X64: {{.*}}\lib\x64
X64: {{.*}}\ucrt\x64
X64: {{.*}}\um\x64
X64: PATH = {{.*}}\bin\{{[Hh]ost[Xx]64}}\x64

lldb/trunk/lit/CMakeLists.txt

# Test runner infrastructure for LLDB. This configures the LLDB test trees # Test runner infrastructure for LLDB. This configures the LLDB test trees
# for use by Lit, and delegates to LLVM's lit test handlers. # for use by Lit, and delegates to LLVM's lit test handlers.
if (CMAKE_CFG_INTDIR STREQUAL ".") if (CMAKE_CFG_INTDIR STREQUAL ".")
set(LLVM_BUILD_MODE ".") set(LLVM_BUILD_MODE ".")
else () else ()
set(LLVM_BUILD_MODE "%(build_mode)s") set(LLVM_BUILD_MODE "%(build_mode)s")
endif () endif ()
if (CMAKE_SIZEOF_VOID_P EQUAL 8)
set(LLDB_IS_64_BITS 1)
endif()
if (NOT LLDB_TEST_USE_CUSTOM_C_COMPILER) if (NOT LLDB_TEST_USE_CUSTOM_C_COMPILER)
string(REPLACE ${CMAKE_CFG_INTDIR} ${LLVM_BUILD_MODE} LLDB_TEST_C_COMPILER ${LLDB_TEST_C_COMPILER}) string(REPLACE ${CMAKE_CFG_INTDIR} ${LLVM_BUILD_MODE} LLDB_TEST_C_COMPILER ${LLDB_TEST_C_COMPILER})
endif () endif ()
if (NOT LLDB_TEST_USE_CUSTOM_CXX_COMPILER) if (NOT LLDB_TEST_USE_CUSTOM_CXX_COMPILER)
string(REPLACE ${CMAKE_CFG_INTDIR} ${LLVM_BUILD_MODE} LLDB_TEST_CXX_COMPILER ${LLDB_TEST_CXX_COMPILER}) string(REPLACE ${CMAKE_CFG_INTDIR} ${LLVM_BUILD_MODE} LLDB_TEST_CXX_COMPILER ${LLDB_TEST_CXX_COMPILER})
endif () endif ()
Show All 26 Linesif(BUILD_SHARED_LIBS)
set(ENABLE_SHARED 1) set(ENABLE_SHARED 1)
else() else()
set(ENABLE_SHARED 0) set(ENABLE_SHARED 0)
endif(BUILD_SHARED_LIBS) endif(BUILD_SHARED_LIBS)
# the value is not canonicalized within LLVM # the value is not canonicalized within LLVM
llvm_canonicalize_cmake_booleans( llvm_canonicalize_cmake_booleans(
LLDB_DISABLE_PYTHON LLDB_DISABLE_PYTHON
LLVM_ENABLE_ZLIB) LLVM_ENABLE_ZLIB
LLDB_IS_64_BITS)
configure_lit_site_cfg( configure_lit_site_cfg(
${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.py.in ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.py.in
${CMAKE_CURRENT_BINARY_DIR}/lit.site.cfg.py ${CMAKE_CURRENT_BINARY_DIR}/lit.site.cfg.py
MAIN_CONFIG MAIN_CONFIG
${CMAKE_CURRENT_SOURCE_DIR}/lit.cfg.py) ${CMAKE_CURRENT_SOURCE_DIR}/lit.cfg.py)
configure_lit_site_cfg( configure_lit_site_cfg(
${CMAKE_CURRENT_SOURCE_DIR}/Unit/lit.site.cfg.py.in ${CMAKE_CURRENT_SOURCE_DIR}/Unit/lit.site.cfg.py.in
Show All 31 Lines

lldb/trunk/lit/helper/build.py

Show All 17 Linesif __name__ != "__main__":
raise RuntimeError("Do not import this script, run it instead") raise RuntimeError("Do not import this script, run it instead")
parser = argparse.ArgumentParser(description='LLDB compilation wrapper') parser = argparse.ArgumentParser(description='LLDB compilation wrapper')
parser.add_argument('--arch', parser.add_argument('--arch',
metavar='arch', metavar='arch',
dest='arch', dest='arch',
required=True, required=True,
help='Specify the architecture to target. Valid values=[32,64]') default='host',
choices=['32', '64', 'host'],
help='Specify the architecture to target.')
parser.add_argument('--compiler', parser.add_argument('--compiler',
metavar='compiler', metavar='compiler',
dest='compiler', dest='compiler',
required=True, required=True,
help='Path to a compiler executable, or one of the values [any, msvc, clang-cl, gcc, clang]') help='Path to a compiler executable, or one of the values [any, msvc, clang-cl, gcc, clang]')
parser.add_argument('--tools-dir', parser.add_argument('--tools-dir',
metavar='directory', metavar='directory',
dest='tools_dir', dest='tools_dir',
required=False, required=False,
action='append', action='append',
help='If specified, a path to search in addition to PATH when --compiler is not an exact path') help='If specified, a path to search in addition to PATH when --compiler is not an exact path')
if sys.platform == 'darwin': if sys.platform == 'darwin':
parser.add_argument('--apple-sdk', parser.add_argument('--apple-sdk',
metavar='apple_sdk', metavar='apple_sdk',
dest='apple_sdk', dest='apple_sdk',
default="macosx", default="macosx",
help='Specify the name of the Apple SDK (macosx, macosx.internal, iphoneos, iphoneos.internal, or path to SDK) and use the appropriate tools from that SDK\'s toolchain.') help='Specify the name of the Apple SDK (macosx, macosx.internal, iphoneos, iphoneos.internal, or path to SDK) and use the appropriate tools from that SDK\'s toolchain.')
parser.add_argument('--output', '-o', parser.add_argument('--output', '-o',
dest='output', dest='output',
metavar='file', metavar='file',
required=True, required=False,
default='',
help='Path to output file') help='Path to output file')
parser.add_argument('--outdir', '-d',
dest='outdir',
metavar='directory',
required=False,
help='Directory for output files')
parser.add_argument('--nodefaultlib', parser.add_argument('--nodefaultlib',
dest='nodefaultlib', dest='nodefaultlib',
action='store_true', action='store_true',
default=False, default=False,
help='When specified, the resulting image should not link against system libraries or include system headers. Useful when writing cross-targeting tests.') help='When specified, the resulting image should not link against system libraries or include system headers. Useful when writing cross-targeting tests.')
parser.add_argument('--opt', parser.add_argument('--opt',
dest='opt', dest='opt',
Show All 14 Linesparser.add_argument('--noclean',
help='Dont clean output file before building') help='Dont clean output file before building')
parser.add_argument('--verbose', parser.add_argument('--verbose',
dest='verbose', dest='verbose',
action='store_true', action='store_true',
default=False, default=False,
help='Print verbose output') help='Print verbose output')
parser.add_argument('input', parser.add_argument('-n', '--dry-run',
dest='dry',
action='store_true',
default=False,
help='Print the commands that would run, but dont actually run them')
parser.add_argument('inputs',
metavar='file', metavar='file',
help='Source file to compile / object file to link') nargs='+',
help='Source file(s) to compile / object file(s) to link')
args = parser.parse_args(args=sys.argv[1:]) args = parser.parse_args(args=sys.argv[1:])
def to_string(b): def to_string(b):
"""Return the parameter as type 'str', possibly encoding it. """Return the parameter as type 'str', possibly encoding it.
Show All 28 Linesdef to_string(b):
# The last type we might expect is the Python2 'unicode' type. There is no # The last type we might expect is the Python2 'unicode' type. There is no
# 'unicode' type in Python3 (all the Python3 cases were already handled). In # 'unicode' type in Python3 (all the Python3 cases were already handled). In
# order to get a 'str' object, we need to encode the 'unicode' object. # order to get a 'str' object, we need to encode the 'unicode' object.
try: try:
return b.encode('utf-8') return b.encode('utf-8')
except AttributeError: except AttributeError:
raise TypeError('not sure how to convert %s to %s' % (type(b), str)) raise TypeError('not sure how to convert %s to %s' % (type(b), str))
def format_text(lines, indent_0, indent_n):
result = ' ' * indent_0 + lines[0]
for next in lines[1:]:
result = result + '\n{0}{1}'.format(' ' * indent_n, next)
return result
def print_environment(env): def print_environment(env):
for e in env: for e in env:
value = env[e] value = env[e]
split = value.split(os.pathsep) lines = value.split(os.pathsep)
print(' {0} = {1}'.format(e, split[0])) formatted_value = format_text(lines, 0, 7 + len(e))
prefix_width = 3 + len(e) print(' {0} = {1}'.format(e, formatted_value))
for next in split[1:]:
print(' {0}{1}'.format(' ' * prefix_width, next))
def find_executable(binary_name, search_paths): def find_executable(binary_name, search_paths):
if sys.platform == 'win32': if sys.platform == 'win32':
binary_name = binary_name + '.exe' binary_name = binary_name + '.exe'
search_paths = os.pathsep.join(search_paths) search_paths = os.pathsep.join(search_paths)
paths = search_paths + os.pathsep + os.environ.get('PATH', '') paths = search_paths + os.pathsep + os.environ.get('PATH', '')
for path in paths.split(os.pathsep): for path in paths.split(os.pathsep):
Show All 37 Lines if name.startswith('gcc') or name.startswith('g++'):
return 'gcc' return 'gcc'
if name == 'cc' or name == 'c++': if name == 'cc' or name == 'c++':
return 'generic' return 'generic'
return 'unknown' return 'unknown'
class Builder(object): class Builder(object):
def __init__(self, toolchain_type, args): def __init__(self, toolchain_type, args):
self.toolchain_type = toolchain_type self.toolchain_type = toolchain_type
self.input = args.input self.inputs = args.inputs
self.arch = args.arch self.arch = args.arch
self.opt = args.opt self.opt = args.opt
self.outdir = args.outdir
self.compiler = args.compiler self.compiler = args.compiler
self.clean = args.clean self.clean = args.clean
self.output = args.output self.output = args.output
self.mode = args.mode self.mode = args.mode
self.nodefaultlib = args.nodefaultlib self.nodefaultlib = args.nodefaultlib
self.verbose = args.verbose self.verbose = args.verbose
class MsvcBuilder(Builder): class MsvcBuilder(Builder):
▲ Show 20 Lines • Show All 257 Lines • ▼ Show 20 Lines def _get_visual_studio_environment(self):
compileenv['INCLUDE'] = os.pathsep.join(includes) compileenv['INCLUDE'] = os.pathsep.join(includes)
compileenv.update(defaultenv) compileenv.update(defaultenv)
if libs: if libs:
linkenv = {} linkenv = {}
linkenv['LIB'] = os.pathsep.join(libs) linkenv['LIB'] = os.pathsep.join(libs)
linkenv.update(defaultenv) linkenv.update(defaultenv)
return (compileenv, linkenv) return (compileenv, linkenv)
def _ilk_file_name(self): def _output_name(self, input, extension):
basename = os.path.basename(input)
basename = os.path.splitext(basename)[0] + extension
output = os.path.join(self.outdir, basename)
return os.path.normpath(output)
def _ilk_file_names(self):
if self.mode == 'link': if self.mode == 'link':
return None return []
return os.path.splitext(self.output)[0] + '.ilk'
def _obj_file_name(self): return [self._output_name(x, '.ilk') for x in self.inputs]
if self.mode == 'compile':
return self.output def _obj_file_names(self):
return os.path.splitext(self.output)[0] + '.obj' if self.mode == 'link':
return self.inputs
if self.mode == 'compile' and self.output:
return [self.output]
return [self._output_name(x, '.obj') for x in self.inputs]
def _pdb_file_name(self): def _pdb_file_name(self):
if self.mode == 'compile': if self.mode == 'compile':
return None return None
return os.path.splitext(self.output)[0] + '.pdb' return os.path.splitext(self.output)[0] + '.pdb'
def _exe_file_name(self): def _exe_file_name(self):
if self.mode == 'compile': if self.mode == 'compile':
return None return None
return self.output return self.output
def _get_compilation_command(self): def _get_compilation_command(self, source, obj):
args = [] args = []
args.append(self.compiler) args.append(self.compiler)
if self.toolchain_type == 'clang-cl': if self.toolchain_type == 'clang-cl':
args.append('-m' + self.arch) args.append('-m' + self.arch)
if self.opt == 'none': if self.opt == 'none':
args.append('/Od') args.append('/Od')
Show All 9 Lines def _get_compilation_command(self, source, obj):
args.append('/GS-') args.append('/GS-')
args.append('/GR-') args.append('/GR-')
args.append('/Z7') args.append('/Z7')
if self.toolchain_type == 'clang-cl': if self.toolchain_type == 'clang-cl':
args.append('-Xclang') args.append('-Xclang')
args.append('-fkeep-static-consts') args.append('-fkeep-static-consts')
args.append('/c') args.append('/c')
args.append('/Fo' + self._obj_file_name()) args.append('/Fo' + obj)
args.append(self.input) args.append(source)
input = os.path.basename(self.input)
output = os.path.basename(self._obj_file_name()) return ('compiling', [source], obj,
return ('compiling {0} -> {1}'.format(input, output),
self.compile_env, self.compile_env,
args) args)
def _get_link_command(self): def _get_link_command(self):
args = [] args = []
args.append(self.linker) args.append(self.linker)
args.append('/DEBUG:FULL') args.append('/DEBUG:FULL')
args.append('/INCREMENTAL:NO') args.append('/INCREMENTAL:NO')
if self.nodefaultlib: if self.nodefaultlib:
args.append('/nodefaultlib') args.append('/nodefaultlib')
args.append('/entry:main') args.append('/entry:main')
args.append('/PDB:' + self._pdb_file_name()) args.append('/PDB:' + self._pdb_file_name())
args.append('/OUT:' + self._exe_file_name()) args.append('/OUT:' + self._exe_file_name())
args.append(self._obj_file_name()) args.extend(self._obj_file_names())
input = os.path.basename(self._obj_file_name()) return ('linking', self._obj_file_names(), self._exe_file_name(),
output = os.path.basename(self._exe_file_name())
return ('linking {0} -> {1}'.format(input, output),
self.link_env, self.link_env,
args) args)
def build_commands(self): def build_commands(self):
commands = [] commands = []
if self.mode == 'compile' or self.mode == 'compile-and-link': if self.mode == 'compile' or self.mode == 'compile-and-link':
commands.append(self._get_compilation_command()) for input, output in zip(self.inputs, self._obj_file_names()):
commands.append(self._get_compilation_command(input, output))
if self.mode == 'link' or self.mode == 'compile-and-link': if self.mode == 'link' or self.mode == 'compile-and-link':
commands.append(self._get_link_command()) commands.append(self._get_link_command())
return commands return commands
def output_files(self): def output_files(self):
outdir = os.path.dirname(self.output) outdir = os.path.dirname(self.output)
file = os.path.basename(self.output) file = os.path.basename(self.output)
name, ext = os.path.splitext(file) name, ext = os.path.splitext(file)
outputs = [] outputs = []
if self.mode == 'compile' or self.mode == 'compile-and-link': if self.mode == 'compile' or self.mode == 'compile-and-link':
outputs.append(self._ilk_file_name()) outputs.extend(self._ilk_file_names())
outputs.append(self._obj_file_name()) outputs.extend(self._obj_file_names())
if self.mode == 'link' or self.mode == 'compile-and-link': if self.mode == 'link' or self.mode == 'compile-and-link':
outputs.append(self._pdb_file_name()) outputs.append(self._pdb_file_name())
outputs.append(self._exe_file_name()) outputs.append(self._exe_file_name())
return [x for x in outputs if x is not None] return [x for x in outputs if x is not None]
class GccBuilder(Builder): class GccBuilder(Builder):
def __init__(self, toolchain_type, args): def __init__(self, toolchain_type, args):
Show All 9 Linesdef indent(text, spaces):
def prefixed_lines(): def prefixed_lines():
prefix = ' ' * spaces prefix = ' ' * spaces
for line in text.splitlines(True): for line in text.splitlines(True):
yield prefix + line yield prefix + line
return ''.join(prefixed_lines()) return ''.join(prefixed_lines())
def build(commands): def build(commands):
global args global args
for (status, env, child_args) in commands: for (status, inputs, output, env, child_args) in commands:
print('\n\n') print('\n\n')
print(status) if not args.verbose:
inputs = [os.path.basename(x) for x in inputs]
output = os.path.basename(output)
print(status + ' {0} -> {1}'.format('+'.join(inputs), output))
if args.verbose: if args.verbose:
print(' Command Line: ' + ' '.join(child_args)) print(' Command Line: ' + ' '.join(child_args))
print(' Env:') print(' Env:')
print_environment(env) print_environment(env)
if args.dry:
continue
popen = subprocess.Popen(child_args, popen = subprocess.Popen(child_args,
stdout=subprocess.PIPE, stdout=subprocess.PIPE,
stderr=subprocess.PIPE, stderr=subprocess.PIPE,
env=env, env=env,
universal_newlines=True) universal_newlines=True)
stdout, stderr = popen.communicate() stdout, stderr = popen.communicate()
res = popen.wait() res = popen.wait()
if res == -signal.SIGINT: if res == -signal.SIGINT:
raise KeyboardInterrupt raise KeyboardInterrupt
print(' STDOUT:') print(' STDOUT:')
print(indent(stdout, 4)) print(indent(stdout, 4))
if res != 0: if res != 0:
print(' STDERR:') print(' STDERR:')
print(indent(stderr, 4)) print(indent(stderr, 4))
sys.exit(res) sys.exit(res)
def clean(files): def clean(files):
global args global args
for o in files: for o in files:
file = o if args.verbose else os.path.basename(o) file = o if args.verbose else os.path.basename(o)
print('Cleaning {0}'.format(file)) print('Cleaning {0}'.format(file))
try: try:
if os.path.exists(o): if os.path.exists(o):
if not args.dry:
os.remove(o) os.remove(o)
if args.verbose: if args.verbose:
print(' The file was successfully cleaned.') print(' The file was successfully cleaned.')
elif args.verbose: elif args.verbose:
print(' The file does not exist.') print(' The file does not exist.')
except: except:
if args.verbose: if args.verbose:
print(' The file could not be removed.') print(' The file could not be removed.')
def fix_arguments(args):
if not args.inputs:
raise ValueError('No input files specified')
if args.output and args.mode == 'compile' and len(args.inputs) > 1:
raise ValueError('Cannot specify -o with mode=compile and multiple source files. Use --outdir instead.')
if not args.dry:
args.inputs = [os.path.abspath(x) for x in args.inputs]
# If user didn't specify the outdir, use the directory of the first input.
if not args.outdir:
if args.output:
args.outdir = os.path.dirname(args.output)
else:
args.outdir = os.path.dirname(args.inputs[0])
args.outdir = os.path.abspath(args.outdir)
args.outdir = os.path.normpath(args.outdir)
# If user specified a non-absolute path for the output file, append the
# output directory to it.
if args.output:
if not os.path.isabs(args.output):
args.output = os.path.join(args.outdir, args.output)
args.output = os.path.normpath(args.output)
fix_arguments(args)
(toolchain_type, toolchain_path) = find_toolchain(args.compiler, args.tools_dir) (toolchain_type, toolchain_path) = find_toolchain(args.compiler, args.tools_dir)
if not toolchain_path or not toolchain_type: if not toolchain_path or not toolchain_type:
print('Unable to find toolchain {0}'.format(args.compiler)) print('Unable to find toolchain {0}'.format(args.compiler))
sys.exit(1) sys.exit(1)
if args.verbose: if args.verbose:
print("Script Environment:") print('Script Arguments:')
print(' Arch: ' + args.arch)
print(' Compiler: ' + args.compiler)
print(' Outdir: ' + args.outdir)
print(' Output: ' + args.output)
print(' Nodefaultlib: ' + str(args.nodefaultlib))
print(' Opt: ' + args.opt)
print(' Mode: ' + args.mode)
print(' Clean: ' + str(args.clean))
print(' Verbose: ' + str(args.verbose))
print(' Dryrun: ' + str(args.dry))
print(' Inputs: ' + format_text(args.inputs, 0, 10))
print('Script Environment:')
print_environment(os.environ) print_environment(os.environ)
args.compiler = toolchain_path args.compiler = toolchain_path
if not os.path.exists(args.compiler): if not os.path.exists(args.compiler) and not args.dry:
raise ValueError('The toolchain {} does not exist.'.format(args.compiler)) raise ValueError('The toolchain {} does not exist.'.format(args.compiler))
if toolchain_type == 'msvc' or toolchain_type=='clang-cl': if toolchain_type == 'msvc' or toolchain_type=='clang-cl':
builder = MsvcBuilder(toolchain_type, args) builder = MsvcBuilder(toolchain_type, args)
else: else:
builder = GccBuilder(toolchain_type, args) builder = GccBuilder(toolchain_type, args)
if args.clean: if args.clean:
clean(builder.output_files()) clean(builder.output_files())
cmds = builder.build_commands() cmds = builder.build_commands()
build(cmds) build(cmds)

lldb/trunk/lit/helper/toolchain.py

Show All 14 Linesdef use_lldb_substitutions(config):
# anywhere else, since they are specifically the programs which are actually being tested. # anywhere else, since they are specifically the programs which are actually being tested.
dsname = 'debugserver' if platform.system() in ['Darwin'] else 'lldb-server' dsname = 'debugserver' if platform.system() in ['Darwin'] else 'lldb-server'
dsargs = [] if platform.system() in ['Darwin'] else ['gdbserver'] dsargs = [] if platform.system() in ['Darwin'] else ['gdbserver']
lldbmi = ToolSubst('%lldbmi', lldbmi = ToolSubst('%lldbmi',
command=FindTool('lldb-mi'), command=FindTool('lldb-mi'),
extra_args=['--synchronous'], extra_args=['--synchronous'],
unresolved='ignore') unresolved='ignore')
build_script = os.path.dirname(__file__) build_script = os.path.dirname(__file__)
build_script = os.path.join(build_script, 'build.py') build_script = os.path.join(build_script, 'build.py')
build_script_args = [build_script, build_script_args = [build_script,
'--compiler=any', # Default to best compiler '--compiler=any', # Default to best compiler
'--arch=64'] # Default to 64-bit, user can override '--arch=' + str(config.lldb_bitness)]
if config.lldb_lit_tools_dir: if config.lldb_lit_tools_dir:
build_script_args.append('--tools-dir={0}'.format(config.lldb_lit_tools_dir)) build_script_args.append('--tools-dir={0}'.format(config.lldb_lit_tools_dir))
if config.lldb_tools_dir: if config.lldb_tools_dir:
build_script_args.append('--tools-dir={0}'.format(config.lldb_tools_dir)) build_script_args.append('--tools-dir={0}'.format(config.lldb_tools_dir))
primary_tools = [ primary_tools = [
ToolSubst('%lldb', ToolSubst('%lldb',
command=FindTool('lldb'), command=FindTool('lldb'),
extra_args=['-S', extra_args=['-S',
os.path.join(config.test_source_root, os.path.join(config.test_source_root,
'lit-lldb-init')]), 'lit-lldb-init')]),
lldbmi, lldbmi,
ToolSubst('%debugserver', ToolSubst('%debugserver',
▲ Show 20 Lines • Show All 80 Lines • Show Last 20 Lines

lldb/trunk/lit/lit.site.cfg.py.in

Show All 10 Lines
config.lldb_tools_dir = "@LLDB_TOOLS_DIR@" config.lldb_tools_dir = "@LLDB_TOOLS_DIR@"
# Since it comes from the command line, it may have backslashes which # Since it comes from the command line, it may have backslashes which
# should not need to be escaped. # should not need to be escaped.
config.lldb_lit_tools_dir = r"@LLDB_LIT_TOOLS_DIR@" config.lldb_lit_tools_dir = r"@LLDB_LIT_TOOLS_DIR@"
config.target_triple = "@TARGET_TRIPLE@" config.target_triple = "@TARGET_TRIPLE@"
config.python_executable = "@PYTHON_EXECUTABLE@" config.python_executable = "@PYTHON_EXECUTABLE@"
config.have_zlib = @LLVM_ENABLE_ZLIB@ config.have_zlib = @LLVM_ENABLE_ZLIB@
config.host_triple = "@LLVM_HOST_TRIPLE@" config.host_triple = "@LLVM_HOST_TRIPLE@"
config.lldb_bitness = 64 if @LLDB_IS_64_BITS@ else 32
# Support substitution of the tools and libs dirs with user parameters. This is # Support substitution of the tools and libs dirs with user parameters. This is
# used when we can't determine the tool dir at configuration time. # used when we can't determine the tool dir at configuration time.
try: try:
config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
config.llvm_libs_dir = config.llvm_libs_dir % lit_config.params config.llvm_libs_dir = config.llvm_libs_dir % lit_config.params
config.llvm_shlib_dir = config.llvm_shlib_dir % lit_config.params config.llvm_shlib_dir = config.llvm_shlib_dir % lit_config.params
config.lldb_libs_dir = config.lldb_libs_dir % lit_config.params config.lldb_libs_dir = config.lldb_libs_dir % lit_config.params
Show All 12 Lines