KotJohansson/palemoon27
1
0
Fork 0
mirror of https://github.com/roytam1/palemoon27.git synced 2026-05-26 14:30:27 +00:00
Code Issues Projects Releases Wiki Activity

import changes from `dev' branch of rmottola/Arctic-Fox:

Browse Source 
- Bug 1159056. Don't store a value in CSS pixels in a variable with type nscoord in ScrollFrameHelper::SetCoordAttribute. r=dholbert (432f06158)
- Bug 1138442 - Disable reftests on Mulet in parity with B2G/B2G Desktop. r=ahal (8b5c55da7)
- Don't clip in ComputeFrameMetrics if the scrollframe won't be layerized. (bug 1148582 part 2, r=tn) (088b6ef29)
- Bug 1157579 - Make sure the visual scroll port size includes the area of scrollbars when used in TryLayout. r=tn (c35244374)
- Bug 1139306: Fix margin sides for right-to-left scrollbars depending on the actual position of the scrollbar as determined by layout.scrollbar.side, r=tn (34328312d)
- Fix some full zoom FrameMetrics changes not propagating to APZ. (bug 1147648, r=botond) (91f009ac4)
- Bug 1159405 - Fix "Found a non-root APZ with no handoff parent". r=botond (09b0ddf07)
- Bug 1161040 - Don't set a scrollparent on the root content document in the child process that points to itself. r=tn (d0dfbffb0)
- Bug 1161040 - Add some assertions to guard against infinite loops in the handoff chains. r=botond (e6bae82e8)
- Bug 1153539 patch 1 - Improve comments for nsLayoutUtils::ComputeSuitableScaleForAnimation. r=roc (1b2bc00cd)
- Bug 1153539 patch 2 - Compute scale for rasterizing off-main-thread transform animations based on ratio to display size in addition to maximum and minimum scale. r=roc (97d632d0f)
- Bug 1160250 - Extract a helper method from the triplicated comp-bounds function to determine if we need to deflate the scrollbars. r=botond,tn (8d79c845f)
- Bug 1160250 - Extract a helper function from the triplicated composition-bounds-updating code. r=botond,tn (5b3f71329)
- Bug 1160250 - Collapse the code from CalculateRootCompositionSize into UpdateCompositionBounds. r=botond,tn (db0f081d6)
- Bug 1160250 - Put in a missing transfomToAncestor scaling factor. r=botond,tn (14837aa3a)
- Bug 1160250 - Fix up the rect being used for the composition size calculation. r=tn (0ba58021f)
- Bug 1139575 - Reset velocity on axes when going from a pan to a pinch. r=botond (2e03d04ab)
- Bug 1136971 - Draw layer borders correctly in the presence of multi-FrameMetrics. r=kats (e72f5b171)
- Bug 1148871 - Only apply checkerboarding background color to opaque layers. r=botond (20e070b84)
- Bug 1157327 - Don't cull out container layers, because their prepared data might be null and cause crashes. r=nical (3d06c925f)
- Bug 1157327 follow-up to add a comment. r=me and DONTBUILD (4cd74c0a1)
- Bug 1160566 - Make FrameMetrics.mCompositionBounds private and add a getter/setter for it. r=botond (277dd9afd)
- Bug 1160566 - Change some return values to be reference-to-const. r=botond (2ce17b63f)
- Bug 1158933 - Prevent scrollbars from jumping while zoomed in the B2G browser. r=botond (eca8aae7d)
- Bug 1158933 - Replace the compositedHeight/scrollableHeight (and width) term with the adjusted scrollThumbRatio. r=botond (236af4c4a)
- Bug 1156401 - Fix painting when building fennec with apzc. r=kats, r=botond (be18764b9)
- Bug 1159305 - patch 1 - Provide logical accessors for nsStylePosition and nsStyleSides fields. r=dbaron (7a90c519d)
- Bug 1159305 - patch 2 - Clean up layout code by using the new accessors. r=smontagu (926eb7dd5)
- Bug 1164406 - Expose scrollbar-area deflation in nsLayoutUtils. r=kats (42e5e8784)
- Bug 1139895 - Add fuzz for 1062792-1.html. r=tn (e38d0ac73)
- Bug 1139893 - Disable reftest bugs/1062108-1.html on mulet. r=jfkthame (930590d73)
- Move the APZ scroll frame clip onto FrameMetrics. (bug 1148582 part 3, r=mstange,tn) (c8e05f8d7)
- Fix PostprocessRetainedLayers to not test occlusion with asynchronous clips. (bug 1148582 part 5, r=tn) (cc71cafe2)
- Bug 1146626 - Refresh the reftest snapshot after adjusting the async scroll offsets. r=roc (1f041955d)
- Bug 1160642. Add reftest-async-zoom to apply an async zoom before taking snapshot. r=dbaron (1e8e54d8d)
- Bug 1160285 - Add a staticruntime version of mfbt and link some stuff against it. r=glandium (c733c49c7)
- Bug 786520 - Install things to $(DIST)/branding from moz.build instead of manual rules in Makefile.ins. r=mshal (42d38eb26)
- Bug 1155776 - move USE_EXTENSION_MANIFEST to moz.build; r=mshal (a87ecc2ca)
- Bug 1155816 - part 0 - remove NO_JS_MANIFEST from testing/mochitest/Makefile.in; r=mshal (af90a362c)
- Bug 1155816 - part 1 - move NO_JS_MANIFEST to moz.build; r=mshal (3a55e99a0)
- Bug 978591 - Remove per-directory uses of MOZ_CHROME_FILE_FORMAT. r=glandium (5084d62b5)
- Bug 991983 - Set GARBAGE for GeneratedSources in the recursivemake backend. r=gps (b8ae57c14)
- Bug 991983 - Emit absolute paths for UnifiedSources. r=gps (4100640d0)
- Bug 991983 - Emit absolute paths for other sources. r=gps (f30ddd773)
- Bug 991983 - Remove commented code in gyp_reader.py. r=gps (0685eca6a)
- Bug 1167474 - Remove ELFHACK_BUILD hack. r=mshal (b9d56c0b2)
- Bug 1097804 - Part 1 - Create a library containing nsISocketTransportService and nsIDNS that can be used to support standalone WebRTC. r=mcmanus (d2f83eb1c)
- Bug 1097804 - Part 2 - Added xpcomrt build of library containing unicode util functions. r=gps (fa9306f34)
- Bug 1162852 - Remove EXTRA_COMPILE_FLAGS. r=gps (340312bc6)
- Bug 1162852 - Remove EXTRA_ASSEMBLER_FLAGS. r=gps (f124c902c)
- Bug 1162852 - Remove the VARIABLES method of GypContext now it is doing nothing. r=gps (b6c1c0246)
- accidental commit (a1bbb1ada)
- Bug 991983 - Define SOURCES as SourcePath. r=gps (678065bce)
- Bug 991983 - Use objdir-relative SOURCES instead of GENERATED_SOURCES. r=gps (b54db0304)
- Bug 991983 - Make TEST_HARNESS_FILES use the *Path classes instead of a separate set of methods to resolve paths. r=gps (025fe1ca9)
- Bug 991983 - Add a deprecation hint for GENERATED_SOURCES. r=gps (67c75f4ab)
- Bug 1172800 - Fixup after bug 991983. r=gps (ce6e8c7a3)
- Bug 1164218 - Allow running individual mochitests and reftests in chaos mode. r=roc,froydnj (f2ec1d9d8)
- Bug 1154231 - Part 1. Use LRU to manage remote layers. r=mattwoodrow (1fe47e64f)
- Bug 1128037 - Minimalist support for wildcards in jar manifests. r=gps (1b07a6ffc)
- Bug 1162569 - default engine files should be in the omni.ja file, r=markh,glandium. (b07e54881)
- Bug 1157279. Escaping CSS identifiers should use lowercase letters for hex digits, not uppercase ones. r=dbaron (6f220936e)
This commit is contained in:
roytam1
2020-12-07 11:03:05 +08:00
parent fa9ed1e11b
commit 729a46113b
277 changed files with 3373 additions and 1834 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/mozbuild/mozbuild/backend/recursivemake.py
+27 -4
View File
@@ -31,6 +31,7 @@ from ..frontend.data import (
ContextDerived,
ContextWrapped,
Defines,
DistFiles,
DirectoryTraversal,
Exports,
ExternalLibrary,
@@ -99,6 +100,7 @@ MOZBUILD_VARIABLES = [
'MODULE',
'NO_DIST_INSTALL',
'NO_EXPAND_LIBS',
'NO_JS_MANIFEST',
'OS_LIBS',
'PARALLEL_DIRS',
'PREF_JS_EXPORTS',
@@ -114,6 +116,8 @@ MOZBUILD_VARIABLES = [
'STATIC_LIBRARY_NAME',
'TEST_DIRS',
'TOOL_DIRS',
# XXX config/Makefile.in specifies this in a make invocation
#'USE_EXTENSION_MANIFEST',
'XPCSHELL_TESTS',
'XPIDL_MODULE',
]
@@ -132,6 +136,7 @@ DEPRECATED_VARIABLES = [
'MOCHITEST_FILES_PARTS',
'MOCHITEST_METRO_FILES',
'MOCHITEST_ROBOCOP_FILES',
'MOZ_CHROME_FILE_FORMAT',
'SHORT_LIBNAME',
'TESTING_JS_MODULES',
'TESTING_JS_MODULE_DIR',
@@ -445,9 +450,16 @@ class RecursiveMakeBackend(CommonBackend):
'.cpp': 'CPPSRCS',
'.S': 'SSRCS',
}
var = suffix_map[obj.canonical_suffix]
variables = [suffix_map[obj.canonical_suffix]]
if isinstance(obj, GeneratedSources):
variables.append('GARBAGE')
base = backend_file.objdir
else:
base = backend_file.srcdir
for f in sorted(obj.files):
backend_file.write('%s += %s\n' % (var, f))
f = mozpath.relpath(f, base)
for var in variables:
backend_file.write('%s += %s\n' % (var, f))
elif isinstance(obj, HostSources):
suffix_map = {
'.c': 'HOST_CSRCS',
@@ -456,7 +468,8 @@ class RecursiveMakeBackend(CommonBackend):
}
var = suffix_map[obj.canonical_suffix]
for f in sorted(obj.files):
backend_file.write('%s += %s\n' % (var, f))
backend_file.write('%s += %s\n' % (
var, mozpath.relpath(f, backend_file.srcdir)))
elif isinstance(obj, VariablePassthru):
# Sorted so output is consistent and we don't bump mtimes.
for k, v in sorted(obj.variables.items()):
@@ -562,6 +575,15 @@ class RecursiveMakeBackend(CommonBackend):
elif isinstance(obj, FinalTargetFiles):
self._process_final_target_files(obj, obj.files, obj.target)
elif isinstance(obj, DistFiles):
# We'd like to install these via manifests as preprocessed files.
# But they currently depend on non-standard flags being added via
# some Makefiles, so for now we just pass them through to the
# underlying Makefile.in.
for f in obj.files:
backend_file.write('DIST_FILES += %s\n' % f)
else:
return
obj.ack()
@@ -909,7 +931,8 @@ class RecursiveMakeBackend(CommonBackend):
INSTALL_TARGETS += %(prefix)s
""" % { 'prefix': prefix,
'dest': '$(DEPTH)/_tests/%s' % path,
'files': ' '.join(files) })
'files': ' '.join(mozpath.relpath(f, backend_file.objdir)
for f in files) })
def _process_resources(self, obj, resources, backend_file):
dep_path = mozpath.join(self.environment.topobjdir, '_build_manifests', '.deps', 'install')
python/mozbuild/mozbuild/frontend/context.py
+38 -46
View File
@@ -211,33 +211,6 @@ class Context(KeyedDefaultDict):
return KeyedDefaultDict.__setitem__(self, key, value)
def resolve_path(self, path):
"""Resolves a path using moz.build conventions.
Paths may be relative to the current srcdir or objdir, or to the
environment's topsrcdir or topobjdir. Different resolution contexts
are denoted by characters at the beginning of the path:
* '/' - relative to topsrcdir;
* '!/' - relative to topobjdir;
* '!' - relative to objdir; and
* any other character - relative to srcdir.
"""
if path.startswith('/'):
resolved = mozpath.join(self.config.topsrcdir, path[1:])
elif path.startswith('!/'):
resolved = mozpath.join(self.config.topobjdir, path[2:])
elif path.startswith('!'):
resolved = mozpath.join(self.objdir, path[1:])
else:
resolved = mozpath.join(self.srcdir, path)
return mozpath.normpath(resolved)
@staticmethod
def is_objdir_path(path):
return path[0] == '!'
def update(self, iterable={}, **kwargs):
"""Like dict.update(), but using the context's setitem.
@@ -660,26 +633,19 @@ VARIABLES = {
populated by calling add_android_eclipse{_library}_project().
""", 'export'),
'SOURCES': (StrictOrderingOnAppendListWithFlagsFactory({'no_pgo': bool, 'flags': List}), list,
'SOURCES': (ContextDerivedTypedListWithItems(Path, StrictOrderingOnAppendListWithFlagsFactory({'no_pgo': bool, 'flags': List})), list,
"""Source code files.
This variable contains a list of source code files to compile.
Accepts assembler, C, C++, Objective C/C++.
""", None),
'GENERATED_SOURCES': (StrictOrderingOnAppendList, list,
"""Generated source code files.
This variable contains a list of generated source code files to
compile. Accepts assembler, C, C++, Objective C/C++.
""", None),
'FILES_PER_UNIFIED_FILE': (int, int,
"""The number of source files to compile into each unified source file.
""", 'None'),
'UNIFIED_SOURCES': (StrictOrderingOnAppendList, list,
'UNIFIED_SOURCES': (ContextDerivedTypedList(SourcePath, StrictOrderingOnAppendList), list,
"""Source code files that can be compiled together.
This variable contains a list of source code files to compile,
@@ -688,15 +654,6 @@ VARIABLES = {
size.
""", None),
'GENERATED_UNIFIED_SOURCES': (StrictOrderingOnAppendList, list,
"""Generated source code files that can be compiled together.
This variable contains a list of generated source code files to
compile, that can be concatenated all together, with UNIFIED_SOURCES,
and built as a single source file. This can help make the build faster
and reduce the debug info size.
""", None),
'GENERATED_FILES': (StrictOrderingOnAppendListWithFlagsFactory({
'script': unicode,
'inputs': list }), list,
@@ -823,6 +780,12 @@ VARIABLES = {
disabled.
""", None),
'DIST_FILES': (StrictOrderingOnAppendList, list,
"""Additional files to place in ``FINAL_TARGET`` (typically ``dist/bin``).
Unlike ``FINAL_TARGET_FILES``, these files are preprocessed.
""", 'libs'),
'EXTRA_COMPONENTS': (StrictOrderingOnAppendList, list,
"""Additional component files to distribute.
@@ -906,7 +869,7 @@ VARIABLES = {
files by the compiler.
""", None),
'HOST_SOURCES': (StrictOrderingOnAppendList, list,
'HOST_SOURCES': (ContextDerivedTypedList(SourcePath, StrictOrderingOnAppendList), list,
"""Source code files to compile with the host compiler.
This variable contains a list of source code files to compile.
@@ -1351,6 +1314,25 @@ VARIABLES = {
the $(DIST_SUBDIR) directory of the otherwise default value is used.
""", None),
'USE_EXTENSION_MANIFEST': (bool, bool,
"""Controls the name of the manifest for JAR files.
By default, the name of the manifest is ${JAR_MANIFEST}.manifest.
Setting this variable to ``True`` changes the name of the manifest to
chrome.manifest.
""", None),
'NO_JS_MANIFEST': (bool, bool,
"""Explicitly disclaims responsibility for manifest listing in EXTRA_COMPONENTS.
Normally, if you have .js files listed in ``EXTRA_COMPONENTS`` or
``EXTRA_PP_COMPONENTS``, you are expected to have a corresponding
.manifest file to go with those .js files. Setting ``NO_JS_MANIFEST``
indicates that the relevant .manifest file and entries for those .js
files are elsehwere (jar.mn, for instance) and this state of affairs
is OK.
""", None),
'GYP_DIRS': (StrictOrderingOnAppendListWithFlagsFactory({
'variables': dict,
'input': unicode,
@@ -1866,6 +1848,16 @@ DEPRECATION_HINTS = {
NO_DIST_INSTALL = True
''',
'GENERATED_SOURCES': '''
Please use
SOURCES += [ '!foo.cpp' ]
instead of
GENERATED_SOURCES += [ 'foo.cpp']
''',
}
# Make sure that all template variables have a deprecation hint.
python/mozbuild/mozbuild/frontend/data.py
+16
View File
@@ -866,6 +866,22 @@ class FinalTargetFiles(ContextDerived):
self.target = target
class DistFiles(ContextDerived):
"""Sandbox container object for FINAL_TARGET_FILES, which is a
HierarchicalStringList.
We need an object derived from ContextDerived for use in the backend, so
this object fills that role. It just has a reference to the underlying
HierarchicalStringList, which is created when parsing DIST_FILES.
"""
__slots__ = ('files', 'target')
def __init__(self, sandbox, files, target):
ContextDerived.__init__(self, sandbox)
self.files = files
self.target = target
class GeneratedFile(ContextDerived):
"""Represents a generated file."""
python/mozbuild/mozbuild/frontend/emitter.py
+84 -46
View File
@@ -28,6 +28,7 @@ from .data import (
ConfigFileSubstitution,
ContextWrapped,
Defines,
DistFiles,
DirectoryTraversal,
Exports,
FinalTargetFiles,
@@ -76,6 +77,10 @@ from .reader import SandboxValidationError
from .context import (
Context,
ObjDirPath,
SourcePath,
ObjDirPath,
Path,
SubContext,
TemplateContext,
)
@@ -541,8 +546,6 @@ class TreeMetadataEmitter(LoggingMixin):
'ANDROID_GENERATED_RESFILES',
'ANDROID_RES_DIRS',
'DISABLE_STL_WRAPPING',
'EXTRA_ASSEMBLER_FLAGS',
'EXTRA_COMPILE_FLAGS',
'EXTRA_COMPONENTS',
'EXTRA_DSO_LDOPTS',
'EXTRA_PP_COMPONENTS',
@@ -555,6 +558,8 @@ class TreeMetadataEmitter(LoggingMixin):
'DEFFILE',
'WIN32_EXE_LDFLAGS',
'LD_VERSION_SCRIPT',
'USE_EXTENSION_MANIFEST',
'NO_JS_MANIFEST',
]
for v in varlist:
if v in context and context[v]:
@@ -656,6 +661,16 @@ class TreeMetadataEmitter(LoggingMixin):
if final_target_files:
yield FinalTargetFiles(context, final_target_files, context['FINAL_TARGET'])
dist_files = context.get('DIST_FILES')
if dist_files:
for f in dist_files:
path = os.path.join(context.srcdir, f)
if not os.path.exists(path):
raise SandboxValidationError('File listed in DIST_FILES '
'does not exist: %s' % f, context)
yield DistFiles(context, dist_files, context['FINAL_TARGET'])
branding_files = context.get('BRANDING_FILES')
if branding_files:
yield BrandingFiles(context, branding_files)
@@ -689,15 +704,37 @@ class TreeMetadataEmitter(LoggingMixin):
return sub
def _process_sources(self, context, passthru):
sources = defaultdict(list)
gen_sources = defaultdict(list)
all_flags = {}
for symbol in ('SOURCES', 'HOST_SOURCES', 'UNIFIED_SOURCES'):
for src in (context[symbol] or []):
if not os.path.exists(mozpath.join(context.srcdir, src)):
srcs = sources[symbol]
gen_srcs = gen_sources[symbol]
context_srcs = context.get(symbol, [])
for f in context_srcs:
full_path = f.full_path
if isinstance(f, SourcePath):
srcs.append(full_path)
else:
assert isinstance(f, ObjDirPath)
gen_srcs.append(full_path)
if symbol == 'SOURCES':
flags = context_srcs[f]
if flags:
all_flags[full_path] = flags
if isinstance(f, SourcePath) and not os.path.exists(full_path):
raise SandboxValidationError('File listed in %s does not '
'exist: \'%s\'' % (symbol, src), context)
'exist: \'%s\'' % (symbol, full_path), context)
# HOST_SOURCES and UNIFIED_SOURCES only take SourcePaths, so
# there should be no generated source in here
assert not gen_sources['HOST_SOURCES']
assert not gen_sources['UNIFIED_SOURCES']
no_pgo = context.get('NO_PGO')
sources = context.get('SOURCES', [])
no_pgo_sources = [f for f in sources if sources[f].no_pgo]
no_pgo_sources = [f for f, flags in all_flags.iteritems()
if flags.no_pgo]
if no_pgo:
if no_pgo_sources:
raise SandboxValidationError('NO_PGO and SOURCES[...].no_pgo '
@@ -735,38 +772,38 @@ class TreeMetadataEmitter(LoggingMixin):
# kinds that can be listed therein.
all_suffixes = list(suffix_map.keys())
varmap = dict(
SOURCES=(Sources, all_suffixes),
HOST_SOURCES=(HostSources, ['.c', '.mm', '.cpp']),
UNIFIED_SOURCES=(UnifiedSources, ['.c', '.mm', '.cpp']),
GENERATED_SOURCES=(GeneratedSources, all_suffixes),
SOURCES=(Sources, GeneratedSources, all_suffixes),
HOST_SOURCES=(HostSources, None, ['.c', '.mm', '.cpp']),
UNIFIED_SOURCES=(UnifiedSources, None, ['.c', '.mm', '.cpp']),
)
for variable, (klass, suffixes) in varmap.items():
for variable, (klass, gen_klass, suffixes) in varmap.items():
allowed_suffixes = set().union(*[suffix_map[s] for s in suffixes])
# First ensure that we haven't been given filetypes that we don't
# recognize.
for f in context[variable]:
for f in itertools.chain(sources[variable], gen_sources[variable]):
ext = mozpath.splitext(f)[1]
if ext not in allowed_suffixes:
raise SandboxValidationError(
'%s has an unknown file type.' % f, context)
if variable.startswith('GENERATED_'):
l = passthru.variables.setdefault('GARBAGE', [])
l.append(f)
# Now sort the files to let groupby work.
sorted_files = sorted(context[variable], key=canonical_suffix_for_file)
for canonical_suffix, files in itertools.groupby(sorted_files, canonical_suffix_for_file):
arglist = [context, list(files), canonical_suffix]
if variable.startswith('UNIFIED_') and 'FILES_PER_UNIFIED_FILE' in context:
arglist.append(context['FILES_PER_UNIFIED_FILE'])
yield klass(*arglist)
for srcs, cls in ((sources[variable], klass),
(gen_sources[variable], gen_klass)):
# Now sort the files to let groupby work.
sorted_files = sorted(srcs, key=canonical_suffix_for_file)
for canonical_suffix, files in itertools.groupby(
sorted_files, canonical_suffix_for_file):
arglist = [context, list(files), canonical_suffix]
if (variable.startswith('UNIFIED_') and
'FILES_PER_UNIFIED_FILE' in context):
arglist.append(context['FILES_PER_UNIFIED_FILE'])
yield cls(*arglist)
sources_with_flags = [f for f in sources if sources[f].flags]
for f in sources_with_flags:
ext = mozpath.splitext(f)[1]
yield PerSourceFlag(context, f, sources[f].flags)
for f, flags in all_flags.iteritems():
if flags.flags:
ext = mozpath.splitext(f)[1]
yield PerSourceFlag(context, f, flags.flags)
def _process_xpidl(self, context):
# XPIDL source files get processed and turned into .h and .xpt files.
@@ -844,27 +881,28 @@ class TreeMetadataEmitter(LoggingMixin):
'Cannot install files to the root of TEST_HARNESS_FILES', context)
for s in strings:
if context.is_objdir_path(s):
if s.startswith('!/'):
objdir_files[path].append('$(DEPTH)/%s' % s[2:])
# Ideally, TEST_HARNESS_FILES would expose Path instances, but
# subclassing HierarchicalStringList to be ContextDerived is
# painful, so until we actually kill HierarchicalStringList,
# just do Path manipulation here.
p = Path(context, s)
if isinstance(p, ObjDirPath):
objdir_files[path].append(p.full_path)
elif '*' in s:
resolved = p.full_path
if s[0] == '/':
pattern_start = resolved.index('*')
base_path = mozpath.dirname(resolved[:pattern_start])
pattern = resolved[len(base_path)+1:]
else:
objdir_files[path].append(s[1:])
base_path = context.srcdir
pattern = s
srcdir_pattern_files[path].append((base_path, pattern));
elif not os.path.exists(p.full_path):
raise SandboxValidationError(
'File listed in TEST_HARNESS_FILES does not exist: %s' % s, context)
else:
resolved = context.resolve_path(s)
if '*' in s:
if s[0] == '/':
pattern_start = resolved.index('*')
base_path = mozpath.dirname(resolved[:pattern_start])
pattern = resolved[len(base_path)+1:]
else:
base_path = context.srcdir
pattern = s
srcdir_pattern_files[path].append((base_path, pattern));
elif not os.path.exists(resolved):
raise SandboxValidationError(
'File listed in TEST_HARNESS_FILES does not exist: %s' % s, context)
else:
srcdir_files[path].append(resolved)
srcdir_files[path].append(p.full_path)
yield TestHarnessFiles(context, srcdir_files,
srcdir_pattern_files, objdir_files)
python/mozbuild/mozbuild/frontend/gyp_reader.py
+32 -26
View File
@@ -7,10 +7,12 @@ import gyp
import sys
import time
import os
from itertools import chain
import mozpack.path as mozpath
from mozpack.files import FileFinder
from .sandbox import alphabetical_sorted
from .context import (
SourcePath,
TemplateContext,
VARIABLES,
)
@@ -62,18 +64,7 @@ class GypContext(TemplateContext):
def __init__(self, config, relobjdir):
self._relobjdir = relobjdir
TemplateContext.__init__(self, template='Gyp',
allowed_variables=self.VARIABLES(), config=config)
@classmethod
@memoize
def VARIABLES(cls):
"""Returns the allowed variables for a GypContext."""
# Using a class method instead of a class variable to hide the content
# from sphinx.
return dict(VARIABLES,
EXTRA_ASSEMBLER_FLAGS=(List, list, '', None),
EXTRA_COMPILE_FLAGS=(List, list, '', None),
)
allowed_variables=VARIABLES, config=config)
def encode(value):
@@ -93,7 +84,6 @@ def read_from_gyp(config, path, output, vars, non_unified_sources = set()):
"""
time_start = time.time()
all_sources = set()
# gyp expects plain str instead of unicode. The frontend code gives us
# unicode strings, so convert them.
@@ -167,14 +157,20 @@ def read_from_gyp(config, path, output, vars, non_unified_sources = set()):
# The context expects an unicode string.
context['LIBRARY_NAME'] = name.decode('utf-8')
# gyp files contain headers and asm sources in sources lists.
sources = set(mozpath.normpath(mozpath.join(context.srcdir, f))
for f in spec.get('sources', [])
if mozpath.splitext(f)[-1] != '.h')
asm_sources = set(f for f in sources if f.endswith('.S'))
sources = []
unified_sources = []
extensions = set()
for f in spec.get('sources', []):
ext = mozpath.splitext(f)[-1]
extensions.add(ext)
s = SourcePath(context, f)
if ext == '.h':
continue
if ext != '.S' and s not in non_unified_sources:
unified_sources.append(s)
else:
sources.append(s)
unified_sources = sources - non_unified_sources - asm_sources
sources -= unified_sources
all_sources |= sources
# The context expects alphabetical order when adding sources
context['SOURCES'] = alphabetical_sorted(sources)
context['UNIFIED_SOURCES'] = alphabetical_sorted(unified_sources)
@@ -203,8 +199,22 @@ def read_from_gyp(config, path, output, vars, non_unified_sources = set()):
continue
context['LOCAL_INCLUDES'] += [include]
context['EXTRA_ASSEMBLER_FLAGS'] = target_conf.get('asflags_mozilla', [])
context['EXTRA_COMPILE_FLAGS'] = target_conf.get('cflags_mozilla', [])
context['ASFLAGS'] = target_conf.get('asflags_mozilla', [])
flags = target_conf.get('cflags_mozilla', [])
if flags:
suffix_map = {
'.c': 'CFLAGS',
'.cpp': 'CXXFLAGS',
'.cc': 'CXXFLAGS',
'.m': 'CMFLAGS',
'.mm': 'CMMFLAGS',
}
variables = (
suffix_map[e]
for e in extensions if e in suffix_map
)
for var in variables:
context[var].extend(flags)
else:
# Ignore other types than static_library because we don't have
# anything using them, and we're not testing them. They can be
@@ -227,7 +237,3 @@ def read_from_gyp(config, path, output, vars, non_unified_sources = set()):
context.execution_time = time.time() - time_start
yield context
time_start = time.time()
# remainder = non_unified_sources - all_sources
# if remainder:
# raise SandboxValidationError('%s defined as non_unified_source, but is '
# 'not defined as a source' % ', '.join(remainder))
python/mozbuild/mozbuild/frontend/reader.py
+3 -2
View File
@@ -66,6 +66,7 @@ from .context import (
FUNCTIONS,
VARIABLES,
DEPRECATION_HINTS,
SourcePath,
SPECIAL_VARIABLES,
SUBCONTEXTS,
SubContext,
@@ -1068,8 +1069,8 @@ class BuildReader(object):
from .gyp_reader import read_from_gyp
non_unified_sources = set()
for s in gyp_dir.non_unified_sources:
source = mozpath.normpath(mozpath.join(curdir, s))
if not os.path.exists(source):
source = SourcePath(context, s)
if not os.path.exists(source.full_path):
raise SandboxValidationError('Cannot find %s.' % source,
context)
non_unified_sources.add(source)
python/mozbuild/mozbuild/frontend/sandbox.py
+1 -1
View File
@@ -213,7 +213,7 @@ class Sandbox(dict):
raise SandboxExecutionError(source_stack, exc[0], exc[1], exc[2])
finally:
self._context._sandbox = old_sandbox
if path:
if path and becomes_current_path:
self._context.pop_source()
def push_subcontext(self, context):
python/mozbuild/mozbuild/jar.py
+25 -1
View File
@@ -24,6 +24,7 @@ from mozbuild.util import (
from mozbuild.preprocessor import Preprocessor
from mozbuild.action.buildlist import addEntriesToListFile
from mozpack.files import FileFinder
if sys.platform == 'win32':
from ctypes import windll, WinError
CreateHardLink = windll.kernel32.CreateHardLinkA
@@ -74,7 +75,7 @@ class JarMaker(object):
regline = re.compile('\%\s+(.*)$')
entryre = '(?P<optPreprocess>\*)?(?P<optOverwrite>\+?)\s+'
entryline = re.compile(entryre
+ '(?P<output>[\w\d.\-\_\\\/\+\@]+)\s*(\((?P<locale>\%?)(?P<source>[\w\d.\-\_\\\/\@]+)\))?\s*$'
+ '(?P<output>[\w\d.\-\_\\\/\+\@]+)\s*(\((?P<locale>\%?)(?P<source>[\w\d.\-\_\\\/\@\*]+)\))?\s*$'
)
def __init__(self, outputFormat='flat', useJarfileManifest=True,
@@ -373,6 +374,29 @@ class JarMaker(object):
# use srcdirs and the objdir (current working dir) for relative paths
src_base = self.sourcedirs + [os.getcwd()]
if '*' in src:
if not out.endswith('/'):
out += '/'
def _prefix(s):
for p in s.split('/'):
if '*' not in p:
yield p + '/'
prefix = ''.join(_prefix(src))
fmt = '%s%s %s%%s (%s%%s)' % (
m.group('optPreprocess') or '',
m.group('optOverwrite') or '',
out,
m.group('locale') or '',
)
for _srcdir in src_base:
finder = FileFinder(_srcdir, find_executables=False)
for path, _ in finder.find(src):
line = fmt % (path[len(prefix):], path)
m = self.entryline.match(line)
if m:
self._processEntryLine(m, outHelper, jf)
return
# check if the source file exists
realsrc = None
for _srcdir in src_base:
python/mozbuild/mozbuild/test/backend/data/dist-files/install.rdf
View File
python/mozbuild/mozbuild/test/backend/data/dist-files/main.js
View File
python/mozbuild/mozbuild/test/backend/data/dist-files/moz.build
+8
View File
@@ -0,0 +1,8 @@
# -*- Mode: python; c-basic-offset: 4; indent-tabs-mode: nil; tab-width: 40 -*-
# Any copyright is dedicated to the Public Domain.
# http://creativecommons.org/publicdomain/zero/1.0/
DIST_FILES += [
'install.rdf',
'main.js',
]
python/mozbuild/mozbuild/test/backend/test_recursivemake.py
+15
View File
@@ -624,6 +624,21 @@ class TestRecursiveMakeBackend(BackendTester):
str.startswith('DIST_SUBDIR')]
self.assertEqual(found, expected_rules)
def test_dist_files(self):
"""Test that DIST_FILES is written to backend.mk correctly."""
env = self._consume('dist-files', RecursiveMakeBackend)
backend_path = mozpath.join(env.topobjdir, 'backend.mk')
lines = [l.strip() for l in open(backend_path, 'rt').readlines()[2:]]
expected = [
'DIST_FILES += install.rdf',
'DIST_FILES += main.js',
]
found = [str for str in lines if str.startswith('DIST_FILES')]
self.assertEqual(found, expected)
def test_config(self):
"""Test that CONFIGURE_SUBST_FILES and CONFIGURE_DEFINE_FILES are
properly handled."""
python/mozbuild/mozbuild/test/frontend/data/dist-files-missing/install.rdf
View File
python/mozbuild/mozbuild/test/frontend/data/dist-files-missing/moz.build
+8
View File
@@ -0,0 +1,8 @@
# -*- Mode: python; c-basic-offset: 4; indent-tabs-mode: nil; tab-width: 40 -*-
# Any copyright is dedicated to the Public Domain.
# http://creativecommons.org/publicdomain/zero/1.0/
DIST_FILES += [
'install.rdf',
'main.js',
]
python/mozbuild/mozbuild/test/frontend/data/dist-files/install.rdf
View File
python/mozbuild/mozbuild/test/frontend/data/dist-files/main.js
View File
python/mozbuild/mozbuild/test/frontend/data/dist-files/moz.build
+8
View File
@@ -0,0 +1,8 @@
# -*- Mode: python; c-basic-offset: 4; indent-tabs-mode: nil; tab-width: 40 -*-
# Any copyright is dedicated to the Public Domain.
# http://creativecommons.org/publicdomain/zero/1.0/
DIST_FILES += [
'install.rdf',
'main.js',
]
python/mozbuild/mozbuild/test/frontend/data/generated-sources/moz.build
+15 -15
View File
@@ -2,29 +2,29 @@
# Any copyright is dedicated to the Public Domain.
# http://creativecommons.org/publicdomain/zero/1.0/
GENERATED_SOURCES += [
'a.cpp',
'b.cc',
'c.cxx',
SOURCES += [
'!a.cpp',
'!b.cc',
'!c.cxx',
]
GENERATED_SOURCES += [
'd.c',
SOURCES += [
'!d.c',
]
GENERATED_SOURCES += [
'e.m',
SOURCES += [
'!e.m',
]
GENERATED_SOURCES += [
'f.mm',
SOURCES += [
'!f.mm',
]
GENERATED_SOURCES += [
'g.S',
SOURCES += [
'!g.S',
]
GENERATED_SOURCES += [
'h.s',
'i.asm',
SOURCES += [
'!h.s',
'!i.asm',
]
python/mozbuild/mozbuild/test/frontend/test_emitter.py
+39 -11
View File
@@ -12,6 +12,7 @@ from mozunit import main
from mozbuild.frontend.data import (
ConfigFileSubstitution,
Defines,
DistFiles,
DirectoryTraversal,
Exports,
GeneratedFile,
@@ -714,18 +715,16 @@ class TestEmitterBasic(unittest.TestCase):
}
for suffix, files in expected.items():
sources = suffix_map[suffix]
self.assertEqual(sources.files, files)
self.assertEqual(
sources.files,
[mozpath.join(reader.config.topsrcdir, f) for f in files])
def test_sources(self):
def test_generated_sources(self):
"""Test that GENERATED_SOURCES works properly."""
reader = self.reader('generated-sources')
objs = self.read_topsrcdir(reader)
self.assertEqual(len(objs), 7)
# GENERATED_SOURCES automatically generate GARBAGE definitions.
garbage = [o for o in objs if isinstance(o, VariablePassthru)]
self.assertEqual(len(garbage), 1)
self.assertEqual(len(objs), 6)
generated_sources = [o for o in objs if isinstance(o, GeneratedSources)]
self.assertEqual(len(generated_sources), 6)
@@ -743,7 +742,9 @@ class TestEmitterBasic(unittest.TestCase):
}
for suffix, files in expected.items():
sources = suffix_map[suffix]
self.assertEqual(sources.files, files)
self.assertEqual(
sources.files,
[mozpath.join(reader.config.topobjdir, f) for f in files])
def test_host_sources(self):
"""Test that HOST_SOURCES works properly."""
@@ -764,7 +765,9 @@ class TestEmitterBasic(unittest.TestCase):
}
for suffix, files in expected.items():
sources = suffix_map[suffix]
self.assertEqual(sources.files, files)
self.assertEqual(
sources.files,
[mozpath.join(reader.config.topsrcdir, f) for f in files])
def test_unified_sources(self):
"""Test that UNIFIED_SOURCES works properly."""
@@ -785,7 +788,9 @@ class TestEmitterBasic(unittest.TestCase):
}
for suffix, files in expected.items():
sources = suffix_map[suffix]
self.assertEqual(sources.files, files)
self.assertEqual(
sources.files,
[mozpath.join(reader.config.topsrcdir, f) for f in files])
self.assertTrue(sources.have_unified_mapping)
def test_unified_sources_non_unified(self):
@@ -807,8 +812,31 @@ class TestEmitterBasic(unittest.TestCase):
}
for suffix, files in expected.items():
sources = suffix_map[suffix]
self.assertEqual(sources.files, files)
self.assertEqual(
sources.files,
[mozpath.join(reader.config.topsrcdir, f) for f in files])
self.assertFalse(sources.have_unified_mapping)
def test_dist_files(self):
"""Test that DIST_FILES works properly."""
reader = self.reader('dist-files')
objs = self.read_topsrcdir(reader)
self.assertEqual(len(objs), 1)
self.assertIsInstance(objs[0], DistFiles)
self.assertEqual(len(objs[0].files), 2)
expected = {'install.rdf', 'main.js'}
for f in objs[0].files:
self.assertTrue(f in expected)
def test_missing_dist_files(self):
"""Test that DIST_FILES with missing files throws errors."""
with self.assertRaisesRegexp(SandboxValidationError, 'File listed in '
'DIST_FILES does not exist'):
reader = self.reader('dist-files-missing')
self.read_topsrcdir(reader)
if __name__ == '__main__':
main()
python/mozbuild/mozbuild/test/test_jarmaker.py
+60 -1
View File
@@ -233,7 +233,7 @@ class TestJarMaker(unittest.TestCase):
def test_a_simple_symlink(self):
'''Test a simple jar.mn with a symlink'''
if not symlinks_supported(self.srcdir):
return
raise unittest.SkipTest('symlinks not supported')
self._create_simple_setup()
jm = JarMaker(outputFormat='symlink')
@@ -247,6 +247,65 @@ class TestJarMaker(unittest.TestCase):
self.assertTrue(is_symlink_to(destfoo, srcbar),
"{0} is not a symlink to {1}".format(destfoo, srcbar))
def _create_wildcard_setup(self):
# create src content
jarf = open(os.path.join(self.srcdir, 'jar.mn'), 'w')
jarf.write('''test.jar:
dir/bar (*.js)
dir/hoge (qux/*)
''')
jarf.close()
open(os.path.join(self.srcdir,'foo.js'),'w').write('foo.js\n')
open(os.path.join(self.srcdir,'bar.js'),'w').write('bar.js\n')
os.makedirs(os.path.join(self.srcdir, 'qux', 'foo'))
open(os.path.join(self.srcdir,'qux', 'foo', '1'),'w').write('1\n')
open(os.path.join(self.srcdir,'qux', 'foo', '2'),'w').write('2\n')
open(os.path.join(self.srcdir,'qux', 'baz'),'w').write('baz\n')
# create reference
refpath = os.path.join(self.refdir, 'chrome', 'test.jar', 'dir')
os.makedirs(os.path.join(refpath, 'bar'))
os.makedirs(os.path.join(refpath, 'hoge', 'foo'))
open(os.path.join(refpath, 'bar', 'foo.js'), 'w').write('foo.js\n')
open(os.path.join(refpath, 'bar', 'bar.js'), 'w').write('bar.js\n')
open(os.path.join(refpath, 'hoge', 'foo', '1'), 'w').write('1\n')
open(os.path.join(refpath, 'hoge', 'foo', '2'), 'w').write('2\n')
open(os.path.join(refpath, 'hoge', 'baz'), 'w').write('baz\n')
def test_a_wildcard_jar(self):
'''Test a wildcard in jar.mn'''
self._create_wildcard_setup()
# call JarMaker
rv = self._jar_and_compare(os.path.join(self.srcdir,'jar.mn'),
sourcedirs = [self.srcdir])
self.assertTrue(not rv, rv)
def test_a_wildcard_symlink(self):
'''Test a wildcard in jar.mn with symlinks'''
if not symlinks_supported(self.srcdir):
raise unittest.SkipTest('symlinks not supported')
self._create_wildcard_setup()
jm = JarMaker(outputFormat='symlink')
jm.sourcedirs = [self.srcdir]
jm.topsourcedir = self.srcdir
jardir = os.path.join(self.builddir, 'chrome')
jm.makeJar(os.path.join(self.srcdir,'jar.mn'), jardir)
expected_symlinks = {
('bar', 'foo.js'): ('foo.js',),
('bar', 'bar.js'): ('bar.js',),
('hoge', 'foo', '1'): ('qux', 'foo', '1'),
('hoge', 'foo', '2'): ('qux', 'foo', '2'),
('hoge', 'baz'): ('qux', 'baz'),
}
for dest, src in expected_symlinks.iteritems():
srcpath = os.path.join(self.srcdir, *src)
destpath = os.path.join(self.builddir, 'chrome', 'test', 'dir',
*dest)
self.assertTrue(is_symlink_to(destpath, srcpath),
"{0} is not a symlink to {1}".format(destpath,
srcpath))
class Test_relativesrcdir(unittest.TestCase):
def setUp(self):