KotJohansson/palemoon27
1
0
Fork 0
mirror of https://github.com/roytam1/palemoon27.git synced 2026-05-26 14:30:27 +00:00
Code Issues Projects Releases Wiki Activity

import changes from `dev' branch of rmottola/Arctic-Fox:

Browse Source 
- Bug 1199885 - Part 2: Add AsyncDragMetrics. r=kats (220a4f445e)
- Bug 1149482 - Localize the 'none' string for APZ in about:support. r=dvander (3e0128f62e)
- Bug 1199885 - Part 3: Add 'apz.drag.enabled' preference for async scrollbars. r=kats (5133061d3f)
- Bug 1199885 - Part 4: Let the hit testing tree know about the scroll thumb. r=kats (26a9a69443)
- Bug 1199885 - Part 5: Make mRootLayerTreeID const to prove that there's no data races. r=kats (88e2eb80f1)
- Bug 1199885 - Part 8: Add FindScrollNode to locate the scrollbar thumb. r=kats (a545ec9569)
- Bug 1199885 - Part 9: Let APZC handle the drag events. r=kats (4cc0f88153)
- Bug 1199885 - Part 9.5: Make the mouse events APZC aware. r=kats (69bd4caf94)
- Bug 1199885 - Part 10: Add APZTeeManager API to start an async scroll. r=kats (e212ee2750)
- Bug 1199885 - Part 12: Add StartScrollbarDrag IPC message. r=kats (705af3b233)
- Bug 1199885 - Part 13: Let nsSliderFrame trigger async scrolling via StartScrollbarDrag. r=kats (d31b5a790d)
- put in gstreamer as of 2015-10-05 Bug 1146482 (7cca45858c)
- pointer style (f8041af438)
- Bug 1197097 - Don't use a context-wide cycle-detection mechanism for detecting cycles during JSON.stringify. This prevents nested (yet separate) JSON.stringify, and it causes that algorithm to be affected by specification-unrelated operations like toSource. r=jonco (09652471c1)
- align to ugly grammaer of 1114782 (c760693693)
- remove space (7158a0c2bf)
- align to FF/TFF (27138a55bb)
- Bug 1205887 - Verify that MOZILLA_VERSION was set correctly, r=glandium (2269e8f31b)
- Bug 1205012 - Allow rust source code in SpiderMonkey; r=mshal (9ebca9ed28)
- Bug 1197281 - Use MOZ_FIND_WINSDK_VERSION for MOZ_WINSDK_MAXVER on mingw. r=glandium (2910b2160e)
- Bug 1207893 - Refactor how build backend execution is summarized. r=gps (56d8fecbc2)
- Bug 1207893 - Allow to create multiple build backends at once. r=gps (d8cba87d2b)
- Bug 1207897 - Add a configure option to build multiple build backends. r=gps (f9c7851d02)
- Bug 1188555 - part 1 - remove write-only configure.in variable NO_LD_ARCHIVE_FLAGS; r=mshal (461958b100)
- Bug 1188555 - part 2 - remove write-only configure.in variables MKSHLIB_{UN,}FORCE_ALL; r=mshal (1813210dda)
- Bug 1207882 - Ensure chrome manifests are created in a directory that exists when processing jar manifests. r=gps (59165a64cb)
- Bug 1204712 - Handle wildcards properly for localized content in jar manifests. r=gps (54dfb632c7)
- Bug 1188551: treat assertion failures as bad mozconfig; r=mshal (e307769de7)
- Bug 1181040 - Set ${var}_IS_SET variables for mk_add_options-defined variables. r=gps (19f169556b)
- Bug 1193015 - Require MOZ_GLX_USE_SURFACE_SHARING to enable WebGL surface sharing on GLX. r=jgilbert (9aa4fa8f41)
- Bug 1211324 (part 1) - Remove BILINEAR and GAUSSIAN filter constants. (3d9290ef02)
- Bug 1211324 (part 2) - Make gfx::FILTER::GOOD convert to GraphicsFilter::FILTER_GOOD. r=mattwoodrow. (16e7607c70)
- Bug 1208365 (part 1) - Remove unused EXTEND_PAD_EDGE. r=Bas. (10920e2bb6)
- Bug 1208365 (part 2) - Remove gfxPattern::Extend(). r=bas. (3703b9748a)
- Bug 1208365 (part 3) - Change gfxPattern::mExtend from a GraphicsExtend to a gfx::ExtendMode. r=bas. (b105d06e91)
- Bug 1208365 (part 4) - Remove gfxPattern::GraphicsExtend. r=bas. (7a16d48995)
- Bug 1211324 (part 3) - Remove GraphicsFilter::FILTER_FAST and replace it with FILTER_BEST. r=mattwoodrow. (b5101e049f)
- Bug 1211324 (part 4) - Replace GraphicsFilter constants with gfx::Filter equivalents. r=mattwoodrow. (c19b6b030e)
- Bug 1190117 - Track mLastProducerID and mLastFrameID explicitly in ImageLayerProperties. r=roc (ed9a5c777f)
- Bug 1211324 (part 5) - Remove GraphicsFilter and gfxGraphicsFilter. r=mattwoodrow. (0bd4ce7160)
- Bug 1194954 - Fix -Wunreachable-code warnings in gfx/layers and gfx/thebes. r=BenWa (1a3d68c490)
- Bug 1180509 - Fix judder of icons in Australis doorhanger menu at the end of its scale-in animation. r=roc (9b4df470fa)
- Bug 1206915 - Make paint dumping to a file e10s-friendly. r=mattwoodrow (87d2e12c2b)
- Bug 1206915 - Handle nested PaintFrame() calls correctly during paint dumping. r=mattwoodrow,BenWa (d30f77fbdd)
- Fix max texture size handling in displayport clamping. (bug 1135907 follow-up, r=kats) (a791894332)
- Fix bogus assertion in nsLayoutUtils::SetDisplayPortMargins. (bug 1156409, r=botond, a=philor) (b635b21c34)
- Bug 1169879 - Use only the critical displayport when computing image visibility. r=tn (9ec91c9527)
- Bug 1169881 - Recompute image visibility when display port margins change. r=tn (35a5bd3a51)
- Bug 1197765 - Compare text content inside frame instead of the content node for ruby autohiding. r=dbaron (00cf5b7674)
- Bug 1173580 - Record content descriptions in APZ test data. r=kats (7a72d1ac0a)
- Bug 1178971 - Changed line snapping behaviour depending on even/odd-ness of stroke width. r=mstange (495b32dc23)
- Bug 1208953: [mp3] Don't parse data we've already parsed. r=cpearce (72eed4309f)
- Bug 1137151: Marked destructor of |android::MediaCodecReader| as protected, r=sotaro (0632b34bc5)
- Bug 1205351 - Replace nsBaseHashtable::Enumerate() calls in dom/media/ with iterators r=cpearce (45976c24c0)
- Bug 1133624 - Add lang-specific ruby rules to ua.css. r=dbaron (46788cc220)
- Bug 1133624 - followup add fuzzy maxdiff on CLOSED TREE (1b6b62aded)
- Bug 1180443 - Consider whitespace collapse when calculating intrinsic isize of ruby. r=dbaron (3862184acd)
- Bug 1153764 - Avoid explicitly doing break before when there is ruby span. r=roc (71b4ec7749)
- add gstreamer parts (7e562556be)
- some preferences aligned to FF (b26d0b389c)
- Bug 1166301 - If APZ is enabled, clip fixed background images at the layer level rather than the display item level. r=mattwoodrow (5644e22090)
- Bug 1144990 - Dump display items with class id and name, r=roc (aa2e227e35)
- Bug 1205087 - Remove LayerManager parameter for ShouldFixToViewport. r=roc (00a1f2e36f)
- Bug 1205087 - Make nsSVGIntegrationUtils paint frame continuations manually since combining them meant that our reference frame wasn't an ancestor of all painted frames. r=roc (e40f6b7b81)
- Bug 1195400 - Check ancestor geometry roots when determining scrollability of a layer. r=mattwoodrow (4699b7e935)
- Bug 1205087 - Cache the AnimatedGeometryRoot on DisplayItem. r=roc (06bba311c3)
- Back out the bits of bug 1205087 that cache the AnimatedGeometryRoot on DisplayItem. r=backout (254057a6b6)
- Bug 1203190 - Don't intersect with the visible rect for fixed background images. r=mattwoodrow (1c5a432459)
- Bug 1208438 - Don't allow layers with scrolling clips to occlusion-cull layers behind them. r=mattwoodrow (c3d77a9846)
- Bug 1205630 - Translate a fixed background display item's clip rect correctly when setting it on the layer. r=mstange (49039f0e2d)
- Bug 1205630 - Reftest. r=mstange (a8db59eaee)
- Bug 1105832 - Also dump inactive layer managers when display list dumping is enabled. r=mattwoodrow (564fe5fcbc)
- Bug 1208661 - Move Dump() up from ContentClient to CompositableClient. r=BenWa (37915312b3)
This commit is contained in:
roytam1
2022-09-22 10:16:52 +08:00
parent 0484be0c56
commit e1776c67f8
196 changed files with 5677 additions and 1164 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/mozbuild/mozbuild/backend/android_eclipse.py
+14 -18
View File
@@ -24,6 +24,7 @@ from ..frontend.data import (
)
from ..makeutil import Makefile
from ..util import ensureParentDir
from mozbuild.base import ExecutionSummary
def pretty_print(element):
@@ -38,22 +39,17 @@ class AndroidEclipseBackend(CommonBackend):
"""Backend that generates Android Eclipse project files.
"""
def _init(self):
CommonBackend._init(self)
def detailed(summary):
s = 'Wrote {:d} Android Eclipse projects to {:s}; ' \
'{:d} created; {:d} updated'.format(
summary.created_count + summary.updated_count,
mozpath.join(self.environment.topobjdir, 'android_eclipse'),
summary.created_count,
summary.updated_count)
return s
# This is a little kludgy and could be improved with a better API.
self.summary.backend_detailed_summary = types.MethodType(detailed,
self.summary)
def summary(self):
return ExecutionSummary(
'AndroidEclipse backend executed in {execution_time:.2f}s\n'
'Wrote {projects:d} Android Eclipse projects to {path:s}; '
'{created:d} created; {updated:d} updated',
execution_time=self._execution_time,
projects=self._created_count + self._updated_count,
path=mozpath.join(self.environment.topobjdir, 'android_eclipse'),
created=self._created_count,
updated=self._updated_count,
)
def consume_object(self, obj):
"""Write out Android Eclipse project files."""
@@ -257,7 +253,7 @@ class AndroidEclipseBackend(CommonBackend):
# When we re-create the build backend, we kill everything that was there.
if os.path.isdir(project_directory):
self.summary.updated_count += 1
self._updated_count += 1
else:
self.summary.created_count += 1
self._created_count += 1
copier.copy(project_directory, skip_if_older=False, remove_unaccounted=True)
python/mozbuild/mozbuild/backend/base.py
+48 -126
View File
@@ -21,102 +21,9 @@ import mozpack.path as mozpath
from ..preprocessor import Preprocessor
from ..pythonutil import iter_modules_in_path
from ..util import FileAvoidWrite
from ..frontend.data import (
ContextDerived,
ReaderSummary,
)
from ..frontend.data import ContextDerived
from .configenvironment import ConfigEnvironment
class BackendConsumeSummary(object):
"""Holds state about what a backend did.
This is used primarily to print a summary of what the backend did
so people know what's going on.
"""
def __init__(self):
# How many moz.build files were read. This includes included files.
self.mozbuild_count = 0
# The number of derived objects from the read moz.build files.
self.object_count = 0
# The number of backend files created.
self.created_count = 0
# The number of backend files updated.
self.updated_count = 0
# The number of unchanged backend files.
self.unchanged_count = 0
# The number of deleted backend files.
self.deleted_count = 0
# The total wall time this backend spent consuming objects. If
# the iterable passed into consume() is a generator, this includes the
# time spent to read moz.build files.
self.wall_time = 0.0
# CPU time spent by during the interval captured by wall_time.
self.cpu_time = 0.0
# The total wall time spent executing moz.build files. This is just
# the read and execute time. It does not cover consume time.
self.mozbuild_execution_time = 0.0
# The total wall time spent emitting objects from sandboxes.
self.emitter_execution_time = 0.0
# The total wall time spent in the backend. This counts the time the
# backend writes out files, etc.
self.backend_execution_time = 0.0
# How much wall time the system spent doing other things. This is
# wall_time - mozbuild_execution_time - emitter_execution_time -
# backend_execution_time.
self.other_time = 0.0
# Mapping of changed file paths to diffs of the changes.
self.file_diffs = {}
@property
def reader_summary(self):
return 'Finished reading {:d} moz.build files in {:.2f}s'.format(
self.mozbuild_count,
self.mozbuild_execution_time)
@property
def emitter_summary(self):
return 'Processed into {:d} build config descriptors in {:.2f}s'.format(
self.object_count, self.emitter_execution_time)
@property
def backend_summary(self):
return 'Backend executed in {:.2f}s'.format(self.backend_execution_time)
def backend_detailed_summary(self):
"""Backend summary to be supplied by BuildBackend implementations."""
return None
@property
def total_summary(self):
efficiency_value = self.cpu_time / self.wall_time if self.wall_time else 100
return 'Total wall time: {:.2f}s; CPU time: {:.2f}s; Efficiency: ' \
'{:.0%}; Untracked: {:.2f}s'.format(
self.wall_time, self.cpu_time, efficiency_value,
self.other_time)
def summaries(self):
yield self.reader_summary
yield self.emitter_summary
yield self.backend_summary
detailed = self.backend_detailed_summary()
if detailed:
yield detailed
yield self.total_summary
from mozbuild.base import ExecutionSummary
class BuildBackend(LoggingMixin):
@@ -135,7 +42,6 @@ class BuildBackend(LoggingMixin):
self.populate_logger()
self.environment = environment
self.summary = BackendConsumeSummary()
# Files whose modification should cause a new read and backend
# generation.
@@ -155,8 +61,44 @@ class BuildBackend(LoggingMixin):
self._environments = {}
self._environments[environment.topobjdir] = environment
# The number of backend files created.
self._created_count = 0
# The number of backend files updated.
self._updated_count = 0
# The number of unchanged backend files.
self._unchanged_count = 0
# The number of deleted backend files.
self._deleted_count = 0
# The total wall time spent in the backend. This counts the time the
# backend writes out files, etc.
self._execution_time = 0.0
# Mapping of changed file paths to diffs of the changes.
self.file_diffs = {}
self._init()
def summary(self):
return ExecutionSummary(
self.__class__.__name__.replace('Backend', '') +
' backend executed in {execution_time:.2f}s\n '
'{total:d} total backend files; '
'{created:d} created; '
'{updated:d} updated; '
'{unchanged:d} unchanged; '
'{deleted:d} deleted',
execution_time=self._execution_time,
total=self._created_count + self._updated_count +
self._unchanged_count,
created=self._created_count,
updated=self._updated_count,
unchanged=self._unchanged_count,
deleted=self._deleted_count)
def _init():
"""Hook point for child classes to perform actions during __init__.
@@ -173,24 +115,14 @@ class BuildBackend(LoggingMixin):
base class consumes objects and calls methods (possibly) implemented by
child classes.
"""
cpu_start = time.clock()
time_start = time.time()
backend_time = 0.0
for obj in objs:
self.summary.object_count += 1
obj_start = time.time()
self.consume_object(obj)
backend_time += time.time() - obj_start
self._execution_time += time.time() - obj_start
if isinstance(obj, ContextDerived):
self.backend_input_files |= obj.context_all_paths
if isinstance(obj, ReaderSummary):
self.summary.mozbuild_count = obj.total_file_count
self.summary.mozbuild_execution_time = obj.total_sandbox_execution_time
self.summary.emitter_execution_time = obj.total_emitter_execution_time
# Pull in all loaded Python as dependencies so any Python changes that
# could influence our output result in a rescan.
self.backend_input_files |= set(iter_modules_in_path(
@@ -198,14 +130,14 @@ class BuildBackend(LoggingMixin):
finished_start = time.time()
self.consume_finished()
backend_time += time.time() - finished_start
self._execution_time += time.time() - finished_start
# Purge backend files created in previous run, but not created anymore
delete_files = self._backend_output_list - self._backend_output_files
for path in delete_files:
try:
os.unlink(mozpath.join(self.environment.topobjdir, path))
self.summary.deleted_count += 1
self._deleted_count += 1
except OSError:
pass
# Remove now empty directories
@@ -216,24 +148,14 @@ class BuildBackend(LoggingMixin):
pass
# Write out the list of backend files generated, if it changed.
if self.summary.deleted_count or self.summary.created_count or \
if self._deleted_count or self._created_count or \
not os.path.exists(self._backend_output_list_file):
with open(self._backend_output_list_file, 'w') as fh:
fh.write('\n'.join(sorted(self._backend_output_files)))
elif self.summary.updated_count:
elif self._updated_count:
with open(self._backend_output_list_file, 'a'):
os.utime(self._backend_output_list_file, None)
self.summary.cpu_time = time.clock() - cpu_start
self.summary.wall_time = time.time() - time_start
self.summary.backend_execution_time = backend_time
self.summary.other_time = self.summary.wall_time - \
self.summary.mozbuild_execution_time - \
self.summary.emitter_execution_time - \
self.summary.backend_execution_time
return self.summary
@abstractmethod
def consume_object(self, obj):
"""Consumes an individual TreeMetadata instance.
@@ -250,7 +172,7 @@ class BuildBackend(LoggingMixin):
"""Context manager to write a file.
This is a glorified wrapper around FileAvoidWrite with integration to
update the BackendConsumeSummary on this instance.
update the summary data on this instance.
Example usage:
@@ -276,13 +198,13 @@ class BuildBackend(LoggingMixin):
self._backend_output_files.add(mozpath.relpath(fh.name, self.environment.topobjdir))
existed, updated = fh.close()
if not existed:
self.summary.created_count += 1
self._created_count += 1
elif updated:
self.summary.updated_count += 1
self._updated_count += 1
if fh.diff:
self.summary.file_diffs[fh.name] = fh.diff
self.file_diffs[fh.name] = fh.diff
else:
self.summary.unchanged_count += 1
self._unchanged_count += 1
@contextmanager
def _get_preprocessor(self, obj):
python/mozbuild/mozbuild/backend/cpp_eclipse.py
+10 -9
View File
@@ -15,6 +15,7 @@ from .common import CommonBackend
from ..frontend.data import (
Defines,
)
from mozbuild.base import ExecutionSummary
# TODO Have ./mach eclipse generate the workspace and index it:
# /Users/bgirard/mozilla/eclipse/eclipse/eclipse/eclipse -application org.eclipse.cdt.managedbuilder.core.headlessbuild -data $PWD/workspace -importAll $PWD/eclipse
@@ -41,15 +42,15 @@ class CppEclipseBackend(CommonBackend):
# Note: We need the C Pre Processor (CPP) flags, not the CXX flags
self._cppflags = self.environment.substs.get('CPPFLAGS', '')
def detailed(summary):
return ('Generated Cpp Eclipse workspace in "%s".\n' + \
'If missing, import the project using File > Import > General > Existing Project into workspace\n' + \
'\n' + \
'Run with: eclipse -data %s\n') \
% (self._workspace_dir, self._workspace_dir)
self.summary.backend_detailed_summary = types.MethodType(detailed,
self.summary)
def summary(self):
return ExecutionSummary(
'CppEclipse backend executed in {execution_time:.2f}s\n'
'Generated Cpp Eclipse workspace in "{workspace:s}".\n'
'If missing, import the project using File > Import > General > Existing Project into workspace\n'
'\n'
'Run with: eclipse -data {workspace:s}\n',
execution_time=self._execution_time,
workspace=self._workspace_dir)
def _get_workspace_path(self):
return CppEclipseBackend.get_workspace_path(self.environment.topsrcdir, self.environment.topobjdir)
python/mozbuild/mozbuild/backend/recursivemake.py
+11 -22
View File
@@ -367,26 +367,8 @@ class RecursiveMakeBackend(CommonBackend):
self._backend_files = {}
self._idl_dirs = set()
def detailed(summary):
s = '{:d} total backend files; ' \
'{:d} created; {:d} updated; {:d} unchanged; ' \
'{:d} deleted; {:d} -> {:d} Makefile'.format(
summary.created_count + summary.updated_count +
summary.unchanged_count,
summary.created_count,
summary.updated_count,
summary.unchanged_count,
summary.deleted_count,
summary.makefile_in_count,
summary.makefile_out_count)
return s
# This is a little kludgy and could be improved with a better API.
self.summary.backend_detailed_summary = types.MethodType(detailed,
self.summary)
self.summary.makefile_in_count = 0
self.summary.makefile_out_count = 0
self._makefile_in_count = 0
self._makefile_out_count = 0
self._test_manifests = {}
@@ -419,6 +401,13 @@ class RecursiveMakeBackend(CommonBackend):
'tools': set(),
}
def summary(self):
summary = super(RecursiveMakeBackend, self).summary()
summary.extend('; {makefile_in:d} -> {makefile_out:d} Makefile',
makefile_in=self._makefile_in_count,
makefile_out=self._makefile_out_count)
return summary
def _get_backend_file_for(self, obj):
if obj.objdir not in self._backend_files:
self._backend_files[obj.objdir] = \
@@ -779,7 +768,7 @@ class RecursiveMakeBackend(CommonBackend):
if not stub:
self.log(logging.DEBUG, 'substitute_makefile',
{'path': makefile}, 'Substituting makefile: {path}')
self.summary.makefile_in_count += 1
self._makefile_in_count += 1
for tier, skiplist in self._may_skip.items():
if bf.relobjdir in skiplist:
@@ -1424,7 +1413,7 @@ INSTALL_TARGETS += %(prefix)s
# the autogenerated one automatically.
self.backend_input_files.add(obj.input_path)
self.summary.makefile_out_count += 1
self._makefile_out_count += 1
def _handle_ipdl_sources(self, ipdl_dir, sorted_ipdl_sources,
unified_ipdl_cppsrcs_mapping):
python/mozbuild/mozbuild/backend/visualstudio.py
+7 -6
View File
@@ -27,6 +27,7 @@ from ..frontend.data import (
Sources,
UnifiedSources,
)
from mozbuild.base import ExecutionSummary
MSBUILD_NAMESPACE = 'http://schemas.microsoft.com/developer/msbuild/2003'
@@ -84,12 +85,12 @@ class VisualStudioBackend(CommonBackend):
self._paths_to_configs = {}
self._libs_to_paths = {}
def detailed(summary):
return 'Generated Visual Studio solution at %s' % (
os.path.join(self._out_dir, 'mozilla.sln'))
self.summary.backend_detailed_summary = types.MethodType(detailed,
self.summary)
def summary(self):
return ExecutionSummary(
'VisualStudio backend executed in {execution_time:.2f}s\n'
'Generated Visual Studio solution at {path:s}',
execution_time=self._execution_time,
path=os.path.join(self._out_dir, 'mozilla.sln'))
def consume_object(self, obj):
# Just acknowledge everything.
python/mozbuild/mozbuild/base.py
+19
View File
@@ -802,3 +802,22 @@ class PathArgument(object):
def objdir_path(self):
return mozpath.join(self.topobjdir, self.relpath())
class ExecutionSummary(dict):
"""Helper for execution summaries."""
def __init__(self, summary_format, **data):
self._summary_format = ''
assert 'execution_time' in data
self.extend(summary_format, **data)
def extend(self, summary_format, **data):
self._summary_format += summary_format
self.update(data)
def __str__(self):
return self._summary_format.format(**self)
def __getattr__(self, key):
return self[key]
python/mozbuild/mozbuild/config_status.py
+83 -37
View File
@@ -11,8 +11,9 @@ from __future__ import absolute_import, print_function
import logging
import os
import sys
import time
from optparse import OptionParser
from argparse import ArgumentParser
from mach.logging import LoggingManager
from mozbuild.backend.configenvironment import ConfigEnvironment
@@ -21,6 +22,7 @@ from mozbuild.base import MachCommandConditions
from mozbuild.frontend.emitter import TreeMetadataEmitter
from mozbuild.frontend.reader import BuildReader
from mozbuild.mozinfo import write_mozinfo
from itertools import chain
log_manager = LoggingManager()
@@ -72,20 +74,31 @@ def config_status(topobjdir='.', topsrcdir='.',
raise Exception('topsrcdir must be defined as an absolute directory: '
'%s' % topsrcdir)
parser = OptionParser()
parser.add_option('--recheck', dest='recheck', action='store_true',
help='update config.status by reconfiguring in the same conditions')
parser.add_option('-v', '--verbose', dest='verbose', action='store_true',
help='display verbose output')
parser.add_option('-n', dest='not_topobjdir', action='store_true',
help='do not consider current directory as top object directory')
parser.add_option('-d', '--diff', action='store_true',
help='print diffs of changed files.')
parser.add_option('-b', '--backend',
choices=['RecursiveMake', 'AndroidEclipse', 'CppEclipse', 'VisualStudio'],
default='RecursiveMake',
help='what backend to build (default: RecursiveMake).')
options, args = parser.parse_args()
default_backends = ['RecursiveMake']
# We have a chicken/egg problem, where we only have a dict for substs after
# creating the ConfigEnvironment, which requires argument parsing to have
# occurred.
for name, value in substs:
if name == 'BUILD_BACKENDS':
default_backends = value
break
parser = ArgumentParser()
parser.add_argument('--recheck', dest='recheck', action='store_true',
help='update config.status by reconfiguring in the same conditions')
parser.add_argument('-v', '--verbose', dest='verbose', action='store_true',
help='display verbose output')
parser.add_argument('-n', dest='not_topobjdir', action='store_true',
help='do not consider current directory as top object directory')
parser.add_argument('-d', '--diff', action='store_true',
help='print diffs of changed files.')
parser.add_argument('-b', '--backend', nargs='+',
choices=['RecursiveMake', 'AndroidEclipse', 'CppEclipse',
'VisualStudio', 'FasterMake'],
default=default_backends,
help='what backend to build (default: %s).' %
' '.join(default_backends))
options = parser.parse_args()
# Without -n, the current directory is meant to be the top object directory
if not options.not_topobjdir:
@@ -100,22 +113,31 @@ def config_status(topobjdir='.', topsrcdir='.',
write_mozinfo(os.path.join(topobjdir, 'mozinfo.json'), env, os.environ)
# Make an appropriate backend instance, defaulting to RecursiveMakeBackend.
backend_cls = RecursiveMakeBackend
if options.backend == 'AndroidEclipse':
from mozbuild.backend.android_eclipse import AndroidEclipseBackend
if not MachCommandConditions.is_android(env):
raise Exception('The Android Eclipse backend is not available with this configuration.')
backend_cls = AndroidEclipseBackend
elif options.backend == 'CppEclipse':
from mozbuild.backend.cpp_eclipse import CppEclipseBackend
backend_cls = CppEclipseBackend
if os.name == 'nt':
raise Exception('Eclipse is not supported on Windows. Consider using Visual Studio instead.')
elif options.backend == 'VisualStudio':
from mozbuild.backend.visualstudio import VisualStudioBackend
backend_cls = VisualStudioBackend
backends_cls = []
for backend in options.backend:
if backend == 'AndroidEclipse':
from mozbuild.backend.android_eclipse import AndroidEclipseBackend
if not MachCommandConditions.is_android(env):
raise Exception('The Android Eclipse backend is not available with this configuration.')
backends_cls.append(AndroidEclipseBackend)
elif backend == 'CppEclipse':
from mozbuild.backend.cpp_eclipse import CppEclipseBackend
backends_cls.append(CppEclipseBackend)
if os.name == 'nt':
raise Exception('Eclipse is not supported on Windows. Consider using Visual Studio instead.')
elif backend == 'VisualStudio':
from mozbuild.backend.visualstudio import VisualStudioBackend
backends_cls.append(VisualStudioBackend)
elif backend == 'FasterMake':
from mozbuild.backend.fastermake import FasterMakeBackend
backends_cls.append(FasterMakeBackend)
else:
backends_cls.append(RecursiveMakeBackend)
the_backend = backend_cls(env)
cpu_start = time.clock()
time_start = time.time()
backends = [cls(env) for cls in backends_cls]
reader = BuildReader(env)
emitter = TreeMetadataEmitter(env)
@@ -131,17 +153,41 @@ def config_status(topobjdir='.', topsrcdir='.',
log_manager.add_terminal_logging(level=log_level)
log_manager.enable_unstructured()
print('Walking the dog...', file=sys.stderr)
summary = the_backend.consume(definitions)
print('Reticulating splines...', file=sys.stderr)
if len(backends) > 1:
definitions = list(definitions)
for line in summary.summaries():
print(line, file=sys.stderr)
for the_backend in backends:
the_backend.consume(definitions)
execution_time = 0.0
for obj in chain((reader, emitter), backends):
summary = obj.summary()
print(summary, file=sys.stderr)
execution_time += summary.execution_time
cpu_time = time.clock() - cpu_start
wall_time = time.time() - time_start
efficiency = cpu_time / wall_time if wall_time else 100
untracked = wall_time - execution_time
print(
'Total wall time: {:.2f}s; CPU time: {:.2f}s; Efficiency: '
'{:.0%}; Untracked: {:.2f}s'.format(
wall_time, cpu_time, efficiency, untracked),
file=sys.stderr
)
if options.diff:
for path, diff in sorted(summary.file_diffs.items()):
print('\n'.join(diff))
for the_backend in backends:
for path, diff in sorted(the_backend.file_diffs.items()):
print('\n'.join(diff))
# Advertise Visual Studio if appropriate.
#if os.name == 'nt' and 'VisualStudio' not in options.backend:
# print(VISUAL_STUDIO_ADVERTISEMENT)
# Advertise Eclipse if it is appropriate.
if MachCommandConditions.is_android(env):
if options.backend == 'RecursiveMake':
if 'AndroidEclipse' not in options.backend:
print(ANDROID_IDE_ADVERTISEMENT)
python/mozbuild/mozbuild/frontend/context.py
-1
View File
@@ -83,7 +83,6 @@ class Context(KeyedDefaultDict):
# a list to be a problem.
self._all_paths = []
self.config = config
self.execution_time = 0
self._sandbox = None
KeyedDefaultDict.__init__(self, self._factory)
python/mozbuild/mozbuild/frontend/data.py
-11
View File
@@ -39,17 +39,6 @@ class TreeMetadata(object):
self._ack = True
class ReaderSummary(TreeMetadata):
"""A summary of what the reader did."""
def __init__(self, total_file_count, total_sandbox_execution_time,
total_emitter_execution_time):
TreeMetadata.__init__(self)
self.total_file_count = total_file_count
self.total_sandbox_execution_time = total_sandbox_execution_time
self.total_emitter_execution_time = total_emitter_execution_time
class ContextDerived(TreeMetadata):
"""Build object derived from a single Context instance.
python/mozbuild/mozbuild/frontend/emitter.py
+15 -12
View File
@@ -59,7 +59,6 @@ from .data import (
PreprocessedTestWebIDLFile,
PreprocessedWebIDLFile,
Program,
ReaderSummary,
Resources,
SharedLibrary,
SimpleProgram,
@@ -86,6 +85,8 @@ from .context import (
TemplateContext,
)
from mozbuild.base import ExecutionSummary
class TreeMetadataEmitter(LoggingMixin):
"""Converts the executed mozbuild files into data structures.
@@ -127,19 +128,27 @@ class TreeMetadataEmitter(LoggingMixin):
# Add security/nss manually, since it doesn't have a subconfigure.
self._external_paths.add('security/nss')
self._emitter_time = 0.0
self._object_count = 0
def summary(self):
return ExecutionSummary(
'Processed into {object_count:d} build config descriptors in '
'{execution_time:.2f}s',
execution_time=self._emitter_time,
object_count=self._object_count)
def emit(self, output):
"""Convert the BuildReader output into data structures.
The return value from BuildReader.read_topsrcdir() (a generator) is
typically fed into this function.
"""
file_count = 0
sandbox_execution_time = 0.0
emitter_time = 0.0
contexts = {}
def emit_objs(objs):
for o in objs:
self._object_count += 1
yield o
if not o._ack:
raise Exception('Unhandled object of type %s' % type(o))
@@ -157,14 +166,10 @@ class TreeMetadataEmitter(LoggingMixin):
start = time.time()
# We need to expand the generator for the timings to work.
objs = list(self.emit_from_context(out))
emitter_time += time.time() - start
self._emitter_time += time.time() - start
for o in emit_objs(objs): yield o
# Update the stats.
file_count += len(out.all_paths)
sandbox_execution_time += out.execution_time
else:
raise Exception('Unhandled output type: %s' % type(out))
@@ -173,12 +178,10 @@ class TreeMetadataEmitter(LoggingMixin):
if self.config.substs.get('COMPILE_ENVIRONMENT', True):
start = time.time()
objs = list(self._emit_libs_derived(contexts))
emitter_time += time.time() - start
self._emitter_time += time.time() - start
for o in emit_objs(objs): yield o
yield ReaderSummary(file_count, sandbox_execution_time, emitter_time)
def _emit_libs_derived(self, contexts):
# First do FINAL_LIBRARY linkage.
for lib in (l for libs in self._libs.values() for l in libs):
python/mozbuild/mozbuild/frontend/gyp_reader.py
-5
View File
@@ -6,7 +6,6 @@ from __future__ import absolute_import, unicode_literals
import gyp
import sys
import time
import os
import mozpack.path as mozpath
from mozpack.files import FileFinder
@@ -83,8 +82,6 @@ def read_from_gyp(config, path, output, vars, non_unified_sources = set()):
processor.
"""
time_start = time.time()
# gyp expects plain str instead of unicode. The frontend code gives us
# unicode strings, so convert them.
path = encode(path)
@@ -234,6 +231,4 @@ def read_from_gyp(config, path, output, vars, non_unified_sources = set()):
context['DEFINES']['_UNICODE'] = True
context['DISABLE_STL_WRAPPING'] = True
context.execution_time = time.time() - time_start
yield context
time_start = time.time()
python/mozbuild/mozbuild/frontend/reader.py
+18 -1
View File
@@ -73,6 +73,9 @@ from .context import (
TemplateContext,
)
from mozbuild.base import ExecutionSummary
if sys.version_info.major == 2:
text_type = unicode
type_type = types.TypeType
@@ -864,6 +867,16 @@ class BuildReader(object):
self._execution_stack = []
self._finder = finder
self._execution_time = 0.0
self._file_count = 0
def summary(self):
return ExecutionSummary(
'Finished reading {file_count:d} moz.build files in '
'{execution_time:.2f}s',
file_count=self._file_count,
execution_time=self._execution_time)
def read_topsrcdir(self):
"""Read the tree of linked moz.build files.
@@ -1098,7 +1111,8 @@ class BuildReader(object):
sandbox = MozbuildSandbox(context, metadata=metadata,
finder=self._finder)
sandbox.exec_file(path)
context.execution_time = time.time() - time_start
self._execution_time += time.time() - time_start
self._file_count += len(context.all_paths)
# Yield main context before doing any processing. This gives immediate
# consumers an opportunity to change state before our remaining
@@ -1136,6 +1150,7 @@ class BuildReader(object):
raise SandboxValidationError('Cannot find %s.' % source,
context)
non_unified_sources.add(source)
time_start = time.time()
for gyp_context in read_from_gyp(context.config,
mozpath.join(curdir, gyp_dir.input),
mozpath.join(context.objdir,
@@ -1144,6 +1159,8 @@ class BuildReader(object):
non_unified_sources = non_unified_sources):
gyp_context.update(gyp_dir.sandbox_vars)
gyp_contexts.append(gyp_context)
self._file_count += len(gyp_context.all_paths)
self._execution_time += time.time() - time_start
for gyp_context in gyp_contexts:
context['DIRS'].append(mozpath.relpath(gyp_context.objdir, context.objdir))
python/mozbuild/mozbuild/jar.py
+3 -1
View File
@@ -20,6 +20,7 @@ from MozZipFile import ZipFile
from cStringIO import StringIO
from mozbuild.util import (
ensureParentDir,
lock_file,
PushbackIter,
)
@@ -200,6 +201,7 @@ class JarMaker(object):
with the given chrome base path, and updates the given manifest file.
'''
ensureParentDir(manifestPath)
lock = lock_file(manifestPath + '.lck')
try:
myregister = dict.fromkeys(map(lambda s: s.replace('%',
@@ -388,7 +390,7 @@ class JarMaker(object):
m.group('optPreprocess') or '',
m.group('optOverwrite') or '',
out,
m.group('locale') or '',
m.group('locale').replace('%', '%%') or '',
)
for _srcdir in src_base:
finder = FileFinder(_srcdir, find_executables=False)
python/mozbuild/mozbuild/mach_commands.py
+9 -6
View File
@@ -526,11 +526,11 @@ class Build(MachCommandBase):
help='Show a diff of changes.')
# It would be nice to filter the choices below based on
# conditions, but that is for another day.
@CommandArgument('-b', '--backend',
choices=['RecursiveMake', 'AndroidEclipse', 'CppEclipse', 'VisualStudio'],
default='RecursiveMake',
help='Which backend to build (default: RecursiveMake).')
def build_backend(self, backend='RecursiveMake', diff=False):
@CommandArgument('-b', '--backend', nargs='+',
choices=['RecursiveMake', 'AndroidEclipse', 'CppEclipse',
'VisualStudio', 'FasterMake'],
help='Which backend to build.')
def build_backend(self, backend, diff=False):
python = self.virtualenv_manager.python_path
config_status = os.path.join(self.topobjdir, 'config.status')
@@ -540,7 +540,10 @@ class Build(MachCommandBase):
% backend)
return 1
args = [python, config_status, '--backend=%s' % backend]
args = [python, config_status]
if backend:
args.append('--backend')
args.extend(backend)
if diff:
args.append('--diff')
python/mozbuild/mozbuild/mozconfig.py
+19 -1
View File
@@ -8,6 +8,7 @@ import filecmp
import os
import re
import subprocess
import traceback
from collections import defaultdict
from mach.mixin.process import ProcessExecutionMixin
@@ -31,6 +32,13 @@ by a command inside your mozconfig failing. Please change your mozconfig
to not error and/or to catch errors in executed commands.
'''.strip()
MOZCONFIG_BAD_OUTPUT = '''
Evaluation of your mozconfig produced unexpected output. This could be
triggered by a command inside your mozconfig failing or producing some warnings
or error messages. Please change your mozconfig to not error and/or to catch
errors in executed commands.
'''.strip()
class MozconfigFindException(Exception):
"""Raised when a mozconfig location is not defined properly."""
@@ -234,7 +242,17 @@ class MozconfigLoader(ProcessExecutionMixin):
raise MozconfigLoadException(path, MOZCONFIG_BAD_EXIT_CODE, lines)
parsed = self._parse_loader_output(output)
try:
parsed = self._parse_loader_output(output)
except AssertionError:
# _parse_loader_output uses assertions to verify the
# well-formedness of the shell output; when these fail, it
# generally means there was a problem with the output, but we
# include the assertion traceback just to be sure.
print('Assertion failed in _parse_loader_output:')
traceback.print_exc()
raise MozconfigLoadException(path, MOZCONFIG_BAD_OUTPUT,
output.splitlines())
def diff_vars(vars_before, vars_after):
set1 = set(vars_before.keys()) - self.IGNORE_SHELL_VARIABLES
python/mozbuild/mozbuild/mozconfig_loader
+14 -1
View File
@@ -36,10 +36,23 @@ ac_add_app_options() {
}
mk_add_options() {
local opt
local opt name op value
for opt; do
echo "------BEGIN_MK_OPTION"
echo $opt
# Remove any leading "export"
opt=${opt#export}
case "$opt" in
*\?=*) op="?=" ;;
*:=*) op=":=" ;;
*+=*) op="+=" ;;
*=*) op="=" ;;
esac
# Remove the operator and the value that follows
name=${opt%%${op}*}
# Note: $(echo ${name}) strips the variable from any leading and trailing
# whitespaces.
eval "$(echo ${name})_IS_SET=1"
echo "------END_MK_OPTION"
done
}
python/mozbuild/mozbuild/test/frontend/test_emitter.py
-6
View File
@@ -25,7 +25,6 @@ from mozbuild.frontend.data import (
JsPreferenceFile,
LocalInclude,
Program,
ReaderSummary,
Resources,
SimpleProgram,
Sources,
@@ -77,17 +76,12 @@ class TestEmitterBasic(unittest.TestCase):
objs = list(ack(o) for o in emitter.emit(reader.read_topsrcdir()))
self.assertGreater(len(objs), 0)
self.assertIsInstance(objs[-1], ReaderSummary)
filtered = []
for obj in objs:
if filter_common and isinstance(obj, DirectoryTraversal):
continue
# Always filter ReaderSummary because it's asserted above.
if isinstance(obj, ReaderSummary):
continue
filtered.append(obj)
return filtered
python/mozbuild/mozbuild/test/test_mozconfig.py
+4
View File
@@ -319,6 +319,10 @@ class TestMozconfigLoader(unittest.TestCase):
self.assertEqual(result['make_flags'], ['-j8', '-s'])
self.assertEqual(result['make_extra'], ['FOO=BAR BAZ', 'BIZ=1'])
vars = result['vars']['added']
for var in ('MOZ_OBJDIR', 'MOZ_MAKE_FLAGS', 'FOO', 'BIZ'):
self.assertEqual(vars.get('%s_IS_SET' % var), '1')
def test_read_empty_mozconfig_objdir_environ(self):
os.environ[b'MOZ_OBJDIR'] = b'obj-firefox'
with NamedTemporaryFile(mode='w') as mozconfig: