From a51002fbcacaffa339c97e1d0d0dac40cb708e1b Mon Sep 17 00:00:00 2001 From: roytam1 Date: Thu, 29 Dec 2022 09:22:12 +0800 Subject: [PATCH] import changes from `dev' branch of rmottola/Arctic-Fox: - Bug 1202386: Add logging macros for HAL IPC, r=shuang (246eb96f38) - Bug 1202386: Output clear HAL IPC errors, r=shuang (eaba0dc9a8) - Bug 1202704: Move Bluetooth IPC pack functions to generic HAL IPC, r=shuang (4c745de79c) - Bug 1202704: Move Bluetooth IPC unpack functions to generic HAL IPC, r=shuang (4f8600030b) - Bug 1202704: Move Bluetooth's |UnpackPDUInitOp| to generic HAL IPC code, r=shuang (345f21c637) - Bug 1209085: Add 6-argument operator () to |UnpackPDUInitOp|, r=joliu (f5f8cf2dd9) - Bug 1123760 - make autocomplete dropmarker in the urlbar actually work when activated through a11y APIs, r=surkov (eee42195f4) - Bug 1123760 - bustage follow-up: remove unused variable, rs=bustage on a CLOSED TREE (6de8519b23) - Bug 1152836 - QR Decoder: Let everywhere, style nits: 580ms -> 400ms r=past (92a7fd6a15) - Bug 1212430 - Remove CrashAtUnhandlableOOM() and replace with AutoEnterOOMUnsafeRegion r=jandem (f595b87951) - Bug 1175755 - Only clear GC statistics aborted flag at the end of the outermost nested GC r=bbouvier (ca73f34c69) - reapply Bug 1221385 - Handle OOM during JitRuntime (e7def65b78) - No bug: Clarify documentation for js::NewObjectMetadataState. DONTBUILD r=fitzgen (41022b4137) - Bug 1211164 - Collect JS deprecated language extension telemetry for Add-ons. r=till,bsmedberg (eacd40ad66) - Bug 1212296 - undo a state change on OOM. r=till (6e272353d0) - Bug 1214006 - Take account of the fact that JSScript::atoms may be null while tracing r=terrence (19f61d7494) - missing bits of Bug 1208665 (c31173b0d4) - Bug 1221891 - "Fix a typo in TraceLoggingGraph.h". r=hv1989 (e1fc11f8df) - Bug 1221460 - "TraceLogger: Enable several new optimizations in 'TLLOG=IonCompiler'". r=hv1989 (ab3398646e) - Bug 1204365 - Repair external view source file name and extension. r=mconley (f033e55a0e) - Bug 1163693 - Fix View Source external editor fallback. r=jryans. (9c8becc93b) - Bug 1207629 - Don't assume that viewSourceUtils.js has Services in scope. r=jryans (07977953c0) - add back some telemetry (1e3b5bde5b) - Bug 1186785 - Replace nsBaseHashtable::EnumerateRead() calls in toolkit/ with iterators. r=froydnj. (e93e098dd8) - add back some crashreporter (80e325b3be) - Bug 1220035 - Fix -Wimplicit-fallthrough warnings in xpcom. r=mccr8 (2ef9ecad5f) - Bug 1215629 - Remove nsDebug logger. r=froydnj (46784f05fd) - Bug 1137963 - Use a spin lock for TraceRefCnt. r=waldo, r=froydnj (b2420c97c0) - Bug 1196430 - part 1 - rename serialNumberRecord to SerialNumberRecord; r=mccr8 (933670070d) - Bug 1196430 - part 2 - give SerialNumberRecord a proper constructor; r=mccr8 (3ddf3d5e69) - Bug 1196430 - part 3 - remove unnecessary nsString.h include from nsTraceRefcnt.cpp; r=mccr8 (034954e692) - Bug 1196430 - part 4 - record allocation stacks for classes in XPCOM_MEM_LOG_CLASSES; r=mccr8 (366b612807) - Bug 1196430 - part 5 - dump allocation stacks for leaked objects in XPCOM_MEM_LOG_CLASSES; r=mccr8 (dce7b9cca2) - Bug 1180745 - Fix logging test screenshotting errors. r=jgriffin (a589f0d322) - Bug 1091285 - move dumpScreen in a new mozscreenshot package. r=jgriffin This also completely remove build/automationutils.py. (6e633359ef) - some crashreporter stuff (2d0bc9c95d) - Bug 1196430 - part 6 - move cut-and-paste stack fixer code into mozrunner; r=wlach (ffc7ccd521) - Bug 1196430 - part 7 - teach process_leak_log how to symbolicate leaked object stacks; r=mccr8 (0b5a4ace7c) - Bug 1196430 - part 8 - use less reinterpret_cast in nsTraceRefcnt.cpp; r=mccr8 (758cfca0aa) - Bug 1196430 follow-up: Hide the usage of gCodeAddressService behind #ifdef MOZ_STACKWALKING (e8d62dd73e) - Bug 487494 - Add an xpcshell selftest for readable stacks from assertions.;r=ted (ea15cf3cbb) - Bug 1156977 - Assert when aClassName is empty in BloatEntry. r=froydnj Bug 1116550 - Part 1: Turn HaveLeaks and Clear into methods. r=froydnj (8d7f88f498) - Bug 1116550 - Part 2: Print out negative values for leaks when there are more dtors than ctors. r=froydnj (7c9e3e7848) - Bug 1190483 - Add a way to record a DMD log late in shutdown. r=erahm (df7c22e64d) - Bug 1174344 - make error message for mismatched leak log entries more helpful; r=mccr8 (7f969e72c0) - Bug 1190483 - Followup to address review comment. (d3873f76fd) - Bug 1186025. Optimize the usage of regions. r=mstange (263080a66e) - Bug 1211841 - Style off the main thread markers differently, r=jsantell (1a183c5d3e) - Bug 1211839 - Don't allow off the main thread markers to nest under main thread markers, r=smaug, jsantell (f4d4b7ccf1) - Bug 1207161 - fix run-by-dir leak in test_bug846906.xul; r=mccr8 (5511752103) - Bug 1205348 - Always do shutdown CCs when NS_FREE_PERMANENT_DATA is defined. r=smaug (7fd7a7455c) - Bug 1208157, part 1 - Add and use nsCycleCollector::IsIdle() predicate. r=smaug (a06c2bd1db) - Bug 1208157, part 2 - Make the fields of nsCycleCollector private. r=smaug (494637fbef) - Bug 1207368 - Use swap() instead of forget() to remove MessageElement::mMessage. r=froydnj (8c58d38a55) - Bug 1181520 - Remove support for passing in reftest arguments via the command line, r=jmaher (75e9440e40) - Bug 1196814 - Fail Android mochitest, robocop, reftests when Fennec is not installed; r=jmaher (6ec15636eb) - Bug 1183717 - Increase default timeout for Android Debug reftests; r=jmaher (1b07fc1c9b) - Bug 1181516 - Allow reftests to take paths to multiple directories containing tests on the command line, r=jmaher (1e27ef0d69) - Bug 1198944 - remove vmware recording support from mochitest; r=khuey (06e79556fb) - Bug 1162003 - Enable run-by-dir mode on Fx desktop opt builds. r=jmaher (0ef288a33c) - Bug 1087629 - Add two new test cases for ICE connection states. r=ekr (374112f2d0) - Bug 1186551 - [mozlog] add structured action process_start/process_exit. r=jgraham (13ce88dbf7) - Bug 1154111 - Colorize SKIP in test logs. r=jgraham (85910e0f8b) - Bug 1191267 - Fix mozlog log buffering command line option, r=chmanchester (2bd6592f9b) - Bug 1185244 - Improve mach support for running mochitests on Valgrind. r=jgraham, njn. (7f5a830fa0) - Bug 1219870 - [mozlog] ensure correct suite state when logging suite_start/suite_end via StructuredLogger.log_raw, r=chmanchester (21710387a4) - Bug 1198257 - Better support for providing a directory name and discovering reftests under that directory, r=jmaher (f6255fc44c) - Bug 1208220 - Remove test of manifest filename that breaks my workflow. r=jgraham (ae4e45946d) - Bug 1186888 - [mozlog] Ability to use a pre-existing logger with commandline.setup_logging(), r=jgraham (80dfa2a8a8) - Bug 1042998 - Use StructuredLog.jsm for mochitest logging, r=chmanchester (8851b1b6f9) - Bug 1218010 - Shorten the polling interval when waiting for httpd.js startup in mochitest. r=ahal (a1f2c81a8e) - Bug 1224305 - Add an option to the mochitest harness to provide a copy of the extra chrome manifest it writes. r=ahal (e617971f41) - Bug 1170342 - Don't disable XInput2 for mochitests on GTK3, off by default now. r=karlt (05d53439da) - Bug 1145375 - Don't kill the debugger if the user Ctrl-C's as running a mochitest; r=ted (6c310500b8) - Bug 1199241 - Average runtime data across platforms instead of keeping it distinct, r=jgriffin (cd497f509c) - Bug 1157852 - Mochitest DevTools test directories run multiple times. r=ahal (415ab41a3a) - Bug 1186791 (part 1) - Replace nsBaseHashtable::EnumerateRead() calls in storage/ with iterators. r=mak. (84f6f1f566) - Bug 1186791 (part 2) - Replace nsBaseHashtable::EnumerateRead() calls in storage/ with iterators. r=mak. (9c67504d0c) - Bug 1186791 (part 3) - Replace nsBaseHashtable::EnumerateRead() calls in storage/ with iterators. r=mak. (02f472c197) - Bug 1219238 - remove AutoArray from mozStorageSQLFunctions.cpp; r=mak (45e751d2b7) - Bug 1166931 - JS Warning in MobileIdentityManager.jsm r=ferjm (2022d4cccd) - var-const (b7800ec532) - Trivial, no bug: add missing semicolon to nsBlocklistService.js to avoid a strict warning. (ee6b8a7593) - Bug 1208242 - Part 1: hook up the blocklist service to b2g web extensions r=mossop,ferjm (d0ad653af4) - Bug 1208242 - Part 3: don't ship things that should not ship r=me (cd75e88080) - Bug 1009795 - Use toLocalString to format download size instead of the decimalSymbol hook; r=mak (a4b4442d2c) - Bug 1009795 - Part 2: Revert to the old gDecimalSymbol hack if the Internationalization API is not available; r=mak (465e23f2c3) - Bug 1116385. r=Mossop (f0a7b7d450) - let-var (cb5d9d1d07) - Bug 1210459: Add originAttributes for tests that implement nsILoadContext. r=bholley (ea6be1490a) - var-let (d7d4533b53) - Bug 1034724 - Fixed Unicode values of prefs in about:support. r=adw (1c8253ac5b) - Bug 1153381. Add a D3D11 ANGLE blacklist. r=mstange (9008483ca5) - add back some WIn XP and 2k3 stuff (896a4a7e9b) - Bug 1141783 - Correct user message for mismatched drivers. Don't mismatch if the DLLs are missing. r=jrmuizelaar (872d0c3aff) - missing members version (4771ff5f24) - bug 1170987 - Fix gfx/2d to build on iOS. r=jrmuizel (1e555cb6b3) - Bug 1201937 - push transform onto cairo context when evaluating path bounds. r=eihrul (6a4d8d98ec) - Bug 1165900 - Make MaybeSnapToDevicePixels return a boolean to indicate whether snapping occurred. r=Bas (8f33f6cf7d) - Bug 1190705 - Add crashtest for canvas 2d. r=Bas (bf3afb2acc) - Bug 1161277 - verify SkPath is finite before doing ContainsPoint queries. r=jmuizelaar (c56f9ef322) - Bug 1218900 - Make shell function startTimingMutator() fail with an error rather than asserting when called at the wrong time r=sfink (856e8678ce) - small type fix (b36cfdf416) - Bug 1214846 - Make SPSProfiler::enter() report OOM to the context r=terrence (6086f60d17) - Bug 1218637 - IonMonkey: MIPS64: Add support into vm. r=arai (ae22538418) - gfx: add back `AddCrashReportAnnotations()` prototype (c13f61cd8a) --- accessible/xul/XULFormControlAccessible.cpp | 23 +- b2g/app/b2g.js | 5 + b2g/installer/package-manifest.in | 5 +- build/Makefile.in | 3 - build/automation.py.in | 35 +- build/automationutils.py | 103 -- build/dumpScreen.py | 30 - build/mach_bootstrap.py | 1 + build/win32/moz.build | 3 - build/win32/vmwarerecordinghelper/moz.build | 15 - .../vmwarerecordinghelper.cpp | 34 - .../vmwarerecordinghelper.def | 8 - .../base/timeline/AbstractTimelineMarker.cpp | 19 + .../base/timeline/AbstractTimelineMarker.h | 9 + .../base/timeline/CompositeTimelineMarker.h | 33 + .../base/timeline/ConsoleTimelineMarker.h | 2 + docshell/base/timeline/EventTimelineMarker.h | 2 + .../base/timeline/JavascriptTimelineMarker.h | 2 + .../base/timeline/RestyleTimelineMarker.h | 2 + docshell/base/timeline/TimelineMarker.cpp | 5 +- .../base/timeline/TimestampTimelineMarker.h | 2 + docshell/base/timeline/WorkerTimelineMarker.h | 2 + docshell/base/timeline/moz.build | 1 + docshell/test/chrome/chrome.ini | 2 +- .../bluedroid/BluetoothDaemonHelpers.cpp | 18 - .../bluedroid/BluetoothDaemonHelpers.h | 660 +------- dom/canvas/crashtests/1161277-1.html | 22 + dom/canvas/crashtests/1190705.html | 17 + dom/canvas/crashtests/crashtests.list | 3 + dom/media/tests/mochitest/ipc/mochitest.ini | 2 +- dom/media/tests/mochitest/mochitest.ini | 3 + dom/media/tests/mochitest/templates.js | 2 +- .../test_peerConnection_closeDuringIce.html | 79 + .../test_peerConnection_iceFailure.html | 83 + dom/tests/mochitest/pointerlock/mochitest.ini | 2 +- dom/webidl/ProfileTimelineMarker.webidl | 3 + extensions/cookie/test/channel_utils.js | 4 + gfx/2d/2D.h | 2 +- gfx/2d/BorrowedContext.h | 2 +- gfx/2d/DrawTargetCG.cpp | 21 + gfx/2d/DrawTargetCG.h | 7 + gfx/2d/DrawTargetCairo.cpp | 2 + gfx/2d/Factory.cpp | 12 +- gfx/2d/MacIOSurface.h | 27 +- gfx/2d/Matrix.h | 7 + gfx/2d/PathCG.h | 5 + gfx/2d/PathCairo.cpp | 22 +- gfx/2d/PathCairo.h | 3 +- gfx/2d/PathHelpers.h | 4 +- gfx/2d/PathSkia.cpp | 8 + gfx/2d/ScaledFontMac.cpp | 18 +- gfx/2d/ScaledFontMac.h | 8 +- gfx/2d/SourceSurfaceCG.cpp | 4 + gfx/2d/SourceSurfaceCG.h | 8 + gfx/2d/moz.build | 6 +- gfx/layers/composite/TextureHost.cpp | 2 +- gfx/thebes/gfxPlatform.cpp | 1 - ipc/hal/DaemonSocketPDUHelpers.cpp | 99 +- ipc/hal/DaemonSocketPDUHelpers.h | 755 ++++++++++ ipc/ipdl/test/cxx/TestDataStructures.cpp | 2 +- js/public/Utility.h | 7 +- js/src/gc/Statistics.cpp | 30 +- js/src/gc/Statistics.h | 2 +- js/src/jit-test/tests/gc/bug-1175755.js | 9 + js/src/jit-test/tests/gc/bug-1214006.js | 17 + js/src/jit-test/tests/gc/bug-1214846.js | 7 + js/src/jit-test/tests/gc/bug-1218900-2.js | 3 + js/src/jit-test/tests/gc/bug-1218900.js | 8 + js/src/jit/JitFrames.cpp | 2 +- js/src/jit/arm/Assembler-arm.h | 2 +- js/src/jsalloc.h | 6 +- js/src/jscntxt.cpp | 8 +- js/src/jscompartment.cpp | 3 +- js/src/jscompartment.h | 8 +- js/src/jsscript.cpp | 12 +- js/src/shell/js.cpp | 6 +- js/src/vm/MallocProvider.h | 39 +- js/src/vm/NativeObject.cpp | 4 +- js/src/vm/Probes-inl.h | 2 +- js/src/vm/Runtime.cpp | 1 + js/src/vm/Runtime.h | 2 +- js/src/vm/SPSProfiler.cpp | 6 +- js/src/vm/SPSProfiler.h | 2 +- js/src/vm/TraceLogging.cpp | 6 +- js/src/vm/TraceLoggingGraph.h | 2 +- layout/base/nsDisplayList.cpp | 4 +- layout/tools/reftest/Makefile.in | 2 +- layout/tools/reftest/b2g_desktop.py | 23 +- layout/tools/reftest/mach_commands.py | 471 ++---- layout/tools/reftest/reftest-cmdline.js | 33 +- layout/tools/reftest/reftest.js | 99 +- layout/tools/reftest/reftestcommandline.py | 740 +++++++++ layout/tools/reftest/remotereftest.py | 235 +-- layout/tools/reftest/runreftest.py | 417 ++---- layout/tools/reftest/runreftestb2g.py | 257 +--- netwerk/test/unit/test_bug826063.js | 1 + netwerk/test/unit/test_cacheflags.js | 1 + services/cloudsync/tests/xpcshell/head.js | 2 +- .../common/modules-testing/bagheeraserver.js | 2 +- services/common/modules-testing/logging.js | 2 +- .../common/modules-testing/storageserver.js | 2 +- services/common/tests/unit/head_global.js | 2 +- .../crypto/component/tests/unit/test_jpake.js | 4 +- services/crypto/tests/unit/head_helpers.js | 8 +- .../tests/xpcshell/test_healthreporter.js | 4 +- .../tests/xpcshell/test_profile.js | 4 +- .../tests/xpcshell/test_provider_addons.js | 4 +- .../tests/xpcshell/test_provider_appinfo.js | 2 +- .../tests/xpcshell/test_provider_hotfix.js | 2 +- .../tests/xpcshell/test_provider_places.js | 2 +- .../tests/xpcshell/test_provider_searches.js | 4 +- .../tests/xpcshell/test_provider_sessions.js | 2 +- .../tests/xpcshell/test_provider_sysinfo.js | 2 +- .../tests/xpcshell/test_metrics_provider.js | 2 +- .../xpcshell/test_metrics_provider_manager.js | 2 +- .../tests/xpcshell/test_metrics_storage.js | 2 +- services/mobileid/MobileIdentityManager.jsm | 6 +- storage/mozStorageBindingParams.cpp | 86 +- storage/mozStorageBindingParams.h | 11 - storage/mozStorageConnection.cpp | 78 +- storage/mozStorageSQLFunctions.cpp | 56 +- testing/config/mozbase_requirements.txt | 1 + testing/config/mozharness/linux_config.py | 3 +- testing/mochitest/Makefile.in | 1 - testing/mochitest/browser-harness.xul | 3 +- testing/mochitest/harness.xul | 2 + testing/mochitest/jar.mn | 1 + testing/mochitest/mochitest_options.py | 65 +- testing/mochitest/moz.build | 1 - testing/mochitest/runrobocop.py | 8 + testing/mochitest/runtests.py | 224 ++- testing/mochitest/runtestsb2g.py | 5 +- testing/mochitest/runtestsremote.py | 17 +- testing/mochitest/server.js | 2 + testing/mochitest/tests/SimpleTest/setup.js | 2 +- testing/mochitest/tests/index.html | 1 + testing/modules/StructuredLog.jsm | 301 ++-- testing/modules/moz.build | 2 + .../tests/xpcshell/test_structuredlog.js | 18 +- testing/mozbase/moz.build | 1 + testing/mozbase/mozdebug/mozdebug/mozdebug.py | 81 +- testing/mozbase/mozleak/mozleak/leaklog.py | 14 +- testing/mozbase/mozlog/mozlog/commandline.py | 40 +- .../mozlog/mozlog/formatters/machformatter.py | 16 + .../mozlog/mozlog/formatters/process.py | 55 + .../mozlog/mozlog/formatters/tbplformatter.py | 18 + .../mozlog/mozlog/handlers/__init__.py | 1 + .../mozlog/mozlog/handlers/valgrindhandler.py | 137 ++ testing/mozbase/mozlog/mozlog/logtypes.py | 2 +- .../mozbase/mozlog/mozlog/structuredlog.py | 63 +- .../mozbase/mozlog/tests/test_structured.py | 41 + testing/mozbase/mozrunner/mozrunner/utils.py | 52 +- .../mozscreenshot/mozscreenshot/__init__.py | 61 + testing/mozbase/mozscreenshot/setup.py | 26 + testing/mozbase/packages.txt | 1 + .../mozharness/mozilla/structuredlog.py | 1 + .../mozharness/mozilla/testing/errors.py | 5 + .../mochitest-browser-chrome.runtimes.json | 1 - .../mochitest-devtools-chrome.runtimes.json | 1 - .../mochitest-browser-chrome.runtimes.json | 1 - .../mochitest-devtools-chrome.runtimes.json | 1 - ...ochitest-e10s-browser-chrome.runtimes.json | 1 - ...chitest-e10s-devtools-chrome.runtimes.json | 1 - .../mochitest-browser-chrome.runtimes.json | 1 - .../mochitest-devtools-chrome.runtimes.json | 1 - .../mochitest-browser-chrome.runtimes.json | 1 - .../mochitest-devtools-chrome.runtimes.json | 1 - ...ochitest-e10s-browser-chrome.runtimes.json | 1 - ...chitest-e10s-devtools-chrome.runtimes.json | 1 - .../mochitest-browser-chrome.runtimes.json | 1 - .../mochitest-devtools-chrome.runtimes.json | 1 - .../mochitest-browser-chrome.runtimes.json | 1 - .../mochitest-devtools-chrome.runtimes.json | 1 - ...ochitest-e10s-browser-chrome.runtimes.json | 1 - .../mochitest-browser-chrome.runtimes.json | 689 +++++++++ .../mochitest-devtools-chrome.runtimes.json | 820 ++++++++++ ...ochitest-e10s-browser-chrome.runtimes.json | 477 ++++++ ...chitest-e10s-devtools-chrome.runtimes.json | 677 +++++++++ .../mochitest-browser-chrome.runtimes.json | 1 - .../mochitest-devtools-chrome.runtimes.json | 1 - .../mochitest-browser-chrome.runtimes.json | 1 - .../mochitest-devtools-chrome.runtimes.json | 1 - ...ochitest-e10s-browser-chrome.runtimes.json | 1 - .../mochitest-browser-chrome.runtimes.json | 1 - .../mochitest-devtools-chrome.runtimes.json | 1 - .../mochitest-browser-chrome.runtimes.json | 1 - .../mochitest-devtools-chrome.runtimes.json | 1 - ...ochitest-e10s-browser-chrome.runtimes.json | 1 - testing/testsuite-targets.mk | 4 +- testing/tools/mach_test_package_bootstrap.py | 1 + testing/xpcshell/Makefile.in | 1 - testing/xpcshell/head.js | 2 +- testing/xpcshell/mach_commands.py | 6 +- testing/xpcshell/runxpcshelltests.py | 25 +- testing/xpcshell/selftest.py | 34 +- .../filewatcher/NativeFileWatcherWin.cpp | 24 +- toolkit/components/places/nsNavHistory.cpp | 31 +- .../components/prompts/src/CommonDialog.jsm | 53 +- .../prompts/src/SharedPromptUtils.jsm | 111 +- .../components/remote/nsGTKRemoteService.cpp | 16 +- .../components/remote/nsGTKRemoteService.h | 5 - .../components/terminator/nsTerminator.cpp | 10 + .../viewsource/content/viewSourceUtils.js | 78 +- toolkit/devtools/performance/docs/markers.md | 9 + .../modules/logic/waterfall-utils.js | 210 ++- .../modules/widgets/marker-view.js | 1 + .../browser_timeline-waterfall-workers.js | 23 +- .../unit/test_waterfall-utils-collapse-05.js | 163 ++ toolkit/devtools/qrcode/decoder/index.js | 1334 ++++++++--------- .../chrome/global/aboutSupport.properties | 3 + toolkit/modules/Troubleshoot.jsm | 69 +- .../tests/browser/browser_Troubleshoot.js | 16 + toolkit/mozapps/downloads/DownloadLastDir.jsm | 9 +- toolkit/mozapps/downloads/DownloadUtils.jsm | 42 +- .../mozapps/downloads/content/downloads.js | 24 +- .../downloads/content/unknownContentType.xul | 1 - toolkit/mozapps/downloads/nsHelperAppDlg.js | 42 +- .../mozapps/downloads/tests/chrome/chrome.ini | 1 + .../test_unknownContentType_delayedbutton.xul | 117 ++ .../mozapps/extensions/extensions.manifest | 3 +- toolkit/mozapps/extensions/moz.build | 2 +- .../mozapps/extensions/nsBlocklistService.js | 54 +- .../shared/devtools/performance.inc.css | 65 +- view/nsView.cpp | 12 +- view/nsViewManager.cpp | 2 +- widget/GfxInfoBase.cpp | 10 +- widget/nsIGfxInfo.idl | 6 +- widget/windows/GfxInfo.cpp | 128 +- widget/windows/GfxInfo.h | 1 + xpcom/base/CycleCollectedJSRuntime.cpp | 18 + xpcom/base/nsConsoleService.cpp | 2 +- xpcom/base/nsConsoleService.h | 7 +- xpcom/base/nsCrashOnException.cpp | 11 + xpcom/base/nsCycleCollector.cpp | 24 +- xpcom/base/nsDebugImpl.cpp | 45 +- xpcom/base/nsTraceRefcnt.cpp | 259 +++- xpcom/ds/nsCheapSets.h | 3 +- xpcom/ds/nsPersistentProperties.cpp | 2 +- xpcom/ds/nsVariant.cpp | 7 +- xpcom/glue/nsTextFormatter.cpp | 4 +- xpcom/io/nsWildCard.cpp | 5 +- 241 files changed, 8266 insertions(+), 3868 deletions(-) delete mode 100644 build/automationutils.py delete mode 100644 build/dumpScreen.py delete mode 100644 build/win32/vmwarerecordinghelper/moz.build delete mode 100644 build/win32/vmwarerecordinghelper/vmwarerecordinghelper.cpp delete mode 100644 build/win32/vmwarerecordinghelper/vmwarerecordinghelper.def create mode 100644 docshell/base/timeline/CompositeTimelineMarker.h create mode 100644 dom/canvas/crashtests/1161277-1.html create mode 100644 dom/canvas/crashtests/1190705.html create mode 100644 dom/media/tests/mochitest/test_peerConnection_closeDuringIce.html create mode 100644 dom/media/tests/mochitest/test_peerConnection_iceFailure.html create mode 100644 js/src/jit-test/tests/gc/bug-1175755.js create mode 100644 js/src/jit-test/tests/gc/bug-1214006.js create mode 100644 js/src/jit-test/tests/gc/bug-1214846.js create mode 100644 js/src/jit-test/tests/gc/bug-1218900-2.js create mode 100644 js/src/jit-test/tests/gc/bug-1218900.js create mode 100644 layout/tools/reftest/reftestcommandline.py create mode 100644 testing/mozbase/mozlog/mozlog/formatters/process.py create mode 100644 testing/mozbase/mozlog/mozlog/handlers/valgrindhandler.py create mode 100644 testing/mozbase/mozscreenshot/mozscreenshot/__init__.py create mode 100644 testing/mozbase/mozscreenshot/setup.py delete mode 100644 testing/runtimes/linux-debug/mochitest-browser-chrome.runtimes.json delete mode 100644 testing/runtimes/linux-debug/mochitest-devtools-chrome.runtimes.json delete mode 100644 testing/runtimes/linux-opt/mochitest-browser-chrome.runtimes.json delete mode 100644 testing/runtimes/linux-opt/mochitest-devtools-chrome.runtimes.json delete mode 100644 testing/runtimes/linux-opt/mochitest-e10s-browser-chrome.runtimes.json delete mode 100644 testing/runtimes/linux-opt/mochitest-e10s-devtools-chrome.runtimes.json delete mode 100644 testing/runtimes/linux64-debug/mochitest-browser-chrome.runtimes.json delete mode 100644 testing/runtimes/linux64-debug/mochitest-devtools-chrome.runtimes.json delete mode 100644 testing/runtimes/linux64-opt/mochitest-browser-chrome.runtimes.json delete mode 100644 testing/runtimes/linux64-opt/mochitest-devtools-chrome.runtimes.json delete mode 100644 testing/runtimes/linux64-opt/mochitest-e10s-browser-chrome.runtimes.json delete mode 100644 testing/runtimes/linux64-opt/mochitest-e10s-devtools-chrome.runtimes.json delete mode 100644 testing/runtimes/macosx64-debug/mochitest-browser-chrome.runtimes.json delete mode 100644 testing/runtimes/macosx64-debug/mochitest-devtools-chrome.runtimes.json delete mode 100644 testing/runtimes/macosx64-opt/mochitest-browser-chrome.runtimes.json delete mode 100644 testing/runtimes/macosx64-opt/mochitest-devtools-chrome.runtimes.json delete mode 100644 testing/runtimes/macosx64-opt/mochitest-e10s-browser-chrome.runtimes.json create mode 100644 testing/runtimes/mochitest-browser-chrome.runtimes.json create mode 100644 testing/runtimes/mochitest-devtools-chrome.runtimes.json create mode 100644 testing/runtimes/mochitest-e10s-browser-chrome.runtimes.json create mode 100644 testing/runtimes/mochitest-e10s-devtools-chrome.runtimes.json delete mode 100644 testing/runtimes/win32-debug/mochitest-browser-chrome.runtimes.json delete mode 100644 testing/runtimes/win32-debug/mochitest-devtools-chrome.runtimes.json delete mode 100644 testing/runtimes/win32-opt/mochitest-browser-chrome.runtimes.json delete mode 100644 testing/runtimes/win32-opt/mochitest-devtools-chrome.runtimes.json delete mode 100644 testing/runtimes/win32-opt/mochitest-e10s-browser-chrome.runtimes.json delete mode 100644 testing/runtimes/win64-debug/mochitest-browser-chrome.runtimes.json delete mode 100644 testing/runtimes/win64-debug/mochitest-devtools-chrome.runtimes.json delete mode 100644 testing/runtimes/win64-opt/mochitest-browser-chrome.runtimes.json delete mode 100644 testing/runtimes/win64-opt/mochitest-devtools-chrome.runtimes.json delete mode 100644 testing/runtimes/win64-opt/mochitest-e10s-browser-chrome.runtimes.json create mode 100644 toolkit/devtools/performance/test/unit/test_waterfall-utils-collapse-05.js create mode 100644 toolkit/mozapps/downloads/tests/chrome/test_unknownContentType_delayedbutton.xul diff --git a/accessible/xul/XULFormControlAccessible.cpp b/accessible/xul/XULFormControlAccessible.cpp index b5b52e5771..d52ce23216 100644 --- a/accessible/xul/XULFormControlAccessible.cpp +++ b/accessible/xul/XULFormControlAccessible.cpp @@ -224,22 +224,27 @@ XULDropmarkerAccessible::DropmarkerOpen(bool aToggleOpen) const { bool isOpen = false; - nsCOMPtr parentButtonElement = - do_QueryInterface(mContent->GetFlattenedTreeParent()); + nsIContent* parent = mContent->GetFlattenedTreeParent(); + + while (parent) { + nsCOMPtr parentButtonElement = + do_QueryInterface(parent); + if (parentButtonElement) { + parentButtonElement->GetOpen(&isOpen); + if (aToggleOpen) + parentButtonElement->SetOpen(!isOpen); + return isOpen; + } - if (parentButtonElement) { - parentButtonElement->GetOpen(&isOpen); - if (aToggleOpen) - parentButtonElement->SetOpen(!isOpen); - } - else { nsCOMPtr parentMenuListElement = - do_QueryInterface(parentButtonElement); + do_QueryInterface(parent); if (parentMenuListElement) { parentMenuListElement->GetOpen(&isOpen); if (aToggleOpen) parentMenuListElement->SetOpen(!isOpen); + return isOpen; } + parent = parent->GetFlattenedTreeParent(); } return isOpen; diff --git a/b2g/app/b2g.js b/b2g/app/b2g.js index 0d39655820..49db6f945e 100644 --- a/b2g/app/b2g.js +++ b/b2g/app/b2g.js @@ -1105,3 +1105,8 @@ pref("dom.performance.enable_notify_performance_timing", true); pref("b2g.multiscreen.chrome_remote_url", "chrome://b2g/content/shell_remote.html"); pref("b2g.multiscreen.system_remote_url", "index_remote.html"); +// Blocklist service +pref("extensions.blocklist.enabled", true); +pref("extensions.blocklist.interval", 86400); +pref("extensions.blocklist.url", "https://blocklist.addons.mozilla.org/blocklist/3/%APP_ID%/%APP_VERSION%/%PRODUCT%/%BUILD_ID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/%PING_COUNT%/%TOTAL_PING_COUNT%/%DAYS_SINCE_LAST_PING%/"); +pref("extensions.blocklist.detailsURL", "https://www.mozilla.com/%LOCALE%/blocklist/"); diff --git a/b2g/installer/package-manifest.in b/b2g/installer/package-manifest.in index 0ec0c1a4eb..eb486b5f82 100644 --- a/b2g/installer/package-manifest.in +++ b/b2g/installer/package-manifest.in @@ -545,15 +545,16 @@ #endif // MOZ_WIDGET_GONK && MOZ_B2G_RIL #ifndef MOZ_WIDGET_GONK -@RESPATH@/components/extensions.manifest @RESPATH@/components/addonManager.js @RESPATH@/components/amContentHandler.js @RESPATH@/components/amInstallTrigger.js @RESPATH@/components/amWebInstallListener.js -@RESPATH@/components/nsBlocklistService.js + @RESPATH@/components/OopCommandLine.js @RESPATH@/components/CommandLine.js #endif +@RESPATH@/components/extensions.manifest +@RESPATH@/components/nsBlocklistService.js @RESPATH@/components/BootstrapCommandLine.js #ifdef MOZ_UPDATER diff --git a/build/Makefile.in b/build/Makefile.in index be6f70eece..becbb86a75 100644 --- a/build/Makefile.in +++ b/build/Makefile.in @@ -95,6 +95,3 @@ endif libs:: automation.py -ifdef ENABLE_TESTS -GARBAGE += $(srcdir)/automationutils.pyc -endif # ENABLE_TESTS diff --git a/build/automation.py.in b/build/automation.py.in index d6e7b85d5f..ab52897f80 100644 --- a/build/automation.py.in +++ b/build/automation.py.in @@ -16,7 +16,6 @@ from datetime import datetime, timedelta SCRIPT_DIR = os.path.abspath(os.path.realpath(os.path.dirname(sys.argv[0]))) sys.path.insert(0, SCRIPT_DIR) -import automationutils # -------------------------------------------------------------- # TODO: this is a hack for mozbase without virtualenv, remove with bug 849900 @@ -31,6 +30,8 @@ if os.path.isdir(mozbase): sys.path.append(package_path) import mozcrash +from mozscreenshot import printstatus, dump_screen + # --------------------------------------------------------------- @@ -63,6 +64,7 @@ _IS_CYGWIN = False #expand _CERTS_SRC_DIR = __CERTS_SRC_DIR__ #expand _IS_TEST_BUILD = __IS_TEST_BUILD__ #expand _IS_DEBUG_BUILD = __IS_DEBUG_BUILD__ +#expand _CRASHREPORTER = __CRASHREPORTER__ == 1 #expand _IS_ASAN = __IS_ASAN__ == 1 @@ -108,6 +110,7 @@ class Automation(object): CERTS_SRC_DIR = _CERTS_SRC_DIR IS_TEST_BUILD = _IS_TEST_BUILD IS_DEBUG_BUILD = _IS_DEBUG_BUILD + CRASHREPORTER = _CRASHREPORTER IS_ASAN = _IS_ASAN # timeout, in seconds @@ -225,6 +228,12 @@ class Automation(object): if dmdPath and dmdLibrary and preloadEnvVar: env[preloadEnvVar] = os.path.join(dmdPath, dmdLibrary) + if crashreporter and not debugger: + env['MOZ_CRASHREPORTER_NO_REPORT'] = '1' + env['MOZ_CRASHREPORTER'] = '1' + else: + env['MOZ_CRASHREPORTER_DISABLE'] = '1' + # Crash on non-local network connections by default. # MOZ_DISABLE_NONLOCAL_CONNECTIONS can be set to "0" to temporarily # enable non-local connections for the purposes of local testing. Don't @@ -359,7 +368,7 @@ class Automation(object): return self.haveDumpedScreen = True; - automationutils.dumpScreen(utilityPath) + dump_screen(utilityPath, self.log) def killAndGetStack(self, processPID, utilityPath, debuggerInfo): @@ -371,6 +380,20 @@ class Automation(object): def killAndGetStackNoScreenshot(self, processPID, utilityPath, debuggerInfo): """Kill the process, preferrably in a way that gets us a stack trace.""" + if self.CRASHREPORTER and not debuggerInfo: + if not self.IS_WIN32: + # ABRT will get picked up by Breakpad's signal handler + os.kill(processPID, signal.SIGABRT) + return + else: + # We should have a "crashinject" program in our utility path + crashinject = os.path.normpath(os.path.join(utilityPath, "crashinject.exe")) + if os.path.exists(crashinject): + status = subprocess.Popen([crashinject, str(processPID)]).wait() + printstatus("crashinject", status) + if status == 0: + return + self.log.info("Can't trigger Breakpad, just killing process") self.killPid(processPID) def waitForFinish(self, proc, utilityPath, timeout, maxTime, startTime, debuggerInfo, symbolsPath): @@ -436,7 +459,7 @@ class Automation(object): self.killAndGetStack(browserProcessId, utilityPath, debuggerInfo) status = proc.wait() - automationutils.printstatus(status, "Main app process") + printstatus("Main app process", status) if status == 0: self.lastTestSeen = "Main app process exited normally" if status != 0 and not didTimeout and not hitMaxTime: @@ -507,7 +530,8 @@ class Automation(object): xrePath = None, certPath = None, debuggerInfo = None, symbolsPath = None, timeout = -1, maxTime = None, onLaunch = None, - detectShutdownLeaks = False, screenshotOnFail=False, testPath=None, bisectChunk=None): + detectShutdownLeaks = False, screenshotOnFail=False, testPath=None, bisectChunk=None, + valgrindPath=None, valgrindArgs=None, valgrindSuppFiles=None): """ Run the app, log the duration it took to execute, return the status code. Kills the app if it runs for longer than |maxTime| seconds, or outputs nothing for |timeout| seconds. @@ -545,7 +569,8 @@ class Automation(object): self.lastTestSeen = "automation.py" proc = self.Process([cmd] + args, - env = self.environment(env, xrePath = xrePath), + env = self.environment(env, xrePath = xrePath, + crashreporter = not debuggerInfo), stdout = outputPipe, stderr = subprocess.STDOUT) self.log.info("INFO | automation.py | Application pid: %d", proc.pid) diff --git a/build/automationutils.py b/build/automationutils.py deleted file mode 100644 index 9884fe700c..0000000000 --- a/build/automationutils.py +++ /dev/null @@ -1,103 +0,0 @@ -# -# This Source Code Form is subject to the terms of the Mozilla Public -# License, v. 2.0. If a copy of the MPL was not distributed with this -# file, You can obtain one at http://mozilla.org/MPL/2.0/. - -from __future__ import with_statement -import logging -from operator import itemgetter -import os -import platform -import re -import signal -import subprocess -import sys -import tempfile -import mozinfo - -__all__ = [ - 'dumpScreen', - "setAutomationLog", - ] - -log = logging.getLogger() -def resetGlobalLog(): - while log.handlers: - log.removeHandler(log.handlers[0]) - handler = logging.StreamHandler(sys.stdout) - log.setLevel(logging.INFO) - log.addHandler(handler) -resetGlobalLog() - -def setAutomationLog(alt_logger): - global log - log = alt_logger - -# Python does not provide strsignal() even in the very latest 3.x. -# This is a reasonable fake. -def strsig(n): - # Signal numbers run 0 through NSIG-1; an array with NSIG members - # has exactly that many slots - _sigtbl = [None]*signal.NSIG - for k in dir(signal): - if k.startswith("SIG") and not k.startswith("SIG_") and k != "SIGCLD" and k != "SIGPOLL": - _sigtbl[getattr(signal, k)] = k - # Realtime signals mostly have no names - if hasattr(signal, "SIGRTMIN") and hasattr(signal, "SIGRTMAX"): - for r in range(signal.SIGRTMIN+1, signal.SIGRTMAX+1): - _sigtbl[r] = "SIGRTMIN+" + str(r - signal.SIGRTMIN) - # Fill in any remaining gaps - for i in range(signal.NSIG): - if _sigtbl[i] is None: - _sigtbl[i] = "unrecognized signal, number " + str(i) - if n < 0 or n >= signal.NSIG: - return "out-of-range signal, number "+str(n) - return _sigtbl[n] - -def printstatus(status, name = ""): - # 'status' is the exit status - if os.name != 'posix': - # Windows error codes are easier to look up if printed in hexadecimal - if status < 0: - status += 2**32 - print "TEST-INFO | %s: exit status %x\n" % (name, status) - elif os.WIFEXITED(status): - print "TEST-INFO | %s: exit %d\n" % (name, os.WEXITSTATUS(status)) - elif os.WIFSIGNALED(status): - # The python stdlib doesn't appear to have strsignal(), alas - print "TEST-INFO | {}: killed by {}".format(name,strsig(os.WTERMSIG(status))) - else: - # This is probably a can't-happen condition on Unix, but let's be defensive - print "TEST-INFO | %s: undecodable exit status %04x\n" % (name, status) - -def dumpScreen(utilityPath): - """dumps a screenshot of the entire screen to a directory specified by - the MOZ_UPLOAD_DIR environment variable""" - - # Need to figure out which OS-dependent tool to use - if mozinfo.isUnix: - utility = [os.path.join(utilityPath, "screentopng")] - utilityname = "screentopng" - elif mozinfo.isMac: - utility = ['/usr/sbin/screencapture', '-C', '-x', '-t', 'png'] - utilityname = "screencapture" - elif mozinfo.isWin: - utility = [os.path.join(utilityPath, "screenshot.exe")] - utilityname = "screenshot" - - # Get dir where to write the screenshot file - parent_dir = os.environ.get('MOZ_UPLOAD_DIR', None) - if not parent_dir: - log.info('Failed to retrieve MOZ_UPLOAD_DIR env var') - return - - # Run the capture - try: - tmpfd, imgfilename = tempfile.mkstemp(prefix='mozilla-test-fail-screenshot_', suffix='.png', dir=parent_dir) - os.close(tmpfd) - returncode = subprocess.call(utility + [imgfilename]) - printstatus(returncode, utilityname) - except OSError, err: - log.info("Failed to start %s for screenshot: %s" % - utility[0], err.strerror) - return diff --git a/build/dumpScreen.py b/build/dumpScreen.py deleted file mode 100644 index 70efe0189f..0000000000 --- a/build/dumpScreen.py +++ /dev/null @@ -1,30 +0,0 @@ -#!/usr/bin/env python - -""" -test dumpScreen functionality -""" - -import automationutils -import optparse -import os -import sys - - -def main(args=sys.argv[1:]): - - # parse CLI options - usage = '%prog [options] path/to/OBJDIR/dist/bin' - parser = optparse.OptionParser(usage=usage) - options, args = parser.parse_args(args) - if len(args) != 1: - parser.error("Please provide utility path") - utilityPath = args[0] - - # dump the screen to a data: URL - uri = automationutils.dumpScreen(utilityPath) - - # print the uri - print uri - -if __name__ == '__main__': - main() diff --git a/build/mach_bootstrap.py b/build/mach_bootstrap.py index 7ebac27330..9139365bf5 100644 --- a/build/mach_bootstrap.py +++ b/build/mach_bootstrap.py @@ -68,6 +68,7 @@ SEARCH_PATHS = [ 'testing/mozbase/mozrunner', 'testing/mozbase/mozsystemmonitor', 'testing/mozbase/mozinfo', + 'testing/mozbase/mozscreenshot', 'testing/mozbase/moztest', 'testing/mozbase/mozversion', 'testing/mozbase/manifestparser', diff --git a/build/win32/moz.build b/build/win32/moz.build index 0d14f247d2..8033e502e2 100644 --- a/build/win32/moz.build +++ b/build/win32/moz.build @@ -4,9 +4,6 @@ # License, v. 2.0. If a copy of the MPL was not distributed with this # file, You can obtain one at http://mozilla.org/MPL/2.0/. -if CONFIG['_MSC_VER'] and CONFIG['OS_TEST'] != 'x86_64': - TEST_DIRS += ['vmwarerecordinghelper'] - TEST_DIRS += ['crashinjectdll'] if CONFIG['ENABLE_TESTS']: diff --git a/build/win32/vmwarerecordinghelper/moz.build b/build/win32/vmwarerecordinghelper/moz.build deleted file mode 100644 index 61e40fe8bf..0000000000 --- a/build/win32/vmwarerecordinghelper/moz.build +++ /dev/null @@ -1,15 +0,0 @@ -# -*- Mode: python; c-basic-offset: 4; indent-tabs-mode: nil; tab-width: 40 -*- -# vim: set filetype=python: -# This Source Code Form is subject to the terms of the Mozilla Public -# License, v. 2.0. If a copy of the MPL was not distributed with this -# file, You can obtain one at http://mozilla.org/MPL/2.0/. - -SOURCES += [ - 'vmwarerecordinghelper.cpp', -] - -SharedLibrary('vmwarerecordinghelper') - -DEFFILE = '%s/%s.def' % (SRCDIR, LIBRARY_NAME) - -USE_STATIC_LIBS = True diff --git a/build/win32/vmwarerecordinghelper/vmwarerecordinghelper.cpp b/build/win32/vmwarerecordinghelper/vmwarerecordinghelper.cpp deleted file mode 100644 index b56bab2abf..0000000000 --- a/build/win32/vmwarerecordinghelper/vmwarerecordinghelper.cpp +++ /dev/null @@ -1,34 +0,0 @@ -/* This Source Code Form is subject to the terms of the Mozilla Public - * License, v. 2.0. If a copy of the MPL was not distributed with this - * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ - -/** - * The following code comes from "Starting and Stopping Recording of Virtual - * Machine Activity from Within the Guest": - * - * http://kb.vmware.com/selfservice/documentLink.do?externalID=1001401 - */ - -void __cdecl -StartRecording() -{ - __asm { - mov eax, 564d5868h - mov ebx, 1 - mov cx, 47 - mov dx, 5658h - in eax, dx - } -} - -void __cdecl -StopRecording() -{ - __asm { - mov eax, 564d5868h - mov ebx, 2 - mov cx, 47 - mov dx, 5658h - in eax, dx - } -} diff --git a/build/win32/vmwarerecordinghelper/vmwarerecordinghelper.def b/build/win32/vmwarerecordinghelper/vmwarerecordinghelper.def deleted file mode 100644 index c0d97f8a7f..0000000000 --- a/build/win32/vmwarerecordinghelper/vmwarerecordinghelper.def +++ /dev/null @@ -1,8 +0,0 @@ -;+# This Source Code Form is subject to the terms of the Mozilla Public -;+# License, v. 2.0. If a copy of the MPL was not distributed with this -;+# file, You can obtain one at http://mozilla.org/MPL/2.0/. - -LIBRARY vmwarerecordinghelper -EXPORTS - StartRecording - StopRecording diff --git a/docshell/base/timeline/AbstractTimelineMarker.cpp b/docshell/base/timeline/AbstractTimelineMarker.cpp index b4cc4f578b..aeeab82078 100644 --- a/docshell/base/timeline/AbstractTimelineMarker.cpp +++ b/docshell/base/timeline/AbstractTimelineMarker.cpp @@ -5,7 +5,10 @@ * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ #include "AbstractTimelineMarker.h" + #include "mozilla/TimeStamp.h" +#include "MainThreadUtils.h" +#include "nsAppRunner.h" namespace mozilla { @@ -13,6 +16,8 @@ AbstractTimelineMarker::AbstractTimelineMarker(const char* aName, MarkerTracingType aTracingType) : mName(aName) , mTracingType(aTracingType) + , mProcessType(XRE_GetProcessType()) + , mIsOffMainThread(!NS_IsMainThread()) { MOZ_COUNT_CTOR(AbstractTimelineMarker); SetCurrentTime(); @@ -23,6 +28,8 @@ AbstractTimelineMarker::AbstractTimelineMarker(const char* aName, MarkerTracingType aTracingType) : mName(aName) , mTracingType(aTracingType) + , mProcessType(XRE_GetProcessType()) + , mIsOffMainThread(!NS_IsMainThread()) { MOZ_COUNT_CTOR(AbstractTimelineMarker); SetCustomTime(aTime); @@ -68,4 +75,16 @@ AbstractTimelineMarker::SetCustomTime(DOMHighResTimeStamp aTime) mTime = aTime; } +void +AbstractTimelineMarker::SetProcessType(GeckoProcessType aProcessType) +{ + mProcessType = aProcessType; +} + +void +AbstractTimelineMarker::SetOffMainThread(bool aIsOffMainThread) +{ + mIsOffMainThread = aIsOffMainThread; +} + } // namespace mozilla diff --git a/docshell/base/timeline/AbstractTimelineMarker.h b/docshell/base/timeline/AbstractTimelineMarker.h index e6a1bca996..16c953c52f 100644 --- a/docshell/base/timeline/AbstractTimelineMarker.h +++ b/docshell/base/timeline/AbstractTimelineMarker.h @@ -9,6 +9,7 @@ #include "TimelineMarkerEnums.h" // for MarkerTracingType #include "nsDOMNavigationTiming.h" // for DOMHighResTimeStamp +#include "nsXULAppAPI.h" // for GeckoProcessType #include "mozilla/UniquePtr.h" struct JSContext; @@ -48,15 +49,23 @@ public: DOMHighResTimeStamp GetTime() const { return mTime; } MarkerTracingType GetTracingType() const { return mTracingType; } + const uint8_t GetProcessType() const { return mProcessType; }; + const bool IsOffMainThread() const { return mIsOffMainThread; }; + private: const char* mName; DOMHighResTimeStamp mTime; MarkerTracingType mTracingType; + uint8_t mProcessType; // @see `enum GeckoProcessType`. + bool mIsOffMainThread; + protected: void SetCurrentTime(); void SetCustomTime(const TimeStamp& aTime); void SetCustomTime(DOMHighResTimeStamp aTime); + void SetProcessType(GeckoProcessType aProcessType); + void SetOffMainThread(bool aIsOffMainThread); }; } // namespace mozilla diff --git a/docshell/base/timeline/CompositeTimelineMarker.h b/docshell/base/timeline/CompositeTimelineMarker.h new file mode 100644 index 0000000000..571f5d7a30 --- /dev/null +++ b/docshell/base/timeline/CompositeTimelineMarker.h @@ -0,0 +1,33 @@ +/* -*- Mode: C++; tab-width: 8; indent-tabs-mode: nil; c-basic-offset: 2 -*- */ +/* vim: set ts=8 sts=2 et sw=2 tw=80: */ +/* This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ + +#ifndef mozilla_CompositeTimelineMarker_h_ +#define mozilla_CompositeTimelineMarker_h_ + +#include "TimelineMarker.h" +#include "mozilla/dom/ProfileTimelineMarkerBinding.h" + +namespace mozilla { + +class CompositeTimelineMarker : public TimelineMarker +{ +public: + explicit CompositeTimelineMarker(const TimeStamp& aTime, + MarkerTracingType aTracingType) + : TimelineMarker("Composite", aTime, aTracingType) + { + // Even though these markers end up being created on the main thread in the + // content or chrome processes, they actually trace down code in the + // compositor parent process. All the information for creating these markers + // is sent along via IPC to an nsView when a composite finishes. + // Mark this as 'off the main thread' to style it differently in frontends. + SetOffMainThread(true); + } +}; + +} // namespace mozilla + +#endif // mozilla_CompositeTimelineMarker_h_ diff --git a/docshell/base/timeline/ConsoleTimelineMarker.h b/docshell/base/timeline/ConsoleTimelineMarker.h index ecf3254560..3903649495 100644 --- a/docshell/base/timeline/ConsoleTimelineMarker.h +++ b/docshell/base/timeline/ConsoleTimelineMarker.h @@ -40,6 +40,8 @@ public: virtual void AddDetails(JSContext* aCx, dom::ProfileTimelineMarker& aMarker) override { + TimelineMarker::AddDetails(aCx, aMarker); + if (GetTracingType() == MarkerTracingType::START) { aMarker.mCauseName.Construct(mCause); } else { diff --git a/docshell/base/timeline/EventTimelineMarker.h b/docshell/base/timeline/EventTimelineMarker.h index 2779f2037a..74dfb5e5c0 100644 --- a/docshell/base/timeline/EventTimelineMarker.h +++ b/docshell/base/timeline/EventTimelineMarker.h @@ -25,6 +25,8 @@ public: virtual void AddDetails(JSContext* aCx, dom::ProfileTimelineMarker& aMarker) override { + TimelineMarker::AddDetails(aCx, aMarker); + if (GetTracingType() == MarkerTracingType::START) { aMarker.mType.Construct(mType); aMarker.mEventPhase.Construct(mPhase); diff --git a/docshell/base/timeline/JavascriptTimelineMarker.h b/docshell/base/timeline/JavascriptTimelineMarker.h index 180a0e7926..124dc82612 100644 --- a/docshell/base/timeline/JavascriptTimelineMarker.h +++ b/docshell/base/timeline/JavascriptTimelineMarker.h @@ -39,6 +39,8 @@ public: virtual void AddDetails(JSContext* aCx, dom::ProfileTimelineMarker& aMarker) override { + TimelineMarker::AddDetails(aCx, aMarker); + aMarker.mCauseName.Construct(mCause); if (!mFunctionName.IsEmpty() || !mFileName.IsEmpty()) { diff --git a/docshell/base/timeline/RestyleTimelineMarker.h b/docshell/base/timeline/RestyleTimelineMarker.h index 87207f9251..b5d5a02ef2 100644 --- a/docshell/base/timeline/RestyleTimelineMarker.h +++ b/docshell/base/timeline/RestyleTimelineMarker.h @@ -26,6 +26,8 @@ public: virtual void AddDetails(JSContext* aCx, dom::ProfileTimelineMarker& aMarker) override { + TimelineMarker::AddDetails(aCx, aMarker); + if (GetTracingType() == MarkerTracingType::START) { aMarker.mRestyleHint.Construct(mRestyleHint); } diff --git a/docshell/base/timeline/TimelineMarker.cpp b/docshell/base/timeline/TimelineMarker.cpp index bf5dd79933..b83e9ceb42 100644 --- a/docshell/base/timeline/TimelineMarker.cpp +++ b/docshell/base/timeline/TimelineMarker.cpp @@ -28,7 +28,10 @@ TimelineMarker::TimelineMarker(const char* aName, void TimelineMarker::AddDetails(JSContext* aCx, dom::ProfileTimelineMarker& aMarker) { - // Nothing to do here for plain markers. + if (GetTracingType() == MarkerTracingType::START) { + aMarker.mProcessType.Construct(GetProcessType()); + aMarker.mIsOffMainThread.Construct(IsOffMainThread()); + } } JSObject* diff --git a/docshell/base/timeline/TimestampTimelineMarker.h b/docshell/base/timeline/TimestampTimelineMarker.h index 73ed91800a..4e588b5769 100644 --- a/docshell/base/timeline/TimestampTimelineMarker.h +++ b/docshell/base/timeline/TimestampTimelineMarker.h @@ -22,6 +22,8 @@ public: virtual void AddDetails(JSContext* aCx, dom::ProfileTimelineMarker& aMarker) override { + TimelineMarker::AddDetails(aCx, aMarker); + if (!mCause.IsEmpty()) { aMarker.mCauseName.Construct(mCause); } diff --git a/docshell/base/timeline/WorkerTimelineMarker.h b/docshell/base/timeline/WorkerTimelineMarker.h index f6ef9e361c..2b2b695003 100644 --- a/docshell/base/timeline/WorkerTimelineMarker.h +++ b/docshell/base/timeline/WorkerTimelineMarker.h @@ -30,6 +30,8 @@ public: virtual void AddDetails(JSContext* aCx, dom::ProfileTimelineMarker& aMarker) override { + TimelineMarker::AddDetails(aCx, aMarker); + if (GetTracingType() == MarkerTracingType::START) { aMarker.mWorkerOperation.Construct(mOperationType); } diff --git a/docshell/base/timeline/moz.build b/docshell/base/timeline/moz.build index 58e9863622..c88e24021f 100644 --- a/docshell/base/timeline/moz.build +++ b/docshell/base/timeline/moz.build @@ -8,6 +8,7 @@ EXPORTS.mozilla += [ 'AbstractTimelineMarker.h', 'AutoGlobalTimelineMarker.h', 'AutoTimelineMarker.h', + 'CompositeTimelineMarker.h', 'ConsoleTimelineMarker.h', 'EventTimelineMarker.h', 'JavascriptTimelineMarker.h', diff --git a/docshell/test/chrome/chrome.ini b/docshell/test/chrome/chrome.ini index 3adcb5eba0..4b82630efa 100644 --- a/docshell/test/chrome/chrome.ini +++ b/docshell/test/chrome/chrome.ini @@ -1,5 +1,5 @@ [DEFAULT] -skip-if = buildapp == 'b2g' +skip-if = buildapp == 'b2g' || os == 'android' support-files = 662200a.html 662200b.html diff --git a/dom/bluetooth/bluedroid/BluetoothDaemonHelpers.cpp b/dom/bluetooth/bluedroid/BluetoothDaemonHelpers.cpp index 76f9e01aa7..73ed67badf 100644 --- a/dom/bluetooth/bluedroid/BluetoothDaemonHelpers.cpp +++ b/dom/bluetooth/bluedroid/BluetoothDaemonHelpers.cpp @@ -993,12 +993,6 @@ Convert(const ConvertArray& aIn, Tout& aOut) // Packing // -nsresult -PackPDU(bool aIn, DaemonSocketPDU& aPDU) -{ - return PackPDU(PackConversion(aIn), aPDU); -} - nsresult PackPDU(const BluetoothAddress& aIn, DaemonSocketPDU& aPDU) { @@ -1366,18 +1360,6 @@ PackPDU(BluetoothGattWriteType aIn, DaemonSocketPDU& aPDU) // Unpacking // -nsresult -UnpackPDU(DaemonSocketPDU& aPDU, bool& aOut) -{ - return UnpackPDU(aPDU, UnpackConversion(aOut)); -} - -nsresult -UnpackPDU(DaemonSocketPDU& aPDU, char& aOut) -{ - return UnpackPDU(aPDU, UnpackConversion(aOut)); -} - nsresult UnpackPDU(DaemonSocketPDU& aPDU, BluetoothA2dpAudioState& aOut) { diff --git a/dom/bluetooth/bluedroid/BluetoothDaemonHelpers.h b/dom/bluetooth/bluedroid/BluetoothDaemonHelpers.h index ef3c7f6dbe..53fb9431e9 100644 --- a/dom/bluetooth/bluedroid/BluetoothDaemonHelpers.h +++ b/dom/bluetooth/bluedroid/BluetoothDaemonHelpers.h @@ -270,9 +270,6 @@ Convert(nsresult aIn, BluetoothStatus& aOut); // Packing // -nsresult -PackPDU(bool aIn, DaemonSocketPDU& aPDU); - nsresult PackPDU(const BluetoothAddress& aIn, DaemonSocketPDU& aPDU); @@ -380,147 +377,11 @@ PackPDU(BluetoothGattWriteType aIn, DaemonSocketPDU& aPDU); nsresult PackPDU(BluetoothTransport aIn, DaemonSocketPDU& aPDU); -/* |PackConversion| is a helper for packing converted values. Pass - * an instance of this structure to |PackPDU| to convert a value from - * the input type to the output type and and write it to the PDU. - */ -template -struct PackConversion { - PackConversion(const Tin& aIn) - : mIn(aIn) - { } - - const Tin& mIn; -}; - -template -inline nsresult -PackPDU(const PackConversion& aIn, DaemonSocketPDU& aPDU) -{ - Tout out; - - nsresult rv = Convert(aIn.mIn, out); - if (NS_FAILED(rv)) { - return rv; - } - return PackPDU(out, aPDU); -} - -/* |PackArray| is a helper for packing arrays. Pass an instance - * of this structure as the first argument to |PackPDU| to pack - * an array. The array's maximum default length is 255 elements. - */ -template -struct PackArray -{ - PackArray(const T* aData, size_t aLength) - : mData(aData) - , mLength(aLength) - { } - - const T* mData; - size_t mLength; -}; - -/* This implementation of |PackPDU| packs the length of an array - * and the elements of the array one-by-one. - */ -template -inline nsresult -PackPDU(const PackArray& aIn, DaemonSocketPDU& aPDU) -{ - for (size_t i = 0; i < aIn.mLength; ++i) { - nsresult rv = PackPDU(aIn.mData[i], aPDU); - if (NS_FAILED(rv)) { - return rv; - } - } - return NS_OK; -} - -template<> -inline nsresult -PackPDU(const PackArray& aIn, DaemonSocketPDU& aPDU) -{ - /* Write raw bytes in one pass */ - return aPDU.Write(aIn.mData, aIn.mLength); -} - -template<> -inline nsresult -PackPDU(const PackArray& aIn, DaemonSocketPDU& aPDU) -{ - /* Write raw bytes in one pass */ - return aPDU.Write(aIn.mData, aIn.mLength); -} - -/* |PackCString0| is a helper for packing 0-terminated C string, - * including the \0 character. Pass an instance of this structure - * as the first argument to |PackPDU| to pack a string. - */ -struct PackCString0 -{ - PackCString0(const nsCString& aString) - : mString(aString) - { } - - const nsCString& mString; -}; - -/* This implementation of |PackPDU| packs a 0-terminated C string. - */ -inline nsresult -PackPDU(const PackCString0& aIn, DaemonSocketPDU& aPDU) -{ - return PackPDU( - PackArray(reinterpret_cast(aIn.mString.get()), - aIn.mString.Length() + 1), aPDU); -} - -/* |PackReversed| is a helper for packing data in reversed order. Pass an - * instance of this structure as the first argument to |PackPDU| to pack data - * in reversed order. - */ -template -struct PackReversed -{ - PackReversed(const T& aValue) - : mValue(aValue) - { } - - const T& mValue; -}; - -/* No general rules to pack data in reversed order. Signal a link error if the - * type |T| of |PackReversed| is not defined explicitly. - */ -template -nsresult -PackPDU(const PackReversed& aIn, DaemonSocketPDU& aPDU); - -/* This implementation of |PackPDU| packs elements in |PackArray| in reversed - * order. (ex. reversed GATT UUID, see bug 1171866) - */ -template -inline nsresult -PackPDU(const PackReversed>& aIn, DaemonSocketPDU& aPDU) -{ - for (size_t i = 0; i < aIn.mValue.mLength; ++i) { - nsresult rv = PackPDU(aIn.mValue.mData[aIn.mValue.mLength - i - 1], aPDU); - if (NS_FAILED(rv)) { - return rv; - } - } - return NS_OK; -} - /* This implementation of |PackPDU| packs |BluetoothUuid| in reversed order. * (ex. reversed GATT UUID, see bug 1171866) */ -template <> inline nsresult -PackPDU(const PackReversed& aIn, - DaemonSocketPDU& aPDU) +PackPDU(const PackReversed& aIn, DaemonSocketPDU& aPDU) { return PackPDU( PackReversed>( @@ -528,256 +389,10 @@ PackPDU(const PackReversed& aIn, aPDU); } -template -inline nsresult -PackPDU(const T1& aIn1, const T2& aIn2, DaemonSocketPDU& aPDU) -{ - nsresult rv = PackPDU(aIn1, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - return PackPDU(aIn2, aPDU); -} - -template -inline nsresult -PackPDU(const T1& aIn1, const T2& aIn2, const T3& aIn3, - DaemonSocketPDU& aPDU) -{ - nsresult rv = PackPDU(aIn1, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn2, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - return PackPDU(aIn3, aPDU); -} - -template -inline nsresult -PackPDU(const T1& aIn1, const T2& aIn2, const T3& aIn3, const T4& aIn4, - DaemonSocketPDU& aPDU) -{ - nsresult rv = PackPDU(aIn1, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn2, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn3, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - return PackPDU(aIn4, aPDU); -} - -template -inline nsresult -PackPDU(const T1& aIn1, const T2& aIn2, const T3& aIn3, - const T4& aIn4, const T5& aIn5, - DaemonSocketPDU& aPDU) -{ - nsresult rv = PackPDU(aIn1, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn2, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn3, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn4, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - return PackPDU(aIn5, aPDU); -} - -template -inline nsresult -PackPDU(const T1& aIn1, const T2& aIn2, const T3& aIn3, - const T4& aIn4, const T5& aIn5, const T6& aIn6, - DaemonSocketPDU& aPDU) -{ - nsresult rv = PackPDU(aIn1, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn2, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn3, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn4, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn5, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - return PackPDU(aIn6, aPDU); -} - -template -inline nsresult -PackPDU(const T1& aIn1, const T2& aIn2, const T3& aIn3, - const T4& aIn4, const T5& aIn5, const T6& aIn6, - const T7& aIn7, DaemonSocketPDU& aPDU) -{ - nsresult rv = PackPDU(aIn1, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn2, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn3, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn4, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn5, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn6, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - return PackPDU(aIn7, aPDU); -} - -template -inline nsresult -PackPDU(const T1& aIn1, const T2& aIn2, const T3& aIn3, - const T4& aIn4, const T5& aIn5, const T6& aIn6, - const T7& aIn7, const T8& aIn8, DaemonSocketPDU& aPDU) -{ - nsresult rv = PackPDU(aIn1, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn2, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn3, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn4, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn5, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn6, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn7, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - return PackPDU(aIn8, aPDU); -} - -template -inline nsresult -PackPDU(const T1& aIn1, const T2& aIn2, const T3& aIn3, - const T4& aIn4, const T5& aIn5, const T6& aIn6, - const T7& aIn7, const T8& aIn8, const T9& aIn9, - const T10& aIn10, const T11& aIn11, const T12& aIn12, - const T13& aIn13, DaemonSocketPDU& aPDU) -{ - nsresult rv = PackPDU(aIn1, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn2, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn3, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn4, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn5, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn6, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn7, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn8, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn9, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn10, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn11, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - rv = PackPDU(aIn12, aPDU); - if (NS_FAILED(rv)) { - return rv; - } - return PackPDU(aIn13, aPDU); -} - // // Unpacking // -nsresult -UnpackPDU(DaemonSocketPDU& aPDU, bool& aOut); - -nsresult -UnpackPDU(DaemonSocketPDU& aPDU, char& aOut); - nsresult UnpackPDU(DaemonSocketPDU& aPDU, BluetoothA2dpAudioState& aOut); @@ -887,156 +502,11 @@ UnpackPDU(DaemonSocketPDU& aPDU, BluetoothGattWriteParam& aOut); nsresult UnpackPDU(DaemonSocketPDU& aPDU, BluetoothGattNotifyParam& aOut); -/* |UnpackConversion| is a helper for convering unpacked values. Pass - * an instance of this structure to |UnpackPDU| to read a value from - * the PDU in the input type and convert it to the output type. - */ -template -struct UnpackConversion { - UnpackConversion(Tout& aOut) - : mOut(aOut) - { } - - Tout& mOut; -}; - -template -inline nsresult -UnpackPDU(DaemonSocketPDU& aPDU, const UnpackConversion& aOut) -{ - Tin in; - nsresult rv = UnpackPDU(aPDU, in); - if (NS_FAILED(rv)) { - return rv; - } - return Convert(in, aOut.mOut); -} - -/* |UnpackArray| is a helper for unpacking arrays. Pass an instance - * of this structure as the second argument to |UnpackPDU| to unpack - * an array. - */ -template -struct UnpackArray -{ - UnpackArray(T* aData, size_t aLength) - : mData(aData) - , mLength(aLength) - { } - - UnpackArray(nsAutoArrayPtr& aData, size_t aLength) - : mData(nullptr) - , mLength(aLength) - { - aData = new T[mLength]; - mData = aData.get(); - } - - UnpackArray(nsAutoArrayPtr& aData, size_t aSize, size_t aElemSize) - : mData(nullptr) - , mLength(aSize / aElemSize) - { - aData = new T[mLength]; - mData = aData.get(); - } - - T* mData; - size_t mLength; -}; - -template -inline nsresult -UnpackPDU(DaemonSocketPDU& aPDU, const UnpackArray& aOut) -{ - for (size_t i = 0; i < aOut.mLength; ++i) { - nsresult rv = UnpackPDU(aPDU, aOut.mData[i]); - if (NS_FAILED(rv)) { - return rv; - } - } - return NS_OK; -} - -template -inline nsresult -UnpackPDU(DaemonSocketPDU& aPDU, UnpackArray& aOut) -{ - for (size_t i = 0; i < aOut.mLength; ++i) { - nsresult rv = UnpackPDU(aPDU, aOut.mData[i]); - if (NS_FAILED(rv)) { - return rv; - } - } - return NS_OK; -} - -template<> -inline nsresult -UnpackPDU(DaemonSocketPDU& aPDU, const UnpackArray& aOut) -{ - /* Read raw bytes in one pass */ - return aPDU.Read(aOut.mData, aOut.mLength); -} - -template -inline nsresult -UnpackPDU(DaemonSocketPDU& aPDU, nsTArray& aOut) -{ - for (typename nsTArray::size_type i = 0; i < aOut.Length(); ++i) { - nsresult rv = UnpackPDU(aPDU, aOut[i]); - if (NS_FAILED(rv)) { - return rv; - } - } - return NS_OK; -} - -/* |UnpackReversed| is a helper for unpacking data in reversed order. Pass an - * instance of this structure as the second argument to |UnpackPDU| to unpack - * data in reversed order. - */ -template -struct UnpackReversed -{ - UnpackReversed(T& aValue) - : mValue(&aValue) - { } - - UnpackReversed(T&& aValue) - : mValue(&aValue) - { } - - T* mValue; -}; - -/* No general rules to unpack data in reversed order. Signal a link error if - * the type |T| of |UnpackReversed| is not defined explicitly. - */ -template -nsresult -UnpackPDU(DaemonSocketPDU& aPDU, const UnpackReversed& aOut); - -template -inline nsresult -UnpackPDU(DaemonSocketPDU& aPDU, const UnpackReversed>& aOut) -{ - for (size_t i = 0; i < aOut.mValue->mLength; ++i) { - nsresult rv = UnpackPDU(aPDU, - aOut.mValue->mData[aOut.mValue->mLength - i - 1]); - if (NS_FAILED(rv)) { - return rv; - } - } - return NS_OK; -} - /* This implementation of |UnpackPDU| unpacks |BluetoothUuid| in reversed * order. (ex. reversed GATT UUID, see bug 1171866) */ -template<> inline nsresult -UnpackPDU(DaemonSocketPDU& aPDU, - const UnpackReversed& aOut) +UnpackPDU(DaemonSocketPDU& aPDU, const UnpackReversed& aOut) { return UnpackPDU( aPDU, @@ -1044,132 +514,6 @@ UnpackPDU(DaemonSocketPDU& aPDU, UnpackArray(aOut.mValue->mUuid, sizeof(aOut.mValue->mUuid)))); } -// -// Init operators -// - -// |UnpackPDUInitOp| is a general-purpose init operator for all variants -// of |BluetoothResultRunnable| and |BluetoothNotificationRunnable|. The -// call operators of |UnpackPDUInitOp| unpack a PDU into the supplied -// arguments. -class UnpackPDUInitOp final : private PDUInitOp -{ -public: - UnpackPDUInitOp(DaemonSocketPDU& aPDU) - : PDUInitOp(aPDU) - { } - - nsresult operator () () const - { - WarnAboutTrailingData(); - return NS_OK; - } - - template - nsresult operator () (T1& aArg1) const - { - nsresult rv = UnpackPDU(GetPDU(), aArg1); - if (NS_FAILED(rv)) { - return rv; - } - WarnAboutTrailingData(); - return NS_OK; - } - - template - nsresult operator () (T1& aArg1, T2& aArg2) const - { - DaemonSocketPDU& pdu = GetPDU(); - - nsresult rv = UnpackPDU(pdu, aArg1); - if (NS_FAILED(rv)) { - return rv; - } - rv = UnpackPDU(pdu, aArg2); - if (NS_FAILED(rv)) { - return rv; - } - WarnAboutTrailingData(); - return NS_OK; - } - - template - nsresult operator () (T1& aArg1, T2& aArg2, T3& aArg3) const - { - DaemonSocketPDU& pdu = GetPDU(); - - nsresult rv = UnpackPDU(pdu, aArg1); - if (NS_FAILED(rv)) { - return rv; - } - rv = UnpackPDU(pdu, aArg2); - if (NS_FAILED(rv)) { - return rv; - } - rv = UnpackPDU(pdu, aArg3); - if (NS_FAILED(rv)) { - return rv; - } - WarnAboutTrailingData(); - return NS_OK; - } - - template - nsresult operator () (T1& aArg1, T2& aArg2, T3& aArg3, T4& aArg4) const - { - DaemonSocketPDU& pdu = GetPDU(); - - nsresult rv = UnpackPDU(pdu, aArg1); - if (NS_FAILED(rv)) { - return rv; - } - rv = UnpackPDU(pdu, aArg2); - if (NS_FAILED(rv)) { - return rv; - } - rv = UnpackPDU(pdu, aArg3); - if (NS_FAILED(rv)) { - return rv; - } - rv = UnpackPDU(pdu, aArg4); - if (NS_FAILED(rv)) { - return rv; - } - WarnAboutTrailingData(); - return NS_OK; - } - - template - nsresult operator () (T1& aArg1, T2& aArg2, T3& aArg3, T4& aArg4, - T5& aArg5) const - { - DaemonSocketPDU& pdu = GetPDU(); - - nsresult rv = UnpackPDU(pdu, aArg1); - if (NS_FAILED(rv)) { - return rv; - } - rv = UnpackPDU(pdu, aArg2); - if (NS_FAILED(rv)) { - return rv; - } - rv = UnpackPDU(pdu, aArg3); - if (NS_FAILED(rv)) { - return rv; - } - rv = UnpackPDU(pdu, aArg4); - if (NS_FAILED(rv)) { - return rv; - } - rv = UnpackPDU(pdu, aArg5); - if (NS_FAILED(rv)) { - return rv; - } - WarnAboutTrailingData(); - return NS_OK; - } -}; - END_BLUETOOTH_NAMESPACE #endif // mozilla_dom_bluetooth_bluedroid_BluetoothDaemonHelpers_h diff --git a/dom/canvas/crashtests/1161277-1.html b/dom/canvas/crashtests/1161277-1.html new file mode 100644 index 0000000000..3e04dc46da --- /dev/null +++ b/dom/canvas/crashtests/1161277-1.html @@ -0,0 +1,22 @@ + + + + + + + diff --git a/dom/canvas/crashtests/1190705.html b/dom/canvas/crashtests/1190705.html new file mode 100644 index 0000000000..64795decb6 --- /dev/null +++ b/dom/canvas/crashtests/1190705.html @@ -0,0 +1,17 @@ + + + + + + + diff --git a/dom/canvas/crashtests/crashtests.list b/dom/canvas/crashtests/crashtests.list index 5704451480..6feb578bbe 100644 --- a/dom/canvas/crashtests/crashtests.list +++ b/dom/canvas/crashtests/crashtests.list @@ -20,4 +20,7 @@ load 896047-2.html load 916128-1.html load 934939-1.html load 1099143-1.html +load 1161277-1.html load 1183363.html +load 1190705.html + diff --git a/dom/media/tests/mochitest/ipc/mochitest.ini b/dom/media/tests/mochitest/ipc/mochitest.ini index 5614de183d..dc1872a73d 100644 --- a/dom/media/tests/mochitest/ipc/mochitest.ini +++ b/dom/media/tests/mochitest/ipc/mochitest.ini @@ -6,4 +6,4 @@ support-files = skip-if = e10s [test_ipc.html] -skip-if = buildapp == 'b2g' || toolkit == 'android' #bug 910661 # b2g(nested ipc not working) b2g-debug(debug-only failure) b2g-desktop(nested ipc not working) +skip-if = buildapp == 'b2g' || toolkit == 'android' #bug 910661 # b2g(nested ipc not working) b2g-debug(debug-only failure) b2g-desktop(nested ipc not working) diff --git a/dom/media/tests/mochitest/mochitest.ini b/dom/media/tests/mochitest/mochitest.ini index 81790ba198..2033013715 100644 --- a/dom/media/tests/mochitest/mochitest.ini +++ b/dom/media/tests/mochitest/mochitest.ini @@ -99,7 +99,10 @@ skip-if = toolkit == 'gonk' || buildapp == 'mulet' # b2g(Bug 960442, video suppo [test_peerConnection_captureStream_canvas_webgl.html] skip-if = toolkit == 'gonk' || buildapp == 'mulet' # b2g(Bug 960442, video support for WebRTC is disabled on b2g) [test_peerConnection_close.html] +[test_peerConnection_closeDuringIce.html] [test_peerConnection_errorCallbacks.html] +[test_peerConnection_iceFailure.html] +skip-if = toolkit == 'gonk' || buildapp == 'mulet' || os == 'linux' || os == 'mac' || os == 'win' || android_version == '18' # Disabling because of test failures on B2G emulator (Bug 1180388 for win, mac and linux), android(Bug 1189784, timeouts on 4.3 emulator) [test_peerConnection_forwarding_basicAudioVideoCombined.html] skip-if = toolkit == 'gonk' || buildapp == 'mulet' # b2g(Bug 960442, video support for WebRTC is disabled on b2g) [test_peerConnection_noTrickleAnswer.html] diff --git a/dom/media/tests/mochitest/templates.js b/dom/media/tests/mochitest/templates.js index f3deaf7696..303e36f12c 100644 --- a/dom/media/tests/mochitest/templates.js +++ b/dom/media/tests/mochitest/templates.js @@ -405,7 +405,7 @@ var commandsPeerConnectionOfferAnswer = [ }, function PC_LOCAL_SET_REMOTE_DESCRIPTION(test) { - test.setRemoteDescription(test.pcLocal, test._remote_answer, STABLE) + return test.setRemoteDescription(test.pcLocal, test._remote_answer, STABLE) .then(() => { is(test.pcLocal.signalingState, STABLE, "signalingState after local setRemoteDescription is 'stable'"); diff --git a/dom/media/tests/mochitest/test_peerConnection_closeDuringIce.html b/dom/media/tests/mochitest/test_peerConnection_closeDuringIce.html new file mode 100644 index 0000000000..00b793dd3d --- /dev/null +++ b/dom/media/tests/mochitest/test_peerConnection_closeDuringIce.html @@ -0,0 +1,79 @@ + + + + + + +
+
+
+ + diff --git a/dom/media/tests/mochitest/test_peerConnection_iceFailure.html b/dom/media/tests/mochitest/test_peerConnection_iceFailure.html new file mode 100644 index 0000000000..115ea023c7 --- /dev/null +++ b/dom/media/tests/mochitest/test_peerConnection_iceFailure.html @@ -0,0 +1,83 @@ + + + + + + +
+
+
+ + diff --git a/dom/tests/mochitest/pointerlock/mochitest.ini b/dom/tests/mochitest/pointerlock/mochitest.ini index 0369312897..d6c5920baa 100644 --- a/dom/tests/mochitest/pointerlock/mochitest.ini +++ b/dom/tests/mochitest/pointerlock/mochitest.ini @@ -25,4 +25,4 @@ support-files = iframe_differentDOM.html [test_pointerlock-api.html] -skip-if = toolkit == "windows" || buildapp == 'b2g' || toolkit == 'android' || e10s # B2G - window.open focus issues using fullscreen, Windows - bug 919106 & bug 931445 +skip-if = buildapp == 'b2g' || toolkit == 'android' || e10s || os == 'linux' || os == 'win' # B2G - window.open focus issues using fullscreen. (For Linux & Win) Bug1180351 diff --git a/dom/webidl/ProfileTimelineMarker.webidl b/dom/webidl/ProfileTimelineMarker.webidl index f6fba5cee5..7158736fe6 100644 --- a/dom/webidl/ProfileTimelineMarker.webidl +++ b/dom/webidl/ProfileTimelineMarker.webidl @@ -38,6 +38,9 @@ dictionary ProfileTimelineMarker { DOMHighResTimeStamp end = 0; object? stack = null; + unsigned short processType; + boolean isOffMainThread; + /* For ConsoleTime, Timestamp and Javascript markers. */ DOMString causeName; diff --git a/extensions/cookie/test/channel_utils.js b/extensions/cookie/test/channel_utils.js index 4f97e4df94..595abee8b9 100644 --- a/extensions/cookie/test/channel_utils.js +++ b/extensions/cookie/test/channel_utils.js @@ -171,6 +171,10 @@ function LoadContextCallback(appId, inBrowserElement, isPrivate, isContent) { this.isInBrowserElement = inBrowserElement; this.usePrivateBrowsing = isPrivate; this.isContent = isContent; + this.originAttributes = { + appId: appId, + inBrowser: inBrowserElement + }; } LoadContextCallback.prototype = { diff --git a/gfx/2d/2D.h b/gfx/2d/2D.h index 83e1dd38c7..3518efdb62 100644 --- a/gfx/2d/2D.h +++ b/gfx/2d/2D.h @@ -1268,7 +1268,7 @@ public: static bool DoesBackendSupportDataDrawtarget(BackendType aType); -#ifdef XP_MACOSX +#ifdef XP_DARWIN static already_AddRefed CreateDrawTargetForCairoCGContext(CGContextRef cg, const IntSize& aSize); static already_AddRefed CreateCGGlyphRenderingOptions(const Color &aFontSmoothingBackgroundColor); diff --git a/gfx/2d/BorrowedContext.h b/gfx/2d/BorrowedContext.h index 477ec9d12d..0b5612ad8e 100644 --- a/gfx/2d/BorrowedContext.h +++ b/gfx/2d/BorrowedContext.h @@ -137,7 +137,7 @@ private: }; #endif -#ifdef XP_MACOSX +#ifdef XP_DARWIN /* This is a helper class that let's you borrow a CGContextRef from a * DrawTargetCG. This is used for drawing themed widgets. * diff --git a/gfx/2d/DrawTargetCG.cpp b/gfx/2d/DrawTargetCG.cpp index d315400107..a45d63f5f4 100644 --- a/gfx/2d/DrawTargetCG.cpp +++ b/gfx/2d/DrawTargetCG.cpp @@ -3,6 +3,7 @@ * License, v. 2.0. If a copy of the MPL was not distributed with this * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ +#include #include "BorrowedContext.h" #include "DataSurfaceHelpers.h" #include "DrawTargetCG.h" @@ -177,6 +178,7 @@ DrawTargetCG::GetType() const BackendType DrawTargetCG::GetBackendType() const { +#ifdef MOZ_WIDGET_COCOA // It may be worth spliting Bitmap and IOSurface DrawTarget // into seperate classes. if (GetContextType(mCg) == CG_CONTEXT_TYPE_IOSURFACE) { @@ -184,15 +186,20 @@ DrawTargetCG::GetBackendType() const } else { return BackendType::COREGRAPHICS; } +#else + return BackendType::COREGRAPHICS; +#endif } already_AddRefed DrawTargetCG::Snapshot() { if (!mSnapshot) { +#ifdef MOZ_WIDGET_COCOA if (GetContextType(mCg) == CG_CONTEXT_TYPE_IOSURFACE) { return MakeAndAddRef(this); } +#endif Flush(); mSnapshot = new SourceSurfaceCGBitmapContext(this); } @@ -1717,12 +1724,14 @@ DrawTargetCG::Init(BackendType aType, mSize = aSize; +#ifdef MOZ_WIDGET_COCOA if (aType == BackendType::COREGRAPHICS_ACCELERATED) { RefPtr ioSurface = MacIOSurface::CreateIOSurface(aSize.width, aSize.height); mCg = ioSurface->CreateIOSurfaceContext(); // If we don't have the symbol for 'CreateIOSurfaceContext' mCg will be null // and we will fallback to software below } +#endif mFormat = SurfaceFormat::B8G8R8A8; @@ -1820,6 +1829,7 @@ EnsureValidPremultipliedData(CGContextRef aContext) void DrawTargetCG::Flush() { +#ifdef MOZ_WIDGET_COCOA if (GetContextType(mCg) == CG_CONTEXT_TYPE_IOSURFACE) { CGContextFlush(mCg); } else if (GetContextType(mCg) == CG_CONTEXT_TYPE_BITMAP && @@ -1835,6 +1845,9 @@ DrawTargetCG::Flush() EnsureValidPremultipliedData(mCg); mMayContainInvalidPremultipliedData = false; } +#else + //TODO +#endif } bool @@ -1874,7 +1887,9 @@ DrawTargetCG::Init(CGContextRef cgContext, const IntSize &aSize) mOriginalTransform = CGContextGetCTM(mCg); mFormat = SurfaceFormat::B8G8R8A8; +#ifdef MOZ_WIDGET_COCOA if (GetContextType(mCg) == CG_CONTEXT_TYPE_BITMAP) { +#endif CGColorSpaceRef colorspace; CGBitmapInfo bitinfo = CGBitmapContextGetBitmapInfo(mCg); colorspace = CGBitmapContextGetColorSpace (mCg); @@ -1883,7 +1898,9 @@ DrawTargetCG::Init(CGContextRef cgContext, const IntSize &aSize) } else if ((bitinfo & kCGBitmapAlphaInfoMask) == kCGImageAlphaNoneSkipFirst) { mFormat = SurfaceFormat::B8G8R8X8; } +#ifdef MOZ_WIDGET_COCOA } +#endif return true; } @@ -1906,12 +1923,16 @@ DrawTargetCG::CreatePathBuilder(FillRule aFillRule) const void* DrawTargetCG::GetNativeSurface(NativeSurfaceType aType) { +#ifdef MOZ_WIDGET_COCOA if ((aType == NativeSurfaceType::CGCONTEXT && GetContextType(mCg) == CG_CONTEXT_TYPE_BITMAP) || (aType == NativeSurfaceType::CGCONTEXT_ACCELERATED && GetContextType(mCg) == CG_CONTEXT_TYPE_IOSURFACE)) { return mCg; } else { return nullptr; } +#else + return mCg; +#endif } void diff --git a/gfx/2d/DrawTargetCG.h b/gfx/2d/DrawTargetCG.h index 30e901bd6b..50c1381800 100644 --- a/gfx/2d/DrawTargetCG.h +++ b/gfx/2d/DrawTargetCG.h @@ -6,7 +6,14 @@ #ifndef mozilla_gfx_DrawTargetCG_h #define mozilla_gfx_DrawTargetCG_h +#ifdef MOZ_WIDGET_COCOA #include +#import +#else +#include +#include +#include +#endif #include "2D.h" #include "Rect.h" diff --git a/gfx/2d/DrawTargetCairo.cpp b/gfx/2d/DrawTargetCairo.cpp index 0b358703fd..910d90a5b5 100644 --- a/gfx/2d/DrawTargetCairo.cpp +++ b/gfx/2d/DrawTargetCairo.cpp @@ -24,8 +24,10 @@ #ifdef CAIRO_HAS_QUARTZ_SURFACE #include "cairo-quartz.h" +#ifdef MOZ_WIDGET_COCOA #include #endif +#endif #ifdef CAIRO_HAS_XLIB_SURFACE #include "cairo-xlib.h" diff --git a/gfx/2d/Factory.cpp b/gfx/2d/Factory.cpp index bbca183afb..535ffff5e3 100644 --- a/gfx/2d/Factory.cpp +++ b/gfx/2d/Factory.cpp @@ -23,12 +23,12 @@ #include "ScaledFontWin.h" #endif -#ifdef XP_MACOSX +#ifdef XP_DARWIN #include "ScaledFontMac.h" #endif -#ifdef XP_MACOSX +#ifdef XP_DARWIN #include "DrawTargetCG.h" #endif @@ -347,7 +347,7 @@ Factory::CreateDrawTarget(BackendType aBackend, const IntSize &aSize, SurfaceFor } break; } -#elif defined XP_MACOSX +#elif defined XP_DARWIN case BackendType::COREGRAPHICS: case BackendType::COREGRAPHICS_ACCELERATED: { @@ -430,7 +430,7 @@ Factory::CreateDrawTargetForData(BackendType aBackend, break; } #endif -#ifdef XP_MACOSX +#ifdef XP_DARWIN case BackendType::COREGRAPHICS: { RefPtr newTarget = new DrawTargetCG(); @@ -537,7 +537,7 @@ Factory::CreateScaledFontForNativeFont(const NativeFont &aNativeFont, Float aSiz } #endif #endif -#ifdef XP_MACOSX +#ifdef XP_DARWIN case NativeFontType::MAC_FONT_FACE: { return MakeAndAddRef(static_cast(aNativeFont.mFont), aSize); @@ -839,7 +839,7 @@ Factory::CreateDrawTargetForCairoSurface(cairo_surface_t* aSurface, const IntSiz return retVal.forget(); } -#ifdef XP_MACOSX +#ifdef XP_DARWIN already_AddRefed Factory::CreateDrawTargetForCairoCGContext(CGContextRef cg, const IntSize& aSize) { diff --git a/gfx/2d/MacIOSurface.h b/gfx/2d/MacIOSurface.h index dc2d773e50..737affceab 100644 --- a/gfx/2d/MacIOSurface.h +++ b/gfx/2d/MacIOSurface.h @@ -6,10 +6,23 @@ #ifndef MacIOSurface_h__ #define MacIOSurface_h__ -#ifdef XP_MACOSX +#ifdef XP_DARWIN #include +#include #include +struct _CGLContextObject; + +typedef _CGLContextObject* CGLContextObj; +typedef struct CGContext* CGContextRef; +typedef struct CGImage* CGImageRef; +typedef uint32_t IOSurfaceID; + +#ifdef XP_IOS +typedef kern_return_t IOReturn; +typedef int CGLError; +#endif + typedef CFTypeRef IOSurfacePtr; typedef IOSurfacePtr (*IOSurfaceCreateFunc) (CFDictionaryRef properties); typedef IOSurfacePtr (*IOSurfaceLookupFunc) (uint32_t io_surface_id); @@ -42,18 +55,16 @@ typedef IOSurfacePtr (*CVPixelBufferGetIOSurfaceFunc)( typedef OSType (*IOSurfacePixelFormatFunc)(IOSurfacePtr io_surface); +#ifdef XP_MACOSX #import +#else +#import +#endif + #include "2D.h" #include "mozilla/RefPtr.h" #include "mozilla/RefCounted.h" -struct _CGLContextObject; - -typedef _CGLContextObject* CGLContextObj; -typedef struct CGContext* CGContextRef; -typedef struct CGImage* CGImageRef; -typedef uint32_t IOSurfaceID; - enum CGContextType { CG_CONTEXT_TYPE_UNKNOWN = 0, // These are found by inspection, it's possible they could be changed diff --git a/gfx/2d/Matrix.h b/gfx/2d/Matrix.h index 2ecc186fe2..b57fc97234 100644 --- a/gfx/2d/Matrix.h +++ b/gfx/2d/Matrix.h @@ -265,6 +265,13 @@ public: return !(*this == other); } + bool ExactlyEquals(const Matrix& o) const + { + return _11 == o._11 && _12 == o._12 && + _21 == o._21 && _22 == o._22 && + _31 == o._31 && _32 == o._32; + } + /* Verifies that the matrix contains no Infs or NaNs. */ bool IsFinite() const { diff --git a/gfx/2d/PathCG.h b/gfx/2d/PathCG.h index e627ab706a..0c8e571e2e 100644 --- a/gfx/2d/PathCG.h +++ b/gfx/2d/PathCG.h @@ -6,7 +6,12 @@ #ifndef MOZILLA_GFX_PATHCG_H_ #define MOZILLA_GFX_PATHCG_H_ +#ifdef MOZ_WIDGET_COCOA #include +#else +#include +#endif + #include "2D.h" namespace mozilla { diff --git a/gfx/2d/PathCairo.cpp b/gfx/2d/PathCairo.cpp index 2d78a27711..9c7ba58746 100644 --- a/gfx/2d/PathCairo.cpp +++ b/gfx/2d/PathCairo.cpp @@ -188,7 +188,7 @@ PathCairo::ContainsPoint(const Point &aPoint, const Matrix &aTransform) const inverse.Invert(); Point transformed = inverse * aPoint; - EnsureContainingContext(); + EnsureContainingContext(aTransform); return cairo_in_fill(mContainingContext, transformed.x, transformed.y); } @@ -202,7 +202,7 @@ PathCairo::StrokeContainsPoint(const StrokeOptions &aStrokeOptions, inverse.Invert(); Point transformed = inverse * aPoint; - EnsureContainingContext(); + EnsureContainingContext(aTransform); SetCairoStrokeOptions(mContainingContext, aStrokeOptions); @@ -212,7 +212,7 @@ PathCairo::StrokeContainsPoint(const StrokeOptions &aStrokeOptions, Rect PathCairo::GetBounds(const Matrix &aTransform) const { - EnsureContainingContext(); + EnsureContainingContext(aTransform); double x1, y1, x2, y2; @@ -225,7 +225,7 @@ Rect PathCairo::GetStrokedBounds(const StrokeOptions &aStrokeOptions, const Matrix &aTransform) const { - EnsureContainingContext(); + EnsureContainingContext(aTransform); double x1, y1, x2, y2; @@ -266,13 +266,21 @@ PathCairo::StreamToSink(PathSink *aSink) const } void -PathCairo::EnsureContainingContext() const +PathCairo::EnsureContainingContext(const Matrix &aTransform) const { if (mContainingContext) { - return; + if (mContainingTransform.ExactlyEquals(aTransform)) { + return; + } + } else { + mContainingContext = cairo_create(DrawTargetCairo::GetDummySurface()); } - mContainingContext = cairo_create(DrawTargetCairo::GetDummySurface()); + mContainingTransform = aTransform; + + cairo_matrix_t mat; + GfxMatrixToCairoMatrix(mContainingTransform, mat); + cairo_set_matrix(mContainingContext, &mat); SetPathOnContext(mContainingContext); } diff --git a/gfx/2d/PathCairo.h b/gfx/2d/PathCairo.h index 8ee76ad95b..70d8fbd16d 100644 --- a/gfx/2d/PathCairo.h +++ b/gfx/2d/PathCairo.h @@ -80,11 +80,12 @@ public: void AppendPathToBuilder(PathBuilderCairo *aBuilder, const Matrix *aTransform = nullptr) const; private: - void EnsureContainingContext() const; + void EnsureContainingContext(const Matrix &aTransform) const; FillRule mFillRule; std::vector mPathData; mutable cairo_t *mContainingContext; + mutable Matrix mContainingTransform; Point mCurrentPoint; }; diff --git a/gfx/2d/PathHelpers.h b/gfx/2d/PathHelpers.h index e51cdacf3b..b7636bf6e8 100644 --- a/gfx/2d/PathHelpers.h +++ b/gfx/2d/PathHelpers.h @@ -401,7 +401,7 @@ inline bool UserToDevicePixelSnapped(Rect& aRect, const DrawTarget& aDrawTarget, * This function has the same behavior as UserToDevicePixelSnapped except that * aRect is not transformed to device space. */ -inline void MaybeSnapToDevicePixels(Rect& aRect, const DrawTarget& aDrawTarget, +inline bool MaybeSnapToDevicePixels(Rect& aRect, const DrawTarget& aDrawTarget, bool aAllowScaleOr90DegreeRotate = false, bool aAllowEmptySnaps = true) { @@ -412,7 +412,9 @@ inline void MaybeSnapToDevicePixels(Rect& aRect, const DrawTarget& aDrawTarget, Matrix mat = aDrawTarget.GetTransform(); mat.Invert(); aRect = mat.TransformBounds(aRect); + return true; } + return false; } } // namespace gfx diff --git a/gfx/2d/PathSkia.cpp b/gfx/2d/PathSkia.cpp index b9fe0b3e50..dc3980bbf4 100644 --- a/gfx/2d/PathSkia.cpp +++ b/gfx/2d/PathSkia.cpp @@ -173,6 +173,10 @@ SkPathContainsPoint(const SkPath& aPath, const Point& aPoint, const Matrix& aTra bool PathSkia::ContainsPoint(const Point &aPoint, const Matrix &aTransform) const { + if (!mPath.isFinite()) { + return false; + } + return SkPathContainsPoint(mPath, aPoint, aTransform); } @@ -181,6 +185,10 @@ PathSkia::StrokeContainsPoint(const StrokeOptions &aStrokeOptions, const Point &aPoint, const Matrix &aTransform) const { + if (!mPath.isFinite()) { + return false; + } + SkPaint paint; StrokeOptionsToPaint(paint, aStrokeOptions); diff --git a/gfx/2d/ScaledFontMac.cpp b/gfx/2d/ScaledFontMac.cpp index 7c68b90d99..75180b913f 100644 --- a/gfx/2d/ScaledFontMac.cpp +++ b/gfx/2d/ScaledFontMac.cpp @@ -13,11 +13,16 @@ #include "DrawTargetCG.h" #include #include +#ifdef MOZ_WIDGET_UIKIT +#include +#endif +#ifdef MOZ_WIDGET_COCOA // prototype for private API extern "C" { CGPathRef CGFontGetGlyphPath(CGFontRef fontRef, CGAffineTransform *textTransform, int unknown, CGGlyph glyph); }; +#endif namespace mozilla { @@ -81,11 +86,12 @@ ScaledFontMac::GetPathForGlyphs(const GlyphBuffer &aBuffer, const DrawTarget *aT { if (aTarget->GetBackendType() == BackendType::COREGRAPHICS || aTarget->GetBackendType() == BackendType::COREGRAPHICS_ACCELERATED) { +#ifdef MOZ_WIDGET_COCOA CGMutablePathRef path = CGPathCreateMutable(); - for (unsigned int i = 0; i < aBuffer.mNumGlyphs; i++) { // XXX: we could probably fold both of these transforms together to avoid extra work CGAffineTransform flip = CGAffineTransformMakeScale(1, -1); + CGPathRef glyphPath = ::CGFontGetGlyphPath(mFont, &flip, 0, aBuffer.mGlyphs[i].mIndex); CGAffineTransform matrix = CGAffineTransformMake(mSize, 0, 0, mSize, @@ -97,6 +103,10 @@ ScaledFontMac::GetPathForGlyphs(const GlyphBuffer &aBuffer, const DrawTarget *aT RefPtr ret = new PathCG(path, FillRule::FILL_WINDING); CGPathRelease(path); return ret.forget(); +#else + //TODO: probably want CTFontCreatePathForGlyph + MOZ_CRASH("This needs implemented"); +#endif } return ScaledFontBase::GetPathForGlyphs(aBuffer, aTarget); } @@ -108,7 +118,7 @@ ScaledFontMac::CopyGlyphsToBuilder(const GlyphBuffer &aBuffer, PathBuilder *aBui ScaledFontBase::CopyGlyphsToBuilder(aBuffer, aBuilder, aBackendType, aTransformHint); return; } - +#ifdef MOZ_WIDGET_COCOA PathBuilderCG *pathBuilderCG = static_cast(aBuilder); // XXX: check builder type @@ -123,6 +133,10 @@ ScaledFontMac::CopyGlyphsToBuilder(const GlyphBuffer &aBuffer, PathBuilder *aBui CGPathAddPath(pathBuilderCG->mCGPath, &matrix, glyphPath); CGPathRelease(glyphPath); } +#else + //TODO: probably want CTFontCreatePathForGlyph + MOZ_CRASH("This needs implemented"); +#endif } uint32_t diff --git a/gfx/2d/ScaledFontMac.h b/gfx/2d/ScaledFontMac.h index 7862a0a497..982b5f7c52 100644 --- a/gfx/2d/ScaledFontMac.h +++ b/gfx/2d/ScaledFontMac.h @@ -6,7 +6,13 @@ #ifndef MOZILLA_GFX_SCALEDFONTMAC_H_ #define MOZILLA_GFX_SCALEDFONTMAC_H_ -#import +#ifdef MOZ_WIDGET_COCOA +#include +#else +#include +#include +#endif + #include "2D.h" #include "ScaledFontBase.h" diff --git a/gfx/2d/SourceSurfaceCG.cpp b/gfx/2d/SourceSurfaceCG.cpp index b2627b4403..fdfde62d8c 100644 --- a/gfx/2d/SourceSurfaceCG.cpp +++ b/gfx/2d/SourceSurfaceCG.cpp @@ -9,7 +9,9 @@ #include "DataSurfaceHelpers.h" #include "mozilla/Types.h" // for decltype +#ifdef MOZ_WIDGET_COCOA #include "MacIOSurface.h" +#endif #include "Tools.h" namespace mozilla { @@ -384,6 +386,7 @@ SourceSurfaceCGBitmapContext::~SourceSurfaceCGBitmapContext() CGImageRelease(mImage); } +#ifdef MOZ_WIDGET_COCOA SourceSurfaceCGIOSurfaceContext::SourceSurfaceCGIOSurfaceContext(DrawTargetCG *aDrawTarget) { CGContextRef cg = (CGContextRef)aDrawTarget->GetNativeSurface(NativeSurfaceType::CGCONTEXT_ACCELERATED); @@ -452,6 +455,7 @@ SourceSurfaceCGIOSurfaceContext::GetData() { return (unsigned char*)mData; } +#endif } // namespace gfx } // namespace mozilla diff --git a/gfx/2d/SourceSurfaceCG.h b/gfx/2d/SourceSurfaceCG.h index 5d8655d503..e365c93e0a 100644 --- a/gfx/2d/SourceSurfaceCG.h +++ b/gfx/2d/SourceSurfaceCG.h @@ -6,11 +6,17 @@ #ifndef _MOZILLA_GFX_SOURCESURFACECG_H #define _MOZILLA_GFX_SOURCESURFACECG_H +#ifdef MOZ_WIDGET_COCOA #include +#else +#include +#endif #include "2D.h" +#ifdef MOZ_WIDGET_COCOA class MacIOSurface; +#endif namespace mozilla { namespace gfx { @@ -163,6 +169,7 @@ private: IntSize mSize; }; +#ifdef MOZ_WIDGET_COCOA class SourceSurfaceCGIOSurfaceContext : public SourceSurfaceCGContext { public: @@ -196,6 +203,7 @@ private: IntSize mSize; }; +#endif } // namespace gfx diff --git a/gfx/2d/moz.build b/gfx/2d/moz.build index def5628f06..3fc089af39 100644 --- a/gfx/2d/moz.build +++ b/gfx/2d/moz.build @@ -48,10 +48,9 @@ EXPORTS.mozilla.gfx += [ 'UserData.h', ] -if CONFIG['MOZ_WIDGET_TOOLKIT'] == 'cocoa': +if CONFIG['MOZ_WIDGET_TOOLKIT'] in ('cocoa', 'uikit'): EXPORTS.mozilla.gfx += [ 'MacIOSurface.h', - 'QuartzSupport.h', ] UNIFIED_SOURCES += [ 'DrawTargetCG.cpp', @@ -163,6 +162,9 @@ SOURCES += [ ] if CONFIG['MOZ_WIDGET_TOOLKIT'] == 'cocoa': + EXPORTS.mozilla.gfx += [ + 'QuartzSupport.h', + ] SOURCES += [ 'MacIOSurface.cpp', 'QuartzSupport.mm', diff --git a/gfx/layers/composite/TextureHost.cpp b/gfx/layers/composite/TextureHost.cpp index e88e178d29..669816da06 100644 --- a/gfx/layers/composite/TextureHost.cpp +++ b/gfx/layers/composite/TextureHost.cpp @@ -410,7 +410,7 @@ BufferTextureHost::UpdatedInternal(const nsIntRegion* aRegion) // If the last frame wasn't uploaded yet, and we -don't- have a partial update, // we still need to update the full surface. if (aRegion && !mNeedsFullUpdate) { - mMaybeUpdatedRegion = mMaybeUpdatedRegion.Or(mMaybeUpdatedRegion, *aRegion); + mMaybeUpdatedRegion.OrWith(*aRegion); } else { mNeedsFullUpdate = true; } diff --git a/gfx/thebes/gfxPlatform.cpp b/gfx/thebes/gfxPlatform.cpp index 1b3da03e51..99b7e562df 100644 --- a/gfx/thebes/gfxPlatform.cpp +++ b/gfx/thebes/gfxPlatform.cpp @@ -2090,7 +2090,6 @@ gfxPlatform::UsesOffMainThreadCompositing() return result; } - /*** * The preference "layout.frame_rate" has 3 meanings depending on the value: * diff --git a/ipc/hal/DaemonSocketPDUHelpers.cpp b/ipc/hal/DaemonSocketPDUHelpers.cpp index 988137e564..6aaf21c339 100644 --- a/ipc/hal/DaemonSocketPDUHelpers.cpp +++ b/ipc/hal/DaemonSocketPDUHelpers.cpp @@ -7,23 +7,60 @@ #include "DaemonSocketPDUHelpers.h" #include +// Enable this constant to abort Gecko on IPC errors. This is helpful +// for debugging, but should *never* be enabled by default. +#define MOZ_HAL_ABORT_ON_IPC_ERRORS (0) + #ifdef CHROMIUM_LOG #undef CHROMIUM_LOG #endif #if defined(MOZ_WIDGET_GONK) + #include -#define CHROMIUM_LOG(args...) __android_log_print(ANDROID_LOG_INFO, "I/O", args); + +#define CHROMIUM_LOG(args...) \ + __android_log_print(ANDROID_LOG_INFO, "HAL-IPC", args); + +#define CHROMIUM_LOG_VA(fmt, ap) \ + __android_log_vprint(ANDROID_LOG_INFO, "HAL-IPC", fmt, ap); + #else + #include + #define IODEBUG true -#define CHROMIUM_LOG(args...) if (IODEBUG) printf(args); +#define CHROMIUM_LOG(args...) if (IODEBUG) { printf(args); } +#define CHROMIUM_LOG_VA(fmt, ap) if (IODEBUG) { vprintf(fmt, ap); } + #endif namespace mozilla { namespace ipc { namespace DaemonSocketPDUHelpers { +// +// Logging +// + +namespace detail { + +void +LogProtocolError(const char* aFmt, ...) +{ + va_list ap; + + va_start(ap, aFmt); + CHROMIUM_LOG_VA(aFmt, ap); + va_end(ap); + + if (MOZ_HAL_ABORT_ON_IPC_ERRORS) { + MOZ_CRASH("HAL IPC protocol error"); + } +} + +} // namespace detail + // // Conversion // @@ -35,7 +72,8 @@ Convert(bool aIn, uint8_t& aOut) [false] = 0x00, [true] = 0x01 }; - if (NS_WARN_IF(aIn >= MOZ_ARRAY_LENGTH(sValue))) { + if (MOZ_HAL_IPC_CONVERT_WARN_IF( + aIn >= MOZ_ARRAY_LENGTH(sValue), bool, uint8_t)) { aOut = 0; return NS_ERROR_ILLEGAL_VALUE; } @@ -59,8 +97,10 @@ Convert(bool aIn, int32_t& aOut) nsresult Convert(int aIn, uint8_t& aOut) { - if (NS_WARN_IF(aIn < std::numeric_limits::min()) || - NS_WARN_IF(aIn > std::numeric_limits::max())) { + if (MOZ_HAL_IPC_CONVERT_WARN_IF( + aIn < std::numeric_limits::min(), int, uint8_t) || + MOZ_HAL_IPC_CONVERT_WARN_IF( + aIn > std::numeric_limits::max(), int, uint8_t)) { aOut = 0; // silences compiler warning return NS_ERROR_ILLEGAL_VALUE; } @@ -71,8 +111,10 @@ Convert(int aIn, uint8_t& aOut) nsresult Convert(int aIn, int16_t& aOut) { - if (NS_WARN_IF(aIn < std::numeric_limits::min()) || - NS_WARN_IF(aIn > std::numeric_limits::max())) { + if (MOZ_HAL_IPC_CONVERT_WARN_IF( + aIn < std::numeric_limits::min(), int, int16_t) || + MOZ_HAL_IPC_CONVERT_WARN_IF( + aIn > std::numeric_limits::max(), int, int16_t)) { aOut = 0; // silences compiler warning return NS_ERROR_ILLEGAL_VALUE; } @@ -83,8 +125,10 @@ Convert(int aIn, int16_t& aOut) nsresult Convert(int aIn, int32_t& aOut) { - if (NS_WARN_IF(aIn < std::numeric_limits::min()) || - NS_WARN_IF(aIn > std::numeric_limits::max())) { + if (MOZ_HAL_IPC_CONVERT_WARN_IF( + aIn < std::numeric_limits::min(), int, int32_t) || + MOZ_HAL_IPC_CONVERT_WARN_IF( + aIn > std::numeric_limits::max(), int, int32_t)) { aOut = 0; // silences compiler warning return NS_ERROR_ILLEGAL_VALUE; } @@ -99,7 +143,8 @@ Convert(uint8_t aIn, bool& aOut) [0x00] = false, [0x01] = true }; - if (NS_WARN_IF(aIn >= MOZ_ARRAY_LENGTH(sBool))) { + if (MOZ_HAL_IPC_CONVERT_WARN_IF( + aIn >= MOZ_ARRAY_LENGTH(sBool), uint8_t, bool)) { return NS_ERROR_ILLEGAL_VALUE; } aOut = sBool[aIn]; @@ -137,8 +182,10 @@ Convert(uint32_t aIn, int& aOut) nsresult Convert(uint32_t aIn, uint8_t& aOut) { - if (NS_WARN_IF(aIn < std::numeric_limits::min()) || - NS_WARN_IF(aIn > std::numeric_limits::max())) { + if (MOZ_HAL_IPC_CONVERT_WARN_IF( + aIn < std::numeric_limits::min(), uint32_t, uint8_t) || + MOZ_HAL_IPC_CONVERT_WARN_IF( + aIn > std::numeric_limits::max(), uint32_t, uint8_t)) { aOut = 0; // silences compiler warning return NS_ERROR_ILLEGAL_VALUE; } @@ -149,7 +196,7 @@ Convert(uint32_t aIn, uint8_t& aOut) nsresult Convert(size_t aIn, uint16_t& aOut) { - if (NS_WARN_IF(aIn >= (1ul << 16))) { + if (MOZ_HAL_IPC_CONVERT_WARN_IF(aIn >= (1ul << 16), size_t, uint16_t)) { aOut = 0; // silences compiler warning return NS_ERROR_ILLEGAL_VALUE; } @@ -161,6 +208,12 @@ Convert(size_t aIn, uint16_t& aOut) // Packing // +nsresult +PackPDU(bool aIn, DaemonSocketPDU& aPDU) +{ + return PackPDU(PackConversion(aIn), aPDU); +} + nsresult PackPDU(const DaemonSocketPDUHeader& aIn, DaemonSocketPDU& aPDU) { @@ -183,6 +236,18 @@ PackPDU(const DaemonSocketPDUHeader& aIn, DaemonSocketPDU& aPDU) // Unpacking // +nsresult +UnpackPDU(DaemonSocketPDU& aPDU, bool& aOut) +{ + return UnpackPDU(aPDU, UnpackConversion(aOut)); +} + +nsresult +UnpackPDU(DaemonSocketPDU& aPDU, char& aOut) +{ + return UnpackPDU(aPDU, UnpackConversion(aOut)); +} + nsresult UnpackPDU(DaemonSocketPDU& aPDU, nsDependentCString& aOut) { @@ -191,19 +256,19 @@ UnpackPDU(DaemonSocketPDU& aPDU, nsDependentCString& aOut) // the string in the PDU, we can copy the actual bytes. const char* str = reinterpret_cast(aPDU.Consume(1)); - if (NS_WARN_IF(!str)) { + if (MOZ_HAL_IPC_UNPACK_WARN_IF(!str, nsDependentCString)) { return NS_ERROR_ILLEGAL_VALUE; // end of PDU } const char* end = static_cast(memchr(str, '\0', aPDU.GetSize() + 1)); - if (NS_WARN_IF(!end)) { + if (MOZ_HAL_IPC_UNPACK_WARN_IF(!end, nsDependentCString)) { return NS_ERROR_ILLEGAL_VALUE; // no string terminator } ptrdiff_t len = end - str; const uint8_t* rest = aPDU.Consume(len); - if (NS_WARN_IF(!rest)) { + if (MOZ_HAL_IPC_UNPACK_WARN_IF(!rest, nsDependentCString)) { // We couldn't consume bytes that should have been there. return NS_ERROR_ILLEGAL_VALUE; } @@ -260,7 +325,7 @@ PDUInitOp::WarnAboutTrailingData() const uint16_t payloadSize; mPDU->GetHeader(service, opcode, payloadSize); - CHROMIUM_LOG( + detail::LogProtocolError( "Unpacked PDU of type (%x,%x) still contains %zu Bytes of data.", service, opcode, size); } diff --git a/ipc/hal/DaemonSocketPDUHelpers.h b/ipc/hal/DaemonSocketPDUHelpers.h index 89a6096717..3bd28c4f21 100644 --- a/ipc/hal/DaemonSocketPDUHelpers.h +++ b/ipc/hal/DaemonSocketPDUHelpers.h @@ -34,6 +34,78 @@ struct DaemonSocketPDUHeader { namespace DaemonSocketPDUHelpers { +// +// Logging +// +// The HAL IPC logging macros below print clear error messages for +// failed IPC operations. Use |MOZ_HAL_IPC_CONVERT_WARN_IF|, +// |MOZ_HAL_IPC_PACK_WARN_IF| and |MOZ_HAL_IPC_UNPACK_WARN_IF| to +// test for failures when processing PDUs. +// +// All macros accept the test condition as their first argument, and +// additional type information: the convert macro takes the input and +// output types, the pack macro takes the input type, and the unpack +// macro takes output type. All macros return the result of the test +// condition. If the test fails (i.e., the condition is true), they +// output a warning to the log. +// +// Don't call the functions in the detail namespace. They are helpers +// and not for general use. +// + +namespace detail { + +void +LogProtocolError(const char*, ...); + +inline bool +ConvertWarnIfImpl(const char* aFile, unsigned long aLine, + bool aCondition, const char* aExpr, const char* aIn, + const char* aOut) +{ + if (MOZ_UNLIKELY(aCondition)) { + LogProtocolError("%s:%d: Convert('%s' to '%s') failed: %s", + aFile, aLine, aIn, aOut, aExpr); + } + return aCondition; +} + +inline bool +PackWarnIfImpl(const char* aFile, unsigned long aLine, + bool aCondition, const char* aExpr, const char* aIn) +{ + if (MOZ_UNLIKELY(aCondition)) { + LogProtocolError("%s:%d: Pack('%s') failed: %s", + aFile, aLine, aIn, aExpr); + } + return aCondition; +} + +inline bool +UnpackWarnIfImpl(const char* aFile, unsigned long aLine, + bool aCondition, const char* aExpr, const char* aOut) +{ + if (MOZ_UNLIKELY(aCondition)) { + LogProtocolError("%s:%d: Unpack('%s') failed: %s", + aFile, aLine, aOut, aExpr); + } + return aCondition; +} + +} // namespace detail + +#define MOZ_HAL_IPC_CONVERT_WARN_IF(condition, in, out) \ + ::mozilla::ipc::DaemonSocketPDUHelpers::detail:: \ + ConvertWarnIfImpl(__FILE__, __LINE__, condition, #condition, #in, #out) + +#define MOZ_HAL_IPC_PACK_WARN_IF(condition, in) \ + ::mozilla::ipc::DaemonSocketPDUHelpers::detail:: \ + PackWarnIfImpl(__FILE__, __LINE__, condition, #condition, #in) + +#define MOZ_HAL_IPC_UNPACK_WARN_IF(condition, out) \ + ::mozilla::ipc::DaemonSocketPDUHelpers::detail:: \ + UnpackWarnIfImpl(__FILE__, __LINE__, condition, #condition, #out) + // // Conversion // @@ -89,6 +161,9 @@ template nsresult PackPDU(T aIn, DaemonSocketPDU& aPDU); +nsresult +PackPDU(bool aIn, DaemonSocketPDU& aPDU); + inline nsresult PackPDU(uint8_t aIn, DaemonSocketPDU& aPDU) { @@ -116,6 +191,380 @@ PackPDU(uint32_t aIn, DaemonSocketPDU& aPDU) nsresult PackPDU(const DaemonSocketPDUHeader& aIn, DaemonSocketPDU& aPDU); +/* |PackConversion| is a helper for packing converted values. Pass + * an instance of this structure to |PackPDU| to convert a value from + * the input type to the output type and and write it to the PDU. + */ +template +struct PackConversion { + PackConversion(const Tin& aIn) + : mIn(aIn) + { } + + const Tin& mIn; +}; + +template +inline nsresult +PackPDU(const PackConversion& aIn, DaemonSocketPDU& aPDU) +{ + Tout out; + + nsresult rv = Convert(aIn.mIn, out); + if (NS_FAILED(rv)) { + return rv; + } + return PackPDU(out, aPDU); +} + +/* |PackArray| is a helper for packing arrays. Pass an instance + * of this structure as the first argument to |PackPDU| to pack + * an array. The array's maximum default length is 255 elements. + */ +template +struct PackArray +{ + PackArray(const T* aData, size_t aLength) + : mData(aData) + , mLength(aLength) + { } + + const T* mData; + size_t mLength; +}; + +/* This implementation of |PackPDU| packs the length of an array + * and the elements of the array one-by-one. + */ +template +inline nsresult +PackPDU(const PackArray& aIn, DaemonSocketPDU& aPDU) +{ + for (size_t i = 0; i < aIn.mLength; ++i) { + nsresult rv = PackPDU(aIn.mData[i], aPDU); + if (NS_FAILED(rv)) { + return rv; + } + } + return NS_OK; +} + +template<> +inline nsresult +PackPDU(const PackArray& aIn, DaemonSocketPDU& aPDU) +{ + /* Write raw bytes in one pass */ + return aPDU.Write(aIn.mData, aIn.mLength); +} + +template<> +inline nsresult +PackPDU(const PackArray& aIn, DaemonSocketPDU& aPDU) +{ + /* Write raw bytes in one pass */ + return aPDU.Write(aIn.mData, aIn.mLength); +} + +/* |PackCString0| is a helper for packing 0-terminated C string, + * including the \0 character. Pass an instance of this structure + * as the first argument to |PackPDU| to pack a string. + */ +struct PackCString0 +{ + PackCString0(const nsCString& aString) + : mString(aString) + { } + + const nsCString& mString; +}; + +/* This implementation of |PackPDU| packs a 0-terminated C string. + */ +inline nsresult +PackPDU(const PackCString0& aIn, DaemonSocketPDU& aPDU) +{ + return PackPDU( + PackArray(reinterpret_cast(aIn.mString.get()), + aIn.mString.Length() + 1), aPDU); +} + +/* |PackReversed| is a helper for packing data in reversed order. Pass an + * instance of this structure as the first argument to |PackPDU| to pack data + * in reversed order. + */ +template +struct PackReversed +{ + PackReversed(const T& aValue) + : mValue(aValue) + { } + + const T& mValue; +}; + +/* No general rules to pack data in reversed order. Signal a link error if the + * type |T| of |PackReversed| is not defined explicitly. + */ +template +nsresult +PackPDU(const PackReversed& aIn, DaemonSocketPDU& aPDU); + +/* This implementation of |PackPDU| packs elements in |PackArray| in reversed + * order. (ex. reversed GATT UUID, see bug 1171866) + */ +template +inline nsresult +PackPDU(const PackReversed>& aIn, DaemonSocketPDU& aPDU) +{ + for (size_t i = 0; i < aIn.mValue.mLength; ++i) { + nsresult rv = PackPDU(aIn.mValue.mData[aIn.mValue.mLength - i - 1], aPDU); + if (NS_FAILED(rv)) { + return rv; + } + } + return NS_OK; +} + +template +inline nsresult +PackPDU(const T1& aIn1, const T2& aIn2, DaemonSocketPDU& aPDU) +{ + nsresult rv = PackPDU(aIn1, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + return PackPDU(aIn2, aPDU); +} + +template +inline nsresult +PackPDU(const T1& aIn1, const T2& aIn2, const T3& aIn3, + DaemonSocketPDU& aPDU) +{ + nsresult rv = PackPDU(aIn1, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn2, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + return PackPDU(aIn3, aPDU); +} + +template +inline nsresult +PackPDU(const T1& aIn1, const T2& aIn2, const T3& aIn3, const T4& aIn4, + DaemonSocketPDU& aPDU) +{ + nsresult rv = PackPDU(aIn1, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn2, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn3, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + return PackPDU(aIn4, aPDU); +} + +template +inline nsresult +PackPDU(const T1& aIn1, const T2& aIn2, const T3& aIn3, + const T4& aIn4, const T5& aIn5, + DaemonSocketPDU& aPDU) +{ + nsresult rv = PackPDU(aIn1, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn2, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn3, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn4, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + return PackPDU(aIn5, aPDU); +} + +template +inline nsresult +PackPDU(const T1& aIn1, const T2& aIn2, const T3& aIn3, + const T4& aIn4, const T5& aIn5, const T6& aIn6, + DaemonSocketPDU& aPDU) +{ + nsresult rv = PackPDU(aIn1, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn2, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn3, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn4, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn5, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + return PackPDU(aIn6, aPDU); +} + +template +inline nsresult +PackPDU(const T1& aIn1, const T2& aIn2, const T3& aIn3, + const T4& aIn4, const T5& aIn5, const T6& aIn6, + const T7& aIn7, DaemonSocketPDU& aPDU) +{ + nsresult rv = PackPDU(aIn1, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn2, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn3, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn4, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn5, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn6, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + return PackPDU(aIn7, aPDU); +} + +template +inline nsresult +PackPDU(const T1& aIn1, const T2& aIn2, const T3& aIn3, + const T4& aIn4, const T5& aIn5, const T6& aIn6, + const T7& aIn7, const T8& aIn8, DaemonSocketPDU& aPDU) +{ + nsresult rv = PackPDU(aIn1, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn2, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn3, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn4, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn5, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn6, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn7, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + return PackPDU(aIn8, aPDU); +} + +template +inline nsresult +PackPDU(const T1& aIn1, const T2& aIn2, const T3& aIn3, + const T4& aIn4, const T5& aIn5, const T6& aIn6, + const T7& aIn7, const T8& aIn8, const T9& aIn9, + const T10& aIn10, const T11& aIn11, const T12& aIn12, + const T13& aIn13, DaemonSocketPDU& aPDU) +{ + nsresult rv = PackPDU(aIn1, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn2, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn3, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn4, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn5, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn6, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn7, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn8, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn9, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn10, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn11, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + rv = PackPDU(aIn12, aPDU); + if (NS_FAILED(rv)) { + return rv; + } + return PackPDU(aIn13, aPDU); +} + // // Unpacking // @@ -125,6 +574,12 @@ template nsresult UnpackPDU(DaemonSocketPDU& aPDU, T& aOut); +nsresult +UnpackPDU(DaemonSocketPDU& aPDU, bool& aOut); + +nsresult +UnpackPDU(DaemonSocketPDU& aPDU, char& aOut); + inline nsresult UnpackPDU(DaemonSocketPDU& aPDU, int8_t& aOut) { @@ -210,6 +665,149 @@ struct UnpackString0 nsresult UnpackPDU(DaemonSocketPDU& aPDU, const UnpackString0& aOut); +/* |UnpackConversion| is a helper for convering unpacked values. Pass + * an instance of this structure to |UnpackPDU| to read a value from + * the PDU in the input type and convert it to the output type. + */ +template +struct UnpackConversion { + UnpackConversion(Tout& aOut) + : mOut(aOut) + { } + + Tout& mOut; +}; + +template +inline nsresult +UnpackPDU(DaemonSocketPDU& aPDU, const UnpackConversion& aOut) +{ + Tin in; + nsresult rv = UnpackPDU(aPDU, in); + if (NS_FAILED(rv)) { + return rv; + } + return Convert(in, aOut.mOut); +} + +/* |UnpackArray| is a helper for unpacking arrays. Pass an instance + * of this structure as the second argument to |UnpackPDU| to unpack + * an array. + */ +template +struct UnpackArray +{ + UnpackArray(T* aData, size_t aLength) + : mData(aData) + , mLength(aLength) + { } + + UnpackArray(nsAutoArrayPtr& aData, size_t aLength) + : mData(nullptr) + , mLength(aLength) + { + aData = new T[mLength]; + mData = aData.get(); + } + + UnpackArray(nsAutoArrayPtr& aData, size_t aSize, size_t aElemSize) + : mData(nullptr) + , mLength(aSize / aElemSize) + { + aData = new T[mLength]; + mData = aData.get(); + } + + T* mData; + size_t mLength; +}; + +template +inline nsresult +UnpackPDU(DaemonSocketPDU& aPDU, const UnpackArray& aOut) +{ + for (size_t i = 0; i < aOut.mLength; ++i) { + nsresult rv = UnpackPDU(aPDU, aOut.mData[i]); + if (NS_FAILED(rv)) { + return rv; + } + } + return NS_OK; +} + +template +inline nsresult +UnpackPDU(DaemonSocketPDU& aPDU, UnpackArray& aOut) +{ + for (size_t i = 0; i < aOut.mLength; ++i) { + nsresult rv = UnpackPDU(aPDU, aOut.mData[i]); + if (NS_FAILED(rv)) { + return rv; + } + } + return NS_OK; +} + +template<> +inline nsresult +UnpackPDU(DaemonSocketPDU& aPDU, const UnpackArray& aOut) +{ + /* Read raw bytes in one pass */ + return aPDU.Read(aOut.mData, aOut.mLength); +} + +template +inline nsresult +UnpackPDU(DaemonSocketPDU& aPDU, nsTArray& aOut) +{ + for (typename nsTArray::size_type i = 0; i < aOut.Length(); ++i) { + nsresult rv = UnpackPDU(aPDU, aOut[i]); + if (NS_FAILED(rv)) { + return rv; + } + } + return NS_OK; +} + +/* |UnpackReversed| is a helper for unpacking data in reversed order. Pass an + * instance of this structure as the second argument to |UnpackPDU| to unpack + * data in reversed order. + */ +template +struct UnpackReversed +{ + UnpackReversed(T& aValue) + : mValue(&aValue) + { } + + UnpackReversed(T&& aValue) + : mValue(&aValue) + { } + + T* mValue; +}; + +/* No general rules to unpack data in reversed order. Signal a link error if + * the type |T| of |UnpackReversed| is not defined explicitly. + */ +template +nsresult +UnpackPDU(DaemonSocketPDU& aPDU, const UnpackReversed& aOut); + +template +inline nsresult +UnpackPDU(DaemonSocketPDU& aPDU, const UnpackReversed>& aOut) +{ + for (size_t i = 0; i < aOut.mValue->mLength; ++i) { + nsresult rv = UnpackPDU(aPDU, + aOut.mValue->mData[aOut.mValue->mLength - i - 1]); + if (NS_FAILED(rv)) { + return rv; + } + } + return NS_OK; +} + // // Init operators // @@ -305,6 +903,163 @@ private: DaemonSocketPDU* mPDU; // Hold pointer to allow for constant instances }; +// |UnpackPDUInitOp| is a general-purpose init operator for all variants +// of |DaemonResultRunnable| and |DaemonNotificationRunnable|. The call +// operators of |UnpackPDUInitOp| unpack a PDU into the supplied +// arguments. +class UnpackPDUInitOp final : private PDUInitOp +{ +public: + UnpackPDUInitOp(DaemonSocketPDU& aPDU) + : PDUInitOp(aPDU) + { } + + nsresult operator () () const + { + WarnAboutTrailingData(); + return NS_OK; + } + + template + nsresult operator () (T1& aArg1) const + { + nsresult rv = UnpackPDU(GetPDU(), aArg1); + if (NS_FAILED(rv)) { + return rv; + } + WarnAboutTrailingData(); + return NS_OK; + } + + template + nsresult operator () (T1& aArg1, T2& aArg2) const + { + DaemonSocketPDU& pdu = GetPDU(); + + nsresult rv = UnpackPDU(pdu, aArg1); + if (NS_FAILED(rv)) { + return rv; + } + rv = UnpackPDU(pdu, aArg2); + if (NS_FAILED(rv)) { + return rv; + } + WarnAboutTrailingData(); + return NS_OK; + } + + template + nsresult operator () (T1& aArg1, T2& aArg2, T3& aArg3) const + { + DaemonSocketPDU& pdu = GetPDU(); + + nsresult rv = UnpackPDU(pdu, aArg1); + if (NS_FAILED(rv)) { + return rv; + } + rv = UnpackPDU(pdu, aArg2); + if (NS_FAILED(rv)) { + return rv; + } + rv = UnpackPDU(pdu, aArg3); + if (NS_FAILED(rv)) { + return rv; + } + WarnAboutTrailingData(); + return NS_OK; + } + + template + nsresult operator () (T1& aArg1, T2& aArg2, T3& aArg3, T4& aArg4) const + { + DaemonSocketPDU& pdu = GetPDU(); + + nsresult rv = UnpackPDU(pdu, aArg1); + if (NS_FAILED(rv)) { + return rv; + } + rv = UnpackPDU(pdu, aArg2); + if (NS_FAILED(rv)) { + return rv; + } + rv = UnpackPDU(pdu, aArg3); + if (NS_FAILED(rv)) { + return rv; + } + rv = UnpackPDU(pdu, aArg4); + if (NS_FAILED(rv)) { + return rv; + } + WarnAboutTrailingData(); + return NS_OK; + } + + template + nsresult operator () (T1& aArg1, T2& aArg2, T3& aArg3, T4& aArg4, + T5& aArg5) const + { + DaemonSocketPDU& pdu = GetPDU(); + + nsresult rv = UnpackPDU(pdu, aArg1); + if (NS_FAILED(rv)) { + return rv; + } + rv = UnpackPDU(pdu, aArg2); + if (NS_FAILED(rv)) { + return rv; + } + rv = UnpackPDU(pdu, aArg3); + if (NS_FAILED(rv)) { + return rv; + } + rv = UnpackPDU(pdu, aArg4); + if (NS_FAILED(rv)) { + return rv; + } + rv = UnpackPDU(pdu, aArg5); + if (NS_FAILED(rv)) { + return rv; + } + WarnAboutTrailingData(); + return NS_OK; + } + + template + nsresult operator () (T1& aArg1, T2& aArg2, T3& aArg3, T4& aArg4, + T5& aArg5, T6& aArg6) const + { + DaemonSocketPDU& pdu = GetPDU(); + + nsresult rv = UnpackPDU(pdu, aArg1); + if (NS_FAILED(rv)) { + return rv; + } + rv = UnpackPDU(pdu, aArg2); + if (NS_FAILED(rv)) { + return rv; + } + rv = UnpackPDU(pdu, aArg3); + if (NS_FAILED(rv)) { + return rv; + } + rv = UnpackPDU(pdu, aArg4); + if (NS_FAILED(rv)) { + return rv; + } + rv = UnpackPDU(pdu, aArg5); + if (NS_FAILED(rv)) { + return rv; + } + rv = UnpackPDU(pdu, aArg6); + if (NS_FAILED(rv)) { + return rv; + } + WarnAboutTrailingData(); + return NS_OK; + } +}; + } // namespace DaemonSocketPDUHelpers } // namespace ipc diff --git a/ipc/ipdl/test/cxx/TestDataStructures.cpp b/ipc/ipdl/test/cxx/TestDataStructures.cpp index eb206dda13..a7286f5bd7 100644 --- a/ipc/ipdl/test/cxx/TestDataStructures.cpp +++ b/ipc/ipdl/test/cxx/TestDataStructures.cpp @@ -971,7 +971,7 @@ TestDataStructuresChild::Test18() ra.SetCapacity(nelements); for (int i = 0; i < nelements; ++i) { nsIntRegion r; - r = r.Or(nsIntRect(0, 0, 10, 10), nsIntRect(10, 10, 10, 10)); + r.Or(nsIntRect(0, 0, 10, 10), nsIntRect(10, 10, 10, 10)); ra.AppendElement(r); } diff --git a/js/public/Utility.h b/js/public/Utility.h index ec58043ce4..d62e62e42d 100644 --- a/js/public/Utility.h +++ b/js/public/Utility.h @@ -181,15 +181,10 @@ static inline bool ShouldFailWithOOM() { return false; } namespace js { -MOZ_NORETURN MOZ_COLD void -CrashAtUnhandlableOOM(const char* reason); - /* Disable OOM testing in sections which are not OOM safe. */ struct MOZ_RAII AutoEnterOOMUnsafeRegion { - void crash(const char* reason) { - CrashAtUnhandlableOOM(reason); - } + MOZ_NORETURN MOZ_COLD void crash(const char* reason); #if defined(DEBUG) || defined(JS_OOM_BREAKPOINT) AutoEnterOOMUnsafeRegion() diff --git a/js/src/gc/Statistics.cpp b/js/src/gc/Statistics.cpp index d377778a99..a516bc3e5f 100644 --- a/js/src/gc/Statistics.cpp +++ b/js/src/gc/Statistics.cpp @@ -935,13 +935,16 @@ Statistics::endGC() for (size_t d = PHASE_DAG_NONE; d < NumTimingArrays; d++) PodZero(&phaseTimes[d][PHASE_GC_BEGIN], PHASE_LIMIT - PHASE_GC_BEGIN); - aborted = false; + // Clear the OOM flag but only if we are not in a nested GC. + if (gcDepth == 1) + aborted = false; } void Statistics::beginSlice(const ZoneGCStats& zoneStats, JSGCInvocationKind gckind, SliceBudget budget, JS::gcreason::Reason reason) { + gcDepth++; this->zoneStats = zoneStats; bool first = !runtime->gc.isIncrementalGCInProgress(); @@ -950,13 +953,13 @@ Statistics::beginSlice(const ZoneGCStats& zoneStats, JSGCInvocationKind gckind, SliceData data(budget, reason, PRMJ_Now(), JS_GetCurrentEmbedderTime(), GetPageFaultCount()); if (!slices.append(data)) { - // OOM testing fails if we CrashAtUnhandlableOOM here. + // If we are OOM, set a flag to indicate we have missing slice data. aborted = true; return; } - // Slice callbacks should only fire for the outermost level - if (++gcDepth == 1) { + // Slice callbacks should only fire for the outermost level. + if (gcDepth == 1) { bool wasFullGC = zoneStats.isCollectingAllZones(); if (sliceCallback) (*sliceCallback)(runtime, first ? JS::GC_CYCLE_BEGIN : JS::GC_SLICE_BEGIN, @@ -977,8 +980,8 @@ Statistics::endSlice() if (last) endGC(); - // Slice callbacks should only fire for the outermost level - if (--gcDepth == 0) { + // Slice callbacks should only fire for the outermost level. + if (gcDepth == 1 && !aborted) { bool wasFullGC = zoneStats.isCollectingAllZones(); if (sliceCallback) (*sliceCallback)(runtime, last ? JS::GC_CYCLE_END : JS::GC_SLICE_END, @@ -988,13 +991,21 @@ Statistics::endSlice() /* Do this after the slice callback since it uses these values. */ if (last) PodArrayZero(counts); + + gcDepth--; + MOZ_ASSERT(gcDepth >= 0); } -void +bool Statistics::startTimingMutator() { - // Should only be called from outside of GC - MOZ_ASSERT(phaseNestingDepth == 0); + if (phaseNestingDepth != 0) { + // Should only be called from outside of GC. + MOZ_ASSERT(phaseNestingDepth == 1); + MOZ_ASSERT(phaseNesting[0] == PHASE_MUTATOR); + return false; + } + MOZ_ASSERT(suspendedPhaseNestingDepth == 0); timedGCTime = 0; @@ -1003,6 +1014,7 @@ Statistics::startTimingMutator() timedGCStart = 0; beginPhase(PHASE_MUTATOR); + return true; } bool diff --git a/js/src/gc/Statistics.h b/js/src/gc/Statistics.h index 2609a20990..722e1f8a22 100644 --- a/js/src/gc/Statistics.h +++ b/js/src/gc/Statistics.h @@ -173,7 +173,7 @@ struct Statistics SliceBudget budget, JS::gcreason::Reason reason); void endSlice(); - void startTimingMutator(); + bool startTimingMutator(); bool stopTimingMutator(double& mutator_ms, double& gc_ms); void reset(const char* reason) { diff --git a/js/src/jit-test/tests/gc/bug-1175755.js b/js/src/jit-test/tests/gc/bug-1175755.js new file mode 100644 index 0000000000..e0a319ce55 --- /dev/null +++ b/js/src/jit-test/tests/gc/bug-1175755.js @@ -0,0 +1,9 @@ +// |jit-test| allow-oom; allow-unhandlable-oom + +if (!('oomAfterAllocations' in this)) + quit(); + +setGCCallback({ + action: "majorGC", +}); +oomAfterAllocations(50); diff --git a/js/src/jit-test/tests/gc/bug-1214006.js b/js/src/jit-test/tests/gc/bug-1214006.js new file mode 100644 index 0000000000..b1bef3d6c0 --- /dev/null +++ b/js/src/jit-test/tests/gc/bug-1214006.js @@ -0,0 +1,17 @@ +if (!('oomAtAllocation' in this && 'resetOOMFailure' in this)) + quit(); + +function f() { + var i = 1; + do { + try { + oomAtAllocation(i); + (function() y)(); + } catch (e) { + x = resetOOMFailure(); + } + i++; + } while (x); +} +f(); +fullcompartmentchecks(true); diff --git a/js/src/jit-test/tests/gc/bug-1214846.js b/js/src/jit-test/tests/gc/bug-1214846.js new file mode 100644 index 0000000000..f147d18070 --- /dev/null +++ b/js/src/jit-test/tests/gc/bug-1214846.js @@ -0,0 +1,7 @@ +if (!('oomTest' in this) || helperThreadCount() === 0) + quit(); + +enableSPSProfiling(); +var s = newGlobal(); +s.offThreadCompileScript('oomTest(() => {});'); +s.runOffThreadScript(); diff --git a/js/src/jit-test/tests/gc/bug-1218900-2.js b/js/src/jit-test/tests/gc/bug-1218900-2.js new file mode 100644 index 0000000000..6e99475266 --- /dev/null +++ b/js/src/jit-test/tests/gc/bug-1218900-2.js @@ -0,0 +1,3 @@ +// |jit-test| error: Error +startTimingMutator(); +startTimingMutator(); diff --git a/js/src/jit-test/tests/gc/bug-1218900.js b/js/src/jit-test/tests/gc/bug-1218900.js new file mode 100644 index 0000000000..a3bc82bd2b --- /dev/null +++ b/js/src/jit-test/tests/gc/bug-1218900.js @@ -0,0 +1,8 @@ +// |jit-test| --fuzzing-safe +readline = function() {}; +Function.prototype.toString = function() { + for (var i = 0; i < 2; i++) { + this() + } +}; +getBacktrace({thisprops: true}); diff --git a/js/src/jit/JitFrames.cpp b/js/src/jit/JitFrames.cpp index ff87e5b40b..dcbb5b2889 100644 --- a/js/src/jit/JitFrames.cpp +++ b/js/src/jit/JitFrames.cpp @@ -2033,7 +2033,7 @@ SnapshotIterator::maybeRead(const RValueAllocation& a, MaybeReadFallback& fallba if (fallback.canRecoverResults()) { if (!initInstructionResults(fallback)) - js::CrashAtUnhandlableOOM("Unable to recover allocations."); + MOZ_CRASH("Unable to recover allocations."); if (allocationReadable(a)) return allocationValue(a); diff --git a/js/src/jit/arm/Assembler-arm.h b/js/src/jit/arm/Assembler-arm.h index 433c9652e1..e826a37e82 100644 --- a/js/src/jit/arm/Assembler-arm.h +++ b/js/src/jit/arm/Assembler-arm.h @@ -993,7 +993,7 @@ class BOffImm { MOZ_ASSERT((offset & 0x3) == 0); if (!IsInRange(offset)) - CrashAtUnhandlableOOM("BOffImm"); + MOZ_CRASH("BOffImm offset out of range"); } explicit BOffImm() diff --git a/js/src/jsalloc.h b/js/src/jsalloc.h index 6d9bf42949..b9ae51901c 100644 --- a/js/src/jsalloc.h +++ b/js/src/jsalloc.h @@ -72,11 +72,11 @@ class TempAllocPolicy void* reallocPtr = nullptr); template - T* onOutOfMemoryTyped(AllocFunction allocFunc, size_t numElems) { + T* onOutOfMemoryTyped(AllocFunction allocFunc, size_t numElems, void* reallocPtr = nullptr) { size_t bytes; if (MOZ_UNLIKELY(!CalculateAllocSize(numElems, &bytes))) return nullptr; - return static_cast(onOutOfMemory(allocFunc, bytes)); + return static_cast(onOutOfMemory(allocFunc, bytes, reallocPtr)); } public: @@ -118,7 +118,7 @@ class TempAllocPolicy T* pod_realloc(T* prior, size_t oldSize, size_t newSize) { T* p2 = maybe_pod_realloc(prior, oldSize, newSize); if (MOZ_UNLIKELY(!p2)) - p2 = onOutOfMemoryTyped(AllocFunction::Realloc, newSize); + p2 = onOutOfMemoryTyped(AllocFunction::Realloc, newSize, prior); return p2; } diff --git a/js/src/jscntxt.cpp b/js/src/jscntxt.cpp index 8b235c33e4..8c585e59ef 100644 --- a/js/src/jscntxt.cpp +++ b/js/src/jscntxt.cpp @@ -1198,13 +1198,15 @@ JS::AutoCheckRequestDepth::~AutoCheckRequestDepth() #endif #ifdef JS_CRASH_DIAGNOSTICS -void CompartmentChecker::check(InterpreterFrame* fp) +void +CompartmentChecker::check(InterpreterFrame* fp) { if (fp) check(fp->scopeChain()); } -void CompartmentChecker::check(AbstractFramePtr frame) +void +CompartmentChecker::check(AbstractFramePtr frame) { if (frame) check(frame.scopeChain()); @@ -1212,7 +1214,7 @@ void CompartmentChecker::check(AbstractFramePtr frame) #endif void -js::CrashAtUnhandlableOOM(const char* reason) +AutoEnterOOMUnsafeRegion::crash(const char* reason) { char msgbuf[1024]; JS_snprintf(msgbuf, sizeof(msgbuf), "[unhandlable oom] %s", reason); diff --git a/js/src/jscompartment.cpp b/js/src/jscompartment.cpp index b297641636..38b5fc7e6e 100644 --- a/js/src/jscompartment.cpp +++ b/js/src/jscompartment.cpp @@ -164,11 +164,12 @@ JSRuntime::createJitRuntime(JSContext* cx) JitRuntime::AutoMutateBackedges amb(jrt); jitRuntime_ = jrt; + AutoEnterOOMUnsafeRegion noOOM; if (!jitRuntime_->initialize(cx)) { // Handling OOM here is complicated: if we delete jitRuntime_ now, we // will destroy the ExecutableAllocator, even though there may still be // JitCode instances holding references to ExecutablePools. - CrashAtUnhandlableOOM("OOM in createJitRuntime"); + noOOM.crash("OOM in createJitRuntime"); } return jitRuntime_; diff --git a/js/src/jscompartment.h b/js/src/jscompartment.h index 1e7e9fba8a..5ffa6aa436 100644 --- a/js/src/jscompartment.h +++ b/js/src/jscompartment.h @@ -149,10 +149,10 @@ typedef HashMapflags(); + dst->setFunction(fun); Rooted lazy(cx); if (fun->isInterpretedLazy()) { @@ -3637,6 +3640,7 @@ js::CloneScriptIntoFunction(JSContext* cx, HandleObject enclosingScope, HandleFu fun->initLazyScript(lazy); else fun->setScript(nullptr); + fun->setFlags(preservedFlags); return nullptr; } @@ -3843,9 +3847,11 @@ JSScript::traceChildren(JSTracer* trc) static_cast(trc)->shouldCheckCompartments(), zone()->isCollecting()); - for (uint32_t i = 0; i < natoms(); ++i) { - if (atoms[i]) - TraceEdge(trc, &atoms[i], "atom"); + if (atoms) { + for (uint32_t i = 0; i < natoms(); ++i) { + if (atoms[i]) + TraceEdge(trc, &atoms[i], "atom"); + } } if (hasObjects()) { diff --git a/js/src/shell/js.cpp b/js/src/shell/js.cpp index 0395d8db8e..15a3c0faa6 100644 --- a/js/src/shell/js.cpp +++ b/js/src/shell/js.cpp @@ -1579,7 +1579,11 @@ StartTimingMutator(JSContext* cx, unsigned argc, Value* vp) return false; } - cx->runtime()->gc.stats.startTimingMutator(); + if (!cx->runtime()->gc.stats.startTimingMutator()) { + JS_ReportError(cx, "StartTimingMutator should only be called from outside of GC"); + return false; + } + args.rval().setUndefined(); return true; } diff --git a/js/src/vm/MallocProvider.h b/js/src/vm/MallocProvider.h index 168b2e50fa..6d6eb1c886 100644 --- a/js/src/vm/MallocProvider.h +++ b/js/src/vm/MallocProvider.h @@ -52,19 +52,17 @@ struct MallocProvider { template T* maybe_pod_malloc(size_t numElems) { - size_t bytes = numElems * sizeof(T); T* p = js_pod_malloc(numElems); if (MOZ_LIKELY(p)) - client()->updateMallocCounter(bytes); + client()->updateMallocCounter(numElems * sizeof(T)); return p; } template T* maybe_pod_calloc(size_t numElems) { - size_t bytes = numElems * sizeof(T); T* p = js_pod_calloc(numElems); if (MOZ_LIKELY(p)) - client()->updateMallocCounter(bytes); + client()->updateMallocCounter(numElems * sizeof(T)); return p; } @@ -90,11 +88,11 @@ struct MallocProvider T* p = maybe_pod_malloc(numElems); if (MOZ_LIKELY(p)) return p; - if (numElems & mozilla::tl::MulOverflowMask::value) { + size_t bytes; + if (MOZ_UNLIKELY(!CalculateAllocSize(numElems, &bytes))) { client()->reportAllocationOverflow(); return nullptr; } - size_t bytes = numElems * sizeof(T); p = (T*)client()->onOutOfMemory(AllocFunction::Malloc, bytes); if (p) client()->updateMallocCounter(bytes); @@ -103,16 +101,12 @@ struct MallocProvider template T* pod_malloc_with_extra(size_t numExtra) { - if (MOZ_UNLIKELY(numExtra & mozilla::tl::MulOverflowMask::value)) { + size_t bytes; + if (MOZ_UNLIKELY((!CalculateAllocSizeWithExtra(numExtra, &bytes)))) { client()->reportAllocationOverflow(); return nullptr; } - size_t bytes = sizeof(T) + numExtra * sizeof(U); - if (MOZ_UNLIKELY(bytes < sizeof(T))) { - client()->reportAllocationOverflow(); - return nullptr; - } - T* p = reinterpret_cast(js_pod_malloc(bytes)); + T* p = static_cast(js_malloc(bytes)); if (MOZ_LIKELY(p)) { client()->updateMallocCounter(bytes); return p; @@ -139,11 +133,11 @@ struct MallocProvider T* p = maybe_pod_calloc(numElems); if (MOZ_LIKELY(p)) return p; - if (numElems & mozilla::tl::MulOverflowMask::value) { + size_t bytes; + if (MOZ_UNLIKELY(!CalculateAllocSize(numElems, &bytes))) { client()->reportAllocationOverflow(); return nullptr; } - size_t bytes = numElems * sizeof(T); p = (T*)client()->onOutOfMemory(AllocFunction::Calloc, bytes); if (p) client()->updateMallocCounter(bytes); @@ -152,16 +146,12 @@ struct MallocProvider template T* pod_calloc_with_extra(size_t numExtra) { - if (MOZ_UNLIKELY(numExtra & mozilla::tl::MulOverflowMask::value)) { + size_t bytes; + if (MOZ_UNLIKELY((!CalculateAllocSizeWithExtra(numExtra, &bytes)))) { client()->reportAllocationOverflow(); return nullptr; } - size_t bytes = sizeof(T) + numExtra * sizeof(U); - if (MOZ_UNLIKELY(bytes < sizeof(T))) { - client()->reportAllocationOverflow(); - return nullptr; - } - T* p = reinterpret_cast(js_pod_calloc(bytes)); + T* p = static_cast(js_calloc(bytes)); if (p) { client()->updateMallocCounter(bytes); return p; @@ -184,11 +174,12 @@ struct MallocProvider T* p = maybe_pod_realloc(prior, oldSize, newSize); if (MOZ_LIKELY(p)) return p; - if (newSize & mozilla::tl::MulOverflowMask::value) { + size_t bytes; + if (MOZ_UNLIKELY(!CalculateAllocSize(newSize, &bytes))) { client()->reportAllocationOverflow(); return nullptr; } - p = (T*)client()->onOutOfMemory(AllocFunction::Realloc, newSize * sizeof(T), prior); + p = (T*)client()->onOutOfMemory(AllocFunction::Realloc, bytes, prior); if (p && newSize > oldSize) client()->updateMallocCounter((newSize - oldSize) * sizeof(T)); return p; diff --git a/js/src/vm/NativeObject.cpp b/js/src/vm/NativeObject.cpp index 436d6b0405..61135a801e 100644 --- a/js/src/vm/NativeObject.cpp +++ b/js/src/vm/NativeObject.cpp @@ -1561,7 +1561,7 @@ js::NativeDefineElement(ExclusiveContext* cx, HandleNativeObject obj, uint32_t i bool js::NativeDefineProperty(ExclusiveContext* cx, HandleNativeObject obj, HandleId id, - HandleValue value, GetterOp getter, SetterOp setter, + HandleValue value, JSGetterOp getter, JSSetterOp setter, unsigned attrs) { ObjectOpResult result; @@ -1581,7 +1581,7 @@ js::NativeDefineProperty(ExclusiveContext* cx, HandleNativeObject obj, HandleId bool js::NativeDefineProperty(ExclusiveContext* cx, HandleNativeObject obj, PropertyName* name, - HandleValue value, GetterOp getter, SetterOp setter, + HandleValue value, JSGetterOp getter, JSSetterOp setter, unsigned attrs) { RootedId id(cx, NameToId(name)); diff --git a/js/src/vm/Probes-inl.h b/js/src/vm/Probes-inl.h index 9c3a86044c..347f842b8e 100644 --- a/js/src/vm/Probes-inl.h +++ b/js/src/vm/Probes-inl.h @@ -39,7 +39,7 @@ probes::EnterScript(JSContext* cx, JSScript* script, JSFunction* maybeFun, JSRuntime* rt = cx->runtime(); if (rt->spsProfiler.enabled()) { - if (!rt->spsProfiler.enter(script, maybeFun)) + if (!rt->spsProfiler.enter(cx, script, maybeFun)) return false; MOZ_ASSERT_IF(!fp->script()->isGenerator(), !fp->hasPushedSPSFrame()); fp->setPushedSPSFrame(); diff --git a/js/src/vm/Runtime.cpp b/js/src/vm/Runtime.cpp index b0f0c6a3a7..58d55e77c1 100644 --- a/js/src/vm/Runtime.cpp +++ b/js/src/vm/Runtime.cpp @@ -44,6 +44,7 @@ #include "jit/arm64/vixl/Simulator-vixl.h" #include "jit/JitCompartment.h" #include "jit/mips32/Simulator-mips32.h" +#include "jit/mips64/Simulator-mips64.h" #include "jit/PcScriptCache.h" #include "js/Date.h" #include "js/MemoryMetrics.h" diff --git a/js/src/vm/Runtime.h b/js/src/vm/Runtime.h index a1b62e537d..99811fa30f 100644 --- a/js/src/vm/Runtime.h +++ b/js/src/vm/Runtime.h @@ -1490,7 +1490,7 @@ struct JSRuntime : public JS::shadow::Runtime, reportAllocationOverflow(); return nullptr; } - return static_cast(onOutOfMemoryCanGC(js::AllocFunction::Realloc, bytes)); + return static_cast(onOutOfMemoryCanGC(js::AllocFunction::Realloc, bytes, p)); } /* diff --git a/js/src/vm/SPSProfiler.cpp b/js/src/vm/SPSProfiler.cpp index a517151768..67d9ddead0 100644 --- a/js/src/vm/SPSProfiler.cpp +++ b/js/src/vm/SPSProfiler.cpp @@ -188,11 +188,13 @@ SPSProfiler::markEvent(const char* event) } bool -SPSProfiler::enter(JSScript* script, JSFunction* maybeFun) +SPSProfiler::enter(JSContext* cx, JSScript* script, JSFunction* maybeFun) { const char* str = profileString(script, maybeFun); - if (str == nullptr) + if (str == nullptr) { + ReportOutOfMemory(cx); return false; + } #ifdef DEBUG // In debug builds, assert the JS pseudo frames already on the stack diff --git a/js/src/vm/SPSProfiler.h b/js/src/vm/SPSProfiler.h index 668a1fe1be..a578f41b04 100644 --- a/js/src/vm/SPSProfiler.h +++ b/js/src/vm/SPSProfiler.h @@ -173,7 +173,7 @@ class SPSProfiler * - exit: this function has ceased execution, and no further * entries/exits will be made */ - bool enter(JSScript* script, JSFunction* maybeFun); + bool enter(JSContext* cx, JSScript* script, JSFunction* maybeFun); void exit(JSScript* script, JSFunction* maybeFun); void updatePC(JSScript* script, jsbytecode* pc) { if (enabled() && *size_ - 1 < max_) { diff --git a/js/src/vm/TraceLogging.cpp b/js/src/vm/TraceLogging.cpp index 10f2f813ef..fe069d98d2 100644 --- a/js/src/vm/TraceLogging.cpp +++ b/js/src/vm/TraceLogging.cpp @@ -688,21 +688,25 @@ TraceLoggerThreadState::init() enabledTextIds[TraceLogger_FoldTests] = true; enabledTextIds[TraceLogger_SplitCriticalEdges] = true; enabledTextIds[TraceLogger_RenumberBlocks] = true; + enabledTextIds[TraceLogger_ScalarReplacement] = true; enabledTextIds[TraceLogger_DominatorTree] = true; enabledTextIds[TraceLogger_PhiAnalysis] = true; - enabledTextIds[TraceLogger_ScalarReplacement] = true; + enabledTextIds[TraceLogger_MakeLoopsContiguous] = true; enabledTextIds[TraceLogger_ApplyTypes] = true; enabledTextIds[TraceLogger_EagerSimdUnbox] = true; enabledTextIds[TraceLogger_AliasAnalysis] = true; enabledTextIds[TraceLogger_GVN] = true; enabledTextIds[TraceLogger_LICM] = true; + enabledTextIds[TraceLogger_Sincos] = true; enabledTextIds[TraceLogger_RangeAnalysis] = true; enabledTextIds[TraceLogger_LoopUnrolling] = true; enabledTextIds[TraceLogger_EffectiveAddressAnalysis] = true; enabledTextIds[TraceLogger_AlignmentMaskAnalysis] = true; enabledTextIds[TraceLogger_EliminateDeadCode] = true; + enabledTextIds[TraceLogger_ReorderInstructions] = true; enabledTextIds[TraceLogger_EdgeCaseAnalysis] = true; enabledTextIds[TraceLogger_EliminateRedundantChecks] = true; + enabledTextIds[TraceLogger_AddKeepAliveInstructions] = true; enabledTextIds[TraceLogger_GenerateLIR] = true; enabledTextIds[TraceLogger_RegisterAllocation] = true; enabledTextIds[TraceLogger_GenerateCode] = true; diff --git a/js/src/vm/TraceLoggingGraph.h b/js/src/vm/TraceLoggingGraph.h index fd83cc5505..dfd71f6670 100644 --- a/js/src/vm/TraceLoggingGraph.h +++ b/js/src/vm/TraceLoggingGraph.h @@ -28,7 +28,7 @@ * binary file. * - treeFormat: The format used to encode the tree. By default "64,64,31,1,32". * There are currently no other formats to save the tree. - * - 64,64,31,1,31 signifies how many bytes are used for the different + * - 64,64,31,1,32 signifies how many bytes are used for the different * parts of the tree. * => 64 bits: Time Stamp Counter of start of event. * => 64 bits: Time Stamp Counter of end of event. diff --git a/layout/base/nsDisplayList.cpp b/layout/base/nsDisplayList.cpp index 9a186a5d67..eca7caa490 100644 --- a/layout/base/nsDisplayList.cpp +++ b/layout/base/nsDisplayList.cpp @@ -3560,8 +3560,8 @@ nsDisplayBoxShadowOuter::ComputeInvalidationRegion(nsDisplayListBuilder* aBuilde oldShadow = geometry->mBounds; newShadow = GetBounds(aBuilder, &snap); } else { - oldShadow = oldShadow.Sub(geometry->mBounds, geometry->mBorderRect); - newShadow = newShadow.Sub(GetBounds(aBuilder, &snap), GetBorderRect()); + oldShadow.Sub(geometry->mBounds, geometry->mBorderRect); + newShadow.Sub(GetBounds(aBuilder, &snap), GetBorderRect()); } aInvalidRegion->Or(oldShadow, newShadow); } diff --git a/layout/tools/reftest/Makefile.in b/layout/tools/reftest/Makefile.in index f318ca328b..f64b2351a2 100644 --- a/layout/tools/reftest/Makefile.in +++ b/layout/tools/reftest/Makefile.in @@ -7,6 +7,7 @@ _DEST_DIR = $(DEPTH)/_tests/reftest _HARNESS_FILES = \ $(srcdir)/runreftest.py \ + $(srcdir)/reftestcommandline.py \ $(srcdir)/remotereftest.py \ $(srcdir)/runreftestb2g.py \ $(srcdir)/b2g_desktop.py \ @@ -19,7 +20,6 @@ _HARNESS_FILES = \ $(topsrcdir)/testing/mozbase/mozdevice/mozdevice/Zeroconf.py \ $(topsrcdir)/testing/mozbase/moznetwork/moznetwork/moznetwork.py \ $(topsrcdir)/build/mobile/b2gautomation.py \ - $(topsrcdir)/build/automationutils.py \ $(topsrcdir)/build/mobile/remoteautomation.py \ $(topsrcdir)/testing/mochitest/server.js \ $(topsrcdir)/build/pgo/server-locations.txt \ diff --git a/layout/tools/reftest/b2g_desktop.py b/layout/tools/reftest/b2g_desktop.py index e0d5e8f44a..20098c5536 100644 --- a/layout/tools/reftest/b2g_desktop.py +++ b/layout/tools/reftest/b2g_desktop.py @@ -9,7 +9,7 @@ import sys here = os.path.abspath(os.path.dirname(__file__)) -from runreftest import RefTest, ReftestOptions +from runreftest import RefTest from marionette_driver import expected from marionette_driver.by import By @@ -51,12 +51,10 @@ class B2GDesktopReftest(RefTest): f.close() self.marionette.execute_script(self.test_script) - def run_tests(self, test_path, options): - reftestlist = self.getManifestPath(test_path) - if not reftestlist.startswith('file://'): - reftestlist = 'file://%s' % reftestlist + def run_tests(self, tests, options): + manifests = self.resolver.resolveManifests(options, tests) - self.profile = self.create_profile(options, reftestlist, + self.profile = self.create_profile(options, manifests, profile_to_clone=options.profile) env = self.buildBrowserEnv(options, self.profile.profile) kp_kwargs = { 'processOutputLine': [self._on_output], @@ -107,8 +105,8 @@ class B2GDesktopReftest(RefTest): log.info("%s | Running tests: end.", os.path.basename(__file__)) return status - def create_profile(self, options, reftestlist, profile_to_clone=None): - profile = RefTest.createReftestProfile(self, options, reftestlist, + def create_profile(self, options, manifests, profile_to_clone=None): + profile = RefTest.createReftestProfile(self, options, manifests, profile_to_clone=profile_to_clone) prefs = {} @@ -136,7 +134,6 @@ class B2GDesktopReftest(RefTest): prefs["network.dns.localDomains"] = "app://test-container.gaiamobile.org" prefs["reftest.browser.iframe.enabled"] = False prefs["reftest.remote"] = False - prefs["reftest.uri"] = "%s" % reftestlist # Set a future policy version to avoid the telemetry prompt. prefs["toolkit.telemetry.prompted"] = 999 prefs["toolkit.telemetry.notifiedOptOut"] = 999 @@ -188,7 +185,7 @@ class MuletReftest(B2GDesktopReftest): Wait(self.marionette, timeout).until(expected.element_present( By.CSS_SELECTOR, '#homescreen[loading-state=false]')) -def run_desktop_reftests(parser, options, args): +def run_desktop_reftests(parser, options): marionette_args = {} if options.marionette: host, port = options.marionette.split(':') @@ -200,9 +197,7 @@ def run_desktop_reftests(parser, options, args): else: reftest = B2GDesktopReftest(marionette_args) - options = ReftestOptions.verifyCommonOptions(parser, options, reftest) - if options == None: - sys.exit(1) + parser.validate(options, reftest) # add a -bin suffix if b2g-bin exists, but just b2g was specified if options.app[-4:] != '-bin': @@ -215,4 +210,4 @@ def run_desktop_reftests(parser, options, args): if options.desktop and not options.profile: raise Exception("must specify --profile when specifying --desktop") - sys.exit(reftest.run_tests(args[0], options)) + sys.exit(reftest.run_tests(options.tests, options)) diff --git a/layout/tools/reftest/mach_commands.py b/layout/tools/reftest/mach_commands.py index e4b13e5f82..7237e05bfd 100644 --- a/layout/tools/reftest/mach_commands.py +++ b/layout/tools/reftest/mach_commands.py @@ -23,8 +23,7 @@ from mach.decorators import ( Command, ) - -DEBUGGER_HELP = 'Debugger binary to run test in. Program name or path.' +import reftestcommandline ADB_NOT_FOUND = ''' The %s command requires the adb binary to be on your path. @@ -81,43 +80,10 @@ class ReftestRunner(MozbuildObject): self.tests_dir = os.path.join(self.topobjdir, '_tests') self.reftest_dir = os.path.join(self.tests_dir, 'reftest') - def _manifest_file(self, suite): - """Returns the manifest file used for a given test suite.""" - files = { - 'reftest': 'reftest.list', - 'reftest-ipc': 'reftest.list', - 'crashtest': 'crashtests.list', - 'crashtest-ipc': 'crashtests.list', - 'jstestbrowser': 'jstests.list' - } - assert suite in files - return files[suite] - - def _find_manifest(self, suite, test_file): - """Return a tuple of (manifest-path, filter-string) for running test_file. - - test_file can be a relative path to a single test file or manifest from - the top source directory, an absolute path to the same, or a directory - containing a manifest. - """ - assert test_file - path_arg = self._wrap_path_argument(test_file) - relpath = path_arg.relpath() - - if os.path.isdir(path_arg.srcdir_path()): - return (mozpath.join(relpath, self._manifest_file(suite)), None) - - if relpath.endswith('.list'): - return (relpath, None) - - return (self._find_manifest(suite, mozpath.dirname(test_file))[0], - mozpath.basename(test_file)) - def _make_shell_string(self, s): return "'%s'" % re.sub("'", r"'\''", s) - def run_b2g_test(self, b2g_home=None, xre_path=None, test_file=None, - suite=None, filter=None, **kwargs): + def run_b2g_test(self, b2g_home=None, xre_path=None, **kwargs): """Runs a b2g reftest. filter is a regular expression (in JS syntax, as could be passed to the @@ -130,324 +96,175 @@ class ReftestRunner(MozbuildObject): suite is the type of reftest to run. It can be one of ('reftest', 'crashtest'). """ - if suite not in ('reftest', 'crashtest'): + if kwargs["suite"] not in ('reftest', 'crashtest'): raise Exception('None or unrecognized reftest suite type.') + sys.path.insert(0, self.reftest_dir) + + test_subdir = {"reftest": os.path.join('layout', 'reftests'), + "crashtest": os.path.join('layout', 'crashtest')}[kwargs["suite"]] + # Find the manifest file - if not test_file: - if suite == 'reftest': - test_file = mozpath.join('layout', 'reftests') - elif suite == 'crashtest': - test_file = mozpath.join('testing', 'crashtest') + if not kwargs["tests"]: + if not os.path.exists(os.path.join(self.topsrcdir, test_subdir)): + test_file = mozpath.relpath(os.path.abspath(test_subdir), + self.topsrcdir) + kwargs["tests"] = [test_subdir] - if not os.path.exists(os.path.join(self.topsrcdir, test_file)): - test_file = mozpath.relpath(os.path.abspath(test_file), - self.topsrcdir) - - (manifest, single_file_filter) = self._find_manifest(suite, test_file) - if not os.path.exists(mozpath.join(self.topsrcdir, manifest)): - raise Exception('No manifest file was found at %s.' % manifest) - if single_file_filter: - if filter: - raise Exception('Cannot run single files in conjunction with --filter') - filter = single_file_filter - - # Need to chdir to reftest_dir otherwise imports fail below. - os.chdir(self.reftest_dir) - - # The imp module can spew warnings if the modules below have - # already been imported, ignore them. - with warnings.catch_warnings(): - warnings.simplefilter('ignore') - - import imp - path = os.path.join(self.reftest_dir, 'runreftestb2g.py') - with open(path, 'r') as fh: - imp.load_module('reftest', fh, path, ('.py', 'r', imp.PY_SOURCE)) - import reftest - - # Set up the reftest options. - parser = reftest.B2GOptions() - options, args = parser.parse_args([]) - - # Tests need to be served from a subdirectory of the server. Symlink - # topsrcdir here to get around this. tests = os.path.join(self.reftest_dir, 'tests') if not os.path.isdir(tests): os.symlink(self.topsrcdir, tests) - args.insert(0, os.path.join('tests', manifest)) - for k, v in kwargs.iteritems(): - setattr(options, k, v) + for i, path in enumerate(kwargs["tests"]): + # Non-absolute paths are relative to the packaged directory, which + # has an extra tests/ at the start + if os.path.exists(os.path.abspath(path)): + path = os.path.relpath(path, os.path.join(self.topsrcdir)) + kwargs["tests"][i] = os.path.join('tests', path) if conditions.is_b2g_desktop(self): - if self.substs.get('ENABLE_MARIONETTE') != '1': - print(MARIONETTE_DISABLED % ('mochitest-b2g-desktop', - self.mozconfig['path'])) - return 1 + return self.run_b2g_desktop(**kwargs) - options.profile = options.profile or os.environ.get('GAIA_PROFILE') - if not options.profile: + return self.run_b2g_remote(b2g_home, xre_path, **kwargs) + + def run_b2g_desktop(self, **kwargs): + if self.substs.get('ENABLE_MARIONETTE') != '1': + print(MARIONETTE_DISABLED % ('mochitest-b2g-desktop', + self.mozconfig['path'])) + return 1 + + if not kwargs["profile"]: + gaia_profile = os.environ.get('GAIA_PROFILE') + if not gaia_profile: print(GAIA_PROFILE_NOT_FOUND % 'reftest-b2g-desktop') return 1 + kwargs["profile"] = gaia_profile - if os.path.isfile(os.path.join(options.profile, 'extensions', \ - 'httpd@gaiamobile.org')): - print(GAIA_PROFILE_IS_DEBUG % ('mochitest-b2g-desktop', - options.profile)) - return 1 - options.desktop = True - options.app = self.get_binary_path() - if options.oop: - options.browser_arg = '-oop' - if not options.app.endswith('-bin'): - options.app = '%s-bin' % options.app - if not os.path.isfile(options.app): - options.app = options.app[:-len('-bin')] + if os.path.isfile(os.path.join(kwargs["profile"], 'extensions', + 'httpd@gaiamobile.org')): + print(GAIA_PROFILE_IS_DEBUG % ('mochitest-b2g-desktop', + kwargs["profile"])) + return 1 - return reftest.run_desktop_reftests(parser, options, args) + kwargs["desktop"] = True + kwargs["app"] = self.get_binary_path() + if kwargs["oop"]: + options.browser_arg = '-oop' + if not kwargs["app"].endswith('-bin'): + kwargs["app"] = '%s-bin' % options.app + if not os.path.isfile(kwargs["app"]): + options.app = kwargs["app"][:-len('-bin')] + return runreftestb2g.run(**kwargs) + + def run_b2g_remote(self, b2g_home, xre_path, **kwargs): + import runreftestb2g try: which.which('adb') except which.WhichError: # TODO Find adb automatically if it isn't on the path - raise Exception(ADB_NOT_FOUND % ('%s-remote' % suite, b2g_home)) + raise Exception(ADB_NOT_FOUND % ('%s-remote' % kwargs["suite"], b2g_home)) - options.b2gPath = b2g_home - options.logdir = self.reftest_dir - options.httpdPath = os.path.join(self.topsrcdir, 'netwerk', 'test', 'httpserver') - options.xrePath = xre_path - options.ignoreWindowSize = True - options.filter = filter + kwargs["b2gPath"] = b2g_home + kwargs["logdir"] = self.reftest_dir + kwargs["httpdPath"] = os.path.join(self.topsrcdir, 'netwerk', 'test', 'httpserver') + kwargs["xrePath"] = xre_path + kwargs["ignoreWindowSize"] = True # Don't enable oop for crashtest until they run oop in automation - if suite == 'reftest': - options.oop = True + if kwargs["suite"] == 'reftest': + kwargs["oop"] = True - return reftest.run_remote_reftests(parser, options, args) + return runreftestb2g.run_remote(**kwargs) - def run_desktop_test(self, test_file=None, filter=None, suite=None, - debugger=None, debugger_args=None, parallel=False, shuffle=False, - e10s=False, extraPrefs=None, this_chunk=None, total_chunks=None): - """Runs a reftest. + def run_desktop_test(self, **kwargs): + """Runs a reftest.""" + import runreftest - test_file is a path to a test file. It can be a relative path from the - top source directory, an absolute filename, or a directory containing - test files. - - filter is a regular expression (in JS syntax, as could be passed to the - RegExp constructor) to select which reftests to run from the manifest. - - suite is the type of reftest to run. It can be one of ('reftest', - 'crashtest', 'jstestbrowser'). - - debugger is the program name (in $PATH) or the full path of the - debugger to run. - - debugger_args are the arguments passed to the debugger. - - parallel indicates whether tests should be run in parallel or not. - - shuffle indicates whether to run tests in random order. - """ - - if suite not in ('reftest', 'reftest-ipc', 'crashtest', 'crashtest-ipc', 'jstestbrowser'): + if kwargs["suite"] not in ('reftest', 'crashtest', 'jstestbrowser'): raise Exception('None or unrecognized reftest suite type.') - env = {} - extra_args = [] + default_manifest = { + "reftest": (self.topsrcdir, "layout", "reftests", "reftest.list"), + "crashtest": (self.topsrcdir, "testing", "crashtest", "crashtests.list"), + "jstestbrowser": (self.topobjdir, "dist", "test-stage", "jsreftest", "tests", + "jstests.list") + } - if test_file: - (path, single_file_filter) = self._find_manifest(suite, test_file) - if not os.path.exists(mozpath.join(self.topsrcdir, path)): - raise Exception('No manifest file was found at %s.' % path) - if single_file_filter: - if filter: - raise Exception('Cannot run single files in conjunction with --filter') - filter = single_file_filter - env[b'TEST_PATH'] = path - if filter: - extra_args.extend(['--filter', self._make_shell_string(filter)]) + kwargs["extraProfileFiles"] = [os.path.join(self.topobjdir, "dist", "plugins")] + kwargs["symbolsPath"] = os.path.join(self.topobjdir, "crashreporter-symbols") - pass_thru = False + if not kwargs["tests"]: + kwargs["tests"] = [os.path.join(*default_manifest[kwargs["suite"]])] - if debugger: - extra_args.append('--debugger=\'%s\'' % debugger) - pass_thru = True - if debugger_args: - # Use _make_shell_string (which quotes) so that we - # handle multiple args being passed to the debugger. - extra_args.extend(['--debugger-args', self._make_shell_string(debugger_args)]) - else: - if debugger_args: - print("--debugger-args passed, but no debugger specified.") - return 1 + if kwargs["suite"] == "jstestbrowser": + kwargs["extraProfileFiles"].append(os.path.join(self.topobjdir, "dist", + "test-stage", "jsreftest", + "tests", "user.js")) - if parallel: - extra_args.append('--run-tests-in-parallel') + if not kwargs["runTestsInParallel"]: + kwargs["logFile"] = "%s.log" % kwargs["suite"] - if shuffle: - extra_args.append('--shuffle') - - if e10s: - extra_args.append('--e10s') - - if extraPrefs: - for pref in extraPrefs: - extra_args.extend(['--setpref', pref]) - - if this_chunk: - extra_args.append('--this-chunk=%s' % this_chunk) - - if total_chunks: - extra_args.append('--total-chunks=%s' % total_chunks) - - if extra_args: - args = [os.environ.get(b'EXTRA_TEST_ARGS', '')] - args.extend(extra_args) - env[b'EXTRA_TEST_ARGS'] = ' '.join(args) - - # TODO hook up harness via native Python - return self._run_make(directory='.', target=suite, append_env=env, - pass_thru=pass_thru, ensure_exit_code=False) - - -def ReftestCommand(func): - """Decorator that adds shared command arguments to reftest commands.""" - - debugger = CommandArgument('--debugger', metavar='DEBUGGER', - help=DEBUGGER_HELP) - func = debugger(func) - - debugger_args = CommandArgument('--debugger-args', metavar='DEBUGGER_ARGS', - help='Arguments to pass to the debugger.') - func = debugger_args(func) - - flter = CommandArgument('--filter', metavar='REGEX', - help='A JS regular expression to match test URLs against, to select ' - 'a subset of tests to run.') - func = flter(func) - - path = CommandArgument('test_file', nargs='?', metavar='MANIFEST', - help='Reftest manifest file, or a directory in which to select ' - 'reftest.list. If omitted, the entire test suite is executed.') - func = path(func) - - parallel = CommandArgument('--parallel', action='store_true', - help='Run tests in parallel.') - func = parallel(func) - - shuffle = CommandArgument('--shuffle', action='store_true', - help='Run tests in random order.') - func = shuffle(func) - - e10s = CommandArgument('--e10s', action='store_true', - help='Use content processes.') - func = e10s(func) - - extraPrefs = CommandArgument('--setpref', action='append', - default=[], dest='extraPrefs', metavar='PREF=VALUE', - help='Set prefs in the reftest profile.') - func = extraPrefs(func) - - totalChunks = CommandArgument('--total-chunks', - help = 'How many chunks to split the tests up into.') - func = totalChunks(func) - - thisChunk = CommandArgument('--this-chunk', - help = 'Which chunk to run between 1 and --total-chunks.') - func = thisChunk(func) - - return func - -def B2GCommand(func): - """Decorator that adds shared command arguments to b2g reftest commands.""" - - busybox = CommandArgument('--busybox', default=None, - help='Path to busybox binary to install on device') - func = busybox(func) - - logdir = CommandArgument('--logdir', default=None, - help='directory to store log files') - func = logdir(func) - - sdcard = CommandArgument('--sdcard', default="10MB", - help='Define size of sdcard: 1MB, 50MB...etc') - func = sdcard(func) - - emulator_res = CommandArgument('--emulator-res', default='800x1000', - help='Emulator resolution of the format \'x\'') - func = emulator_res(func) - - marionette = CommandArgument('--marionette', default=None, - help='host:port to use when connecting to Marionette') - func = marionette(func) - - totalChunks = CommandArgument('--total-chunks', dest='totalChunks', - type = int, - help = 'How many chunks to split the tests up into.') - func = totalChunks(func) - - thisChunk = CommandArgument('--this-chunk', dest='thisChunk', - type = int, - help = 'Which chunk to run between 1 and --total-chunks.') - func = thisChunk(func) - - flter = CommandArgument('--filter', metavar='REGEX', - help='A JS regular expression to match test URLs against, to select ' - 'a subset of tests to run.') - func = flter(func) - - oop = CommandArgument('--enable-oop', action='store_true', dest='oop', - help = 'Run tests in out-of-process mode.') - func = oop(func) - - path = CommandArgument('test_file', default=None, nargs='?', - metavar='TEST', - help='Test to run. Can be specified as a single file, a ' \ - 'directory, or omitted. If omitted, the entire test suite is ' \ - 'executed.') - func = path(func) - - return func + #Remove the stdout handler from the internal logger and let mach deal with it + runreftest.log.removeHandler(runreftest.log.handlers[0]) + self.log_manager.enable_unstructured() + rv = runreftest.run(**kwargs) + self.log_manager.disable_unstructured() + return rv @CommandProvider class MachCommands(MachCommandBase): - @Command('reftest', category='testing', description='Run reftests (layout and graphics correctness).') - @ReftestCommand - def run_reftest(self, test_file, **kwargs): - return self._run_reftest(test_file, suite='reftest', **kwargs) + @Command('reftest', + category='testing', + description='Run reftests (layout and graphics correctness).', + parser=reftestcommandline.DesktopArgumentsParser) + def run_reftest(self, **kwargs): + kwargs["suite"] = "reftest" + return self._run_reftest(**kwargs) - @Command('jstestbrowser', category='testing', - description='Run js/src/tests in the browser.') - @ReftestCommand - def run_jstestbrowser(self, test_file, **kwargs): - return self._run_reftest(test_file, suite='jstestbrowser', **kwargs) + @Command('jstestbrowser', + category='testing', + description='Run js/src/tests in the browser.', + parser=reftestcommandline.DesktopArgumentsParser) + def run_jstestbrowser(self, **kwargs): + self._mach_context.commands.dispatch("build", + self._mach_context, + what=["stage-jstests"]) + kwargs["suite"] = "jstestbrowser" + return self._run_reftest(**kwargs) - @Command('reftest-ipc', category='testing', - description='Run IPC reftests (layout and graphics correctness, separate process).') - @ReftestCommand - def run_ipc(self, test_file, **kwargs): - return self._run_reftest(test_file, suite='reftest-ipc', **kwargs) + @Command('reftest-ipc', + category='testing', + description='Run IPC reftests (layout and graphics correctness, separate process).', + parser=reftestcommandline.DesktopArgumentsParser) + def run_ipc(self, **kwargs): + kwargs["ipc"] = True + kwargs["suite"] = "reftest" + return self._run_reftest(**kwargs) - @Command('crashtest', category='testing', - description='Run crashtests (Check if crashes on a page).') - @ReftestCommand - def run_crashtest(self, test_file, **kwargs): - return self._run_reftest(test_file, suite='crashtest', **kwargs) + @Command('crashtest', + category='testing', + description='Run crashtests (Check if crashes on a page).', + parser=reftestcommandline.DesktopArgumentsParser) + def run_crashtest(self, **kwargs): + kwargs["suite"] = "crashtest" + return self._run_reftest(**kwargs) - @Command('crashtest-ipc', category='testing', - description='Run IPC crashtests (Check if crashes on a page, separate process).') - @ReftestCommand - def run_crashtest_ipc(self, test_file, **kwargs): - return self._run_reftest(test_file, suite='crashtest-ipc', **kwargs) + @Command('crashtest-ipc', + category='testing', + description='Run IPC crashtests (Check if crashes on a page, separate process).', + parser=reftestcommandline.DesktopArgumentsParser) + def run_crashtest_ipc(self, **kwargs): + kwargs["ipc"] = True + kwargs["suite"] = "crashtest" + return self._run_reftest(**kwargs) - def _run_reftest(self, test_file=None, suite=None, **kwargs): + def _run_reftest(self, **kwargs): reftest = self._spawn(ReftestRunner) - return reftest.run_desktop_test(test_file, suite=suite, **kwargs) + return reftest.run_desktop_test(**kwargs) # TODO For now b2g commands will only work with the emulator, @@ -466,27 +283,30 @@ class B2GCommands(MachCommandBase): setattr(self, attr, getattr(context, attr, None)) @Command('reftest-remote', category='testing', - description='Run a remote reftest (b2g layout and graphics correctness, remote device).', - conditions=[conditions.is_b2g, is_emulator]) - @B2GCommand - def run_reftest_remote(self, test_file, **kwargs): - return self._run_reftest(test_file, suite='reftest', **kwargs) + description='Run a remote reftest (b2g layout and graphics correctness, remote device).', + conditions=[conditions.is_b2g, is_emulator], + parser=reftestcommandline.B2GArgumentParser) + def run_reftest_remote(self, **kwargs): + kwargs["suite"] = "reftest" + return self._run_reftest(**kwargs) @Command('reftest-b2g-desktop', category='testing', - description='Run a b2g desktop reftest (b2g desktop layout and graphics correctness).', - conditions=[conditions.is_b2g_desktop]) - @B2GCommand - def run_reftest_b2g_desktop(self, test_file, **kwargs): - return self._run_reftest(test_file, suite='reftest', **kwargs) + description='Run a b2g desktop reftest (b2g desktop layout and graphics correctness).', + conditions=[conditions.is_b2g_desktop], + parser=reftestcommandline.B2GArgumentParser) + def run_reftest_b2g_desktop(self, **kwargs): + kwargs["suite"] = "reftest" + return self._run_reftest(**kwargs) @Command('crashtest-remote', category='testing', - description='Run a remote crashtest (Check if b2g crashes on a page, remote device).', - conditions=[conditions.is_b2g, is_emulator]) - @B2GCommand + description='Run a remote crashtest (Check if b2g crashes on a page, remote device).', + conditions=[conditions.is_b2g, is_emulator], + parser=reftestcommandline.B2GArgumentParser) def run_crashtest_remote(self, test_file, **kwargs): - return self._run_reftest(test_file, suite='crashtest', **kwargs) + kwargs["suite"] = "crashtest" + return self._run_reftest(**kwargs) - def _run_reftest(self, test_file=None, suite=None, **kwargs): + def _run_reftest(self, **kwargs): if self.device_name: if self.device_name.startswith('emulator'): emulator = 'arm' @@ -495,5 +315,4 @@ class B2GCommands(MachCommandBase): kwargs['emulator'] = emulator reftest = self._spawn(ReftestRunner) - return reftest.run_b2g_test(self.b2g_home, self.xre_path, - test_file, suite=suite, **kwargs) + return reftest.run_b2g_test(self.b2g_home, self.xre_path, **kwargs) diff --git a/layout/tools/reftest/reftest-cmdline.js b/layout/tools/reftest/reftest-cmdline.js index fde70c9c04..5094f6b8bd 100644 --- a/layout/tools/reftest/reftest-cmdline.js +++ b/layout/tools/reftest/reftest-cmdline.js @@ -21,37 +21,6 @@ RefTestCmdLineHandler.prototype = /* nsICommandLineHandler */ handle : function handler_handle(cmdLine) { - var args = { }; - args.wrappedJSObject = args; - try { - var uristr = cmdLine.handleFlagWithParam("reftest", false); - if (uristr == null) - return; - try { - args.uri = cmdLine.resolveURI(uristr).spec; - } - catch (e) { - return; - } - } - catch (e) { - cmdLine.handleFlag("reftest", true); - } - - try { - var nocache = cmdLine.handleFlag("reftestnocache", false); - args.nocache = nocache; - } - catch (e) { - } - - try { - var skipslowtests = cmdLine.handleFlag("reftestskipslowtests", false); - args.skipslowtests = skipslowtests; - } - catch (e) { - } - /* Ignore the platform's online/offline status while running reftests. */ var ios = Components.classes["@mozilla.org/network/io-service;1"] .getService(Components.interfaces.nsIIOService2); @@ -78,7 +47,7 @@ RefTestCmdLineHandler.prototype = function loadReftests() { wwatch.openWindow(null, "chrome://reftest/content/reftest.xul", "_blank", - "chrome,dialog=no,all", args); + "chrome,dialog=no,all", {}); } var remote = false; diff --git a/layout/tools/reftest/reftest.js b/layout/tools/reftest/reftest.js index 2f6e0d792a..1364f851ee 100644 --- a/layout/tools/reftest/reftest.js +++ b/layout/tools/reftest/reftest.js @@ -48,7 +48,7 @@ var gShuffle = false; var gTotalChunks = 0; var gThisChunk = 0; var gContainingWindow = null; -var gURLFilterRegex = null; +var gURLFilterRegex = {}; const FOCUS_FILTER_ALL_TESTS = "all"; const FOCUS_FILTER_NEEDS_FOCUS_TESTS = "needs-focus"; const FOCUS_FILTER_NON_NEEDS_FOCUS_TESTS = "non-needs-focus"; @@ -69,6 +69,7 @@ var gCanvas1, gCanvas2; // RecordResult. var gCurrentCanvas = null; var gURLs; +var gManifestsLoaded = {}; // Map from URI spec to the number of times it remains to be used var gURIUseCounts; // Map from URI spec to the canvas rendered for that URI @@ -389,10 +390,6 @@ function InitAndStartRefTests() gThisChunk = 0; } - try { - gURLFilterRegex = new RegExp(prefs.getCharPref("reftest.filter")); - } catch(e) {} - try { gFocusFilterMode = prefs.getCharPref("reftest.focusFilterMode"); } catch(e) {} @@ -449,8 +446,7 @@ function Shuffle(array) function StartTests() { - var uri; -#if BOOTSTRAP + var manifests; /* These prefs are optional, so we don't need to spit an error to the log */ try { var prefs = Components.classes["@mozilla.org/preferences-service;1"]. @@ -477,41 +473,32 @@ function StartTests() gRunSlowTests = false; } - try { - uri = prefs.getCharPref("reftest.uri"); - } catch(e) { - uri = ""; - } - - if (uri == "") { - gDumpLog("REFTEST TEST-UNEXPECTED-FAIL | | Unable to find reftest.uri pref. Please ensure your profile is setup properly\n"); - DoneTests(); - } -#else - try { - // Need to read the manifest once we have gHttpServerPort.. - var args = window.arguments[0].wrappedJSObject; - - if ("nocache" in args && args["nocache"]) - gNoCanvasCache = true; - - if ("skipslowtests" in args && args.skipslowtests) - gRunSlowTests = false; - - uri = args.uri; - } catch (e) { - ++gTestResults.Exception; - gDumpLog("REFTEST TEST-UNEXPECTED-FAIL | | EXCEPTION: " + ex + "\n"); - DoneTests(); - } -#endif - if (gShuffle) { gNoCanvasCache = true; } + gURLs = []; + try { - ReadTopManifest(uri); + var manifests = JSON.parse(prefs.getCharPref("reftest.manifests")); + gURLFilterRegex = manifests[null]; + } catch(e) { + gDumpLog("REFTEST TEST-UNEXPECTED-FAIL | | Unable to find reftest.manifests pref. Please ensure your profile is setup properly\n"); + DoneTests(); + } + + try { + var globalFilter = manifests.hasOwnProperty("") ? new RegExp(manifests[""]) : null; + var manifestURLs = Object.keys(manifests); + + // Ensure we read manifests from higher up the directory tree first so that we + // process includes before reading the included manifest again + manifestURLs.sort(function(a,b) {return a.length - b.length}) + manifestURLs.forEach(function(manifestURL) { + gDumpLog("Readings manifest" + manifestURL + "\n"); + var filter = manifests[manifestURL] ? new RegExp(manifests[manifestURL]) : null; + ReadTopManifest(manifestURL, [globalFilter, filter, false]); + }); BuildUseCounts(); // Filter tests which will be skipped to get a more even distribution when chunking @@ -795,18 +782,25 @@ function AddPrefSettings(aWhere, aPrefName, aPrefValExpression, aSandbox, aTestP return true; } -function ReadTopManifest(aFileURL) +function ReadTopManifest(aFileURL, aFilter) { - gURLs = new Array(); var url = gIOService.newURI(aFileURL, null, null); if (!url) throw "Expected a file or http URL for the manifest."; - ReadManifest(url, EXPECTED_PASS); + ReadManifest(url, EXPECTED_PASS, aFilter); } -function AddTestItem(aTest) +function AddTestItem(aTest, aFilter) { - if (gURLFilterRegex && !gURLFilterRegex.test(aTest.url1.spec)) + if (!aFilter) + aFilter = [null, [], false]; + + globalFilter = aFilter[0]; + manifestFilter = aFilter[1]; + invertManifest = aFilter[2]; + if ((globalFilter && !globalFilter.test(aTest.url1.spec)) || + (manifestFilter && + !(invertManifest ^ manifestFilter.test(aTest.url1.spec)))) return; if (gFocusFilterMode == FOCUS_FILTER_NEEDS_FOCUS_TESTS && !aTest.needsFocus) @@ -819,8 +813,19 @@ function AddTestItem(aTest) // Note: If you materially change the reftest manifest parsing, // please keep the parser in print-manifest-dirs.py in sync. -function ReadManifest(aURL, inherited_status) +function ReadManifest(aURL, inherited_status, aFilter) { + // Ensure each manifest is only read once. This assumes that manifests that are + // included with an unusual inherited_status or filters will be read via their + // include before they are read directly in the case of a duplicate + if (gManifestsLoaded.hasOwnProperty(aURL.spec)) { + if (gManifestsLoaded[aURL.spec] === null) + return; + else + aFilter = [aFilter[0], aFilter[1], true]; + } + gManifestsLoaded[aURL.spec] = aFilter[1]; + var secMan = CC[NS_SCRIPTSECURITYMANAGER_CONTRACTID] .getService(CI.nsIScriptSecurityManager); @@ -1036,7 +1041,7 @@ function ReadManifest(aURL, inherited_status) var incURI = gIOService.newURI(items[1], null, listURL); secMan.checkLoadURIWithPrincipal(principal, incURI, CI.nsIScriptSecurityManager.DISALLOW_SCRIPT); - ReadManifest(incURI, expected_status); + ReadManifest(incURI, expected_status, aFilter); } else if (items[0] == TYPE_LOAD) { if (items.length != 2) throw "Error in manifest file " + aURL.spec + " line " + lineNo + ": incorrect number of arguments to load"; @@ -1066,7 +1071,7 @@ function ReadManifest(aURL, inherited_status) fuzzyMaxPixels: fuzzy_max_pixels, url1: testURI, url2: null, - chaosMode: chaosMode }); + chaosMode: chaosMode }, aFilter); } else if (items[0] == TYPE_SCRIPT) { if (items.length != 2) throw "Error in manifest file " + aURL.spec + " line " + lineNo + ": incorrect number of arguments to script"; @@ -1093,7 +1098,7 @@ function ReadManifest(aURL, inherited_status) fuzzyMaxPixels: fuzzy_max_pixels, url1: testURI, url2: null, - chaosMode: chaosMode }); + chaosMode: chaosMode }, aFilter); } else if (items[0] == TYPE_REFTEST_EQUAL || items[0] == TYPE_REFTEST_NOTEQUAL) { if (items.length != 3) throw "Error in manifest file " + aURL.spec + " line " + lineNo + ": incorrect number of arguments to " + items[0]; @@ -1123,7 +1128,7 @@ function ReadManifest(aURL, inherited_status) fuzzyMaxPixels: fuzzy_max_pixels, url1: testURI, url2: refURI, - chaosMode: chaosMode }); + chaosMode: chaosMode }, aFilter); } else { throw "Error in manifest file " + aURL.spec + " line " + lineNo + ": unknown test type " + items[0]; } diff --git a/layout/tools/reftest/reftestcommandline.py b/layout/tools/reftest/reftestcommandline.py new file mode 100644 index 0000000000..b258f9d486 --- /dev/null +++ b/layout/tools/reftest/reftestcommandline.py @@ -0,0 +1,740 @@ +import argparse +import os +from collections import OrderedDict +from urlparse import urlparse + +here = os.path.abspath(os.path.dirname(__file__)) + + +class ReftestArgumentsParser(argparse.ArgumentParser): + def __init__(self, **kwargs): + super(ReftestArgumentsParser, self).__init__(**kwargs) + + # Try to import a MozbuildObject. Success indicates that we are + # running from a source tree. This allows some defaults to be set + # from the source tree. + try: + from mozbuild.base import MozbuildObject + self.build_obj = MozbuildObject.from_environment(cwd=here) + except ImportError: + self.build_obj = None + + self.add_argument("--xre-path", + action="store", + type=str, + dest="xrePath", + # individual scripts will set a sane default + default=None, + help="absolute path to directory containing XRE (probably xulrunner)") + + self.add_argument("--symbols-path", + action="store", + type=str, + dest="symbolsPath", + default=None, + help="absolute path to directory containing breakpad symbols, or the URL of a zip file containing symbols") + + self.add_argument("--debugger", + action="store", + dest="debugger", + help="use the given debugger to launch the application") + + self.add_argument("--debugger-args", + action="store", + dest="debuggerArgs", + help="pass the given args to the debugger _before_ " + "the application on the command line") + + self.add_argument("--debugger-interactive", + action="store_true", + dest="debuggerInteractive", + help="prevents the test harness from redirecting " + "stdout and stderr for interactive debuggers") + + self.add_argument("--appname", + action="store", + type=str, + dest="app", + default=None, + help="absolute path to application, overriding default") + + self.add_argument("--extra-profile-file", + action="append", + dest="extraProfileFiles", + default=[], + help="copy specified files/dirs to testing profile") + + self.add_argument("--timeout", + action="store", + dest="timeout", + type=int, + default=5 * 60, # 5 minutes per bug 479518 + help="reftest will timeout in specified number of seconds. [default %(default)s].") + + self.add_argument("--leak-threshold", + action="store", + type=int, + dest="defaultLeakThreshold", + default=0, + help="fail if the number of bytes leaked in default " + "processes through refcounted objects (or bytes " + "in classes with MOZ_COUNT_CTOR and MOZ_COUNT_DTOR) " + "is greater than the given number") + + self.add_argument("--utility-path", + action="store", + type=str, + dest="utilityPath", + default="bindir", + help="absolute path to directory containing utility " + "programs (xpcshell, ssltunnel, certutil)") + + self.add_argument("--total-chunks", + type=int, + dest="totalChunks", + help="how many chunks to split the tests up into") + + self.add_argument("--this-chunk", + type=int, + dest="thisChunk", + help="which chunk to run between 1 and --total-chunks") + + self.add_argument("--log-file", + action="store", + type=str, + dest="logFile", + default=None, + help="file to log output to in addition to stdout") + + self.add_argument("--skip-slow-tests", + dest="skipSlowTests", + action="store_true", + default=False, + help="skip tests marked as slow when running") + + self.add_argument("--ignore-window-size", + dest="ignoreWindowSize", + action="store_true", + default=False, + help="ignore the window size, which may cause spurious failures and passes") + + self.add_argument("--install-extension", + action="append", + dest="extensionsToInstall", + default=[], + help="install the specified extension in the testing profile. " + "The extension file's name should be .xpi where is " + "the extension's id as indicated in its install.rdf. " + "An optional path can be specified too.") + + self.add_argument("--setenv", + action="append", + type=str, + default=[], + dest="environment", + metavar="NAME=VALUE", + help="sets the given variable in the application's " + "environment") + + self.add_argument("--filter", + action="store", + type=str, + dest="filter", + help="specifies a regular expression (as could be passed to the JS " + "RegExp constructor) to test against URLs in the reftest manifest; " + "only test items that have a matching test URL will be run.") + + self.add_argument("--shuffle", + action="store_true", + default=False, + dest="shuffle", + help="run reftests in random order") + + self.add_argument("--focus-filter-mode", + action="store", + type=str, + dest="focusFilterMode", + default="all", + help="filters tests to run by whether they require focus. " + "Valid values are `all', `needs-focus', or `non-needs-focus'. " + "Defaults to `all'.") + + self.add_argument("--e10s", + action="store_true", + default=False, + dest="e10s", + help="enables content processes") + + self.add_argument("--setpref", + action="append", + type=str, + default=[], + dest="extraPrefs", + metavar="PREF=VALUE", + help="defines an extra user preference") + + self.add_argument("--reftest-extension-path", + action="store", + dest="reftestExtensionPath", + help="Path to the reftest extension") + + self.add_argument("--special-powers-extension-path", + action="store", + dest="specialPowersExtensionPath", + help="Path to the special powers extension") + + self.add_argument("--suite", + choices=["reftest", "crashtest", "jstestbrowser"], + default=None, + help=argparse.SUPPRESS) + + self.add_argument("tests", + metavar="TEST_PATH", + nargs="*", + help="Path to test file, manifest file, or directory containing tests") + + def get_ip(self): + import moznetwork + if os.name != "nt": + return moznetwork.get_ip() + else: + self.error( + "ERROR: you must specify a --remote-webserver=\n") + + def set_default_suite(self, options): + manifests = OrderedDict([("reftest.list", "reftest"), + ("crashtests.list", "crashtest"), + ("jstests.list", "jstestbrowser")]) + + for test_path in options.tests: + file_name = os.path.basename(test_path) + if file_name in manifests: + options.suite = manifests[file_name] + return + + for test_path in options.tests: + for manifest_file, suite in manifests.iteritems(): + if os.path.exists(os.path.join(test_path, manifest_file)): + options.suite = suite + return + + self.error("Failed to determine test suite; supply --suite to set this explicitly") + + def validate(self, options, reftest): + import sys + + if not options.tests: + # Can't just set this in the argument parser because mach will set a default + self.error("Must supply at least one path to a manifest file, test directory, or test file to run.") + + if options.suite is None: + self.set_default_suite(options) + + if options.totalChunks is not None and options.thisChunk is None: + self.error( + "thisChunk must be specified when totalChunks is specified") + + if options.totalChunks: + if not 1 <= options.thisChunk <= options.totalChunks: + self.error("thisChunk must be between 1 and totalChunks") + + if options.logFile: + options.logFile = reftest.getFullPath(options.logFile) + + if options.xrePath is not None: + if not os.access(options.xrePath, os.F_OK): + self.error("--xre-path '%s' not found" % options.xrePath) + if not os.path.isdir(options.xrePath): + self.error("--xre-path '%s' is not a directory" % + options.xrePath) + options.xrePath = reftest.getFullPath(options.xrePath) + + if options.reftestExtensionPath is None: + if self.build_obj is not None: + options.reftestExtensionPath = os.path.join(self.build_obj.topobjdir, "_tests", + "reftest", "reftest") + else: + options.reftestExtensionPath = os.path.join(here, "reftest") + + if (options.specialPowersExtensionPath is None and + options.suite in ["crashtest", "jstestbrowser"]): + if self.build_obj is not None: + options.specialPowersExtensionPath = os.path.join(self.build_obj.topobjdir, "_tests", + "reftest", "specialpowers") + else: + options.specialPowersExtensionPath = os.path.join( + here, "specialpowers") + + options.leakThresholds = { + "default": options.defaultLeakThreshold, + "tab": 5000, # See dependencies of bug 1051230. + } + + +class DesktopArgumentsParser(ReftestArgumentsParser): + def __init__(self, **kwargs): + super(DesktopArgumentsParser, self).__init__(**kwargs) + + self.add_argument("--run-tests-in-parallel", + action="store_true", + default=False, + dest="runTestsInParallel", + help="run tests in parallel if possible") + + self.add_argument("--ipc", + action="store_true", + default=False, + help="Run in out-of-processes mode") + + def _prefs_oop(self): + import mozinfo + prefs = ["layers.async-pan-zoom.enabled=true", + "browser.tabs.remote.autostart=true"] + if mozinfo.os == "win": + prefs.append("layers.acceleration.disabled=true") + + return prefs + + def _prefs_gpu(self): + if mozinfo.os != "win": + return ["layers.acceleration.force-enabled=true"] + return [] + + def validate(self, options, reftest): + super(DesktopArgumentsParser, self).validate(options, reftest) + + if options.ipc: + for item in self._prefs_oop(): + if item not in options.extraPrefs: + options.extraPrefs.append(item) + + if options.runTestsInParallel: + if options.logFile is not None: + self.error("cannot specify logfile with parallel tests") + if options.totalChunks is not None or options.thisChunk is not None: + self.error( + "cannot specify thisChunk or totalChunks with parallel tests") + if options.focusFilterMode != "all": + self.error("cannot specify focusFilterMode with parallel tests") + if options.debugger is not None: + self.error("cannot specify a debugger with parallel tests") + + if not options.tests: + self.error("No test files specified.") + + if options.app is None: + bin_dir = (self.build_obj.get_binary_path() if + self.build_obj and self.build_obj.substs[ + 'MOZ_BUILD_APP'] != 'mobile/android' + else None) + + if bin_dir: + options.app = bin_dir + else: + self.error( + "could not find the application path, --appname must be specified") + + options.app = reftest.getFullPath(options.app) + if not os.path.exists(options.app): + self.error("""Error: Path %(app)s doesn't exist. + Are you executing $objdir/_tests/reftest/runreftest.py?""" + % {"app": options.app}) + + if options.xrePath is None: + options.xrePath = os.path.dirname(options.app) + + if options.symbolsPath and len(urlparse(options.symbolsPath).scheme) < 2: + options.symbolsPath = reftest.getFullPath(options.symbolsPath) + + options.utilityPath = reftest.getFullPath(options.utilityPath) + + +class B2GArgumentParser(ReftestArgumentsParser): + def __init__(self, **kwargs): + super(B2GArgumentParser, self).__init__(**kwargs) + + self.add_argument("--browser-arg", + action="store", + type=str, + dest="browser_arg", + help="Optional command-line arg to pass to the browser") + + self.add_argument("--b2gpath", + action="store", + type=str, + dest="b2gPath", + help="path to B2G repo or qemu dir") + + self.add_argument("--marionette", + action="store", + type=str, + dest="marionette", + help="host:port to use when connecting to Marionette") + + self.add_argument("--emulator", + action="store", + type=str, + dest="emulator", + help="Architecture of emulator to use: x86 or arm") + + self.add_argument("--emulator-res", + action="store", + type=str, + dest="emulator_res", + help="Emulator resolution of the format 'x'") + + self.add_argument("--no-window", + action="store_true", + dest="noWindow", + default=False, + help="Pass --no-window to the emulator") + + self.add_argument("--adbpath", + action="store", + type=str, + dest="adb_path", + default="adb", + help="path to adb") + + self.add_argument("--deviceIP", + action="store", + type=str, + dest="deviceIP", + help="ip address of remote device to test") + + self.add_argument("--devicePort", + action="store", + type=str, + dest="devicePort", + default="20701", + help="port of remote device to test") + + self.add_argument("--remote-logfile", + action="store", + type=str, + dest="remoteLogFile", + help="Name of log file on the device relative to the device root. PLEASE ONLY USE A FILENAME.") + + self.add_argument("--remote-webserver", + action="store", + type=str, + dest="remoteWebServer", + help="ip address where the remote web server is hosted at") + + self.add_argument("--http-port", + action="store", + type=str, + dest="httpPort", + help="ip address where the remote web server is hosted at") + + self.add_argument("--ssl-port", + action="store", + type=str, + dest="sslPort", + help="ip address where the remote web server is hosted at") + + self.add_argument("--pidfile", + action="store", + type=str, + dest="pidFile", + default="", + help="name of the pidfile to generate") + + self.add_argument("--gecko-path", + action="store", + type=str, + dest="geckoPath", + help="the path to a gecko distribution that should " + "be installed on the emulator prior to test") + + self.add_argument("--logdir", + action="store", + type=str, + dest="logdir", + help="directory to store log files") + + self.add_argument('--busybox', + action='store', + type=str, + dest='busybox', + help="Path to busybox binary to install on device") + + self.add_argument("--httpd-path", + action="store", + type=str, + dest="httpdPath", + help="path to the httpd.js file") + + self.add_argument("--profile", + action="store", + type=str, + dest="profile", + help="for desktop testing, the path to the " + "gaia profile to use") + + self.add_argument("--desktop", + action="store_true", + dest="desktop", + default=False, + help="Run the tests on a B2G desktop build") + + self.add_argument("--mulet", + action="store_true", + dest="mulet", + default=False, + help="Run the tests on a B2G desktop build") + + self.add_argument("--enable-oop", + action="store_true", + dest="oop", + default=False, + help="Run the tests out of process") + + self.set_defaults(remoteTestRoot=None, + logFile="reftest.log", + autorun=True, + closeWhenDone=True, + testPath="") + + def validate_remote(self, options, automation): + if not options.app: + options.app = automation.DEFAULT_APP + + if not options.remoteTestRoot: + options.remoteTestRoot = automation._devicemanager.deviceRoot + \ + "/reftest" + + options.remoteProfile = options.remoteTestRoot + "/profile" + + productRoot = options.remoteTestRoot + "/" + automation._product + if options.utilityPath is None: + options.utilityPath = productRoot + "/bin" + + if not options.httpPort: + options.httpPort = automation.DEFAULT_HTTP_PORT + + if not options.sslPort: + options.sslPort = automation.DEFAULT_SSL_PORT + + if options.remoteWebServer is None: + options.remoteWebServer = self.get_ip() + + options.webServer = options.remoteWebServer + + if options.geckoPath and not options.emulator: + self.error( + "You must specify --emulator if you specify --gecko-path") + + if options.logdir and not options.emulator: + self.error("You must specify --emulator if you specify --logdir") + + if options.remoteLogFile is None: + options.remoteLogFile = "reftest.log" + + options.localLogName = options.remoteLogFile + options.remoteLogFile = options.remoteTestRoot + \ + '/' + options.remoteLogFile + + # Ensure that the options.logfile (which the base class uses) is set to + # the remote setting when running remote. Also, if the user set the + # log file name there, use that instead of reusing the remotelogfile as + # above. + if (options.logFile): + # If the user specified a local logfile name use that + options.localLogName = options.logFile + options.logFile = options.remoteLogFile + + # Only reset the xrePath if it wasn't provided + if options.xrePath is None: + options.xrePath = options.utilityPath + options.xrePath = os.path.abspath(options.xrePath) + + if options.pidFile != "": + f = open(options.pidFile, 'w') + f.write("%s" % os.getpid()) + f.close() + + # httpd-path is specified by standard makefile targets and may be specified + # on the command line to select a particular version of httpd.js. If not + # specified, try to select the one from from the xre bundle, as + # required in bug 882932. + if not options.httpdPath: + options.httpdPath = os.path.join(options.xrePath, "components") + + return options + + +class RemoteArgumentsParser(ReftestArgumentsParser): + def __init__(self, **kwargs): + super(RemoteArgumentsParser, self).__init__() + + # app, xrePath and utilityPath variables are set in main function + self.set_defaults(logFile="reftest.log", + app="", + xrePath="", + utilityPath="", + localLogName=None) + + self.add_argument("--remote-app-path", + action="store", + type=str, + dest="remoteAppPath", + help="Path to remote executable relative to device root using only forward slashes. Either this or app must be specified, but not both.") + + self.add_argument("--deviceIP", + action="store", + type=str, + dest="deviceIP", + help="ip address of remote device to test") + + self.add_argument("--deviceSerial", + action="store", + type=str, + dest="deviceSerial", + help="adb serial number of remote device to test") + + self.add_argument("--devicePort", + action="store", + type=str, + default="20701", + dest="devicePort", + help="port of remote device to test") + + self.add_argument("--remote-product-name", + action="store", + type=str, + dest="remoteProductName", + default="fennec", + help="Name of product to test - either fennec or firefox, defaults to fennec") + + self.add_argument("--remote-webserver", + action="store", + type=str, + dest="remoteWebServer", + help="IP Address of the webserver hosting the reftest content") + + self.add_argument("--http-port", + action="store", + type=str, + dest="httpPort", + help="port of the web server for http traffic") + + self.add_argument("--ssl-port", + action="store", + type=str, + dest="sslPort", + help="Port for https traffic to the web server") + + self.add_argument("--remote-logfile", + action="store", + type=str, + dest="remoteLogFile", + default="reftest.log", + help="Name of log file on the device relative to device root. PLEASE USE ONLY A FILENAME.") + + self.add_argument("--pidfile", + action="store", + type=str, + dest="pidFile", + default="", + help="name of the pidfile to generate") + + self.add_argument("--bootstrap", + action="store_true", + dest="bootstrap", + default=False, + help="test with a bootstrap addon required for native Fennec") + + self.add_argument("--dm_trans", + action="store", + type=str, + dest="dm_trans", + default="sut", + help="the transport to use to communicate with device: [adb|sut]; default=sut") + + self.add_argument("--remoteTestRoot", + action="store", + type=str, + dest="remoteTestRoot", + help="remote directory to use as test root (eg. /mnt/sdcard/tests or /data/local/tests)") + + self.add_argument("--httpd-path", + action="store", + type=str, + dest="httpdPath", + help="path to the httpd.js file") + + def validate_remote(self, options, automation): + # Ensure our defaults are set properly for everything we can infer + if not options.remoteTestRoot: + options.remoteTestRoot = automation._devicemanager.deviceRoot + \ + '/reftest' + options.remoteProfile = options.remoteTestRoot + "/profile" + + if options.remoteWebServer is None: + options.remoteWebServer = self.get_ip() + + # Verify that our remotewebserver is set properly + if options.remoteWebServer == '127.0.0.1': + self.error("ERROR: Either you specified the loopback for the remote webserver or ", + "your local IP cannot be detected. Please provide the local ip in --remote-webserver") + + if not options.httpPort: + options.httpPort = automation.DEFAULT_HTTP_PORT + + if not options.sslPort: + options.sslPort = automation.DEFAULT_SSL_PORT + + # One of remoteAppPath (relative path to application) or the app (executable) must be + # set, but not both. If both are set, we destroy the user's selection for app + # so instead of silently destroying a user specificied setting, we + # error. + if options.remoteAppPath and options.app: + self.error( + "ERROR: You cannot specify both the remoteAppPath and the app") + elif options.remoteAppPath: + options.app = options.remoteTestRoot + "/" + options.remoteAppPath + elif options.app is None: + # Neither remoteAppPath nor app are set -- error + self.error("ERROR: You must specify either appPath or app") + + if options.xrePath is None: + self.error( + "ERROR: You must specify the path to the controller xre directory") + else: + # Ensure xrepath is a full path + options.xrePath = os.path.abspath(options.xrePath) + + options.localLogName = options.remoteLogFile + options.remoteLogFile = options.remoteTestRoot + \ + '/' + options.remoteLogFile + + # Ensure that the options.logfile (which the base class uses) is set to + # the remote setting when running remote. Also, if the user set the + # log file name there, use that instead of reusing the remotelogfile as + # above. + if options.logFile: + # If the user specified a local logfile name use that + options.localLogName = options.logFile + + options.logFile = options.remoteLogFile + + if options.pidFile != "": + with open(options.pidFile, 'w') as f: + f.write(str(os.getpid())) + + # httpd-path is specified by standard makefile targets and may be specified + # on the command line to select a particular version of httpd.js. If not + # specified, try to select the one from hostutils.zip, as required in + # bug 882932. + if not options.httpdPath: + options.httpdPath = os.path.join(options.utilityPath, "components") + + if not options.ignoreWindowSize: + parts = automation._devicemanager.getInfo( + 'screen')['screen'][0].split() + width = int(parts[0].split(':')[1]) + height = int(parts[1].split(':')[1]) + if (width < 1050 or height < 1050): + self.error("ERROR: Invalid screen resolution %sx%s, please adjust to 1366x1050 or higher" % ( + width, height)) diff --git a/layout/tools/reftest/remotereftest.py b/layout/tools/reftest/remotereftest.py index e7430bade7..7ca02b0ad4 100644 --- a/layout/tools/reftest/remotereftest.py +++ b/layout/tools/reftest/remotereftest.py @@ -9,169 +9,36 @@ import tempfile import traceback # We need to know our current directory so that we can serve our test files from it. -SCRIPT_DIRECTORY = os.path.abspath(os.path.realpath(os.path.dirname(sys.argv[0]))) +SCRIPT_DIRECTORY = os.path.abspath(os.path.realpath(os.path.dirname(__file__))) -from runreftest import RefTest -from runreftest import ReftestOptions +from runreftest import RefTest, ReftestResolver from automation import Automation import devicemanager import droid +import mozinfo import moznetwork from remoteautomation import RemoteAutomation, fennecLogcatFilters -class RemoteOptions(ReftestOptions): - def __init__(self, automation): - ReftestOptions.__init__(self) - self.automation = automation +import reftestcommandline - defaults = {} - defaults["logFile"] = "reftest.log" - # app, xrePath and utilityPath variables are set in main function - defaults["app"] = "" - defaults["xrePath"] = "" - defaults["utilityPath"] = "" - defaults["runTestsInParallel"] = False - - self.add_option("--remote-app-path", action="store", - type = "string", dest = "remoteAppPath", - help = "Path to remote executable relative to device root using only forward slashes. Either this or app must be specified, but not both.") - defaults["remoteAppPath"] = None - - self.add_option("--deviceIP", action="store", - type = "string", dest = "deviceIP", - help = "ip address of remote device to test") - defaults["deviceIP"] = None - - self.add_option("--deviceSerial", action="store", - type = "string", dest = "deviceSerial", - help = "adb serial number of remote device to test") - defaults["deviceSerial"] = None - - self.add_option("--devicePort", action="store", - type = "string", dest = "devicePort", - help = "port of remote device to test") - defaults["devicePort"] = 20701 - - self.add_option("--remote-product-name", action="store", - type = "string", dest = "remoteProductName", - help = "Name of product to test - either fennec or firefox, defaults to fennec") - defaults["remoteProductName"] = "fennec" - - self.add_option("--remote-webserver", action="store", - type = "string", dest = "remoteWebServer", - help = "IP Address of the webserver hosting the reftest content") - defaults["remoteWebServer"] = moznetwork.get_ip() - - self.add_option("--http-port", action = "store", - type = "string", dest = "httpPort", - help = "port of the web server for http traffic") - defaults["httpPort"] = automation.DEFAULT_HTTP_PORT - - self.add_option("--ssl-port", action = "store", - type = "string", dest = "sslPort", - help = "Port for https traffic to the web server") - defaults["sslPort"] = automation.DEFAULT_SSL_PORT - - self.add_option("--remote-logfile", action="store", - type = "string", dest = "remoteLogFile", - help = "Name of log file on the device relative to device root. PLEASE USE ONLY A FILENAME.") - defaults["remoteLogFile"] = None - - self.add_option("--pidfile", action = "store", - type = "string", dest = "pidFile", - help = "name of the pidfile to generate") - defaults["pidFile"] = "" - - self.add_option("--bootstrap", action="store_true", dest = "bootstrap", - help = "test with a bootstrap addon required for native Fennec") - defaults["bootstrap"] = False - - self.add_option("--dm_trans", action="store", - type = "string", dest = "dm_trans", - help = "the transport to use to communicate with device: [adb|sut]; default=sut") - defaults["dm_trans"] = "sut" - - self.add_option("--remoteTestRoot", action = "store", - type = "string", dest = "remoteTestRoot", - help = "remote directory to use as test root (eg. /mnt/sdcard/tests or /data/local/tests)") - defaults["remoteTestRoot"] = None - - self.add_option("--httpd-path", action = "store", - type = "string", dest = "httpdPath", - help = "path to the httpd.js file") - defaults["httpdPath"] = None - - defaults["localLogName"] = None - - self.set_defaults(**defaults) - - def verifyRemoteOptions(self, options): - if options.runTestsInParallel: - self.error("Cannot run parallel tests here") - - # Ensure our defaults are set properly for everything we can infer - if not options.remoteTestRoot: - options.remoteTestRoot = self.automation._devicemanager.deviceRoot + '/reftest' - options.remoteProfile = options.remoteTestRoot + "/profile" - - # Verify that our remotewebserver is set properly - if (options.remoteWebServer == None or - options.remoteWebServer == '127.0.0.1'): - print "ERROR: Either you specified the loopback for the remote webserver or ", - print "your local IP cannot be detected. Please provide the local ip in --remote-webserver" - return None - - # One of remoteAppPath (relative path to application) or the app (executable) must be - # set, but not both. If both are set, we destroy the user's selection for app - # so instead of silently destroying a user specificied setting, we error. - if (options.remoteAppPath and options.app): - print "ERROR: You cannot specify both the remoteAppPath and the app" - return None - elif (options.remoteAppPath): - options.app = options.remoteTestRoot + "/" + options.remoteAppPath - elif (options.app == None): - # Neither remoteAppPath nor app are set -- error - print "ERROR: You must specify either appPath or app" - return None - - if (options.xrePath == None): - print "ERROR: You must specify the path to the controller xre directory" - return None +class RemoteReftestResolver(ReftestResolver): + def absManifestPath(self, path): + script_abs_path = os.path.join(SCRIPT_DIRECTORY, path) + if os.path.exists(script_abs_path): + rv = script_abs_path + elif os.path.exists(os.path.abspath(path)): + rv = os.path.abspath(path) else: - # Ensure xrepath is a full path - options.xrePath = os.path.abspath(options.xrePath) + print >> sys.stderr, "Could not find manifest %s" % script_abs_path + sys.exit(1) + return os.path.normpath(rv) - # Default to /reftest/reftest.log - if (options.remoteLogFile == None): - options.remoteLogFile = 'reftest.log' + def manifestURL(self, options, path): + # Dynamically build the reftest URL if possible, beware that args[0] should exist 'inside' the webroot + # It's possible for this url to have a leading "..", but reftest.js will fix that up + relPath = os.path.relpath(path, SCRIPT_DIRECTORY) + return "http://%s:%s/%s" % (options.remoteWebServer, options.httpPort, relPath) - options.localLogName = options.remoteLogFile - options.remoteLogFile = options.remoteTestRoot + '/' + options.remoteLogFile - - # Ensure that the options.logfile (which the base class uses) is set to - # the remote setting when running remote. Also, if the user set the - # log file name there, use that instead of reusing the remotelogfile as above. - if (options.logFile): - # If the user specified a local logfile name use that - options.localLogName = options.logFile - - options.logFile = options.remoteLogFile - - if (options.pidFile != ""): - f = open(options.pidFile, 'w') - f.write("%s" % os.getpid()) - f.close() - - # httpd-path is specified by standard makefile targets and may be specified - # on the command line to select a particular version of httpd.js. If not - # specified, try to select the one from hostutils.zip, as required in bug 882932. - if not options.httpdPath: - options.httpdPath = os.path.join(options.utilityPath, "components") - - # TODO: Copied from main, but I think these are no longer used in a post xulrunner world - #options.xrePath = options.remoteTestRoot + self.automation._product + '/xulrunner' - #options.utilityPath = options.testRoot + self.automation._product + '/bin' - return options class ReftestServer: """ Web server used to serve Reftests, for closer fidelity to the real web. @@ -259,6 +126,7 @@ class ReftestServer: class RemoteReftest(RefTest): remoteApp = '' + resolver_cls = RemoteReftestResolver def __init__(self, automation, devicemanager, options, scriptDir): RefTest.__init__(self) @@ -341,8 +209,12 @@ class RemoteReftest(RefTest): def stopWebServer(self, options): self.server.stop() - def createReftestProfile(self, options, reftestlist): - profile = RefTest.createReftestProfile(self, options, reftestlist, server=options.remoteWebServer, port=options.httpPort) + def createReftestProfile(self, options, manifest): + profile = RefTest.createReftestProfile(self, + options, + manifest, + server=options.remoteWebServer, + port=options.httpPort) profileDir = profile.profile prefs = {} @@ -354,7 +226,6 @@ class RemoteReftest(RefTest): # Set a future policy version to avoid the telemetry prompt. prefs["toolkit.telemetry.prompted"] = 999 prefs["toolkit.telemetry.notifiedOptOut"] = 999 - prefs["reftest.uri"] = "%s" % reftestlist prefs["datareporting.policy.dataSubmissionPolicyBypassAcceptance"] = True # Point the url-classifier to the local testing server for fast failures @@ -404,9 +275,6 @@ class RemoteReftest(RefTest): print "Automation Error: Failed to copy extra files to device" raise - def getManifestPath(self, path): - return path - def printDeviceInfo(self, printLogcat=False): try: if printLogcat: @@ -437,7 +305,8 @@ class RemoteReftest(RefTest): def runApp(self, profile, binary, cmdargs, env, timeout=None, debuggerInfo=None, - symbolsPath=None, options=None): + symbolsPath=None, options=None, + valgrindPath=None, valgrindArgs=None, valgrindSuppFiles=None): status = self.automation.runApp(None, env, binary, profile.profile, @@ -467,10 +336,10 @@ class RemoteReftest(RefTest): except: print "Warning: cleaning up pidfile '%s' was unsuccessful from the test harness" % self.pidFile -def main(args): +def main(): automation = RemoteAutomation(None) - parser = RemoteOptions(automation) - options, args = parser.parse_args() + parser = reftestcommandline.RemoteArgumentsParser() + options = parser.parse_args() if (options.dm_trans == 'sut' and options.deviceIP == None): print "Error: If --dm_trans = sut, you must provide a device IP to connect to via the --deviceIP option" @@ -496,39 +365,28 @@ def main(args): automation.setProduct(options.remoteProductName) # Set up the defaults and ensure options are set - options = parser.verifyRemoteOptions(options) - if (options == None): - print "ERROR: Invalid options specified, use --help for a list of valid options" - return 1 + parser.validate_remote(options, automation) - if not options.ignoreWindowSize: - parts = dm.getInfo('screen')['screen'][0].split() - width = int(parts[0].split(':')[1]) - height = int(parts[1].split(':')[1]) - if (width < 1050 or height < 1050): - print "ERROR: Invalid screen resolution %sx%s, please adjust to 1366x1050 or higher" % (width, height) - return 1 + # Check that Firefox is installed + expected = options.app.split('/')[-1] + installed = dm.shellCheckOutput(['pm', 'list', 'packages', expected]) + if expected not in installed: + print "%s is not installed on this device" % expected + return 1 automation.setAppName(options.app) automation.setRemoteProfile(options.remoteProfile) automation.setRemoteLog(options.remoteLogFile) reftest = RemoteReftest(automation, dm, options, SCRIPT_DIRECTORY) - options = parser.verifyCommonOptions(options, reftest) + parser.validate(options, reftest) + + if mozinfo.info['debug']: + print "changing timeout for remote debug reftests from %s to 600 seconds" % options.timeout + options.timeout = 600 # Hack in a symbolic link for jsreftest os.system("ln -s ../jsreftest " + str(os.path.join(SCRIPT_DIRECTORY, "jsreftest"))) - # Dynamically build the reftest URL if possible, beware that args[0] should exist 'inside' the webroot - manifest = args[0] - if os.path.exists(os.path.join(SCRIPT_DIRECTORY, args[0])): - manifest = "http://" + str(options.remoteWebServer) + ":" + str(options.httpPort) + "/" + args[0] - elif os.path.exists(args[0]): - manifestPath = os.path.abspath(args[0]).split(SCRIPT_DIRECTORY)[1].strip('/') - manifest = "http://" + str(options.remoteWebServer) + ":" + str(options.httpPort) + "/" + manifestPath - else: - print "ERROR: Could not find test manifest '%s'" % manifest - return 1 - # Start the webserver retVal = reftest.startWebServer(options) if retVal: @@ -544,11 +402,8 @@ def main(args): # manifest = "http://" + options.remoteWebServer + "/reftests/layout/reftests/reftest-sanity/reftest.list" retVal = 0 try: - cmdlineArgs = ["-reftest", manifest] - if options.bootstrap: - cmdlineArgs = [] dm.recordLogcat() - retVal = reftest.runTests(manifest, options, cmdlineArgs) + retVal = reftest.runTests(options.tests, options) except: print "Automation Error: Exception caught while running tests" traceback.print_exc() @@ -561,5 +416,5 @@ def main(args): return retVal if __name__ == "__main__": - sys.exit(main(sys.argv[1:])) + sys.exit(main()) diff --git a/layout/tools/reftest/runreftest.py b/layout/tools/reftest/runreftest.py index 382fca95bb..a9aadbc325 100644 --- a/layout/tools/reftest/runreftest.py +++ b/layout/tools/reftest/runreftest.py @@ -6,9 +6,8 @@ Runs the reftest test harness. """ -from optparse import OptionParser -from urlparse import urlparse import collections +import json import multiprocessing import os import re @@ -19,13 +18,10 @@ import sys import threading SCRIPT_DIRECTORY = os.path.abspath( - os.path.realpath(os.path.dirname(sys.argv[0]))) -sys.path.insert(0, SCRIPT_DIRECTORY) + os.path.realpath(os.path.dirname(__file__))) +if SCRIPT_DIRECTORY not in sys.path: + sys.path.insert(0, SCRIPT_DIRECTORY) -from automationutils import ( - dumpScreen, - printstatus -) import mozcrash import mozdebug import mozinfo @@ -33,15 +29,10 @@ import mozleak import mozprocess import mozprofile import mozrunner -from mozrunner.utils import test_environment +from mozrunner.utils import get_stack_fixer_function, test_environment +from mozscreenshot import printstatus, dump_screen -here = os.path.abspath(os.path.dirname(__file__)) - -try: - from mozbuild.base import MozbuildObject - build_obj = MozbuildObject.from_environment(cwd=here) -except ImportError: - build_obj = None +import reftestcommandline # set up logging handler a la automation.py.in for compatability import logging @@ -134,14 +125,83 @@ class ReftestThread(threading.Thread): if summaryHeadRegex.search(line) is None: yield line +class ReftestResolver(object): + def defaultManifest(self, suite): + return {"reftest": "reftest.list", + "crashtest": "crashtests.list", + "jstestbrowser": "jstests.list"}[suite] + + def directoryManifest(self, suite, path): + return os.path.join(path, self.defaultManifest(suite)) + + def findManifest(self, suite, test_file, subdirs=True): + """Return a tuple of (manifest-path, filter-string) for running test_file. + + test_file is a path to a test or a manifest file + """ + rv = [] + default_manifest = self.defaultManifest(suite) + if not os.path.isabs(test_file): + test_file = self.absManifestPath(test_file) + + if os.path.isdir(test_file): + for dirpath, dirnames, filenames in os.walk(test_file): + if default_manifest in filenames: + rv.append((os.path.join(dirpath, default_manifest), None)) + # We keep recursing into subdirectories which means that in the case + # of include directives we get the same manifest multiple times. + # However reftest.js will only read each manifest once + + elif test_file.endswith('.list'): + if os.path.exists(test_file): + rv = [(test_file, None)] + else: + dirname, pathname = os.path.split(test_file) + found = True + while not os.path.exists(os.path.join(dirname, default_manifest)): + dirname, suffix = os.path.split(dirname) + pathname = os.path.join(suffix, pathname) + if os.path.dirname(dirname) == dirname: + found = False + break + if found: + rv = [(os.path.join(dirname, default_manifest), + r".*(?:/|\\)%s$" % pathname)] + + return rv + + def absManifestPath(self, path): + return os.path.normpath(os.path.abspath(path)) + + def manifestURL(self, options, path): + return "file://%s" % path + + def resolveManifests(self, options, tests): + suite = options.suite + manifests = {} + for testPath in tests: + for manifest, filter_str in self.findManifest(suite, testPath): + manifest = self.manifestURL(options, manifest) + if manifest not in manifests: + manifests[manifest] = set() + manifests[manifest].add(filter_str) + + for key in manifests.iterkeys(): + if None in manifests[key]: + manifests[key] = None + else: + manifests[key] = "|".join(list(manifests[key])) + return manifests class RefTest(object): oldcwd = os.getcwd() + resolver_cls = ReftestResolver def __init__(self): self.update_mozinfo() self.lastTestSeen = 'reftest' self.haveDumpedScreen = False + self.resolver = self.resolver_cls() def update_mozinfo(self): """walk up directories to find mozinfo.json update the info""" @@ -160,30 +220,15 @@ class RefTest(object): "Get an absolute path relative to self.oldcwd." return os.path.normpath(os.path.join(self.oldcwd, os.path.expanduser(path))) - def getManifestPath(self, path): - "Get the path of the manifest, and for remote testing this function is subclassed to point to remote manifest" - path = self.getFullPath(path) - if os.path.isdir(path): - defaultManifestPath = os.path.join(path, 'reftest.list') - if os.path.exists(defaultManifestPath): - path = defaultManifestPath - else: - defaultManifestPath = os.path.join(path, 'crashtests.list') - if os.path.exists(defaultManifestPath): - path = defaultManifestPath - return path + def createReftestProfile(self, options, manifests, server='localhost', port=0, + profile_to_clone=None): + """Sets up a profile for reftest. - def makeJSString(self, s): - return '"%s"' % re.sub(r'([\\"])', r'\\\1', s) - - def createReftestProfile(self, options, manifest, server='localhost', port=0, - special_powers=True, profile_to_clone=None): - """ - Sets up a profile for reftest. - 'manifest' is the path to the reftest.list file we want to test with. This is used in - the remote subclass in remotereftest.py so we can write it to a preference for the - bootstrap extension. - """ + :param options: Object containing command line options + :param manifests: Dictionary of the form {manifest_path: [filters]} + :param server: Server name to use for http tests + :param profile_to_clone: Path to a profile to use as the basis for the + test profile""" locations = mozprofile.permissions.ServerLocations() locations.add_host(server, scheme='http', port=port) @@ -202,11 +247,10 @@ class RefTest(object): prefs['reftest.logFile'] = options.logFile if options.ignoreWindowSize: prefs['reftest.ignoreWindowSize'] = True - if options.filter: - prefs['reftest.filter'] = options.filter if options.shuffle: prefs['reftest.shuffle'] = True prefs['reftest.focusFilterMode'] = options.focusFilterMode + prefs['reftest.manifests'] = json.dumps(manifests) # Ensure that telemetry is disabled, so we don't connect to the telemetry # server in the middle of the tests. @@ -248,13 +292,10 @@ class RefTest(object): thispref[1].strip()) # install the reftest extension bits into the profile - addons = [] - addons.append(os.path.join(SCRIPT_DIRECTORY, "reftest")) + addons = [options.reftestExtensionPath] - # I would prefer to use "--install-extension reftest/specialpowers", but that requires tight coordination with - # release engineering and landing on multiple branches at once. - if special_powers and (manifest.endswith('crashtests.list') or manifest.endswith('jstests.list')): - addons.append(os.path.join(SCRIPT_DIRECTORY, 'specialpowers')) + if options.specialPowersExtensionPath is not None: + addons.append(options.specialPowersExtensionPath) # SpecialPowers requires insecure automation-only features that we # put behind a pref. prefs['security.turn_off_all_security_so_that_viruses_can_take_over_this_computer'] = True @@ -335,7 +376,7 @@ class RefTest(object): if profileDir: shutil.rmtree(profileDir, True) - def runTests(self, testPath, options, cmdlineArgs=None): + def runTests(self, tests, options, cmdlineArgs=None): # Despite our efforts to clean up servers started by this script, in practice # we still see infrequent cases where a process is orphaned and interferes # with future tests, typically because the old server is keeping the port in use. @@ -344,8 +385,12 @@ class RefTest(object): self.killNamedOrphans('ssltunnel') self.killNamedOrphans('xpcshell') - if not options.runTestsInParallel: - return self.runSerialTests(testPath, options, cmdlineArgs) + manifests = self.resolver.resolveManifests(options, tests) + if options.filter: + manifests[""] = options.filter + + if not hasattr(options, "runTestsInParallel") or not options.runTestsInParallel: + return self.runSerialTests(manifests, options, cmdlineArgs) cpuCount = multiprocessing.cpu_count() @@ -376,7 +421,6 @@ class RefTest(object): jobArgs.remove("--run-tests-in-parallel") except: pass - jobArgs.insert(-1, "--no-run-tests-in-parallel") jobArgs[0:0] = [sys.executable, "-u"] threads = [ReftestThread(args) for args in perProcessArgs[1:]] @@ -435,7 +479,7 @@ class RefTest(object): log.info("Not taking screenshot here: see the one that was previously logged") return self.haveDumpedScreen = True - dumpScreen(utilityPath) + dump_screen(utilityPath, log) def killAndGetStack(self, process, utilityPath, debuggerInfo, dump_screen=False): """ @@ -455,7 +499,7 @@ class RefTest(object): if os.path.exists(crashinject): status = subprocess.Popen( [crashinject, str(process.pid)]).wait() - printstatus(status, "crashinject") + printstatus("crashinject", status) if status == 0: return else: @@ -502,48 +546,9 @@ class RefTest(object): def stack_fixer(self): """ - return stackFixerFunction, if any, to use on the output lines + return get_stack_fixer_function, if any, to use on the output lines """ - - if not mozinfo.info.get('debug'): - return None - - stack_fixer_function = None - - def import_stack_fixer_module(module_name): - sys.path.insert(0, self.utilityPath) - module = __import__(module_name, globals(), locals(), []) - sys.path.pop(0) - return module - - if self.symbolsPath and os.path.exists(self.symbolsPath): - # Run each line through a function in fix_stack_using_bpsyms.py (uses breakpad symbol files). - # This method is preferred for Tinderbox builds, since native - # symbols may have been stripped. - stack_fixer_module = import_stack_fixer_module( - 'fix_stack_using_bpsyms') - stack_fixer_function = lambda line: stack_fixer_module.fixSymbols( - line, self.symbolsPath) - - elif mozinfo.isMac: - # Run each line through fix_macosx_stack.py (uses atos). - # This method is preferred for developer machines, so we don't - # have to run "make buildsymbols". - stack_fixer_module = import_stack_fixer_module( - 'fix_macosx_stack') - stack_fixer_function = lambda line: stack_fixer_module.fixSymbols( - line) - - elif mozinfo.isLinux: - # Run each line through fix_linux_stack.py (uses addr2line). - # This method is preferred for developer machines, so we don't - # have to run "make buildsymbols". - stack_fixer_module = import_stack_fixer_module( - 'fix_linux_stack') - stack_fixer_function = lambda line: stack_fixer_module.fixSymbols( - line) - - return stack_fixer_function + return get_stack_fixer_function(self.utilityPath, self.symbolsPath) # output line handlers: # these take a line and return a line @@ -573,7 +578,8 @@ class RefTest(object): def runApp(self, profile, binary, cmdargs, env, timeout=None, debuggerInfo=None, - symbolsPath=None, options=None): + symbolsPath=None, options=None, + valgrindPath=None, valgrindArgs=None, valgrindSuppFiles=None): def timeoutHandler(): self.handleTimeout( @@ -638,7 +644,7 @@ class RefTest(object): status = 1 return status - def runSerialTests(self, testPath, options, cmdlineArgs=None): + def runSerialTests(self, manifests, options, cmdlineArgs=None): debuggerInfo = None if options.debugger: debuggerInfo = mozdebug.get_debugger_info(options.debugger, options.debuggerArgs, @@ -646,10 +652,9 @@ class RefTest(object): profileDir = None try: - reftestlist = self.getManifestPath(testPath) if cmdlineArgs == None: - cmdlineArgs = ['-reftest', reftestlist] - profile = self.createReftestProfile(options, reftestlist) + cmdlineArgs = [] + profile = self.createReftestProfile(options, manifests) profileDir = profile.profile # name makes more sense # browser environment @@ -668,7 +673,10 @@ class RefTest(object): debuggerInfo=debuggerInfo) mozleak.process_leak_log(self.leakLogFile, leak_thresholds=options.leakThresholds, - log=log) + log=log, + stack_fixer=get_stack_fixer_function(options.utilityPath, + options.symbolsPath), + ) log.info("\nREFTEST INFO | runreftest.py | Running tests: end.") finally: self.cleanup(profileDir) @@ -694,209 +702,26 @@ class RefTest(object): continue -class ReftestOptions(OptionParser): - - def __init__(self): - OptionParser.__init__(self) - defaults = {} - self.add_option("--xre-path", - action="store", type="string", dest="xrePath", - # individual scripts will set a sane default - default=None, - help="absolute path to directory containing XRE (probably xulrunner)") - self.add_option("--symbols-path", - action="store", type="string", dest="symbolsPath", - default=None, - help="absolute path to directory containing breakpad symbols, or the URL of a zip file containing symbols") - self.add_option("--debugger", - action="store", dest="debugger", - help="use the given debugger to launch the application") - self.add_option("--debugger-args", - action="store", dest="debuggerArgs", - help="pass the given args to the debugger _before_ " - "the application on the command line") - self.add_option("--debugger-interactive", - action="store_true", dest="debuggerInteractive", - help="prevents the test harness from redirecting " - "stdout and stderr for interactive debuggers") - self.add_option("--appname", - action="store", type="string", dest="app", - help="absolute path to application, overriding default") - # Certain paths do not make sense when we're cross compiling Fennec. This - # logic is cribbed from the example in - # python/mozbuild/mozbuild/mach_commands.py. - defaults['app'] = build_obj.get_binary_path() if \ - build_obj and build_obj.substs['MOZ_BUILD_APP'] != 'mobile/android' else None - - self.add_option("--extra-profile-file", - action="append", dest="extraProfileFiles", - default=[], - help="copy specified files/dirs to testing profile") - self.add_option("--timeout", - action="store", dest="timeout", type="int", - default=5 * 60, # 5 minutes per bug 479518 - help="reftest will timeout in specified number of seconds. [default %default s].") - self.add_option("--leak-threshold", - action="store", type="int", dest="defaultLeakThreshold", - default=0, - help="fail if the number of bytes leaked in default " - "processes through refcounted objects (or bytes " - "in classes with MOZ_COUNT_CTOR and MOZ_COUNT_DTOR) " - "is greater than the given number") - self.add_option("--utility-path", - action="store", type="string", dest="utilityPath", - help="absolute path to directory containing utility " - "programs (xpcshell, ssltunnel, certutil)") - defaults["utilityPath"] = build_obj.bindir if \ - build_obj and build_obj.substs['MOZ_BUILD_APP'] != 'mobile/android' else None - - self.add_option("--total-chunks", - type="int", dest="totalChunks", - help="how many chunks to split the tests up into") - defaults["totalChunks"] = None - - self.add_option("--this-chunk", - type="int", dest="thisChunk", - help="which chunk to run between 1 and --total-chunks") - defaults["thisChunk"] = None - - self.add_option("--log-file", - action="store", type="string", dest="logFile", - default=None, - help="file to log output to in addition to stdout") - defaults["logFile"] = None - - self.add_option("--skip-slow-tests", - dest="skipSlowTests", action="store_true", - help="skip tests marked as slow when running") - defaults["skipSlowTests"] = False - - self.add_option("--ignore-window-size", - dest="ignoreWindowSize", action="store_true", - help="ignore the window size, which may cause spurious failures and passes") - defaults["ignoreWindowSize"] = False - - self.add_option("--install-extension", - action="append", dest="extensionsToInstall", - help="install the specified extension in the testing profile. " - "The extension file's name should be .xpi where is " - "the extension's id as indicated in its install.rdf. " - "An optional path can be specified too.") - defaults["extensionsToInstall"] = [] - - self.add_option("--run-tests-in-parallel", - action="store_true", dest="runTestsInParallel", - help="run tests in parallel if possible") - self.add_option("--no-run-tests-in-parallel", - action="store_false", dest="runTestsInParallel", - help="do not run tests in parallel") - defaults["runTestsInParallel"] = False - - self.add_option("--setenv", - action="append", type="string", - dest="environment", metavar="NAME=VALUE", - help="sets the given variable in the application's " - "environment") - defaults["environment"] = [] - - self.add_option("--filter", - action="store", type="string", dest="filter", - help="specifies a regular expression (as could be passed to the JS " - "RegExp constructor) to test against URLs in the reftest manifest; " - "only test items that have a matching test URL will be run.") - defaults["filter"] = None - - self.add_option("--shuffle", - action="store_true", dest="shuffle", - help="run reftests in random order") - defaults["shuffle"] = False - - self.add_option("--focus-filter-mode", - action="store", type="string", dest="focusFilterMode", - help="filters tests to run by whether they require focus. " - "Valid values are `all', `needs-focus', or `non-needs-focus'. " - "Defaults to `all'.") - defaults["focusFilterMode"] = "all" - - self.add_option("--e10s", - action="store_true", - dest="e10s", - help="enables content processes") - defaults["e10s"] = False - - self.add_option("--setpref", - action="append", type="string", - default=[], - dest="extraPrefs", metavar="PREF=VALUE", - help="defines an extra user preference") - - self.set_defaults(**defaults) - - def verifyCommonOptions(self, options, reftest): - if options.totalChunks is not None and options.thisChunk is None: - self.error("thisChunk must be specified when totalChunks is specified") - - if options.totalChunks: - if not 1 <= options.thisChunk <= options.totalChunks: - self.error("thisChunk must be between 1 and totalChunks") - - if options.logFile: - options.logFile = reftest.getFullPath(options.logFile) - - if options.xrePath is not None: - if not os.access(options.xrePath, os.F_OK): - self.error("--xre-path '%s' not found" % options.xrePath) - if not os.path.isdir(options.xrePath): - self.error("--xre-path '%s' is not a directory" % - options.xrePath) - options.xrePath = reftest.getFullPath(options.xrePath) - - if options.runTestsInParallel: - if options.logFile is not None: - self.error("cannot specify logfile with parallel tests") - if options.totalChunks is not None and options.thisChunk is None: - self.error("cannot specify thisChunk or totalChunks with parallel tests") - if options.focusFilterMode != "all": - self.error("cannot specify focusFilterMode with parallel tests") - if options.debugger is not None: - self.error("cannot specify a debugger with parallel tests") - - options.leakThresholds = { - "default": options.defaultLeakThreshold, - "tab": 5000, # See dependencies of bug 1051230. - } - - return options +def run(**kwargs): + # Mach gives us kwargs; this is a way to turn them back into an + # options object + parser = reftestcommandline.DesktopArgumentsParser() + reftest = RefTest() + parser.set_defaults(**kwargs) + options = parser.parse_args(kwargs["tests"]) + parser.validate(options, reftest) + return reftest.runTests(options.tests, options) def main(): - parser = ReftestOptions() + parser = reftestcommandline.DesktopArgumentsParser() reftest = RefTest() - options, args = parser.parse_args() - if len(args) != 1: - print >>sys.stderr, "No reftest.list specified." - sys.exit(1) + options = parser.parse_args() + parser.validate(options, reftest) - options = parser.verifyCommonOptions(options, reftest) - if options.app is None: - parser.error("could not find the application path, --appname must be specified") + sys.exit(reftest.runTests(options.tests, options)) - options.app = reftest.getFullPath(options.app) - if not os.path.exists(options.app): - print """Error: Path %(app)s doesn't exist. -Are you executing $objdir/_tests/reftest/runreftest.py?""" \ - % {"app": options.app} - sys.exit(1) - - if options.xrePath is None: - options.xrePath = os.path.dirname(options.app) - - if options.symbolsPath and len(urlparse(options.symbolsPath).scheme) < 2: - options.symbolsPath = reftest.getFullPath(options.symbolsPath) - options.utilityPath = reftest.getFullPath(options.utilityPath) - - sys.exit(reftest.runTests(args[0], options)) if __name__ == "__main__": main() diff --git a/layout/tools/reftest/runreftestb2g.py b/layout/tools/reftest/runreftestb2g.py index a40565de18..6ec0069a82 100644 --- a/layout/tools/reftest/runreftestb2g.py +++ b/layout/tools/reftest/runreftestb2g.py @@ -10,207 +10,18 @@ import traceback # We need to know our current directory so that we can serve our test files from it. here = os.path.abspath(os.path.dirname(__file__)) +if here not in sys.path: + sys.path.insert(0, here) from automation import Automation from b2gautomation import B2GRemoteAutomation from b2g_desktop import run_desktop_reftests +from remotereftest import RemoteReftestResolver, ReftestServer from runreftest import RefTest -from runreftest import ReftestOptions -from remotereftest import ReftestServer +import reftestcommandline from mozdevice import DeviceManagerADB, DMError from marionette import Marionette -import moznetwork - -class B2GOptions(ReftestOptions): - - def __init__(self, **kwargs): - defaults = {} - ReftestOptions.__init__(self) - # This is only used for procName in run_remote_reftests. - defaults["app"] = Automation.DEFAULT_APP - - self.add_option("--browser-arg", action="store", - type = "string", dest = "browser_arg", - help = "Optional command-line arg to pass to the browser") - defaults["browser_arg"] = None - - self.add_option("--b2gpath", action="store", - type = "string", dest = "b2gPath", - help = "path to B2G repo or qemu dir") - defaults["b2gPath"] = None - - self.add_option("--marionette", action="store", - type = "string", dest = "marionette", - help = "host:port to use when connecting to Marionette") - defaults["marionette"] = None - - self.add_option("--emulator", action="store", - type="string", dest = "emulator", - help = "Architecture of emulator to use: x86 or arm") - defaults["emulator"] = None - self.add_option("--emulator-res", action="store", - type="string", dest = "emulator_res", - help = "Emulator resolution of the format 'x'") - defaults["emulator_res"] = None - - self.add_option("--no-window", action="store_true", - dest = "noWindow", - help = "Pass --no-window to the emulator") - defaults["noWindow"] = False - - self.add_option("--adbpath", action="store", - type = "string", dest = "adb_path", - help = "path to adb") - defaults["adb_path"] = "adb" - - self.add_option("--deviceIP", action="store", - type = "string", dest = "deviceIP", - help = "ip address of remote device to test") - defaults["deviceIP"] = None - - self.add_option("--devicePort", action="store", - type = "string", dest = "devicePort", - help = "port of remote device to test") - defaults["devicePort"] = 20701 - - self.add_option("--remote-logfile", action="store", - type = "string", dest = "remoteLogFile", - help = "Name of log file on the device relative to the device root. PLEASE ONLY USE A FILENAME.") - defaults["remoteLogFile"] = None - - self.add_option("--remote-webserver", action = "store", - type = "string", dest = "remoteWebServer", - help = "ip address where the remote web server is hosted at") - defaults["remoteWebServer"] = None - - self.add_option("--http-port", action = "store", - type = "string", dest = "httpPort", - help = "ip address where the remote web server is hosted at") - defaults["httpPort"] = None - - self.add_option("--ssl-port", action = "store", - type = "string", dest = "sslPort", - help = "ip address where the remote web server is hosted at") - defaults["sslPort"] = None - - self.add_option("--pidfile", action = "store", - type = "string", dest = "pidFile", - help = "name of the pidfile to generate") - defaults["pidFile"] = "" - self.add_option("--gecko-path", action="store", - type="string", dest="geckoPath", - help="the path to a gecko distribution that should " - "be installed on the emulator prior to test") - defaults["geckoPath"] = None - self.add_option("--logdir", action="store", - type="string", dest="logdir", - help="directory to store log files") - defaults["logdir"] = None - self.add_option('--busybox', action='store', - type='string', dest='busybox', - help="Path to busybox binary to install on device") - defaults['busybox'] = None - self.add_option("--httpd-path", action = "store", - type = "string", dest = "httpdPath", - help = "path to the httpd.js file") - defaults["httpdPath"] = None - self.add_option("--profile", action="store", - type="string", dest="profile", - help="for desktop testing, the path to the " - "gaia profile to use") - defaults["profile"] = None - self.add_option("--desktop", action="store_true", - dest="desktop", - help="Run the tests on a B2G desktop build") - defaults["desktop"] = False - self.add_option("--mulet", action="store_true", - dest="mulet", - help="Run the tests on a B2G desktop build") - defaults["mulet"] = False - self.add_option("--enable-oop", action="store_true", - dest="oop", - help="Run the tests out of process") - defaults["oop"] = False - defaults["remoteTestRoot"] = None - defaults["logFile"] = "reftest.log" - defaults["autorun"] = True - defaults["closeWhenDone"] = True - defaults["testPath"] = "" - defaults["runTestsInParallel"] = False - - self.set_defaults(**defaults) - - def verifyRemoteOptions(self, options, auto): - if options.runTestsInParallel: - self.error("Cannot run parallel tests here") - - if not options.remoteTestRoot: - options.remoteTestRoot = auto._devicemanager.deviceRoot + "/reftest" - - options.remoteProfile = options.remoteTestRoot + "/profile" - - productRoot = options.remoteTestRoot + "/" + auto._product - if options.utilityPath is None: - options.utilityPath = productRoot + "/bin" - - if options.remoteWebServer == None: - if os.name != "nt": - options.remoteWebServer = moznetwork.get_ip() - else: - print "ERROR: you must specify a --remote-webserver=\n" - return None - - options.webServer = options.remoteWebServer - - if not options.httpPort: - options.httpPort = auto.DEFAULT_HTTP_PORT - - if not options.sslPort: - options.sslPort = auto.DEFAULT_SSL_PORT - - if options.geckoPath and not options.emulator: - self.error("You must specify --emulator if you specify --gecko-path") - - if options.logdir and not options.emulator: - self.error("You must specify --emulator if you specify --logdir") - - #if not options.emulator and not options.deviceIP: - # print "ERROR: you must provide a device IP" - # return None - - if options.remoteLogFile == None: - options.remoteLogFile = "reftest.log" - - options.localLogName = options.remoteLogFile - options.remoteLogFile = options.remoteTestRoot + '/' + options.remoteLogFile - - # Ensure that the options.logfile (which the base class uses) is set to - # the remote setting when running remote. Also, if the user set the - # log file name there, use that instead of reusing the remotelogfile as above. - if (options.logFile): - # If the user specified a local logfile name use that - options.localLogName = options.logFile - options.logFile = options.remoteLogFile - - # Only reset the xrePath if it wasn't provided - if options.xrePath == None: - options.xrePath = options.utilityPath - options.xrePath = os.path.abspath(options.xrePath) - - if options.pidFile != "": - f = open(options.pidFile, 'w') - f.write("%s" % os.getpid()) - f.close() - - # httpd-path is specified by standard makefile targets and may be specified - # on the command line to select a particular version of httpd.js. If not - # specified, try to select the one from from the xre bundle, as required in bug 882932. - if not options.httpdPath: - options.httpdPath = os.path.join(options.xrePath, "components") - - return options - class ProfileConfigParser(ConfigParser.RawConfigParser): """Subclass of RawConfigParser that outputs .ini files in the exact @@ -243,6 +54,7 @@ class B2GRemoteReftest(RefTest): localProfile = None remoteApp = '' profile = None + resolver_cls = RemoteReftestResolver def __init__(self, automation, devicemanager, options, scriptDir): RefTest.__init__(self) @@ -418,10 +230,9 @@ class B2GRemoteReftest(RefTest): pass - def createReftestProfile(self, options, reftestlist): - profile = RefTest.createReftestProfile(self, options, reftestlist, - server=options.remoteWebServer, - special_powers=False) + def createReftestProfile(self, options, manifests): + profile = RefTest.createReftestProfile(self, options, manifests, + server=options.remoteWebServer) profileDir = profile.profile prefs = {} @@ -437,7 +248,7 @@ class B2GRemoteReftest(RefTest): prefs["network.dns.localDomains"] = "app://test-container.gaiamobile.org" prefs["reftest.browser.iframe.enabled"] = False prefs["reftest.remote"] = True - prefs["reftest.uri"] = "%s" % reftestlist + # Set a future policy version to avoid the telemetry prompt. prefs["toolkit.telemetry.prompted"] = 999 prefs["toolkit.telemetry.notifiedOptOut"] = 999 @@ -493,15 +304,13 @@ class B2GRemoteReftest(RefTest): print "Automation Error: Failed to copy extra files to device" raise - def getManifestPath(self, path): - return path - def environment(self, **kwargs): return self.automation.environment(**kwargs) def runApp(self, profile, binary, cmdargs, env, timeout=None, debuggerInfo=None, - symbolsPath=None, options=None): + symbolsPath=None, options=None, + valgrindPath=None, valgrindArgs=None, valgrindSuppFiles=None): status = self.automation.runApp(None, env, binary, profile.profile, @@ -514,7 +323,7 @@ class B2GRemoteReftest(RefTest): return status -def run_remote_reftests(parser, options, args): +def run_remote_reftests(parser, options): auto = B2GRemoteAutomation(None, "fennec", context_chrome=True) # create our Marionette instance @@ -557,11 +366,7 @@ def run_remote_reftests(parser, options, args): dm = DeviceManagerADB(**kwargs) auto.setDeviceManager(dm) - options = parser.verifyRemoteOptions(options, auto) - - if (options == None): - print "ERROR: Invalid options specified, use --help for a list of valid options" - sys.exit(1) + parser.validate_remote(options, auto) # TODO fix exception if not options.ignoreWindowSize: @@ -578,7 +383,7 @@ def run_remote_reftests(parser, options, args): auto.logFinish = "REFTEST TEST-START | Shutdown" reftest = B2GRemoteReftest(auto, dm, options, here) - options = parser.verifyCommonOptions(options, reftest) + parser.validate(options, reftest) logParent = os.path.dirname(options.remoteLogFile) dm.mkDir(logParent); @@ -588,16 +393,6 @@ def run_remote_reftests(parser, options, args): # Hack in a symbolic link for jsreftest os.system("ln -s %s %s" % (os.path.join('..', 'jsreftest'), os.path.join(here, 'jsreftest'))) - # Dynamically build the reftest URL if possible, beware that args[0] should exist 'inside' the webroot - manifest = args[0] - if os.path.exists(os.path.join(here, args[0])): - manifest = "http://%s:%s/%s" % (options.remoteWebServer, options.httpPort, args[0]) - elif os.path.exists(args[0]): - manifestPath = os.path.abspath(args[0]).split(here)[1].strip('/') - manifest = "http://%s:%s/%s" % (options.remoteWebServer, options.httpPort, manifestPath) - else: - print "ERROR: Could not find test manifest '%s'" % manifest - return 1 # Start the webserver retVal = 1 @@ -609,11 +404,7 @@ def run_remote_reftests(parser, options, args): if (dm.processExist(procName)): dm.killProcess(procName) - cmdlineArgs = ["-reftest", manifest] - if getattr(options, 'bootstrap', False): - cmdlineArgs = [] - - retVal = reftest.runTests(manifest, options, cmdlineArgs) + retVal = reftest.runTests(options.tests, options) except: print "Automation Error: Exception caught while running tests" traceback.print_exc() @@ -627,13 +418,21 @@ def run_remote_reftests(parser, options, args): reftest.stopWebServer(options) return retVal -def main(args=sys.argv[1:]): - parser = B2GOptions() - options, args = parser.parse_args(args) +def run_remote(**kwargs): + # Tests need to be served from a subdirectory of the server. Symlink + # topsrcdir here to get around this. + parser = reftestcommandline.B2GArgumentParser() + parser.set_defaults(**kwargs) + options = parser.parse_args(kwargs["tests"]) + return run_remote_reftests(parser, options) + +def main(): + parser = reftestcommandline.B2GArgumentParser() + options = parser.parse_args() if options.desktop or options.mulet: - return run_desktop_reftests(parser, options, args) - return run_remote_reftests(parser, options, args) + return run_desktop_reftests(parser, options) + return run_remote_reftests(parser, options) if __name__ == "__main__": diff --git a/netwerk/test/unit/test_bug826063.js b/netwerk/test/unit/test_bug826063.js index 5abee9fba1..ec886a09b4 100644 --- a/netwerk/test/unit/test_bug826063.js +++ b/netwerk/test/unit/test_bug826063.js @@ -19,6 +19,7 @@ function LoadContext(usePrivateBrowsing) { this.usePrivateBrowsing = usePrivateBrowsing; } LoadContext.prototype = { + originAttributes: {}, QueryInterface: XPCOMUtils.generateQI([Ci.nsILoadContext, Ci.nsIInterfaceRequestor]), getInterface: XPCOMUtils.generateQI([Ci.nsILoadContext]) }; diff --git a/netwerk/test/unit/test_cacheflags.js b/netwerk/test/unit/test_cacheflags.js index 050546edea..84e9362a15 100644 --- a/netwerk/test/unit/test_cacheflags.js +++ b/netwerk/test/unit/test_cacheflags.js @@ -20,6 +20,7 @@ function LoadContext(usePrivateBrowsing) { } LoadContext.prototype = { + originAttributes: {}, usePrivateBrowsing: false, // don't bother defining rest of nsILoadContext fields: don't need 'em diff --git a/services/cloudsync/tests/xpcshell/head.js b/services/cloudsync/tests/xpcshell/head.js index e5b2b9001a..bd517cafaa 100644 --- a/services/cloudsync/tests/xpcshell/head.js +++ b/services/cloudsync/tests/xpcshell/head.js @@ -1,7 +1,7 @@ /* Any copyright is dedicated to the Public Domain. * http://creativecommons.org/publicdomain/zero/1.0/ */ -const {classes: Cc, interfaces: Ci, results: Cr, utils: Cu} = Components; +var {classes: Cc, interfaces: Ci, results: Cr, utils: Cu} = Components; "use strict"; diff --git a/services/common/modules-testing/bagheeraserver.js b/services/common/modules-testing/bagheeraserver.js index 9feea746e5..81328b40bf 100644 --- a/services/common/modules-testing/bagheeraserver.js +++ b/services/common/modules-testing/bagheeraserver.js @@ -4,7 +4,7 @@ "use strict"; -const {utils: Cu} = Components; +var {utils: Cu} = Components; this.EXPORTED_SYMBOLS = ["BagheeraServer"]; diff --git a/services/common/modules-testing/logging.js b/services/common/modules-testing/logging.js index 604cc48a3f..3ff2c396c4 100644 --- a/services/common/modules-testing/logging.js +++ b/services/common/modules-testing/logging.js @@ -9,7 +9,7 @@ this.EXPORTED_SYMBOLS = [ "initTestLogging", ]; -const {utils: Cu} = Components; +var {utils: Cu} = Components; Cu.import("resource://gre/modules/Log.jsm"); diff --git a/services/common/modules-testing/storageserver.js b/services/common/modules-testing/storageserver.js index 0f5446f6b9..94c787c4e2 100644 --- a/services/common/modules-testing/storageserver.js +++ b/services/common/modules-testing/storageserver.js @@ -8,7 +8,7 @@ * The server should not be used for any production purposes. */ -const {classes: Cc, interfaces: Ci, utils: Cu} = Components; +var {classes: Cc, interfaces: Ci, utils: Cu} = Components; this.EXPORTED_SYMBOLS = [ "ServerBSO", diff --git a/services/common/tests/unit/head_global.js b/services/common/tests/unit/head_global.js index 642e536b42..b278780bb2 100644 --- a/services/common/tests/unit/head_global.js +++ b/services/common/tests/unit/head_global.js @@ -1,7 +1,7 @@ /* Any copyright is dedicated to the Public Domain. * http://creativecommons.org/publicdomain/zero/1.0/ */ -const {classes: Cc, interfaces: Ci, results: Cr, utils: Cu, manager: Cm} = Components; +var {classes: Cc, interfaces: Ci, results: Cr, utils: Cu, manager: Cm} = Components; var gSyncProfile = do_get_profile(); diff --git a/services/crypto/component/tests/unit/test_jpake.js b/services/crypto/component/tests/unit/test_jpake.js index 8e6a9b0948..4e9b25e1b2 100644 --- a/services/crypto/component/tests/unit/test_jpake.js +++ b/services/crypto/component/tests/unit/test_jpake.js @@ -1,5 +1,5 @@ -const Cc = Components.classes; -const Ci = Components.interfaces; +var Cc = Components.classes; +var Ci = Components.interfaces; // Ensure PSM is initialized. Cc["@mozilla.org/psm;1"].getService(Ci.nsISupports); diff --git a/services/crypto/tests/unit/head_helpers.js b/services/crypto/tests/unit/head_helpers.js index a4b48650f5..3b547cc37a 100644 --- a/services/crypto/tests/unit/head_helpers.js +++ b/services/crypto/tests/unit/head_helpers.js @@ -1,7 +1,7 @@ -const Cc = Components.classes; -const Ci = Components.interfaces; -const Cr = Components.results; -const Cu = Components.utils; +var Cc = Components.classes; +var Ci = Components.interfaces; +var Cr = Components.results; +var Cu = Components.utils; Cu.import("resource://gre/modules/XPCOMUtils.jsm"); diff --git a/services/healthreport/tests/xpcshell/test_healthreporter.js b/services/healthreport/tests/xpcshell/test_healthreporter.js index 835730a4c8..8057639030 100644 --- a/services/healthreport/tests/xpcshell/test_healthreporter.js +++ b/services/healthreport/tests/xpcshell/test_healthreporter.js @@ -3,7 +3,7 @@ "use strict"; -const {classes: Cc, interfaces: Ci, utils: Cu} = Components; +var {classes: Cc, interfaces: Ci, utils: Cu} = Components; Cu.import("resource://services-common/observers.js"); Cu.import("resource://services-common/utils.js"); @@ -11,7 +11,7 @@ Cu.import("resource://gre/modules/Promise.jsm"); Cu.import("resource://gre/modules/Metrics.jsm"); Cu.import("resource://gre/modules/osfile.jsm"); Cu.import("resource://gre/modules/Preferences.jsm"); -let bsp = Cu.import("resource://gre/modules/services/healthreport/healthreporter.jsm"); +var bsp = Cu.import("resource://gre/modules/services/healthreport/healthreporter.jsm"); Cu.import("resource://gre/modules/services/healthreport/providers.jsm"); Cu.import("resource://gre/modules/services/datareporting/policy.jsm"); Cu.import("resource://gre/modules/Services.jsm"); diff --git a/services/healthreport/tests/xpcshell/test_profile.js b/services/healthreport/tests/xpcshell/test_profile.js index 4458d17a74..c26f064c5c 100644 --- a/services/healthreport/tests/xpcshell/test_profile.js +++ b/services/healthreport/tests/xpcshell/test_profile.js @@ -3,13 +3,13 @@ "use strict"; -const {utils: Cu} = Components; +var {utils: Cu} = Components; const MILLISECONDS_PER_DAY = 24 * 60 * 60 * 1000; // Create profile directory before use. // It can be no older than a day ago…. -let profile_creation_lower = Date.now() - MILLISECONDS_PER_DAY; +var profile_creation_lower = Date.now() - MILLISECONDS_PER_DAY; do_get_profile(); Cu.import("resource://gre/modules/Promise.jsm"); diff --git a/services/healthreport/tests/xpcshell/test_provider_addons.js b/services/healthreport/tests/xpcshell/test_provider_addons.js index e5775b5740..92a8137e4d 100644 --- a/services/healthreport/tests/xpcshell/test_provider_addons.js +++ b/services/healthreport/tests/xpcshell/test_provider_addons.js @@ -3,7 +3,7 @@ "use strict"; -const {utils: Cu, classes: Cc, interfaces: Ci} = Components; +var {utils: Cu, classes: Cc, interfaces: Ci} = Components; Cu.import("resource://gre/modules/Metrics.jsm"); @@ -12,7 +12,7 @@ Cu.import("resource://gre/modules/services/healthreport/providers.jsm"); // The hack, it burns. This could go away if extensions code exposed its // test environment setup functions as a testing-only JSM. See similar // code in Sync's head_helpers.js. -let gGlobalScope = this; +var gGlobalScope = this; function loadAddonManager() { let ns = {}; Cu.import("resource://gre/modules/Services.jsm", ns); diff --git a/services/healthreport/tests/xpcshell/test_provider_appinfo.js b/services/healthreport/tests/xpcshell/test_provider_appinfo.js index 6f51151104..cf5082284e 100644 --- a/services/healthreport/tests/xpcshell/test_provider_appinfo.js +++ b/services/healthreport/tests/xpcshell/test_provider_appinfo.js @@ -3,7 +3,7 @@ "use strict"; -const {interfaces: Ci, results: Cr, utils: Cu, classes: Cc} = Components; +var {interfaces: Ci, results: Cr, utils: Cu, classes: Cc} = Components; Cu.import("resource://gre/modules/Metrics.jsm"); Cu.import("resource://gre/modules/Services.jsm"); diff --git a/services/healthreport/tests/xpcshell/test_provider_hotfix.js b/services/healthreport/tests/xpcshell/test_provider_hotfix.js index fab6c7c38e..1657455962 100644 --- a/services/healthreport/tests/xpcshell/test_provider_hotfix.js +++ b/services/healthreport/tests/xpcshell/test_provider_hotfix.js @@ -3,7 +3,7 @@ "use strict"; -const {utils: Cu} = Components; +var {utils: Cu} = Components; Cu.import("resource://gre/modules/osfile.jsm"); Cu.import("resource://gre/modules/Metrics.jsm"); diff --git a/services/healthreport/tests/xpcshell/test_provider_places.js b/services/healthreport/tests/xpcshell/test_provider_places.js index c883c15b21..2712d8b883 100644 --- a/services/healthreport/tests/xpcshell/test_provider_places.js +++ b/services/healthreport/tests/xpcshell/test_provider_places.js @@ -3,7 +3,7 @@ "use strict"; -const {utils: Cu} = Components; +var {utils: Cu} = Components; Cu.import("resource://gre/modules/Metrics.jsm"); Cu.import("resource://gre/modules/services/healthreport/providers.jsm"); diff --git a/services/healthreport/tests/xpcshell/test_provider_searches.js b/services/healthreport/tests/xpcshell/test_provider_searches.js index a95872655c..b3f94a7874 100644 --- a/services/healthreport/tests/xpcshell/test_provider_searches.js +++ b/services/healthreport/tests/xpcshell/test_provider_searches.js @@ -3,11 +3,11 @@ "use strict"; -const {utils: Cu} = Components; +var {utils: Cu} = Components; Cu.import("resource://gre/modules/Metrics.jsm"); Cu.import("resource://gre/modules/Services.jsm"); -let bsp = Cu.import("resource://gre/modules/services/healthreport/providers.jsm"); +var bsp = Cu.import("resource://gre/modules/services/healthreport/providers.jsm"); const DEFAULT_ENGINES = [ {name: "Amazon.com", identifier: "amazondotcom"}, diff --git a/services/healthreport/tests/xpcshell/test_provider_sessions.js b/services/healthreport/tests/xpcshell/test_provider_sessions.js index 59e190d0eb..d56b708dd8 100644 --- a/services/healthreport/tests/xpcshell/test_provider_sessions.js +++ b/services/healthreport/tests/xpcshell/test_provider_sessions.js @@ -3,7 +3,7 @@ "use strict"; -const {utils: Cu} = Components; +var {utils: Cu} = Components; Cu.import("resource://gre/modules/Promise.jsm"); diff --git a/services/healthreport/tests/xpcshell/test_provider_sysinfo.js b/services/healthreport/tests/xpcshell/test_provider_sysinfo.js index 25670ac17a..0a98555649 100644 --- a/services/healthreport/tests/xpcshell/test_provider_sysinfo.js +++ b/services/healthreport/tests/xpcshell/test_provider_sysinfo.js @@ -3,7 +3,7 @@ "use strict"; -const {interfaces: Ci, results: Cr, utils: Cu} = Components; +var {interfaces: Ci, results: Cr, utils: Cu} = Components; Cu.import("resource://gre/modules/Metrics.jsm"); Cu.import("resource://gre/modules/XPCOMUtils.jsm"); diff --git a/services/metrics/tests/xpcshell/test_metrics_provider.js b/services/metrics/tests/xpcshell/test_metrics_provider.js index 46d0d51518..2ee6631824 100644 --- a/services/metrics/tests/xpcshell/test_metrics_provider.js +++ b/services/metrics/tests/xpcshell/test_metrics_provider.js @@ -3,7 +3,7 @@ "use strict"; -const {utils: Cu} = Components; +var {utils: Cu} = Components; Cu.import("resource://gre/modules/Metrics.jsm"); Cu.import("resource://gre/modules/Preferences.jsm"); diff --git a/services/metrics/tests/xpcshell/test_metrics_provider_manager.js b/services/metrics/tests/xpcshell/test_metrics_provider_manager.js index bbcb448af4..bad8e67bcc 100644 --- a/services/metrics/tests/xpcshell/test_metrics_provider_manager.js +++ b/services/metrics/tests/xpcshell/test_metrics_provider_manager.js @@ -3,7 +3,7 @@ "use strict"; -const {classes: Cc, interfaces: Ci, utils: Cu} = Components; +var {classes: Cc, interfaces: Ci, utils: Cu} = Components; Cu.import("resource://gre/modules/Promise.jsm"); Cu.import("resource://gre/modules/Metrics.jsm"); diff --git a/services/metrics/tests/xpcshell/test_metrics_storage.js b/services/metrics/tests/xpcshell/test_metrics_storage.js index 0fa49ba705..2ee2ad6d2c 100644 --- a/services/metrics/tests/xpcshell/test_metrics_storage.js +++ b/services/metrics/tests/xpcshell/test_metrics_storage.js @@ -3,7 +3,7 @@ "use strict"; -const {utils: Cu} = Components; +var {utils: Cu} = Components; Cu.import("resource://gre/modules/Promise.jsm"); Cu.import("resource://gre/modules/Metrics.jsm"); diff --git a/services/mobileid/MobileIdentityManager.jsm b/services/mobileid/MobileIdentityManager.jsm index 52cf73acd5..b3a731f1fd 100644 --- a/services/mobileid/MobileIdentityManager.jsm +++ b/services/mobileid/MobileIdentityManager.jsm @@ -233,8 +233,9 @@ this.MobileIdentityManager = { } return this._iccInfo; -#endif +#else return null; +#endif }, get iccIds() { @@ -253,8 +254,9 @@ this.MobileIdentityManager = { } return this._iccIds; -#endif +#else return null; +#endif }, get credStore() { diff --git a/storage/mozStorageBindingParams.cpp b/storage/mozStorageBindingParams.cpp index 4de803eaba..526f177fe2 100644 --- a/storage/mozStorageBindingParams.cpp +++ b/storage/mozStorageBindingParams.cpp @@ -155,49 +155,6 @@ BindingParams::getOwner() const return mOwningArray; } -PLDHashOperator -AsyncBindingParams::iterateOverNamedParameters(const nsACString &aName, - nsIVariant *aValue, - void *voidClosureThunk) -{ - NamedParameterIterationClosureThunk *closureThunk = - static_cast(voidClosureThunk); - - // We do not accept any forms of names other than ":name", but we need to add - // the colon for SQLite. - nsAutoCString name(":"); - name.Append(aName); - int oneIdx = ::sqlite3_bind_parameter_index(closureThunk->statement, - name.get()); - - if (oneIdx == 0) { - nsAutoCString errMsg(aName); - errMsg.AppendLiteral(" is not a valid named parameter."); - closureThunk->err = new Error(SQLITE_RANGE, errMsg.get()); - return PL_DHASH_STOP; - } - - // XPCVariant's AddRef and Release are not thread-safe and so we must not do - // anything that would invoke them here on the async thread. As such we can't - // cram aValue into self->mParameters using ReplaceObjectAt so that we can - // freeload off of the BindingParams::Bind implementation. - int rc = variantToSQLiteT(BindingColumnData(closureThunk->statement, - oneIdx - 1), - aValue); - if (rc != SQLITE_OK) { - // We had an error while trying to bind. Now we need to create an error - // object with the right message. Note that we special case - // SQLITE_MISMATCH, but otherwise get the message from SQLite. - const char *msg = "Could not covert nsIVariant to SQLite type."; - if (rc != SQLITE_MISMATCH) - msg = ::sqlite3_errmsg(::sqlite3_db_handle(closureThunk->statement)); - - closureThunk->err = new Error(rc, msg); - return PL_DHASH_STOP; - } - return PL_DHASH_NEXT; -} - //////////////////////////////////////////////////////////////////////////////// //// nsISupports @@ -241,13 +198,44 @@ AsyncBindingParams::bind(sqlite3_stmt * aStatement) if (!mNamedParameters.Count()) return BindingParams::bind(aStatement); - // Enumerate over everyone in the map, propagating them into mParameters if - // we can and creating an error immediately when we cannot. - NamedParameterIterationClosureThunk closureThunk = {this, aStatement, nullptr}; - (void)mNamedParameters.EnumerateRead(iterateOverNamedParameters, - (void *)&closureThunk); + nsCOMPtr err; - return closureThunk.err.forget(); + for (auto iter = mNamedParameters.Iter(); !iter.Done(); iter.Next()) { + const nsACString &key = iter.Key(); + + // We do not accept any forms of names other than ":name", but we need to + // add the colon for SQLite. + nsAutoCString name(":"); + name.Append(key); + int oneIdx = ::sqlite3_bind_parameter_index(aStatement, name.get()); + + if (oneIdx == 0) { + nsAutoCString errMsg(key); + errMsg.AppendLiteral(" is not a valid named parameter."); + err = new Error(SQLITE_RANGE, errMsg.get()); + break; + } + + // XPCVariant's AddRef and Release are not thread-safe and so we must not + // do anything that would invoke them here on the async thread. As such we + // can't cram aValue into mParameters using ReplaceObjectAt so that + // we can freeload off of the BindingParams::Bind implementation. + int rc = variantToSQLiteT(BindingColumnData(aStatement, oneIdx - 1), + iter.UserData()); + if (rc != SQLITE_OK) { + // We had an error while trying to bind. Now we need to create an error + // object with the right message. Note that we special case + // SQLITE_MISMATCH, but otherwise get the message from SQLite. + const char *msg = "Could not covert nsIVariant to SQLite type."; + if (rc != SQLITE_MISMATCH) { + msg = ::sqlite3_errmsg(::sqlite3_db_handle(aStatement)); + } + err = new Error(rc, msg); + break; + } + } + + return err.forget(); } diff --git a/storage/mozStorageBindingParams.h b/storage/mozStorageBindingParams.h index fa08ebb2fc..86d00b02b6 100644 --- a/storage/mozStorageBindingParams.h +++ b/storage/mozStorageBindingParams.h @@ -105,17 +105,6 @@ public: private: nsInterfaceHashtable mNamedParameters; - - struct NamedParameterIterationClosureThunk - { - AsyncBindingParams *self; - sqlite3_stmt *statement; - nsCOMPtr err; - }; - - static PLDHashOperator iterateOverNamedParameters(const nsACString &aName, - nsIVariant *aValue, - void *voidClosureThunk); }; } // namespace storage diff --git a/storage/mozStorageConnection.cpp b/storage/mozStorageConnection.cpp index f5aad00c4e..31ead70cf0 100644 --- a/storage/mozStorageConnection.cpp +++ b/storage/mozStorageConnection.cpp @@ -194,48 +194,6 @@ void tracefunc (void *aClosure, const char *aStmt) aStmt)); } -struct FFEArguments -{ - nsISupports *target; - bool found; -}; -PLDHashOperator -findFunctionEnumerator(const nsACString &aKey, - Connection::FunctionInfo aData, - void *aUserArg) -{ - FFEArguments *args = static_cast(aUserArg); - if (aData.function == args->target) { - args->found = true; - return PL_DHASH_STOP; - } - return PL_DHASH_NEXT; -} - -PLDHashOperator -copyFunctionEnumerator(const nsACString &aKey, - Connection::FunctionInfo aData, - void *aUserArg) -{ - NS_PRECONDITION(aData.type == Connection::FunctionInfo::SIMPLE || - aData.type == Connection::FunctionInfo::AGGREGATE, - "Invalid function type!"); - - Connection *connection = static_cast(aUserArg); - if (aData.type == Connection::FunctionInfo::SIMPLE) { - mozIStorageFunction *function = - static_cast(aData.function.get()); - (void)connection->CreateFunction(aKey, aData.numArgs, function); - } - else { - mozIStorageAggregateFunction *function = - static_cast(aData.function.get()); - (void)connection->CreateAggregateFunction(aKey, aData.numArgs, function); - } - - return PL_DHASH_NEXT; -} - void basicFunctionHelper(sqlite3_context *aCtx, int aArgc, @@ -864,9 +822,13 @@ bool Connection::findFunctionByInstance(nsISupports *aInstance) { sharedDBMutex.assertCurrentThreadOwns(); - FFEArguments args = { aInstance, false }; - (void)mFunctions.EnumerateRead(findFunctionEnumerator, &args); - return args.found; + + for (auto iter = mFunctions.Iter(); !iter.Done(); iter.Next()) { + if (iter.UserData().function == aInstance) { + return true; + } + } + return false; } /* static */ int @@ -1352,7 +1314,31 @@ Connection::initializeClone(Connection* aClone, bool aReadOnly) // Copy any functions that have been added to this connection. SQLiteMutexAutoLock lockedScope(sharedDBMutex); - (void)mFunctions.EnumerateRead(copyFunctionEnumerator, aClone); + for (auto iter = mFunctions.Iter(); !iter.Done(); iter.Next()) { + const nsACString &key = iter.Key(); + Connection::FunctionInfo data = iter.UserData(); + + MOZ_ASSERT(data.type == Connection::FunctionInfo::SIMPLE || + data.type == Connection::FunctionInfo::AGGREGATE, + "Invalid function type!"); + + if (data.type == Connection::FunctionInfo::SIMPLE) { + mozIStorageFunction *function = + static_cast(data.function.get()); + rv = aClone->CreateFunction(key, data.numArgs, function); + if (NS_FAILED(rv)) { + NS_WARNING("Failed to copy function to cloned connection"); + } + + } else { + mozIStorageAggregateFunction *function = + static_cast(data.function.get()); + rv = aClone->CreateAggregateFunction(key, data.numArgs, function); + if (NS_FAILED(rv)) { + NS_WARNING("Failed to copy aggregate function to cloned connection"); + } + } + } return NS_OK; } diff --git a/storage/mozStorageSQLFunctions.cpp b/storage/mozStorageSQLFunctions.cpp index ebe4b4fdf4..71f9259823 100644 --- a/storage/mozStorageSQLFunctions.cpp +++ b/storage/mozStorageSQLFunctions.cpp @@ -116,48 +116,6 @@ likeCompare(nsAString::const_iterator aPatternItr, return aStringItr == aStringEnd; } -/** - * This class manages a dynamic array. It can represent an array of any - * reasonable size, but if the array is "N" elements or smaller, it will be - * stored using fixed space inside the auto array itself. If the auto array - * is a local variable, this internal storage will be allocated cheaply on the - * stack, similar to nsAutoString. If a larger size is requested, the memory - * will be dynamically allocated from the heap. Since the destructor will - * free any heap-allocated memory, client code doesn't need to care where the - * memory came from. - */ -template class AutoArray -{ - -public: - - explicit AutoArray(size_t size) - : mBuffer(size <= N ? mAutoBuffer : new T[size]) - { - } - - ~AutoArray() - { - if (mBuffer != mAutoBuffer) - delete[] mBuffer; - } - - /** - * Return the pointer to the allocated array. - * @note If the array allocation failed, get() will return nullptr! - * - * @return the pointer to the allocated array - */ - T *get() - { - return mBuffer; - } - -private: - T *mBuffer; // Points to mAutoBuffer if we can use it, heap otherwise. - T mAutoBuffer[N]; // The internal memory buffer that we use if we can. -}; - /** * Compute the Levenshtein Edit Distance between two strings. * @@ -206,17 +164,13 @@ levenshteinDistance(const nsAString &aStringS, // left-to-right within each row. The computation only requires that // we be able to see the current row and the previous one. - // Allocate memory for two rows. Use AutoArray's to manage the memory - // so we don't have to explicitly free it, and so we can avoid the expense - // of memory allocations for relatively small strings. - AutoArray row1(sLen + 1); - AutoArray row2(sLen + 1); + // Allocate memory for two rows. + nsAutoTArray row1; + nsAutoTArray row2; // Declare the raw pointers that will actually be used to access the memory. - int *prevRow = row1.get(); - NS_ENSURE_TRUE(prevRow, SQLITE_NOMEM); - int *currRow = row2.get(); - NS_ENSURE_TRUE(currRow, SQLITE_NOMEM); + int *prevRow = row1.AppendElements(sLen + 1); + int *currRow = row2.AppendElements(sLen + 1); // Initialize the first row. for (uint32_t i = 0; i <= sLen; i++) diff --git a/testing/config/mozbase_requirements.txt b/testing/config/mozbase_requirements.txt index 743d5ab3c3..07e47c0cd7 100644 --- a/testing/config/mozbase_requirements.txt +++ b/testing/config/mozbase_requirements.txt @@ -12,5 +12,6 @@ ../mozbase/mozprocess ../mozbase/mozprofile ../mozbase/mozrunner +../mozbase/mozscreenshot ../mozbase/moztest ../mozbase/mozversion diff --git a/testing/config/mozharness/linux_config.py b/testing/config/mozharness/linux_config.py index 8613c70fa5..40aea809ca 100644 --- a/testing/config/mozharness/linux_config.py +++ b/testing/config/mozharness/linux_config.py @@ -107,7 +107,8 @@ config = { "options": [ "--symbols-path=%(symbols_path)s", "--test-plugin-path=%(test_plugin_path)s", - "--log-raw=%(raw_log_file)s" + "--log-raw=%(raw_log_file)s", + "--utility-path=tests/bin", ], "run_filename": "runxpcshelltests.py", "testsdir": "xpcshell" diff --git a/testing/mochitest/Makefile.in b/testing/mochitest/Makefile.in index 64eddfaa58..92bd858378 100644 --- a/testing/mochitest/Makefile.in +++ b/testing/mochitest/Makefile.in @@ -33,7 +33,6 @@ ifeq ($(OS_ARCH),WINNT) TEST_HARNESS_BINS += \ crashinject$(BIN_SUFFIX) \ crashinjectdll$(DLL_SUFFIX) \ - vmwarerecordinghelper$(DLL_SUFFIX) \ $(NULL) endif diff --git a/testing/mochitest/browser-harness.xul b/testing/mochitest/browser-harness.xul index dfe2dd44d3..a513b66cf0 100644 --- a/testing/mochitest/browser-harness.xul +++ b/testing/mochitest/browser-harness.xul @@ -11,6 +11,7 @@ width="1024"> diff --git a/testing/modules/StructuredLog.jsm b/testing/modules/StructuredLog.jsm index 1a08706044..830f59f78a 100644 --- a/testing/modules/StructuredLog.jsm +++ b/testing/modules/StructuredLog.jsm @@ -5,7 +5,8 @@ "use strict"; this.EXPORTED_SYMBOLS = [ - "StructuredLogger" + "StructuredLogger", + "StructuredFormatter" ]; /** @@ -22,148 +23,198 @@ this.EXPORTED_SYMBOLS = [ * These will each be called with the complete object to log as an * argument. */ -this.StructuredLogger = function (name, dumpFun=dump, mutators=[]) { +this.StructuredLogger = function(name, dumpFun=dump, mutators=[]) { this.name = name; this._dumpFun = dumpFun; this._mutatorFuns = mutators; - this._runningTests = new Set(); } /** * Log functions producing messages in the format specified by mozlog */ -StructuredLogger.prototype.testStart = function (test) { - this._runningTests.add(test); - let data = {test: test}; - this._logData("test_start", data); -} +StructuredLogger.prototype = { + testStart: function (test) { + var data = {test: test}; + this._logData("test_start", data); + }, -StructuredLogger.prototype.testStatus = function (test, subtest, status, expected="PASS", - message=null, stack=null, extra=null) { - let data = { - test: test, - subtest: subtest, - status: status, - }; + testStatus: function (test, subtest, status, expected="PASS", + message=null, stack=null, extra=null) { - if (expected != status && status != "SKIP") { - data.expected = expected; - } - if (message !== null) { - data.message = message; - } - if (stack !== null) { - data.stack = stack; - } - if (extra !== null) { - data.extra = extra; - } + if (subtest === null || subtest === undefined) { + // Fix for assertions that don't pass in a name + subtest = "undefined assertion name"; + } - this._logData("test_status", data); -} + var data = { + test: test, + subtest: subtest, + status: status, + }; -StructuredLogger.prototype.testEnd = function (test, status, expected="OK", message=null, - stack=null, extra=null) { - let data = {test: test, status: status}; + if (expected != status && status != "SKIP") { + data.expected = expected; + } + if (message !== null) { + data.message = String(message); + } + if (stack !== null) { + data.stack = stack; + } + if (extra !== null) { + data.extra = extra; + } - if (expected != status && status != "SKIP") { - data.expected = expected; + this._logData("test_status", data); + }, + + testEnd: function (test, status, expected="OK", message=null, stack=null, extra=null) { + var data = {test: test, status: status}; + + if (expected != status && status != "SKIP") { + data.expected = expected; + } + if (message !== null) { + data.message = String(message); + } + if (stack !== null) { + data.stack = stack; + } + if (extra !== null) { + data.extra = extra; + } + + this._logData("test_end", data); + }, + + suiteStart: function (tests, runinfo=null) { + var data = {tests: tests}; + if (runinfo !== null) { + data.runinfo = runinfo; + } + + this._logData("suite_start", data); + }, + + suiteEnd: function () { + this._logData("suite_end"); + }, + + + /** + * Unstructured logging functions. The "extra" parameter can always by used to + * log suite specific data. If a "stack" field is provided it is logged at the + * top level of the data object for the benefit of mozlog's formatters. + */ + log: function (level, message, extra=null) { + var data = { + level: level, + message: String(message), + }; + + if (extra !== null) { + data.extra = extra; + if ("stack" in extra) { + data.stack = extra.stack; + } + } + + this._logData("log", data); + }, + + debug: function (message, extra=null) { + this.log("DEBUG", message, extra); + }, + + info: function (message, extra=null) { + this.log("INFO", message, extra); + }, + + warning: function (message, extra=null) { + this.log("WARNING", message, extra); + }, + + error: function (message, extra=null) { + this.log("ERROR", message, extra); + }, + + critical: function (message, extra=null) { + this.log("CRITICAL", message, extra); + }, + + _logData: function (action, data={}) { + var allData = { + action: action, + time: Date.now(), + thread: null, + pid: null, + source: this.name + }; + + for (var field in data) { + allData[field] = data[field]; + } + + for (var fun of this._mutatorFuns) { + fun(allData); + } + + this._dumpFun(allData); } - if (message !== null) { - data.message = message; - } - if (stack !== null) { - data.stack = stack; - } - if (extra !== null) { - data.extra = extra; - } - - if (!this._runningTests.has(test)) { - this.error("Test \"" + test + "\" was ended more than once or never started. " + - "Ended with this data: " + JSON.stringify(data)); - return; - } - - this._runningTests.delete(test); - this._logData("test_end", data); -} - -StructuredLogger.prototype.suiteStart = function (tests, runinfo=null) { - - let data = {tests: tests}; - if (runinfo !== null) { - data.runinfo = runinfo; - } - - this._logData("suite_start", data); }; -StructuredLogger.prototype.suiteEnd = function () { - this._logData("suite_end"); -}; /** - * Unstructured logging functions. The "extra" parameter can always by used to - * log suite specific data. If a "stack" field is provided it is logged at the - * top level of the data object for the benefit of mozlog's formatters. + * StructuredFormatter: Formatter class turning structured messages + * into human-readable messages. */ -StructuredLogger.prototype.log = function (level, message, extra=null) { - let data = { - level: level, - message: message, - }; - - if (extra !== null) { - data.extra = extra; - if ("stack" in extra) { - data.stack = extra.stack; - } - } - - this._logData("log", data); -} - -StructuredLogger.prototype.debug = function (message, extra=null) { - this.log("DEBUG", message, extra); -} - -StructuredLogger.prototype.info = function (message, extra=null) { - this.log("INFO", message, extra); -} - -StructuredLogger.prototype.warning = function (message, extra=null) { - this.log("WARNING", message, extra); -} - -StructuredLogger.prototype.error = function (message, extra=null) { - this.log("ERROR", message, extra); -} - -StructuredLogger.prototype.critical = function (message, extra=null) { - this.log("CRITICAL", message, extra); -} - -StructuredLogger.prototype._logData = function (action, data={}) { - let allData = { - action: action, - time: Date.now(), - thread: null, - pid: null, - source: this.name - }; - - for (let field in data) { - allData[field] = data[field]; - } - - for (let fun of this._mutatorFuns) { - fun(allData); - } - - this._dumpMessage(allData); +this.StructuredFormatter = function() { + this.testStartTimes = {}; +}; + +StructuredFormatter.prototype = { + + log: function(message) { + return message.message; + }, + + suite_start: function(message) { + this.suiteStartTime = message.time; + return "SUITE-START | Running " + message.tests.length + " tests"; + }, + + test_start: function(message) { + this.testStartTimes[message.test] = new Date().getTime(); + return "TEST-START | " + message.test; + }, + + test_status: function(message) { + var statusInfo = message.test + " | " + message.subtest + + (message.message ? " | " + message.message : ""); + if (message.expected) { + return "TEST-UNEXPECTED-" + message.status + " | " + statusInfo + + " - expected: " + message.expected; + } else { + return "TEST-" + message.status + " | " + statusInfo; + } + }, + + test_end: function(message) { + var startTime = this.testStartTimes[message.test]; + delete this.testStartTimes[message.test]; + var statusInfo = message.test + (message.message ? " | " + String(message.message) : ""); + var result; + if (message.expected) { + result = "TEST-UNEXPECTED-" + message.status + " | " + statusInfo + + " - expected: " + message.expected; + } else { + return "TEST-" + message.status + " | " + statusInfo; + } + result = " | took " + message.time - startTime + "ms"; + }, + + suite_end: function(message) { + return "SUITE-END | took " + message.time - this.suiteStartTime + "ms"; + } }; -StructuredLogger.prototype._dumpMessage = function (message) { - this._dumpFun(JSON.stringify(message)); -} diff --git a/testing/modules/moz.build b/testing/modules/moz.build index fd9247f803..8e3b707379 100644 --- a/testing/modules/moz.build +++ b/testing/modules/moz.build @@ -14,3 +14,5 @@ TESTING_JS_MODULES += [ 'StructuredLog.jsm', 'TestUtils.jsm', ] + +TEST_HARNESS_FILES.testing.mochitest.tests.SimpleTest += ['StructuredLog.jsm'] diff --git a/testing/modules/tests/xpcshell/test_structuredlog.js b/testing/modules/tests/xpcshell/test_structuredlog.js index 97778fa5d5..3a6fbb8ed8 100644 --- a/testing/modules/tests/xpcshell/test_structuredlog.js +++ b/testing/modules/tests/xpcshell/test_structuredlog.js @@ -7,7 +7,7 @@ function run_test () { let testBuffer = []; let appendBuffer = function (msg) { - testBuffer.push(msg); + testBuffer.push(JSON.stringify(msg)); } let assertLastMsg = function (refData) { @@ -48,14 +48,12 @@ function run_test () { // Test end / start actions logger.testStart("aTest"); - ok(logger._runningTests.has("aTest")); assertLastMsg({ test: "aTest", action: "test_start", }); logger.testEnd("aTest", "OK"); - ok(!logger._runningTests.has("aTest")); assertLastMsg({ test: "aTest", action: "test_end", @@ -93,20 +91,6 @@ function run_test () { status: "SKIP" }); - // Ending a test twice logs an error. - logger.testEnd("aTest", "PASS"); - assertLastMsg({ - action: "log", - level: "ERROR" - }); - - // Ending a test than never started logs an error. - logger.testEnd("errantTest", "PASS"); - assertLastMsg({ - action: "log", - level: "ERROR" - }); - logger.testStatus("aTest", "foo", "PASS", "PASS", "Passed test"); ok(!JSON.parse(testBuffer[testBuffer.length - 1]).hasOwnProperty("expected")); assertLastMsg({ diff --git a/testing/mozbase/moz.build b/testing/mozbase/moz.build index 2c6ab5e2d9..351324358c 100644 --- a/testing/mozbase/moz.build +++ b/testing/mozbase/moz.build @@ -23,6 +23,7 @@ python_modules = [ 'mozprocess', 'mozprofile', 'mozrunner', + 'mozscreenshot', 'mozsystemmonitor', 'moztest', 'mozversion', diff --git a/testing/mozbase/mozdebug/mozdebug/mozdebug.py b/testing/mozbase/mozdebug/mozdebug/mozdebug.py index 3f3a54d6be..02b4746678 100644 --- a/testing/mozbase/mozdebug/mozdebug/mozdebug.py +++ b/testing/mozbase/mozdebug/mozdebug/mozdebug.py @@ -11,7 +11,8 @@ from distutils.spawn import find_executable __all__ = ['get_debugger_info', 'get_default_debugger_name', - 'DebuggerSearch'] + 'DebuggerSearch', + 'get_default_valgrind_args'] ''' Map of debugging programs to information about them, like default arguments @@ -49,20 +50,6 @@ _DEBUGGER_INFO = { 'wdexpress.exe': { 'interactive': True, 'args': ['-debugexe'] - }, - - # valgrind doesn't explain much about leaks unless you set the - # '--leak-check=full' flag. But there are a lot of objects that are - # semi-deliberately leaked, so we set '--show-possibly-lost=no' to avoid - # uninteresting output from those objects. We set '--smc-check==all-non-file' - # and '--vex-iropt-register-updates=allregs-at-mem-access' so that valgrind - # deals properly with JIT'd JavaScript code. - 'valgrind': { - 'interactive': False, - 'args': ['--leak-check=full', - '--show-possibly-lost=no', - '--smc-check=all-non-file', - '--vex-iropt-register-updates=allregs-at-mem-access'] } } @@ -166,3 +153,67 @@ def get_default_debugger_name(search=DebuggerSearch.OnlyFirst): return None return None + +# Defines default values for Valgrind flags. +# +# --smc-check=all-non-file is required to deal with code generation and +# patching by the various JITS. Note that this is only necessary on +# x86 and x86_64, but not on ARM. This flag is only necessary for +# Valgrind versions prior to 3.11. +# +# --vex-iropt-register-updates=allregs-at-mem-access is required so that +# Valgrind generates correct register values whenever there is a +# segfault that is caught and handled. In particular OdinMonkey +# requires this. More recent Valgrinds (3.11 and later) provide +# --px-default=allregs-at-mem-access and +# --px-file-backed=unwindregs-at-mem-access +# which provide a significantly cheaper alternative, by restricting the +# precise exception behaviour to JIT generated code only. +# +# --trace-children=yes is required to get Valgrind to follow into +# content and other child processes. The resulting output can be +# difficult to make sense of, and --child-silent-after-fork=yes +# helps by causing Valgrind to be silent for the child in the period +# after fork() but before its subsequent exec(). +# +# --trace-children-skip lists processes that we are not interested +# in tracing into. +# +# --leak-check=full requests full stack traces for all leaked blocks +# detected at process exit. +# +# --show-possibly-lost=no requests blocks for which only an interior +# pointer was found to be considered not leaked. +# +# +# TODO: pass in the user supplied args for V (--valgrind-args=) and +# use this to detect if a different tool has been selected. If so +# adjust tool-specific args appropriately. +# +# TODO: pass in the path to the Valgrind to be used (--valgrind=), and +# check what flags it accepts. Possible args that might be beneficial: +# +# --num-transtab-sectors=24 [reduces re-jitting overheads in long runs] +# --px-default=allregs-at-mem-access +# --px-file-backed=unwindregs-at-mem-access +# [these reduce PX overheads as described above] +# +def get_default_valgrind_args(): + return (['--fair-sched=yes', + '--smc-check=all-non-file', + '--vex-iropt-register-updates=allregs-at-mem-access', + '--trace-children=yes', + '--child-silent-after-fork=yes', + '--leak-check=full', + '--show-possibly-lost=no', + ('--trace-children-skip=' + + '/usr/bin/hg,/bin/rm,*/bin/certutil,*/bin/pk12util,' + + '*/bin/ssltunnel,*/bin/uname,*/bin/which,*/bin/ps,' + + '*/bin/grep,*/bin/java'), + ] + + get_default_valgrind_tool_specific_args()) + +def get_default_valgrind_tool_specific_args(): + return [ + '--partial-loads-ok=yes' + ] diff --git a/testing/mozbase/mozleak/mozleak/leaklog.py b/testing/mozbase/mozleak/mozleak/leaklog.py index 8feddbddd8..9dee506188 100644 --- a/testing/mozbase/mozleak/mozleak/leaklog.py +++ b/testing/mozbase/mozleak/mozleak/leaklog.py @@ -5,7 +5,10 @@ import os import re +import sys +import mozinfo +import mozrunner.utils def _raw_log(): import logging @@ -13,7 +16,8 @@ def _raw_log(): def process_single_leak_file(leakLogFileName, processType, leakThreshold, - ignoreMissingLeaks, log=None): + ignoreMissingLeaks, log=None, + stackFixer=None): """Process a single leak log. """ @@ -44,7 +48,8 @@ def process_single_leak_file(leakLogFileName, processType, leakThreshold, matches = lineRe.match(line) if not matches: # eg: the leak table header row - log.info(line.rstrip()) + strippedLine = line.rstrip() + log.info(stackFixer(strippedLine) if stackFixer else strippedLine) continue name = matches.group("name").rstrip() size = int(matches.group("size")) @@ -136,7 +141,8 @@ def process_single_leak_file(leakLogFileName, processType, leakThreshold, def process_leak_log(leak_log_file, leak_thresholds=None, - ignore_missing_leaks=None, log=None): + ignore_missing_leaks=None, log=None, + stack_fixer=None): """Process the leak log, including separate leak logs created by child processes. @@ -206,4 +212,4 @@ def process_leak_log(leak_log_file, leak_thresholds=None, leakThreshold = leakThresholds.get(processType, 0) process_single_leak_file(thisFile, processType, leakThreshold, processType in ignoreMissingLeaks, - log=log) + log=log, stackFixer=stack_fixer) diff --git a/testing/mozbase/mozlog/mozlog/commandline.py b/testing/mozbase/mozlog/mozlog/commandline.py index 0fcab9fb0e..012304da62 100644 --- a/testing/mozbase/mozlog/mozlog/commandline.py +++ b/testing/mozbase/mozlog/mozlog/commandline.py @@ -37,7 +37,10 @@ def buffer_handler_wrapper(handler, buffer_limit): buffer_limit = None else: buffer_limit = int(buffer_limit) - return handlers.BufferingLogFilter(handler, buffer_limit) + return handlers.BufferHandler(handler, buffer_limit) + +def valgrind_handler_wrapper(handler): + return handlers.ValgrindHandler(handler) def default_formatter_options(log_type, overrides): formatter_option_defaults = { @@ -151,28 +154,37 @@ def setup_handlers(logger, formatters, formatter_options): for fmt, streams in formatters.iteritems(): formatter_cls = log_formatters[fmt][0] formatter = formatter_cls() - handler_wrapper, handler_option = None, "" + handler_wrappers_and_options = [] + for option, value in formatter_options[fmt].iteritems(): - if option == "buffer": - handler_wrapper, handler_option = fmt_options[option][0], value + wrapper, wrapper_args = None, () + if option == "valgrind": + wrapper = valgrind_handler_wrapper + elif option == "buffer": + wrapper, wrapper_args = fmt_options[option][0], (value,) else: formatter = fmt_options[option][0](formatter, value) + if wrapper is not None: + handler_wrappers_and_options.append((wrapper, wrapper_args)) + for value in streams: handler = handlers.StreamHandler(stream=value, formatter=formatter) - if handler_wrapper: - handler = handler_wrapper(handler, handler_option) + for wrapper, wrapper_args in handler_wrappers_and_options: + handler = wrapper(handler, *wrapper_args) logger.add_handler(handler) -def setup_logging(suite, args, defaults=None, formatter_defaults=None): +def setup_logging(logger, args, defaults=None, formatter_defaults=None): """ Configure a structuredlogger based on command line arguments. The created structuredlogger will also be set as the default logger, and can be retrieved with :py:func:`~mozlog.get_default_logger`. - :param suite: The name of the testsuite being run + :param logger: A StructuredLogger instance or string name. If a string, a + new StructuredLogger instance will be created using + `logger` as the name. :param args: A dictionary of {argument_name:value} produced from parsing the command line arguments for the application :param defaults: A dictionary of {formatter name: output stream} to apply @@ -185,7 +197,9 @@ def setup_logging(suite, args, defaults=None, formatter_defaults=None): :rtype: StructuredLogger """ - logger = StructuredLogger(suite) + if not isinstance(logger, StructuredLogger): + logger = StructuredLogger(logger) + # Keep track of any options passed for formatters. formatter_options = {} # Keep track of formatters and list of streams specified. @@ -207,7 +221,9 @@ def setup_logging(suite, args, defaults=None, formatter_defaults=None): parts = name.split('_') if len(parts) > 3: continue - # Our args will be ['log', ] or ['log', ,